DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The following action is in response to the original filing of 03/22/2023.
Claims 1-20 are pending and have been considered below.
Information Disclosure Statement
The IDS of 03/22/2023 has been only partially considered. At least the NPL Listings of (4), (5) and (8) fail to provide any dates.
Claim Objections
Claims 1 and 11 are objected to because of the following informalities: use of “based on the data” in line 5 appears to refer to the input data in line 4. As no other “data” is recited, it is clear that “the data” refers to the “input data”. Appropriate correction to state “based on the input data” would provide further clarity.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3, 11 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 11, the use of “re-train the neural network” in line 12 appears to refer to the “trained neural network” of lines 6-7 but could also refer to the pre-trained “neural network” of line 4. It is unclear to which neural network the re-training occurs or if the neural network are all the same neural network in a pre-trained, trained and re-trained state.
Regarding claims 3 and 13, the use of “second neural networks” in line 1 is unclear as it could refer to either the pre-trained, trained or re-trained neural network of parent claim 1 or new neural networks which are not the neural network of claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-8, 11-14 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over DALLY et al., US 2020/0249674 A1 published 08/06/2020 [“DALLY”] in view of BAI et al., US 2019/0147350 A1 published 05/16/2019 [“BAI”].
Regarding of claim 1, DALLY discloses a system, comprising: a computer that includes a processor and a memory, the memory including instructions executable by the processor to ([0047]):
train a neural network to input data and output a prediction ([0013], “The selected path and related data can be used to update one or more machine learning models that were used for the determination, such as by sending the relevant data to a remote server capable of further training the models, which can then be used for future determinations.”. [0057], “In this example, a set of classified data 702 is provided as input to function as training data”. [0058] “The model can be trained in a number of different ways, as may depend in part upon the type of model selected. For example, in one embodiment a machine learning algorithm can be provided with a set of training data, where the model is a model artifact created by the training process. The learning algorithm finds patterns in the training data that map the input data attributes to the target, the answer to be predicted, and a machine learning model is output that captures these patterns”. Fig. 7. – a neural network uses data as input and is trained to output the predictions);
generate a policy based on the data ([0030], “In one embodiment, a policy function (part of a policy network) can be used to predict the best, or most probable, options at each level so not all options need to be explored further” – the policy function and data are used to generate the best policy at each level/data)
generate force features based on the policy ([0044], “Other information can be determined as well, such as the velocities and directions of movement of the other vehicles, presence of road signals or traffic lights, brake lights, turn signals, presence of pedestrians, presence of construction zones, road status, or weather conditions, as may be determined using one or more vehicle sensors as discussed herein or obtained from other appropriate sources. Based at least in part upon the determined vehicle characteristics, one or more sequences of possible actions can be determined 404, where those sequences include alternating levels of possible actions of the first vehicle and probable responsive actions of the other vehicles. As mentioned, in some embodiments only the most probable responsive actions may be considered in some embodiments “. [0036] “In the present example, the goal is to change lanes to the right within some target distance. Other goals might be to change lanes to the left, to exit (left or right), to maximize velocity (in any lane), or to turn at an intersection (left or right), among other such options.” Also see [0029-0030], “It should be understood that leaf nodes may exist at other levels as well, such as where a vehicle might reach a destination, collision, or other endpoint along a given path. As mentioned, a highest value leaf can be determined, and the corresponding sequence of actions for that path is provided as the five to ten second plan, which can be provided to an optimizer or controller in some embodiments to determine the actions to be taken over the next short time interval, such as the next SOms. The optimizer can take the path data, smooth out the actions, and produce a trajectory for the vehicle with a much finer granularity. In some embodiments the time interval output by the five-to-ten second planner may be variable, and may be based at least in part upon the type of action to be taken or various environmental factors. [0030] In some embodiments deep learning can be used to make the tree more efficient as discussed in more detail elsewhere herein. In one embodiment, a policy function (part of a policy network) can be used to predict the best, or most probable, options at each level so not all options need to be explored further. ” – the force features include ex. velocity, the policy function is used to help generating predictions for the best trajectory for the vehicle using forces around the vehicle);
generate a decision tree based on the decision nodes ([0005, Fig. 3], FIG. 3 “illustrates an example decision tree that can be utilized to determine a highest value navigation option in accordance with various embodiments”. [0028] “The result may then be a tree structure 300 such as illustrated in the example of FIG. 3. In this example there are many levels of nodes, with each non-leaf node having a number of branches extending from that node.” – a decision tree with decision nodes.)
train decision nodes based on force features from the trained neural network and binary vector from the trained neural network, ([0013], “For each possible action of the present object, one or more probable responsive or reactive actions of the nearby objects (i.e., actors) can be determined. This can take the form of a decision tree in some embodiments, with alternating levels of nodes corresponding to possible actions of the present vehicle and probable responsive actions of one or more other vehicles or actors. See [0054], “A deep neural network (DNN) model includes multiple layers of many connected perceptrons (e.g., nodes) that can be trained with enormous amounts of input data to quickly solve complex problems with high accuracy. In one example, a first layer of the DLL model breaks down an input image of an automobile into various sections and looks for basic patterns such as lines and angles. The second layer assembles the lines to look for higher level patterns such as wheels, windshields, and mirrors. The next layer identifies the type of vehicle, and the final few layers generate a label for the input image, identifying the model of a specific automobile brand.” Also see [0075], “An example CNN uses three different filter sizes, which represent potentially different n-gram sizes. The number of filters per filter size can correspond to the depth of the filter. Each filter attempts to learn something different from the structure of the instance, such as the sentence structure for textual data. In the convolutional layer, the activation function can be a rectified linear unit and the pooling type set as max pooling. The results can then be concatenated into a single dimensional vector, and the last layer is fully connected onto a two-dimensional output. This corresponds to the binary classification to which an optimization function can be applied” – a CNN, DNN can be trained using a single dimensional vector, and the single dimensional vector is corresponds to the binary classification, and reactive actions can be interpreted as force features);
generate a decision by inputting the policy to the decision tree ([0045], “One or more move generators, such as may include neural networks, can be used to generate 504 a decision tree, where the decision tree includes alternative levels of actions for the first vehicle and responsive actions for at least the second vehicle. The probability of the responsive actions can be determined 506 at the various levels, where only actions with at least a minimum probability may be considered. In various embodiments this is performed using a policy network that uses the characterizations of the various other objects to suggest, using one or more relevant policies that accept the characterization classifications, parameters, or scalars to determine the probable responsive actions for those vehicles. In various embodiments this can approximate a Monte Carlo-based approach to determining probabilities of various actions. A selected value function can be used to calculate 508 a path value for each path of the decision tree.“ [0037] “In some embodiments the move generator can be augmented with a policy network-based tree search, as discussed herein, to generate better moves than using the reinforcement learning scheme directly. The tree search can function as a multi-modal probability determination that can determine the risk and uncertainty for potential responsive actions“. –policies are used/input to generate a decision tree by one or more move generator);
re-train the neural network ([0070], “In some embodiments the model will be continually trained as new data is available, but in other embodiments the models will be retrained periodically, such as once a day or week, depending upon factors such as the size of the data set or complexity of the model.” – the neural network can be retrained)
DALLY fails to explicitly disclose that the re-training of the neural network is based on a difference between a comparison of the decision and the prediction.
BAI discloses methods for adjusting prediction models that include determined decision trees, an analogous art ([0022]). In particular, BAI discloses comparing a decision to a prediction and re-train a neural network based on a difference between the decision and the prediction. ([0025] “for adjusting a prediction model, comprising: presenting the prediction model using the method described as the first aspect above or in combination with various possible embodiments of the first aspect above; adjusting at least one prediction model training sample on a basis of which the prediction model is trained, in response to an input operation performed by a user with respect to the presented decision tree model; and retraining the prediction model using the adjusted at least one prediction model training sample.” [0059], [0080], [00114-00118] and [Fig. 5] – The user adjusts the input operation using predicted sample, the model then updates/adjust the decision tree so that it contains the predicted sample and then compare which direction is better).
Therefore, it would have been obvious to one having ordinary skill in the art and the teachings of DALLY and BAI before the effective filing of the claimed invention to combine a comparison of a decision of a model with a prediction of the model then retrain the model based on the difference between decision and prediction, as taught by BAI with the re-training of the neural network model including decisions and predictions of DALLY. One would have been motivated to make this combination in order to allow user to learn more data information, understand the models more or complete more related settings by performing operations on the decision tree model, as suggested by BAI [0020], [0021], [0118]).
Regarding claim 2, DALLY and BAI disclose the system of claim 1, and DALLY further discloses wherein the decision tree is determined based on user input ([0078] – User can input command to adjust the decision tree).
Regarding claim 3, DALLY and BAI disclose the system of claim 1, and BAI further discloses wherein the decision nodes are second neural networks ([0024], [0025], [0026] - trained neural network is used as decision nodes)
Regarding claim 4, DALLY and BAI teach the system of claim 1, and DALLY further teaches wherein the decision nodes are trained using supervised learning based on a binary vector output from the trained neural network and the force features based on the policy ([0077], [0073], – the decision nodes are trained using supervised learning).
Regarding claim 6, DALLY and BAI teach the system of claim 1, and DALLY further teaches wherein the decision nodes output binary values ([0059] – the model predicts binary outcome.).
Regarding claim 7, DALLY and BAI teach the system of claim 1, and DALLY further discloses wherein the trained neural network inputs data and outputs predictions regarding categorical variables that are directives usable for determining a trajectory ([0015] – the trajectory is predicted based on factors/categorical variables and is used directly).
Regarding claim 8, DALLY and BAI teach the system of claim 7, and DALLY further discloses wherein the trained neural network is output to a second computer in a vehicle and the trajectory is used to operate the vehicle ([0081], [0078], [0015], Fig. 9 – the model outputs to display elements and the trajectory is used to maneuvers).
Regarding claims 11-14 and 16-18, claims 11-14 and 16-18 recite limitations similar to claims 1-4, 6-8, respectively and are similarly rejected.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over DALLY in view of BAI and in further view Qiao, Litao, Weijia Wang, and Bill Lin. "Learning accurate and interpretable decision rule sets from neural networks." Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 5. 2021 [“QIAO”].
Regarding claim 5, DALLY and BAI disclose the system of claim 1.
DALLY and BAI fails to disclose wherein the decision nodes determine rules in disjunctive normal form.
QIAO discloses methods for modifying classification decision tree models, an analogous art (page 4304 col 1 pp. 3-4: “In contrast, our neural net based approach does not require rules mining and can take advantage of well-developed neural net training techniques to de rive better interpretable models. By connecting interpretable rule-based learning to a neural network based formulation, we hope to open a new line of research that will lead to further fruitful results in the future. Our experimental results show that our method can generate more accurate decision rule sets than other state-of-the art rule-learning algorithms with better accuracy-simplicity trade-offs. Further, when compared with uninterpretable black box machine learning approaches such as random forests and full-precision deep neural networks, our approach can easily find interpretable decision rule sets that have comparable predictive performance.”). In particular, QIAO discloses determining rules of decision nodes in disjunctive normal form (page 4304 col 2 pp. 3: “In this section, we introduce the architecture of our Decision Rules Network (DR-Net), which is a simple two-layer fully-connected neural network. The first layer, called the Rules Layer, consists of trainable neurons that map to logical IF-THEN rules, and the second layer, called the OR Layer, contains a trainable output neuron that maps to a disjunction of the first-layer rules to form the decision rule set. The goal of the design of this network is to simulate the logical formula in disjunctive normal form so that a trained DR Net can be directly mapped to a set of interpretable decision rules”).
Therefore, it would have been obvious to one having ordinary skill in the art and the teachings of DALLY, BAI and QIAIO before them before the effective filing of the claimed invention to combine the determination of rules in disjunctive normal form for decision nodes, as taught by QIAO, with the decision node decision generation of DALLY and BAI. One would have been motivated to make this combination in order to modify the model to provide better decision making such as through the use of Boolean logic predictions, as suggested by QIAO (page 4304 col 1 pp. 4: “Our experimental results show that our method can generate more accurate decision rule sets than other state-of-the art rule-learning algorithms with better accuracy-simplicity trade-offs.”).
Regarding claim 15, claim 15 recites similar limitations to claim 5 and is similarly rejected.
Claims 9-10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over DALLY in view of BAI and in further view Caldwell et al., US 2024/0092398 A1 effective filing of 08/31/2022 [“CALDWELL”].
Regarding claim 9, DALLY and BAI disclose the system of claim 1.
DALLY and BAI fail to disclose combining the force features into a surface plot.
CALDWELL discloses using decision tree models to facilitate movement actions ([0009]), an analogous art. In particular, CALDWELL discloses combining force features into a surface plot (Fig. 1 – the object, other vehicle’s direction are also considered force features, Fig. 2, [0042] - the model comparing 2 trajectories from the surface plot and decides which trajectory to take).
Therefore, it would have been obvious to one having ordinary skill in the art and the teachings of DALLY, BAI and CALDWELL to combine a surface plot including force features, as taught by CALDWELL, with the force feature decision system of DALLY and BAI. One would have been motivated to make this combination in order to aid using illustration when getting information of the environment so the model can make better decisions, as suggested by CALDWELL ([0023]).
Regarding claim 10, DALLY, BAI and CALDWELL disclose the system of claim 9, and CALDWELL further discloses comparing the prediction output from the neural network to the surface plot (Fig. 2, [0042] - the model comparing 2 trajectories from the surface plot and decides which trajectory to take).
Regarding claims 19-20, claims 19-20 recite limitations similar to claims 9-10, respectively, and are similarly rejected.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xu, Ming-wei et al.
CN 111898692 A
CONVERSION METHOD OF NEURAL NETWORK TO DECISION TREE, STORAGE MEDIUM AND ELECTRONIC DEVICE
Ferguson; David Ian Franklin et al.
US 10173673 B1
DISTRIBUTION DECISION TREES
Bender; Michael et al.
US 10703381 B2
INTELLIGENT VEHICLE ACTION DECISIONS
Ogale; Abhijit et al.
US 20190034794 A1
NEURAL NETWORKS FOR VEHICLE TRAJECTORY PLANNING
Nishimura; Kazuya et al.
US 20190126932 A1
EVALUATING DRIVER TRAFFIC TRAJECTORY USING COMPARISONS
KIZUMI; Yuki
US 20200081436 A1
POLICY GENERATION DEVICE AND VEHICLE
Seo; Hae-Jong et al.
US 20200090322 A1
DEEP NEURAL NETWORK PROCESSING FOR SENSOR BLINDNESS DETECTION IN AUTONOMOUS MACHINE APPLICATIONS
Bhalla S et al.
US 20200174471 A1
REINFORCEMENT LEARNING AND PLANNING (RLP) SYSTEM FOR HOST VEHICLE, HAS PROCESSOR THAT EXECUTES INSTRUCTIONS OF RLP ALGORITHM FOR COLLABORATIVE ACTION PLANNING BETWEEN HOST AND OTHER VEHICLES BASED ON OUTPUT OF EVALUATION AND TARGET NETWORKS
Gao; Jiyang et al.
US 20210191395 A1
VEHICLE INTENT PREDICTION NEURAL NETWORK
Solmaz; Gürkan et al.
US 20220161818 A1
METHOD AND SYSTEM FOR SUPPORTING AUTONOMOUS DRIVING OF AN AUTONOMOUS VEHICLE
Pulver; Henry et al.
US 20230089978 A1
PLANNING IN MOBILE ROBOTS
Agrawal; Roli et al.
US 20230359449 A1
LEARNING-AUGMENTED APPLICATION DEPLOYMENT PIPELINE
Wang; Zhen et al.
US 20240166200 A1
VEHICLE CONTROL METHOD AND APPARATUS THEREOF
Bastani, Osbert, Yewen Pu, and Armando Solar-Lezama. "Verifiable reinforcement learning via policy extraction." Advances in neural information processing systems 31 (2018).
Huang, Yanjun, et al. "A survey on trajectory-prediction methods for autonomous driving." IEEE transactions on intelligent vehicles 7.3 (2022): 652-674.
Yildirim, Mustafa, et al. "Prediction based decision making for autonomous highway driving." 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022.
Arifuzzaman, Mohammad, et al. "An advanced decision tree-based deep neural network in nonlinear data classification." Technologies 11.1 (2023): 24.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW L TANK whose telephone number is (571)270-1692. The examiner can normally be reached Monday-Thursday 9a-6p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW L TANK/ Primary Examiner, Art Unit 2141
Prosecution Insights