Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This is the first Office action on the merits. Claims 1-20 are currently pending and addressed below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/10/2024 has been received. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Regarding claim 1, “driving path planner” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “planner”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Regarding claim 1, “driving controller” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “controller”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Regarding claim 11, “driving path planner” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “planner”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Regarding claim 11, “driving controller” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “controller”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Regarding claim 20, “driving path planner” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “planner”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Regarding claim 20, “driving controller” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “controller”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0025) as a processor.
Claim Rejections - 35 USC § 101
Under 35 U.S.C. § 101 a claim is directed to non-statutory subject matter if:
It does not fall within one of the four statutory categories of invention (Process, Machine, Manufacture, or Composition of Matter) or
Meets a three-prong test for determining that
The claim recites a judicial exception (such as: a law of nature, a natural phenomenon, an abstract idea)
Without integration into a practical application, and
Does not recite additional elements that provide significantly more than the recited judicial exception
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
101 Analysis – Step 1: Statutory Category – Is the claim directed to one of the four statutory categories (a process, a machine, a manufacture, or a composition of matter)?
Claim 1 is directed to a method (i.e. a process), claim 11 is directed to a device (i.e. a machine), and claim 20 is directed to a vehicle (i.e. a machine). Therefore, claim(s) 1-20 are within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong I – Is the claim directed to a judicial exception?
The judicial exceptions are as follows:
Abstract ideas (mathematical concepts, mental processes, and certain methods of organizing human activity)
Laws of nature (e.g., naturally occurring correlations, scientific principles)
Natural phenomena (e.g., wind)
Products of nature (e.g., a plant found in the wild, minerals)
Regarding Prong I of the Step 2A analysis in the 2019 Patent Eligibility Guidance (PEG), the claims are to be analyzed to determine whether they recite subject matter that falls within one of the
follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human
activity, and/or c) mental processes.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the limitations can
be “performed in the human mind, or by a human using a pen and paper”. See MPEP 2106.04(a)(2)(III).
Independent claim 1 includes limitations that recite an abstract idea (emphasized below).
Claim 1 recites:
1. A driving path optimization training method of a vehicle, the driving path optimization training method comprising:
receiving a first data set including a driving path and an associated driving environment information;
generating a second data set from the first data by performing data augmentation on the first data;
training a driving path planner based on the second data set; and
training a driving controller based on a training result of the training of the driving path planner.
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “receiving…” in the context of this claim encompasses a person (driver) looking at data collected or viewing an environment and “generating…” encompasses the driver determining changes to the actual state which could be possible by forming a simple judgement. Accordingly, the claim recites at least one abstract idea.
Independent claim 11 includes limitations that recite an abstract idea (emphasized below).
Claim 11 recites:
11. An electronic device, comprising:
a memory storing instructions; and
one or more processors,
wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
receive a first data set including a driving path and an associated driving environment information;
generate a second data set from the first data set by performing data augmentation on the first data set;
train a driving path planner based on the second data set; and
train a driving controller based on a training result of the driving path planner.
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “receive…” in the context of this claim encompasses a person (driver) looking at data collected or viewing an environment and “generate…” encompasses the driver determining changes to the actual state which could be possible by forming a simple judgement. Accordingly, the claim recites at least one abstract idea.
Independent claim 20 includes limitations that recite an abstract idea (emphasized below).
Claim 20 recites:
20. A vehicle comprising:
a memory storing instructions; and
one or more processors configured by the instructions to execute a driving path planner and a driving controller,
wherein the driving path planner is configured to:
receive a first data set including a driving path and an associated driving environment information;
generate a second data set from the first data set by performing data augmentation on the first data set; and
be trained based on the second data set; and
wherein the driving controller is configured to be trained based on the driving path planner as trained based on the second data set.
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “receive…” in the context of this claim encompasses a person (driver) looking at data collected or viewing an environment and “generate…” encompasses the driver determining changes to the actual state which could be possible by forming a simple judgement. Accordingly, the claim recites at least one abstract idea.
101 Analysis – Step 2A, Prong II – Does the claim, as a whole, integrate the abstract idea into a practical
application?
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into a practical application. As noted in MPEP 2106.04(d), it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. The courts have indicated that additional elements such as: merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application”.
The guidelines provide the following (non-exhaustive) list of exemplary considerations which are indicative that an additional element (or combination of elements) may have integrated the judicial element into a practical application:
An additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
An additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition
An additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture which is integral to the claim;
An additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception
The following is a (non-exhaustive) list of examples in which a judicial exception has not been integrated into a practical application:
An additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
An additional element adds insignificant extra-solutionary activity to the judicial exception;
An additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
The Office submits that the foregoing underlined limitation(s) recite additional elements that do
not integrate the recited judicial exception into a practical application.
Independent claim 1 includes limitations that recite additional limitations (emphasized below).
Claim 1 recites:
1. A driving path optimization training method of a vehicle, the driving path optimization training method comprising:
receiving a first data set including a driving path and an associated driving environment information;
generating a second data set from the first data by performing data augmentation on the first data;
training a driving path planner based on the second data set; and
training a driving controller based on a training result of the training of the driving path planner.
For the following reason(s), the examiner submits that the above identified additional
limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities that merely use a computer (processor) to perform the process. The training merely describes how to generally “apply” the otherwise mental judgements in a generic or general purpose vehicle control environment. The training is recited at a high level of generality and merely automates the mathematical process.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Independent claim 11 includes limitations that recite additional limitations (emphasized below).
Claim 11 recites:
11. An electronic device, comprising:
a memory storing instructions; and
one or more processors,
wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
receive a first data set including a driving path and an associated driving environment information;
generate a second data set from the first data set by performing data augmentation on the first data set;
train a driving path planner based on the second data set; and
train a driving controller based on a training result of the driving path planner.
For the following reason(s), the examiner submits that the above identified additional
limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities that merely use a computer (processor) to perform the process. Lastly, the “processor” merely describes how to generally “apply” the otherwise mental judgements in a generic or general purpose vehicle control environment. The processor is recited at a high level of generality and merely automates the receiving, generating and training steps.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Independent claim 20 includes limitations that recite additional limitations (emphasized below).
Claim 20 recites:
20. A vehicle comprising:
a memory storing instructions; and
one or more processors configured by the instructions to execute a driving path planner and a driving controller,
wherein the driving path planner is configured to:
receive a first data set including a driving path and an associated driving environment information;
generate a second data set from the first data set by performing data augmentation on the first data set; and
be trained based on the second data set; and
wherein the driving controller is configured to be trained based on the driving path planner as trained based on the second data set.
For the following reason(s), the examiner submits that the above identified additional
limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities that merely use a computer (processor) to perform the process. Lastly, the “processor” merely describes how to generally “apply” the otherwise mental judgements in a generic or general purpose vehicle control environment. The processor is recited at a high level of generality and merely automates the receiving, generating and training steps.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B – Do the additional elements incorporate an inventive concept to the claim?
In Step 2B of the 2019 PEG, a claim is to be evaluated as to whether the claim, as a whole, amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
Regarding independent claim 1:
Regarding Step 2B of the 2019 PEG, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a processor to perform the receiving, generating, and training amounts to nothing more than applying the exception using a generic computer component. Generally applying an exception using a generic computer component cannot provide an inventive concept. And as discussed above, the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities.
Further, a conclusion that an additional element is insignificant extra-solution activity in
Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well understood, routine, conventional activity in the field. The additional limitations of “training a driving path planner” and “training a driving controller” are well-understood, routine, and conventional activities because the background does not indicate how the training is beyond general application to technology using conventional training methods, and the specification does not provide any indication that the processor is anything other than a conventional computer within a vehicle. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner. Hence, the claim is not patent eligible.
Thus, the independent claim 1 as well as the dependent claims are directed toward an abstract idea, not integrated into a practical application, and do not comprise significantly more than the recited abstract idea.
Regarding independent claim 11:
Regarding Step 2B of the 2019 PEG, representative independent claim 11 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a processor to perform the receiving, generating, and training amounts to nothing more than applying the exception using a generic computer component. Generally applying an exception using a generic computer component cannot provide an inventive concept. And as discussed above, the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities.
Further, a conclusion that an additional element is insignificant extra-solution activity in
Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well understood, routine, conventional activity in the field. The additional limitations of “training a driving path planner” and “training a driving controller” are well-understood, routine, and conventional activities because the background does not indicate how the training is beyond general application to technology using conventional training methods, and the specification does not provide any indication that the processor is anything other than a conventional computer within a vehicle. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner. Hence, the claim is not patent eligible.
Thus, the independent claim 11 as well as the dependent claims are directed toward an abstract idea, not integrated into a practical application, and do not comprise significantly more than the recited abstract idea.
Regarding independent claim 20:
Regarding Step 2B of the 2019 PEG, representative independent claim 20 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a processor to perform the receiving, generating, and training amounts to nothing more than applying the exception using a generic computer component. Generally applying an exception using a generic computer component cannot provide an inventive concept. And as discussed above, the additional limitations of “training a driving path planner” and “training a driving controller” the examiner submits that these limitations are insignificant extra-solution activities.
Further, a conclusion that an additional element is insignificant extra-solution activity in
Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well understood, routine, conventional activity in the field. The additional limitations of “training a driving path planner” and “training a driving controller” are well-understood, routine, and conventional activities because the background does not indicate how the training is beyond general application to technology using conventional training methods, and the specification does not provide any indication that the processor is anything other than a conventional computer within a vehicle. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner. Hence, the claim is not patent eligible.
Thus, the independent claim 20 as well as the dependent claims are directed toward an abstract idea, not integrated into a practical application, and do not comprise significantly more than the recited abstract idea.
101 Analysis – Dependent Claims and Conclusion
Dependent claim(s) 2-10 and 12-19 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of dependent claims are directed toward additional aspects of the judicial exception and/or well-understood, routine and conventional additional elements that do not integrate the judicial exception into a practical application. Therefore, dependent claims 2-10 and 12-19 are not patent eligible under the same rationale as provided for in the rejection of claim(s) 1, 11, and 20.
Therefore, claim(s) 1-20 is/are ineligible under 35 USC §101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-5, 10-11, 13-15, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elmahgiubi et al. (US 20250162611 A1), hereinafter Elmahgiubi in view of Caldwell et al. (US 11565709 B1), hereinafter Caldwell.
Regarding claim 1, Elmahgiubi teaches:
1. A driving path optimization training method of a vehicle, the driving path optimization training method comprising:
receiving a first data set including a driving path and an associated driving environment information;
generating a second data set from the first data by performing data augmentation on the first data; (Paragraph 0021, "Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories.")
training a driving path planner based on the second data set; (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") and
…
Elmahgiubi does not specifically teach training a driving controller based on an outcome of a path planner. However, Caldwell, in the same field of endeavor of autonomous control, teaches:
… training a driving controller based on a training result of the training of the driving path planner. (Column 4, Lines 48-61, "In various examples, the computing system may associate the metrics with particular actions. In such examples, the computing may be configured to determine one or more high-performing actions (e.g., successful actions) for a particular scenario. In some examples, the computing system may be configured to determine an optimal action (e.g., best metrics) for a particular scenario. In various examples, the computing system may train the autonomous controller, such as utilizing machine learning techniques (e.g., reinforcement learning), to perform the optimal action in the scenario. In such examples, the computing system may cause the autonomous controller to bias toward high-scoring (e.g., successful) actions when it encounters a substantially similar scenario." as well as Column 22 Line 65 - Column 23 Line 29, "In general, the planning component 424 may determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 424 may determine various routes and vehicle trajectories and various levels of detail. For example, the planning component 424 may determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route may include a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 424 may generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 424 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instruction may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
In at least one example, the vehicle computing device(s) 404 may include one or more system controllers 426, which may be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. The system controller(s) 426 may communicate with and/or control corresponding systems of the drive system(s) 414 and/or other components of the vehicle 402.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system and methods of control as taught by Elmahgiubi with the ability to separately train the driving controller as taught by Caldwell. This would ensure that the system, along with traveling an optimal path, may operate efficiently with respect to all vehicle control systems such as braking, steering, propulsion. This may further be used to provide a more enjoyable experience for users riding within the vehicle.
Regarding claim 3, where all the limitations of claim 1 are discussed above, Elmahgiubi further teaches:
3. The driving path optimization training method of claim 1, wherein
the performing the data augmentation comprises adding noise to a posture, a location, a speed, a steering angle, a steering rate, or acceleration data of the vehicle included in the first data set. (Paragraph 0021, "Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories.")
Regarding claim 4, where all the limitations of claim 1 are discussed above, Elmahgiubi further teaches:
4. The driving path optimization training method of claim 1, wherein
the second data set comprises optimal paths generated based on data to which noise is added to the first data set, (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") based on a vehicle dynamics model (Paragraphs 0135-0136, "At step 712, according to certain non-limiting embodiments of the present technology, using the augmented dataset generated at step 710, the server 202 can be configured to execute the motion planner model training procedure 210, described above, to train the motion planner model to simulate motion of the autonomous vehicle.
Thus, certain embodiments of the method 700 allow training the motion planner model to simulate the motion of the vehicle more comprehensively. This can enable to model more hypothetical traffic scenarios of navigating the autonomous vehicle, thereby mitigating risks of accidents thereof or improving its passengers' comfort.") and an objective function. (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.")
Regarding claim 5, where all the limitations of claim 4 are discussed above, Elmahgiubi further teaches:
5. The driving path optimization training method of claim 4, wherein
the objective function induces generation of the optimal paths to minimize an error between a target path of the vehicle changed by the noise and an optimal path for the first data set. (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.")
Regarding claim 10, where all the limitations of claim 1 are discussed above, Elmahgiubi further teaches:
10. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1. (Paragraph 0067, “Moreover, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Thus, for example, it will be appreciated by those skilled in the art that any block diagram herein represents conceptual views of illustrative circuitry embodying the principles of the present technology. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes that may be substantially represented in non-transitory computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.”)
Regarding claim 11, Elmahgiubi further teaches:
11. An electronic device, comprising:
a memory storing instructions; (Paragraph 0030, “In accordance with a second broad aspect of the present technology, there is provided a system for motion planning of an autonomous vehicle. The system comprises at least one processor and at least one non-transitory computer-readable memory storing executable instructions, which, when executed by the at least one processor, cause the system to: receive an original dataset including a plurality of trajectories for the autonomous vehicle, a given trajectory of the plurality of trajectories being associated with a respective behavior of a predetermined plurality of behaviors; the plurality of trajectories being distributed over the predetermined plurality of behaviors in the original dataset according to an initial distribution; acquire an indication of a target distribution of the plurality of trajectories over the predetermined plurality of behaviors; determine a difference between the initial and target distributions, thereby identifying those of the predetermined plurality of behaviors, for which additional trajectories are to be generated for adjusting the initial distribution of the plurality of trajectories in the original dataset to the target distribution; generate a given additional trajectory of the additional trajectories by: acquiring an indication of a 3D scene for simulating motion of the autonomous vehicle along the given additional trajectory; and feeding the indication of the 3D scene along with an indication of one of the predetermined plurality of behaviors associated with the given additional trajectory to a predictive model, the predictive model has been pre-trained, based on the plurality of trajectories, to generate trajectories for input 3D scenes and one of the predetermined plurality of behaviors; generate, using the plurality of trajectories in the original dataset and the additional trajectories, an augmented dataset having the target distribution of trajectories over the predetermined plurality of behaviors; and train a motion planner model to simulate motion of the autonomous vehicle based on the augmented dataset instead of the original dataset.”) and
one or more processors, (Paragraph 0067-0068, “Moreover, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Thus, for example, it will be appreciated by those skilled in the art that any block diagram herein represents conceptual views of illustrative circuitry embodying the principles of the present technology. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes that may be substantially represented in non-transitory computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures, including any functional block labelled as a “processor” or “processing unit”, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. In some embodiments of the present technology, the processor may be a general-purpose processor, such as a central processing unit (CPU) or a processor dedicated to a specific purpose, such as a digital signal processor (DSP). Moreover, explicit use of the term a “processor” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included.”)
wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
receive a first data set including a driving path and an associated driving environment information; (Paragraphs 0020-0021, "Referring now to FIG. 2, a method of generating and using simulated training data for an autonomous vehicle model is shown. Block 202 trains a NeRF model to generate a 3D environment based on a 2D image input. This training may be based on both video information and LiDAR information, which are used to inform an RGB loss and a depth loss respectively. Block 204 then simulates scenarios using the trained NeRF model.
Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories." as well as Paragraph 0027, "Additional criteria are applied to generate photorealistic samples during the simulation of block 204. Block 204 applies these criteria to guide the sampling of positions and poses within a given 3D scenario generated from an original 2D scenario input. A first criterion is that the novel view poses should stay near the original trajectory. While the depth loss effectively captures correct geometrical information, it struggles to accurately generate unseen regions. Moving far from the original trajectory exposes those regions, so keeping the new pose close to the original trajectory keeps the scenario in a territory that can be reasonably predicted.")
generate a second data set from the first data set by performing data augmentation on the first data set; (Paragraph 0021, "Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories.")
train a driving path planner based on the second data set; (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") and
…
Elmahgiubi does not specifically teach training a driving controller based on an outcome of a path planner. However, Caldwell, in the same field of endeavor of autonomous control, teaches:
… train a driving controller based on a training result of the driving path planner. (Column 4, Lines 48-61, "In various examples, the computing system may associate the metrics with particular actions. In such examples, the computing may be configured to determine one or more high-performing actions (e.g., successful actions) for a particular scenario. In some examples, the computing system may be configured to determine an optimal action (e.g., best metrics) for a particular scenario. In various examples, the computing system may train the autonomous controller, such as utilizing machine learning techniques (e.g., reinforcement learning), to perform the optimal action in the scenario. In such examples, the computing system may cause the autonomous controller to bias toward high-scoring (e.g., successful) actions when it encounters a substantially similar scenario." as well as Column 22 Line 65 - Column 23 Line 29, "In general, the planning component 424 may determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 424 may determine various routes and vehicle trajectories and various levels of detail. For example, the planning component 424 may determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route may include a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 424 may generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 424 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instruction may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
In at least one example, the vehicle computing device(s) 404 may include one or more system controllers 426, which may be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. The system controller(s) 426 may communicate with and/or control corresponding systems of the drive system(s) 414 and/or other components of the vehicle 402.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system and methods of control as taught by Elmahgiubi with the ability to separately train the driving controller as taught by Caldwell. This would ensure that the system, along with traveling an optimal path, may operate efficiently with respect to all vehicle control systems such as braking, steering, propulsion. This may further be used to provide a more enjoyable experience for users riding within the vehicle.
Regarding claim 13, where all the limitations of claim 11 are discussed above, Elmahgiubi further teaches:
13. The electronic device of claim 11, wherein
the performing the data augmentation comprises adding noise to a posture, a location, a speed, a steering angle, a steering rate, or acceleration data of a vehicle included in the first data set. (Paragraph 0021, "Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories.")
Regarding claim 14, where all the limitations of claim 11 are discussed above, Elmahgiubi further teaches:
14. The electronic device of claim 11, wherein
the second data set comprises optimal paths generated based on data to which noise is added to the first data set, (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") based on a vehicle dynamics model (Paragraphs 0135-0136, "At step 712, according to certain non-limiting embodiments of the present technology, using the augmented dataset generated at step 710, the server 202 can be configured to execute the motion planner model training procedure 210, described above, to train the motion planner model to simulate motion of the autonomous vehicle.
Thus, certain embodiments of the method 700 allow training the motion planner model to simulate the motion of the vehicle more comprehensively. This can enable to model more hypothetical traffic scenarios of navigating the autonomous vehicle, thereby mitigating risks of accidents thereof or improving its passengers' comfort.") and an objective function. (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.")
Regarding claim 15, where all the limitations of claim 14 are discussed above, Elmahgiubi further teaches:
15. The electronic device of claim 14, wherein
the objective function induces generation of the optimal path data to minimize an error between a target path of a vehicle changed by the noise and an optimal path for the first data set. (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.")
Regarding claim20, Elmahgiubi further teaches:
20. A vehicle comprising:
a memory storing instructions; (Paragraph 0030, “In accordance with a second broad aspect of the present technology, there is provided a system for motion planning of an autonomous vehicle. The system comprises at least one processor and at least one non-transitory computer-readable memory storing executable instructions, which, when executed by the at least one processor, cause the system to: receive an original dataset including a plurality of trajectories for the autonomous vehicle, a given trajectory of the plurality of trajectories being associated with a respective behavior of a predetermined plurality of behaviors; the plurality of trajectories being distributed over the predetermined plurality of behaviors in the original dataset according to an initial distribution; acquire an indication of a target distribution of the plurality of trajectories over the predetermined plurality of behaviors; determine a difference between the initial and target distributions, thereby identifying those of the predetermined plurality of behaviors, for which additional trajectories are to be generated for adjusting the initial distribution of the plurality of trajectories in the original dataset to the target distribution; generate a given additional trajectory of the additional trajectories by: acquiring an indication of a 3D scene for simulating motion of the autonomous vehicle along the given additional trajectory; and feeding the indication of the 3D scene along with an indication of one of the predetermined plurality of behaviors associated with the given additional trajectory to a predictive model, the predictive model has been pre-trained, based on the plurality of trajectories, to generate trajectories for input 3D scenes and one of the predetermined plurality of behaviors; generate, using the plurality of trajectories in the original dataset and the additional trajectories, an augmented dataset having the target distribution of trajectories over the predetermined plurality of behaviors; and train a motion planner model to simulate motion of the autonomous vehicle based on the augmented dataset instead of the original dataset.”) and
one or more processors configured by the instructions to execute a driving path planner and a driving controller, (Paragraph 0067-0068, “Moreover, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Thus, for example, it will be appreciated by those skilled in the art that any block diagram herein represents conceptual views of illustrative circuitry embodying the principles of the present technology. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes that may be substantially represented in non-transitory computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures, including any functional block labelled as a “processor” or “processing unit”, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. In some embodiments of the present technology, the processor may be a general-purpose processor, such as a central processing unit (CPU) or a processor dedicated to a specific purpose, such as a digital signal processor (DSP). Moreover, explicit use of the term a “processor” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included.”)
wherein the driving path planner is configured to:
receive a first data set including a driving path and an associated driving environment information; (Paragraphs 0020-0021, "Referring now to FIG. 2, a method of generating and using simulated training data for an autonomous vehicle model is shown. Block 202 trains a NeRF model to generate a 3D environment based on a 2D image input. This training may be based on both video information and LiDAR information, which are used to inform an RGB loss and a depth loss respectively. Block 204 then simulates scenarios using the trained NeRF model.
Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories." as well as Paragraph 0027, "Additional criteria are applied to generate photorealistic samples during the simulation of block 204. Block 204 applies these criteria to guide the sampling of positions and poses within a given 3D scenario generated from an original 2D scenario input. A first criterion is that the novel view poses should stay near the original trajectory. While the depth loss effectively captures correct geometrical information, it struggles to accurately generate unseen regions. Moving far from the original trajectory exposes those regions, so keeping the new pose close to the original trajectory keeps the scenario in a territory that can be reasonably predicted.")
generate a second data set from the first data set by performing data augmentation on the first data set; (Paragraph 0021, "Thus the training data may include original data with RGB video frames, camera poses, LiDAR information, object trajectories, and timestamps associated with each frame. This original data is used in block 202 to train the NeRF model. A second category of training data is generated in block 204 using the trained NeRF model and includes new poses, timestamps, and object trajectories.") and
be trained based on the second data set; (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") and
…
Elmahgiubi does not specifically teach training a driving controller based on an outcome of a path planner. However, Caldwell, in the same field of endeavor of autonomous control, teaches:
… wherein the driving controller is configured to be trained based on the driving path planner as trained based on the second data set. (Column 4, Lines 48-61, "In various examples, the computing system may associate the metrics with particular actions. In such examples, the computing may be configured to determine one or more high-performing actions (e.g., successful actions) for a particular scenario. In some examples, the computing system may be configured to determine an optimal action (e.g., best metrics) for a particular scenario. In various examples, the computing system may train the autonomous controller, such as utilizing machine learning techniques (e.g., reinforcement learning), to perform the optimal action in the scenario. In such examples, the computing system may cause the autonomous controller to bias toward high-scoring (e.g., successful) actions when it encounters a substantially similar scenario." as well as Column 22 Line 65 - Column 23 Line 29, "In general, the planning component 424 may determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 424 may determine various routes and vehicle trajectories and various levels of detail. For example, the planning component 424 may determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route may include a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 424 may generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 424 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instruction may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
In at least one example, the vehicle computing device(s) 404 may include one or more system controllers 426, which may be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. The system controller(s) 426 may communicate with and/or control corresponding systems of the drive system(s) 414 and/or other components of the vehicle 402.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system and methods of control as taught by Elmahgiubi with the ability to separately train the driving controller as taught by Caldwell. This would ensure that the system, along with traveling an optimal path, may operate efficiently with respect to all vehicle control systems such as braking, steering, propulsion. This may further be used to provide a more enjoyable experience for users riding within the vehicle.
Claim(s) 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elmahgiubi in view of Caldwell and in further view of Bansal et al. (Learning to Drive by Imitating the Best and Synthesizing the Worst), hereinafter Bansal.
Regarding claim 2, where all the limitations of claim 1 are discussed above, Elmahgiubi does not specifically discuss collecting expert data with an optimal path from vehicles. However, Li, in the same field of endeavor of autonomous vehicle control, teaches:
2. The driving path optimization training method of claim 1, wherein
the first data set comprises an expert data set collected by an arbitrary vehicle and data about an optimal path associated with the expert data set. (Section 6.1, "The training data to train our model was obtained by randomly sampling segments of real world expert driving and removing segments where the car was stationary for long periods
of time. Our input field of view is 80m × 80m (Wφ = 80) and with the agent positioned at (u0,v0), we get an effective forward sensing range of Rforward = 64m. Therefore, for the experiments in this work we also removed any segments of highway driving given the longer sensing range requirement that entails. Our dataset contains approximately 26 million examples which amount to about 60 days of continuous driving.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system as taught by Elmahgiubi with the ability to train using expert data collected as taught by Bansal. This would provide a base data set to teach the system to plan effective trajectories and ensure the system is able to determine the best approach to a variety of situations both ideal and not ideal.
Regarding claim 12, where all the limitations of claim 11 are discussed above, Elmahgiubi does not specifically discuss collecting expert data with an optimal path from vehicles. However, Li, in the same field of endeavor of autonomous vehicle control, teaches:
12. The electronic device of claim 11, wherein
the first data set comprises an expert data set collected by an arbitrary vehicle and data about an optimal path associated with the expert data set. (Section 6.1, "The training data to train our model was obtained by randomly sampling segments of real world expert driving and removing segments where the car was stationary for long periods
of time. Our input field of view is 80m × 80m (Wφ = 80) and with the agent positioned at (u0,v0), we get an effective forward sensing range of Rforward = 64m. Therefore, for the experiments in this work we also removed any segments of highway driving given the longer sensing range requirement that entails. Our dataset contains approximately 26 million examples which amount to about 60 days of continuous driving.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system as taught by Elmahgiubi with the ability to train using expert data collected as taught by Bansal. This would provide a base data set to teach the system to plan effective trajectories and ensure the system is able to determine the best approach to a variety of situations both ideal and not ideal.
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elmahgiubi in view of Caldwell and in further view of Pulver et al. (US 20230089978 A1), hereinafter Pulver.
Regarding claim 6, where all the limitations of claim 4 are discussed above, Elmahgiubi further teaches:
6. The driving path optimization training method of claim 4, wherein
the optimal paths are generated (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.") …
Elmahgiubi does not specifically teach using a non-linear optimization method. However, Pulver, in the same field of endeavor of autonomous vehicle path planning, teaches:
… based on a nonlinear optimization method. (Paragraph 0625, "PILOT is used with the two stage optimisation (2s-OPT) approach of FIG. 3 as the base planner. Referring to FIG. 15 (bottom), a convolutional neural network is 900 to take as input a graphical representation 902 of the reference path-projected planning situation (including predictions of other road users) in addition to other scalar parameters of the problem (e.g., speed of the ego vehicle), and output a smooth trajectory that imitates the output of the optimiser when presented with the same problem. 2s-OPT is run on a dataset of problems to initiate the training, and uses to label the new planning problems generated by the learner in simulations. The post-hoc optimizer 712 implements a nonlinear constrained optimisation stage, similar to the second stage 304 in 2s-OPT to maintain safety and smoothness guarantees.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Elmahgiubi with the non-linear optimization methods as taught by Pulver. Implementing the optimization methods as taught by Pulver would ensure that the final trajectory is smooth and safe.
Regarding claim 16, where all the limitations of claim 14 are discussed above, Elmahgiubi further teaches:
16. The electronic device of claim 14, wherein
the optimal paths are generated (Paragraph 0099, "Further, in some non-limiting embodiments of the present technology, the server 202 can be configured to feed each one of the first plurality of training digital objects to the predictive model, minimizing, at each training iteration, a difference between a currently predicted trajectory and the respective label including the given trajectory. According to certain non-limiting embodiments of the present technology, the difference between the predictions of the predictive model and the respective labels can be expressed by a loss function. Various non-limiting examples of the loss function include: a Cross-Entropy Loss function, a Mean Squared Error Loss function, a Huber Loss function, a Hinge Loss function, and others. Thus, by doing so, the server 202 can be configured to train the predictive model to generate trajectories for input scenes and desired behavior clusters.") …
Elmahgiubi does not specifically teach using a non-linear optimization method. However, Pulver, in the same field of endeavor of autonomous vehicle path planning, teaches:
… based on a nonlinear optimization method. (Paragraph 0625, "PILOT is used with the two stage optimisation (2s-OPT) approach of FIG. 3 as the base planner. Referring to FIG. 15 (bottom), a convolutional neural network is 900 to take as input a graphical representation 902 of the reference path-projected planning situation (including predictions of other road users) in addition to other scalar parameters of the problem (e.g., speed of the ego vehicle), and output a smooth trajectory that imitates the output of the optimiser when presented with the same problem. 2s-OPT is run on a dataset of problems to initiate the training, and uses to label the new planning problems generated by the learner in simulations. The post-hoc optimizer 712 implements a nonlinear constrained optimisation stage, similar to the second stage 304 in 2s-OPT to maintain safety and smoothness guarantees.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Elmahgiubi with the non-linear optimization methods as taught by Pulver. Implementing the optimization methods as taught by Pulver would ensure that the final trajectory is smooth and safe.
Claim(s) 7-8 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elmahgiubi in view of Caldwell and in further view of Smolyanskiy et al. (US 20220138568 A1), hereinafter Smolyanskiy.
Regarding claim 7, where all the limitations of claim 1 are discussed above, Elmahgiubi further teaches:
7. The driving path optimization training method of claim 1, wherein
the training of the driving path planner comprises performing training of the driving path planner (Paragraph 0031, "After one or more simulated scenarios are generated, block 206 uses these scenarios to train a model for an autonomous vehicle 102. For example, the training may use imitation learning to provide a set of realistic scenarios that may inform a policy, which an agent may subsequently use to select actions based on their present circumstances. Block 208 deploys the trained model, for example including parameters of a trained policy, to a target, such as the autonomous driving system of an autonomous vehicle.") …
Elmahgiubi does not specifically teach using an open-loop simulation training method. However, Smolyanskiy, in the same field of endeavor of trajectory planning, teaches:
… based on an open-loop simulation training method. (Paragraphs 0077-0078, "The DNN 415 may be configured to receive any number of time slices worth of past information and predict any number of time slices worth of future information, depending on the embodiments. For example, the DNN 415 may generate a trajectory that includes information from the past two seconds and two seconds into the future—e.g., where a trajectory point is output every second, every half second, four times a second, eight times a second, and so on. For example, the inputs 408 may include a tensor(s) corresponding to past and/or predicted future locations of actors, a tensor(s) corresponding to wait conditions 414, a tensor(s) corresponding to map information 412, etc. The outputs 418 may include a tensor(s) corresponding to a confidence field (e.g., indicated by the gradients shown for the outputs 204 in FIG. 2), a tensor(s) corresponding to a vector field(s), etc. In some embodiments, because the inputs 408 in the closed-loop mode may be based on actual or simulated (e.g., ground truth) locations of one or more actors in the environment, the outputs 418 in the closed-loop mode may be more precise—e.g., may include a smaller area of potential locations for the actors which may be closer to a 1:1 correspondence between the input 408 and the output 418. In addition, because the inputs 408 in the open-loop mode may be based on future predictions or simulators of locations of one or more of the actors, the outputs 418 in the open-loop mode may be less precise—e.g., may include a larger area of potential locations for the actors.
The DNN 416 may include a past closed-loop mode and a future open-loop mode. In some embodiments, the past closed-loop mode may take as inputs 408 actual real-world or simulated past location(s) 110 of actors in the environment (in addition to other inputs 408, such as the map information 412, the wait conditions 414, etc.) in order to generate the outputs 418—e.g., as indicated by square boxes on the inputs 408A and 408B.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle systems as aught by Elmahgiubi with the ability to use open-loop simulation mode as taught by Smolyanskiy. This would provide the system with efficient training in order to reinforce the optimal path planning abilities.
Regarding claim 8, where all the limitations of claim 1 are discussed above, Elmahgiubi does not specifically discuss training a driving controller based on an outcome of a path planner or using closed-loop reinforcement training. However, Caldwell, in the same field of endeavor of autonomous control, teaches:
8. The driving path optimization training method of claim 1, wherein
the training of the driving controller comprises performing training of the driving controller ("In various examples, the computing system may associate the metrics with particular actions. In such examples, the computing may be configured to determine one or more high-performing actions (e.g., successful actions) for a particular scenario. In some examples, the computing system may be configured to determine an optimal action (e.g., best metrics) for a particular scenario. In various examples, the computing system may train the autonomous controller, such as utilizing machine learning techniques (e.g., reinforcement learning), to perform the optimal action in the scenario. In such examples, the computing system may cause the autonomous controller to bias toward high-scoring (e.g., successful) actions when it encounters a substantially similar scenario." as well as "In general, the planning component 424 may determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 424 may determine various routes and vehicle trajectories and various levels of detail. For example, the planning component 424 may determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route may include a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 424 may generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 424 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instruction may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
In at least one example, the vehicle computing device(s) 404 may include one or more system controllers 426, which may be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. The system controller(s) 426 may communicate with and/or control corresponding systems of the drive system(s) 414 and/or other components of the vehicle 402.") …
However, Smolyanskiy, in the same field of endeavor of trajectory planning, teaches:
… based on a closed-loop reinforcement training method. (Paragraphs 0077-0078, "The DNN 415 may be configured to receive any number of time slices worth of past information and predict any number of time slices worth of future information, depending on the embodiments. For example, the DNN 415 may generate a trajectory that includes information from the past two seconds and two seconds into the future—e.g., where a trajectory point is output every second, every half second, four times a second, eight times a second, and so on. For example, the inputs 408 may include a tensor(s) corresponding to past and/or predicted future locations of actors, a tensor(s) corresponding to wait conditions 414, a tensor(s) corresponding to map information 412, etc. The outputs 418 may include a tensor(s) corresponding to a confidence field (e.g., indicated by the gradients shown for the outputs 204 in FIG. 2), a tensor(s) corresponding to a vector field(s), etc. In some embodiments, because the inputs 408 in the closed-loop mode may be based on actual or simulated (e.g., ground truth) locations of one or more actors in the environment, the outputs 418 in the closed-loop mode may be more precise—e.g., may include a smaller area of potential locations for the actors which may be closer to a 1:1 correspondence between the input 408 and the output 418. In addition, because the inputs 408 in the open-loop mode may be based on future predictions or simulators of locations of one or more of the actors, the outputs 418 in the open-loop mode may be less precise—e.g., may include a larger area of potential locations for the actors.
The DNN 416 may include a past closed-loop mode and a future open-loop mode. In some embodiments, the past closed-loop mode may take as inputs 408 actual real-world or simulated past location(s) 110 of actors in the environment (in addition to other inputs 408, such as the map information 412, the wait conditions 414, etc.) in order to generate the outputs 418—e.g., as indicated by square boxes on the inputs 408A and 408B.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system and methods of control as taught by Elmahgiubi with the ability to separately train the driving controller as taught by Caldwell as well as with the ability to use closed-loop reinforcement training as taught by Smolyanskiy. This would ensure that the system, along with traveling an optimal path, may operate efficiently with respect to all vehicle control systems such as braking, steering, propulsion, etc.. Using closed-loop reinforcement would provide highly accurate results. This combination may be used to provide a more enjoyable experience for users riding within the vehicle.
Regarding claim 17, where all the limitations of claim 11 are discussed above, Elmahgiubi further teaches:
17. The electronic device of claim 11, wherein
the instructions, when executed by the one or more processors, cause the one or more processors to perform training of the driving path planner …
Elmahgiubi does not specifically teach using an open-loop simulation training method. However, Smolyanskiy, in the same field of endeavor of trajectory planning, teaches:
… based on an open-loop imitation training method. (Paragraphs 0077-0078, "The DNN 415 may be configured to receive any number of time slices worth of past information and predict any number of time slices worth of future information, depending on the embodiments. For example, the DNN 415 may generate a trajectory that includes information from the past two seconds and two seconds into the future—e.g., where a trajectory point is output every second, every half second, four times a second, eight times a second, and so on. For example, the inputs 408 may include a tensor(s) corresponding to past and/or predicted future locations of actors, a tensor(s) corresponding to wait conditions 414, a tensor(s) corresponding to map information 412, etc. The outputs 418 may include a tensor(s) corresponding to a confidence field (e.g., indicated by the gradients shown for the outputs 204 in FIG. 2), a tensor(s) corresponding to a vector field(s), etc. In some embodiments, because the inputs 408 in the closed-loop mode may be based on actual or simulated (e.g., ground truth) locations of one or more actors in the environment, the outputs 418 in the closed-loop mode may be more precise—e.g., may include a smaller area of potential locations for the actors which may be closer to a 1:1 correspondence between the input 408 and the output 418. In addition, because the inputs 408 in the open-loop mode may be based on future predictions or simulators of locations of one or more of the actors, the outputs 418 in the open-loop mode may be less precise—e.g., may include a larger area of potential locations for the actors.
The DNN 416 may include a past closed-loop mode and a future open-loop mode. In some embodiments, the past closed-loop mode may take as inputs 408 actual real-world or simulated past location(s) 110 of actors in the environment (in addition to other inputs 408, such as the map information 412, the wait conditions 414, etc.) in order to generate the outputs 418—e.g., as indicated by square boxes on the inputs 408A and 408B.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle systems as aught by Elmahgiubi with the ability to use open-loop simulation mode as taught by Smolyanskiy. This would provide the system with efficient training in order to reinforce the optimal path planning abilities.
Regarding claim 18, where all the limitations of claim 11 are discussed above, Elmahgiubi does not specifically discuss training a driving controller based on an outcome of a path planner or using closed-loop reinforcement training. However, Caldwell, in the same field of endeavor of autonomous control, teaches:
18. The electronic device of claim 11, wherein
the instructions, when executed by the one or more processors, cause the one or more processors to perform training of the driving controller (Column 4, Lines 48-61, "In various examples, the computing system may associate the metrics with particular actions. In such examples, the computing may be configured to determine one or more high-performing actions (e.g., successful actions) for a particular scenario. In some examples, the computing system may be configured to determine an optimal action (e.g., best metrics) for a particular scenario. In various examples, the computing system may train the autonomous controller, such as utilizing machine learning techniques (e.g., reinforcement learning), to perform the optimal action in the scenario. In such examples, the computing system may cause the autonomous controller to bias toward high-scoring (e.g., successful) actions when it encounters a substantially similar scenario." as well as Column 22 Line 65 - Column 23 Line 29, "In general, the planning component 424 may determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 424 may determine various routes and vehicle trajectories and various levels of detail. For example, the planning component 424 may determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route may include a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 424 may generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 424 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instruction may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
In at least one example, the vehicle computing device(s) 404 may include one or more system controllers 426, which may be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. The system controller(s) 426 may communicate with and/or control corresponding systems of the drive system(s) 414 and/or other components of the vehicle 402.") …
However, Smolyanskiy, in the same field of endeavor of trajectory planning, teaches:
… based on a closed-loop reinforcement training method. (Paragraphs 0077-0078, "The DNN 415 may be configured to receive any number of time slices worth of past information and predict any number of time slices worth of future information, depending on the embodiments. For example, the DNN 415 may generate a trajectory that includes information from the past two seconds and two seconds into the future—e.g., where a trajectory point is output every second, every half second, four times a second, eight times a second, and so on. For example, the inputs 408 may include a tensor(s) corresponding to past and/or predicted future locations of actors, a tensor(s) corresponding to wait conditions 414, a tensor(s) corresponding to map information 412, etc. The outputs 418 may include a tensor(s) corresponding to a confidence field (e.g., indicated by the gradients shown for the outputs 204 in FIG. 2), a tensor(s) corresponding to a vector field(s), etc. In some embodiments, because the inputs 408 in the closed-loop mode may be based on actual or simulated (e.g., ground truth) locations of one or more actors in the environment, the outputs 418 in the closed-loop mode may be more precise—e.g., may include a smaller area of potential locations for the actors which may be closer to a 1:1 correspondence between the input 408 and the output 418. In addition, because the inputs 408 in the open-loop mode may be based on future predictions or simulators of locations of one or more of the actors, the outputs 418 in the open-loop mode may be less precise—e.g., may include a larger area of potential locations for the actors.
The DNN 416 may include a past closed-loop mode and a future open-loop mode. In some embodiments, the past closed-loop mode may take as inputs 408 actual real-world or simulated past location(s) 110 of actors in the environment (in addition to other inputs 408, such as the map information 412, the wait conditions 414, etc.) in order to generate the outputs 418—e.g., as indicated by square boxes on the inputs 408A and 408B.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the autonomous vehicle system and methods of control as taught by Elmahgiubi with the ability to separately train the driving controller as taught by Caldwell as well as with the ability to use closed-loop reinforcement training as taught by Smolyanskiy. This would ensure that the system, along with traveling an optimal path, may operate efficiently with respect to all vehicle control systems such as braking, steering, propulsion, etc.. Using closed-loop reinforcement would provide highly accurate results. This combination may be used to provide a more enjoyable experience for users riding within the vehicle.
Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elmahgiubi in view of Caldwell and in further view of Li et al. (CN 118212808 A), hereinafter Li.
Regarding claim 9, where all the limitations of claim 8 are discussed above, Elmahgiubi does not specifically teach using a soft actor critic algorithm for training. However, Li, in the same field of endeavor of task and trajectory planning, teaches:
9. The driving path optimization training method of claim 8, wherein
the closed-loop reinforcement training method comprises a behavior cloned soft actor-critic (BC-SAC) algorithm. (Page 15, Paragraph 5, "The design combines the method for adjusting the reward and punishment strategy based on the risk sensing result with the basic SAC algorithm, which is named as RA-SAC algorithm. Soft Actor-Critic (SAC) is a depth-strengthening learning algorithm that combines off-line policy, Actor-Critic structure and maximum entropy (Maximum Entropy). Compared with the optimal strategy capable of maximizing the accumulated return, the strategy selected by the SAC has the maximum entropy of each output action, which ensures the task result and improves the strategy randomness. The SAC host network architecture includes an Actor network and four Critic networks, and the host network architecture is shown in FIG. 9.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Elmahgiubi with the ability to utilize a soft actor critic learning algorithm which maximizes entropy as taught by Li. This would ensure that the system is trained on a sufficient variety of scenarios and data allowing it to be effective in more diverse situations.
Regarding claim 19, where all the limitations of claim 18 are discussed above, Elmahgiubi does not specifically teach using a soft actor critic algorithm for training. However, Li, in the same field of endeavor of task and trajectory planning, teaches:
19. The electronic device of claim 18, wherein
the closed-loop reinforcement training method comprises a behavior cloned soft actor-critic (BC-SAC) algorithm. (Page 15, Paragraph 5, "The design combines the method for adjusting the reward and punishment strategy based on the risk sensing result with the basic SAC algorithm, which is named as RA-SAC algorithm. Soft Actor-Critic (SAC) is a depth-strengthening learning algorithm that combines off-line policy, Actor-Critic structure and maximum entropy (Maximum Entropy). Compared with the optimal strategy capable of maximizing the accumulated return, the strategy selected by the SAC has the maximum entropy of each output action, which ensures the task result and improves the strategy randomness. The SAC host network architecture includes an Actor network and four Critic networks, and the host network architecture is shown in FIG. 9.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Elmahgiubi with the ability to utilize a soft actor critic learning algorithm which maximizes entropy as taught by Li. This would ensure that the system is trained on a sufficient variety of scenarios and data allowing it to be effective in more diverse situations.
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATHER KENIRY whose telephone number is (571)270-5468. The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657