DETAILED ACTION
This action is in response to the Applicant Response filed 17 April 2026 for application 17/219,350 filed 31 March 2021.
Claim(s) 1, 6, 11, 16-17 is/are currently amended.
Claim(s) 1-20 is/are pending.
Claim(s) 1-20 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments regarding the objections to the claims have been fully considered and, in light of the amendments to the claims, are persuasive. However, in light of the amendments to the claims, new claim objections have arisen, as noted below.
Applicant’s arguments regarding the 35 U.S.C. 101 rejection of the claims are based on the newly amended subject matter. All arguments are addressed in the 35 U.S.C. 101 rejection of the claims below.
Applicant’s arguments regarding the 35 U.S.C. 102 and/or 35 U.S.C. 103 rejections of the claims are based on the newly amended subject matter. All arguments are addressed in the 35 U.S.C. 102 and/or 35 U.S.C. 103 rejections of the claims below.
Claim Objections
Claim(s) 1-10 is/are objected to because of the following informalities:
Claim 1, lines 10-11, the domain randomized one or more instances of output should read “the domain randomized one or more instances”
Claims 2-10 are objected to due to their dependence, either directly or indirectly, on claim 1
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the validation dataset while failing to provide a proper antecedent basis for the term. It is suggested that the term be amended to recite “the ground truth data.” Correction or clarification is required.
Claims 2-10 are rejected under 35 U.S.C. 112(b) due to their dependence, either directly or indirectly, on claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of generating … one or more inferences based on a ground truth data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining, based on at least the ground truth data and the one or more inferences, one or more instances of incorrect inferences generated by the first machine learning model that satisfy a determined set of criteria, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computer-implemented. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – first machine learning model, synthetic imitation generator, domain randomization. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites loading the one or more instances to a synthetic imitation generator; applying domain randomization to the one or more instances using the synthetic imitation generator to generate, based on at least the domain randomized one or more instances of output, at least one of new synthetic data or new ground truth data which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
The claim recites training the first machine learning model using the validation dataset and the new synthetic data which is simply generic training to perform the abstract idea of model creation and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computer-implemented amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model and generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
first machine learning model, synthetic imitation generator, domain randomization amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 2, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 2 carries out the method of claim 1 but for the recitation of additional element(s) of applying at least one of domain adaptation or transfer learning when a difference between the validation dataset and the new synthetic data exceeds a pre-determined threshold.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – domain adaptation, transfer learning. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites applying at least one of domain adaptation or transfer learning when a difference between the validation dataset and the new synthetic data exceeds a pre-determined threshold which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
domain adaptation, transfer learning amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 3, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 3 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the set of criteria comprises at least one of a threshold of false positives or a threshold of false negatives.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 4, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 4 carries out the method of claim 1 but for the recitation of additional element(s) of wherein applying domain randomization adds variation to the ground truth data for one or more parameters.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 5, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 5 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the one or more parameters comprise at least one of: a weather parameter; a time of day parameter; an object type parameter; a location parameter; an orientation parameter; or a speed parameter.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the parameters and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the parameter do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 6, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 6 carries out the method of claim 1 but for the recitation of additional element(s) of using a second machine learning model to perform one or more operations of an autonomous machine.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – autonomous machine. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites using the second machine learning model to perform one or more operations of an autonomous machine which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
autonomous machine amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 7, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 7 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the new synthetic data comprises at least one of: synthetic camera data; synthetic radar data; synthetic lidar data; or synthetic ultrasonic data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 8, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 8 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the synthetic imitation generator creates a synthetic scene based on at least one of human labeled ground truth data or automatically-created ground truth data generated based on the synthetic data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the synthetic image generator and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the synthetic image generator do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 9, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 8 is applicable here since claim 9 carries out the method of claim 8 but for the recitation of additional element(s) of wherein the automatically-created ground truth data comprises three-dimensional (3D) information of one or more dynamic objects in the synthetic scene.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 10, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 8 is applicable here since claim 10 carries out the method of claim 8 but for the recitation of additional element(s) of wherein the synthetic scene comprises one or more domain randomized parameters.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 11, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor.
The limitation of ... implement a technique for creating synthetic scenes mimicking a real scene fulfilling a set of criteria ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – processor, one or more processing units. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – synthetic image generator, machine learning model. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the synthetic generator generates, for training a machine learning model, new synthetic data based on at least one or more instances of incorrect inferences produced by the machine learning model in response to an initial validation dataset which is simply applying the model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
processor, one or more processing units amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying the model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
synthetic image generator, machine learning model amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 12, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 12 carries out the processor of claim 11 but for the recitation of additional element(s) of wherein the synthetic image generator is implemented as a simulator based on a game engine.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the synthetic image generator and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the synthetic image generator do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 13, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 12 carries out the processor of claim 11 but for the recitation of additional element(s) of wherein the synthetic scenes comprise at least one of: synthetic camera data; synthetic radar data; synthetic lidar data; or synthetic ultrasonic data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the synthetic scenes and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the synthetic scenes do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 14, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 14 carries out the processor of claim 11 but for the recitation of additional element(s) of wherein the synthetic image generator creates a synthetic scene based on at least one of human labeled ground truth data or automatically-created ground truth data generated based on the synthetic scenes.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the synthetic image generator and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the synthetic image generator do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 15, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor. The Step 2A Prong One Analysis for claim 14 is applicable here since claim 15 carries out the processor of claim 14 but for the recitation of additional element(s) of wherein the synthetic image generator creates a plurality of instances of the synthetic scene, each instance of the synthetic scene having a different set of domain randomized parameters.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the synthetic image generator and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the synthetic image generator do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 16, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a(n) processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) processor. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 16 carries out the processor of claim 11 but for the recitation of additional element(s) of wherein the one or more processing units further apply the synthetic scenes to train a machine learning module.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – machine learning module. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the one or more processing units are further apply the synthetic scenes to train a machine learning module which is simply generic training to perform the abstract idea of model creation and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
machine learning module amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 17, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of generate … one or more inferences based on a ground truth data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of generate, … based on at least the one or more incorrect inferences, synthetic data having one or more domain variations from the provided one or more incorrect inferences, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – system, at least one processor, memory, instructions. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – machine learning model, synthetic data generator. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites provide one or more of incorrect inferences from the machine learning model that satisfy a determined set of criteria to a synthetic data generator which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
The claim recites further train the machine learning model using a validation dataset and the generated synthetic data which is simply generic training to perform the abstract idea of model creation and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
system, at least one processor, memory, instructions amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model and generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
machine learning model, synthetic data generator amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 18, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 17 is applicable here since claim 18 carries out the system of claim 17 but for the recitation of additional element(s) of apply at least one of domain adaptation or transfer learning when a difference between the validation dataset and the generated synthetic data exceeds a pre-determined threshold.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – domain adaptation, transfer learning. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites applying at least one of domain adaptation or transfer learning when a difference between the validation dataset and the new synthetic data exceeds a pre-determined threshold which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
domain adaptation, transfer learning amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 19, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 17 carries out the system of claim 17 but for the recitation of additional element(s) of wherein the failure cases comprise at least one of false positives or false negatives.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 20, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 20 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 17 is applicable here since claim 20 carries out the system of claim 17 but for the recitation of additional element(s) of wherein the system comprises at least one of: a system for performing graphical rendering operations; a system for performing simulation operations; a system for performing simulation operations to test or validate autonomous machine applications; a system for performing deep learning operations; a system implemented using an edge device; a system incorporating one or more Virtual Machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the system and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the system do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-5, 7-11, 13-17, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kumar et al. (U.S. Pat. No. 11,188,790 B1 – Generation of Synthetic Datasets for Machine Learning Models, hereinafter referred to as “Kumar”).
Regarding claim 1 (Currently Amended), Kumar teaches a computer-implemented method (Col. 7:52-Col. 8:10 - teaches processor executing code; see also Kumar, col. 4:17-32, Fig. 1), comprising:
generating, by a first machine learning model, one or more inferences based on a ground truth data (Kumar, col. 5:47-64 – inputting image of a jumpsuit to a model trained on jumpsuits [based on ground truth] to generate an inference of jumpsuit recognition);
determining, based on at least the ground truth data and the one or more inferences, one or more instances of incorrect inferences generated by the first machine learning model that satisfy a determined set of criteria (Kumar, col. 5:47-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator);
loading the one or more instances to a synthetic imitation generator (Kumar, col. 5:47-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator);
applying domain randomization to the one or more instances using the synthetic imitation generator to generate, based on at least the domain randomized one or more instances of output, at least one of new synthetic data or new ground truth data (Kumar, col. 6:6-37 – teaches applying domain randomization to the incorrect image using the synthetic dataset generation tool to generate synthetic images); and
training the first machine learning model using the validation dataset and the new synthetic data (Kumar, col. 6:59-col. 7:18 – teaches combining the synthetic dataset with the real-world dataset to train a machine learning model; see also Kumar, col. 3:66-col. 4:16 – teaches using synthetic data to further train the original poorly performing machine learning model).
Regarding claim 3 (Original), Kumar teaches all of the limitations of the method of claim 1 as noted above. Kumar further teaches wherein the set of criteria comprises at least one of a threshold of false positives or a threshold of false negatives (Kumar, col. 3:53-col. 4:16 - teaches the ML model providing poor results; Kumar, col. 5:47-64 - teaches misclassifying a jumpsuit [Misclassification means the classifier is providing false negatives or false positive. Identifying poor accuracy means that some threshold of "poor" is being used.]).
Regarding claim 4 (Original), Kumar teaches all of the limitations of the method of claim 1 as noted above. Kumar further teaches wherein applying domain randomization adds variation to the ground truth data for one or more parameters (Kumar, col. 6:6-37 - teaches domain randomization of various parameters including fabric type, target location, pose).
Regarding claim 5 (Original), Kumar teaches all of the limitations of the method of claim 4 as noted above. Kumar further teaches wherein the one or more parameters comprise at least one of: a weather parameter; a time of day parameter; an object type parameter; a location parameter; an orientation parameter; or a speed parameter (Kumar, col. 6:6-37 - teaches domain randomization of various parameters including fabric type [object type], target location [location], pose [orientation ]).
Regarding claim 7 (Original), Kumar teaches all of the limitations of the method of claim 1 as noted above. Kumar further teaches wherein the new synthetic data comprises at least one of: synthetic camera data; synthetic radar data; synthetic lidar data; or synthetic ultrasonic data (Kumar, col. 6:6-37 - teaches synthetic camera data for 3D fashion scenes).
Regarding claim 8 (Original), Kumar teaches all of the limitations of the method of claim 1 as noted above. Kumar further teaches wherein the synthetic imitation generator creates a synthetic scene based on at least one of human labeled ground truth data or automatically-created ground truth data generated based on the synthetic data (Kumar, col. 6:6-37 – teaches synthetic dataset generation tool creating synthetic 3D scenes based on synthetic data created from incorrect image with optional manual adjustments).
Regarding claim 9 (Original), Kumar teaches all of the limitations of the method of claim 8 as noted above. Kumar further teaches wherein the automatically-created ground truth data comprises three-dimensional (3D) information of one or more dynamic objects in the synthetic scene (Kumar, col. 6:6-37 – teaches 3D scenes with 3D dynamic objects just as humans).
Regarding claim 10 (Original), Kumar teaches all of the limitations of the method of claim 8 as noted above. Kumar further teaches wherein the synthetic scene comprises one or more domain randomized parameters (Kumar, col. 6:6-37 – teaches synthetic 3D scenes created with domain randomized parameters).
Regarding claim 11 (Currently Amended), Kumar teaches a processor (Col. 7:52-Col. 8:10 - teaches processor executing code; see also Kumar, col. 4:17-32, Fig. 1) comprising:
one or more processing units (Col. 7:52-Col. 8:10 - teaches processor executing code; see also Kumar, col. 4:17-32, Fig. 1) to implement a technique for creating synthetic scenes mimicking a real scene (Kumar, col. 6:6-67 – teaches synthetic dataset generation tool creating synthetic 3D scenes mimicking real-word data; see also Kumar, col. 5:47-64) fulfilling a set of criteria (Kumar, col. 5:37-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator) using a synthetic image generator (Kumar, col. 6:6-67 – teaches synthetic dataset generation tool creating synthetic 3D scenes mimicking real-word data), wherein the synthetic image generator generates, for training a machine learning model (Kumar, col. 6:59-col. 7:18 – teaches combining the synthetic dataset with the real-world dataset to train a machine learning model; see also Kumar, col. 3:66-col. 4:16 – teaches using synthetic data to further train the original poorly performing machine learning model), new synthetic data (Kumar, col. 6:6-37 – teaches applying domain randomization to the incorrect image using the synthetic dataset generation tool to generate synthetic images) based on at least one or more instances of incorrect inferences produced by the machine learning model (Kumar, col. 5:37-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator) in response to an initial validation dataset (Kumar, col. 5:47-64 – inputting image of a jumpsuit to a model trained on jumpsuits [validation dataset] to generate an inference of jumpsuit recognition).
Regarding claim 13 (Original), Kumar teaches all of the limitations of the processor of claim 11 as noted above. Kumar further teaches wherein the synthetic scenes comprise at least one of: synthetic camera data; synthetic radar data; synthetic lidar data; or synthetic ultrasonic data (Kumar, col. 6:6-37 - teaches synthetic camera data for 3D fashion scenes).
Regarding claim 14 (Previously Presented), Kumar teaches all of the limitations of the processor of claim 11 as noted above. Kumar further teaches wherein the synthetic image generator creates a synthetic scene based on at least one of human labeled ground truth data or automatically-created ground truth data generated based on the synthetic scenes (Kumar, col. 6:6-37 – teaches synthetic dataset generation tool creating synthetic 3D scenes based on synthetic data created from incorrect image with optional manual adjustments).
Regarding claim 15 (Original), Kumar teaches all of the limitations of the processor of claim 14 as noted above. Kumar further teaches wherein the synthetic image generator creates a plurality of instances of the synthetic scene, each instance of the synthetic scene having a different set of domain randomized parameters (Kumar, col. 6:6-37 – teaches synthetic dataset generation tool creating various different synthetic 3D scenes based on synthetic data created from incorrect image with domain randomization).
Regarding claim 16 (Currently Amended), Kumar teaches all of the limitations of the processor of claim 11 as noted above. Kumar further teaches wherein the one or more processing units further apply the synthetic scenes to train a machine learning module (Kumar, col. 6:59-col. 7:18 – teaches combining the synthetic dataset, including 3D synthetic scenes, with the real-world dataset to train a machine learning model).
Regarding claim 17 (Currently Amended), Kumar teaches a system, comprising:
at least one processor (Col. 7:52-Col. 8:10 - teaches processor executing code; see also Kumar, col. 4:17-32, Fig. 1); and
memory including instructions that, when executed by the at least one processor (Col. 7:52-Col. 8:10 - teaches processor executing code; see also Kumar, col. 4:17-32, Fig. 1), cause the system to:
generate, by a machine learning model, one or more inferences based on a ground truth data (Kumar, col. 5:47-64 – inputting image of a jumpsuit to a model trained on jumpsuits [based on ground truth] to generate an inference of jumpsuit recognition);
provide one or more of incorrect inferences from the machine learning model that satisfy a determined set of criteria (Kumar, col. 5:47-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator) to a synthetic data generator (Kumar, col. 5:47-64 – teaches identifying the mistaken image classification as not meaningfully impacting the model accuracy and sending the image to a synthetic image generator);
generate, by the synthetic data generator based on at least the one or more incorrect inferences, synthetic data having one or more domain variations from the provided one or more incorrect inferences (Kumar, col. 6:6-37 – teaches applying domain randomization to the incorrect image using the synthetic dataset generation tool to generate synthetic images); and
further train the machine learning model using a validation dataset and the generated synthetic data (Kumar, col. 6:59-col. 7:18 – teaches combining the synthetic dataset with the real-world dataset to train a machine learning model; see also Kumar, col. 3:66-col. 4:16 – teaches using synthetic data to further train the original poorly performing machine learning model).
Regarding claim 19 (Original), Kumar teaches all of the limitations of the system of claim 17 as noted above. Kumar further teaches wherein the failure cases comprise at least one of false positives or false negatives (Kumar, col. 3:53-col. 4:16 - teaches the ML model providing poor results; Kumar, col. 5:47-64 - teaches misclassifying a jumpsuit [Misclassification means the classifier is providing false negatives or false positive.]).
Regarding claim 20 (Original), Kumar teaches all of the limitations of the system of claim 17 as noted above. Kumar further teaches wherein the system comprises at least one of: a system for performing graphical rendering operations; a system for performing simulation operations; a system for performing simulation operations to test or validate autonomous machine applications; a system for performing deep learning operations; a system implemented using an edge device; a system incorporating one or more Virtual Machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (Kumar, col. 3:40-52 - teaches deep learning; Kumar, col.3:66-col.4:16 - teaches simulating real world situations [simulation]; Kumar, col. 4:17-32 - teaches server [cloud] and client [edge] devices; Kumar, col. 6:6-37 – teaches synthetic dataset generation tool creating synthetic 3D scenes based on synthetic data [graphical rendering]).
Claim(s) 2, 6, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Müller et al. (Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications, hereinafter referred to as “Muller”).
Regarding claim 2 (Original), Kumar teaches all of the limitations of the method of claim 1 as noted above. However, Kumar does not explicitly teach applying at least one of domain adaptation or transfer learning when a difference between the validation dataset and the new synthetic data exceeds a pre-determined threshold.
Muller teaches applying at least one of domain adaptation or transfer learning when a difference between the validation dataset and the new synthetic data exceeds a pre-determined threshold (Muller, section 6 – teaches applying deep transfer learning techniques when the differences between real-world [validation] and simulated [synthetic] datasets need to be reconciled to enable a smooth transition to the real-world [Reconciling differences between dataset for a smooth transition means the difference must be greater than a given threshold.]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Kumar with the teachings of Muller in order to generate synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provide extensive synthetic data variety in the field of synthetic data generation (Muller, Abstract – “We present a photo-realistic training and evaluation simulator (Sim4CV) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network (DNN) architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.”).
Regarding claim 6 (Currently Amended), Kumar teaches all of the limitations of the method of claim 1 as noted above. However, Kumar does not explicitly teach using the second machine learning model to perform one or more operations of an autonomous machine.
Muller teaches using a second machine learning model to perform one or more operations of an autonomous machine (Muller, section 5 – teaches using the synthetic data to train a DNN for autonomous driving).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Kumar with the teachings of Muller in order to generate synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provide extensive synthetic data variety in the field of synthetic data generation (Muller, Abstract – “We present a photo-realistic training and evaluation simulator (Sim4CV) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network (DNN) architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.”).
Regarding claim 18 (Original), Kumar teaches all of the limitations of the system of claim 17 as noted above. However, Kumar does not explicitly teach apply at least one of domain adaptation or transfer learning when a difference between the validation dataset and the generated synthetic data exceeds a pre-determined threshold.
Muller teaches apply at least one of domain adaptation or transfer learning when a difference between the validation dataset and the generated synthetic data exceeds a pre-determined threshold (Muller, section 6 – teaches applying deep transfer learning techniques when the differences between real-world [validation] and simulated [synthetic] datasets need to be reconciled to enable a smooth transition to the real-world [Reconciling differences between dataset for a smooth transition means the difference must be greater than a given threshold.]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Kumar with the teachings of Muller in order to generate synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provide extensive synthetic data variety in the field of synthetic data generation (Muller, Abstract – “We present a photo-realistic training and evaluation simulator (Sim4CV) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network (DNN) architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.”).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Prakash et al. (Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data, hereinafter referred to as “Prakash”).
Regarding claim 12 (Original), Kumar teaches all of the limitations of the processor of claim 11 as noted above. However, Kumar does not explicitly teach wherein the synthetic image generator is implemented as a simulator based on a game engine.
Prakash teaches wherein the synthetic image generator is implemented as a simulator based on a game engine (Prakash, section III – teaches generating synthetic images as a simulator based a game engine).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Kumar with the teachings of Prakash in order to provide structured domain randomization in order to account for the structure and context of a scene to enables a neural network to take the context around an object into consideration in the field of synthetic data generation (Prakash, Abstract – “We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure and context of the scene. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125
Prosecution Insights