DETAILED ACTION
This action is in response to the application files 05/02/2023. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/22/2024, 12/19/2024, and 02/25/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5, 7-18, and 20 rejected under 35 U.S.C. 101 because they are directed to an abstract idea that does not amount to significantly more.
Regarding Claim 1:
Subject Matter of Eligibility Analysis Step 1:
the claim recites a method and is directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites generating one or more feature vectors for the sensor input, comprising processing the sensor input … to generate a prediction output (this limitation is a mental process since a human can mentally generate feature vectors from an input).
generating one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the senor input to generate the prediction output(this limitation is a mental process since a human can mentally generate feature vectors from an input).
The claim recites processing each of the one or more feature vectors … to generate a density score for the feature vector (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (this limitation is a mental process since a human can mentally create a training dataset based on a rareness score).
Therefore, claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim recites obtaining a sensor input (this limitation is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
The claim recites using a prediction neural network (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The claim recites using a density estimation model (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
Therefore, claim 1 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
The method of obtaining a sensor input is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
The method of using a prediction neural network is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The method of using a density estimation model is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
Therefore, claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter of Eligibility Analysis Step 1:
Claim 2 recites a method and is directed to a process, which is one of the four statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites process the sensor input to generate an intermediate feature map (this limitation is a mental process since a human can mentally calculate a feature map from the sensor input, if an equation was given).
The claim recites process the intermediate feature map to generate a prediction output for the sensor input, wherein the prediction output characterizes one or more of (i) one or more regions of the sensor data or (ii) one or more objects depicted in the one or more regions (this limitation is a mental process since a human can mentally organize the feature map into regions/objects of the sensor data).
Therefore, claim 2 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim does not recite any additional elements.
Therefore, claim 2 is not integrated into a practical application
Subject Matter of Eligibility Analysis Step 2b:
The claim does not recite any additional elements, thus claim 2 does not provide significantly more.
Therefore, claim 2 is subject-matter ineligible.
Regarding Claim 4:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 4 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 can be applied here. Therefore, claim 4 recites an abstract idea
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object (this limitation does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f))).
Therefore, claim 4 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 4 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 4 is subject matter ineligible.
Regarding Claim 5:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 5 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 can be applied here. Therefore, claim 5 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent (this limitation does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f))).
Therefore, claim 5 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 5 is subject matter ineligible.
Regarding Claim 7:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 7 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 can be applied here. Therefore, claim 7 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the downstream task is three-dimensional object detection task (this limitation references a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f)))).
The claim recites the downstream neural network is the same neural network as the prediction neural network (this limitation references a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f)))).
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 7 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the downstream task is three-dimensional object detection task is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
the downstream neural network is the same neural network as the prediction neural network is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 7 is subject matter ineligible.
Regarding Claim 8:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
the claim recites generating the training set of feature vectors by generating, for each sensor input in a second set of sensor data and using the object detection neural network, a respective feature vector for each of one or more regions in the sensor input that are predicted by the trained object detection neural network to depict an object (this limitation is a mental process since a human can mentally categorize features based on the region(s) in a sensor input).
Therefore, claim 8 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
the claim recites training the density estimation model on the training set of feature vectors to maximize an expected log density score of the feature vectors in the training set (this limitation does not integrate the abstract idea into a practical application because it amounts to mere “apply it” (see MPEP 2106.05(f)))).
Therefore, claim 8 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The additional elements in claim 8 do not provide significantly more than the abstract idea itself, taken alone and in combination because
training the density estimation model on the training set of feature vectors to maximize an expected log density score of the feature vectors in the training set uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 8 is subject matter ineligible.
Regarding Claim 9:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 9 is a dependent claim of claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 of claim 1 applies to this claim.
Therefore, claim 9 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the density estimation model is a normalizing flow model (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f))).
Therefore, claim 9 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2b:
The additional elements in claim 6 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the density estimation model is a normalizing flow model is an instruction that performs an abstract idea and cannot provide significantly more.
Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter of Eligibility Analysis Step 1:
The claim is directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
The claim recites the rareness score for the feature vector is inversely proportional to the density score for the feature vector (this limitation is mathematical concept since there is a mathematical relationship between the rareness score and density score).
Therefore, claim 10 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim does not recite any additional elements.
Therefore, claim 10 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2b:
The claim does not recite any additional elements, thus claim 10 does not provide significantly more.
Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and is directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
The claim recites ranking respective feature vectors generated from the plurality of sensor inputs by rareness scores (this limitation is a mental process since a human can mentally rank the vectors based on a number, which is the rareness score).
The claim recites selecting a proper subset of respective feature vectors having the highest rareness scores according to the ranking (this limitation is a mental process since a human can mentally choose vectors with the highest scores based on a ranking).
Therefore, claim 11 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim does not recite any additional elements.
Therefore, claim 11 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2b:
The claim does not recite any additional elements, thus claim 11 does not provide significantly more.
Therefore, claim 11 is subject-matter ineligible.
Regarding Claim 12:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a method and is directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
The claim recites generating, from the sensor input and other sensor inputs, one or more test scripts for a software module (this limitation is a mental process since a human can mentally create test scripts).
The claim recites evaluating a performance of the software module by using the software module to process the one or more test scripts (this limitation is a mental process since a human can mentally rate/evaluate a software from a test script).
Therefore, claim 12 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim does not recite any additional elements.
Therefore, claim 12 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2b:
The claim does not recite any additional elements, thus claim 12 does not provide significantly more.
Therefore, claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter of Eligibility Analysis Step 1:
the claim recites a method and is directed to a process, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites processing the sensor input…to generate one or more feature vectors for the sensor input (This limitation is a mental process as it encompasses a human mentally creating feature vectors based on the sensor input).
The claim recites processing each of the one or more feature vectors … to generate a density score for the feature vector… (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a predicted behavior of an agent depicted in the sensor input is rare relative to other objects (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (this limitation is a mental process since a human can mentally create a training dataset based on a rareness score).
Therefore, claim 13 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim recites obtaining a sensor input (this limitation is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
The claim recites using an encoder neural network (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The claim recites using a density estimation model (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The claim recites …wherein the density score represents an estimate of a density of the feature vector in a training set of feature vectors used to train the density estimation model (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 13 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
The method of obtaining a sensor input is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
The method of using an encoder network is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The method of using a density estimation model is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The the method of …wherein the density score represents an estimate of a density of the feature vector in a training set of feature vectors used to train the density estimation model recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 13 is subject-matter ineligible.
Regarding Claim 14:
Subject Matter of Eligibility Analysis Step 1:
the claim recites a system and is directed to a machine, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites generating one or more feature vectors for the sensor input, comprising processing the sensor input … to generate a prediction output (this limitation is a mental process since a human can mentally generate feature vectors from an input).
generating one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the senor input to generate the prediction output(this limitation is a mental process since a human can mentally generate feature vectors from an input).
The claim recites processing each of the one or more feature vectors … to generate a density score for the feature vector (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (this limitation is a mental process since a human can mentally create a training dataset based on a rareness score).
Therefore, claim 14 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim recites A system comprising: one or more computers; and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations (This element amounts to mere “apply it on computer(s)” (see MPEP 2106.05(f))).
The claim recites obtaining a sensor input (this limitation is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
The claim recites using a prediction neural network (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The claim recites using a density estimation model (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
Therefore, claim 14 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 14 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A system comprising: one or more computers; and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations is a mere “apply it on computer(s)” (see MPEP 2106.05(f))).
The method of obtaining a sensor input is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
The method of using a prediction neural network is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The method using a density estimation model is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
Therefore, claim 14 is subject-matter ineligible.
Regarding Claim 15:
Subject Matter of Eligibility Analysis Step 1:
Claim 15 recites a system and is directed to a machine, which is one of the four statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites process the sensor input to generate an intermediate feature map (this limitation is a mental process since a human can mentally calculate a feature map from the sensor input, if an equation was given).
The claim recites process the intermediate feature map to generate a prediction output for the sensor input, wherein the prediction output characterizes one or more of (i) one or more regions of the sensor data or (ii) one or more objects depicted in the one or more regions (this limitation is a mental process since a human can mentally organize the feature map into regions/objects of the sensor data).
Therefore, claim 15 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim does not recite any additional elements.
Therefore, claim 15 is not integrated into a practical application
Subject Matter of Eligibility Analysis Step 2b:
The claim does not recite any additional elements, thus claim 15 does not provide significantly more.
Therefore, claim 15 is subject-matter ineligible.
Regarding Claim 17:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a system and directed to a machine, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 17 is dependent on claim 14, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 14 can be applied here. Therefore, claim 17 recites an abstract idea
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object (this limitation does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)))).
Therefore, claim 17 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 17 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 4 is subject matter ineligible.
Regarding Claim 18:
Subject Matter of Eligibility Analysis Step 1:
The claim recites a system and directed to a machine, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Since claim 18 is dependent on claim 14, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 14 can be applied here. Therefore, claim 18 recites an abstract idea
Subject Matter of Eligibility Analysis Step 2A Prong 2:
The claim recites the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent (this limitation does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)))).
Therefore, claim 18 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 18 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 18 is subject matter ineligible.
Regarding Claim 20:
Subject Matter of Eligibility Analysis Step 1:
the claim recites a computer program stored in a non-transitory computer-readable
storage medium and is directed to an article of manufacture, which is one of the statutory categories of invention.
Subject Matter of Eligibility Analysis Step 2A prong 1:
The claim recites generating one or more feature vectors for the sensor input, comprising processing the sensor input … to generate a prediction output (this limitation is a mental process since a human can mentally generate feature vectors from an input).
generating one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the senor input to generate the prediction output(this limitation is a mental process since a human can mentally generate feature vectors from an input).
The claim recites processing each of the one or more feature vectors … to generate a density score for the feature vector (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects (this limitation is both a mathematical equation since the equation for the density estimation model is given with the specification. It is also a mental process since a human can use that equation to calculate the density score).
The claim recites generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (this limitation is a mental process since a human can mentally create a training dataset based on a rareness score).
Therefore, claim 20 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A prong 2:
The claim recites One or more non-transitory computer-readable storage media storing instructions (This element recites a generic computing component on which to perform the abstract idea (see MPEP 2106.05(f))).
The claim recites obtaining a sensor input (this limitation is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
The claim recites using a prediction neural network (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f))).
The claim recites using a density estimation model (this limitation is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction” (see MPEP 2106.05(f))).
Therefore, claim 20 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements in claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
One or more non-transitory computer-readable storage media storing instructions uses computer(s) as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f))).
The method of obtaining a sensor input is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
The method of using a prediction neural network is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
The method using a density estimation model is referencing a generic term, which does not integrate the abstract idea into a practical application because it amounts to mere “instruction to apply” (see MPEP 2106.05(f))).
Therefore, claim 20 is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-6, 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas et al. (US 12319319 B2) (hereafter referred to as Casas) in view of Cobb et al. (US 20110052068 A1) (hereafter referred to as Cobb) and Frtunikj et al. (US 11769318 B2) (hereafter referred to as Frtunikj).
Regarding claim 1, Casas teaches
Obtaining a sensor input (Casas, paragraph 0015, “a computing system associated with a vehicle (e.g., an autonomous vehicle) can receive sensor data from one or more sensors that generate sensor data relative to the autonomous vehicle. In order to autonomously navigate, the autonomous vehicle can include a plurality of sensors (e.g., a LIDAR system, a RADAR system, cameras, etc.) configured to obtain sensor data associated with the autonomous vehicle's surrounding environment as well as the position and movement of the autonomous vehicle”).
generating one or more feature vectors for the sensor input, comprising processing the sensor input using a prediction neural network to generate a prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network...The operations also include receiving, in response to providing the sensor data and map data as input to the machine-learned convolutional neural network, a jointly determined prediction from the machine-learned convolutional neural network for multiple outputs”. Examiner notes that the convolutional neural network maps to the prediction neural network).
generating the one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the sensor input to generate the prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network”. Examiner notes that the shared layer maps to the hidden layer).
Casas does not teach, but Cobb does teach
processing each of the one or more feature vectors using a density estimation model to generate a density score for the feature vector, (Cobb, page 6, paragraph 0047, “The anomaly detection component 322 is configured to compute a probability density function based on the existing clusters in the ART 325 and compute a probability density value for the micro-feature vector”).
and generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects, (Cobb, page 8, paragraph 0064, “At step 476 the anomaly detection component 322 determines a rareness measure for the micro-feature vector. That is, the anomaly detection component 322 estimates a measure of the likelihood of observing the particular micro-feature vector, based on the probability density function and the probability micro-feature vector”).
Casas and Cobb are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas to use the probability density function to determine rareness from Cobb. One of the ordinary skill in the art would have known to apply the known technique of determining rareness in object detection. Therefore, applying Cobb’s technique would yield the predictable result of improving accuracy of the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Casas and Cobb do not teach, but Frtunikj does teach
generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Cobb to use create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Frtunikj’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Regarding Claim 2, Casas, Cobb, and Frtunikj the method of claim 1, Casas further teaches:
process the sensor input to generate the intermediate feature map, (Casas, claim 1, “processing the sensor data and the map data with a machine-learned model comprising (i) one or more shared layers that are trained to generate intermediate features”).
process the intermediate feature map to generate the prediction output for the sensor input, wherein the prediction output characterizes one or more of (i) one or more regions of the sensor data or (ii) one or more objects depicted in the one or more regions, (Casas, paragraph, 0020, “In some implementations, for example, corresponding forecasting outputs can include a trajectory indicative of the object's expected path towards a goal location. For instance, the trajectory can be represented by trajectory data comprising a sequence of bounding shapes (e.g., bounding boxes) at a plurality of timestamps. In some implementations, the bounding shapes can be indicative of past and current locations for an object or current and future locations for an object”. Examiner notes that the bounding shapes maps to section (ii) limitation).
Regarding Claim 4, Casas, Cobb, and Frtunikj teach the method of claim 1, Casa further teaches:
the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object, (Casas, page 4, paragraph 0038, “the computer vision engine is configured to classify each tracked object as being one of a known category of objects using training data that defines a plurality of object types” and “the classification of "other" represents an affirmative assertion that the object is neither a "person" nor a "vehicle." Additionally, the estimator/identifier component may identify characteristics of the tracked object, e.g., for a person, a prediction of gender, an estimation of a pose (e.g., standing or sitting) or an indication of whether the person is carrying an object”).
Regarding Claim 5, Casas, Cobb, and Frtunikj teach the method of claim 1, Casa further teaches:
the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent, (Casas, Abstract, “The computing system can receive a jointly determined prediction from the machine-learned intent model for multiple outputs including at least one detection output indicative of one or more objects detected within the surrounding environment of the autonomous vehicle, a first corresponding forecasting output descriptive of a trajectory indicative of an expected path of the one or more objects towards a goal location”).
Regarding Claim 6, Casas, Cobb, and Frtunikj teach the method of claim 1, Casas and Cobb do not teach, but Frtunikj does teach:
for each selected feature vector, generating a training example that includes the sensor input from which the selected feature vector is generated and including the training example in the training data (Frtunikj, paragraph 0005, “The methods then include identifying one or more trends associated with a training dataset (and/or other datasets such as validation dataset/test dataset) that includes a plurality of labeled data logs, and determining a function for assigning an importance score to each of the plurality of unlabeled sensor data logs using the one or more trends. The training dataset is used for training a machine learning model”).
training a downstream neural network on the training data for the downstream task, (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Cobb to use create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Frtunikj’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Regarding Claim 14, Casas teaches:
One or more computers; and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations (Casas, paragraph 004, “One example aspect of the present disclosure is directed to a computing system that includes one or more processors, a machine-learned intent model, and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations”).
Obtaining a sensor input (Casas, paragraph 0015, “a computing system associated with a vehicle (e.g., an autonomous vehicle) can receive sensor data from one or more sensors that generate sensor data relative to the autonomous vehicle. In order to autonomously navigate, the autonomous vehicle can include a plurality of sensors (e.g., a LIDAR system, a RADAR system, cameras, etc.) configured to obtain sensor data associated with the autonomous vehicle's surrounding environment as well as the position and movement of the autonomous vehicle”).
generating one or more feature vectors for the sensor input, comprising processing the sensor input using a prediction neural network to generate a prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network...The operations also include receiving, in response to providing the sensor data and map data as input to the machine-learned convolutional neural network, a jointly determined prediction from the machine-learned convolutional neural network for multiple outputs”. Examiner notes that the convolutional neural network maps to the prediction neural network).
generating the one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the sensor input to generate the prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network”. Examiner notes that the shared layer maps to the hidden layer).
Casas does not teach, but Cobb does teach
processing each of the one or more feature vectors using a density estimation model to generate a density score for the feature vector, (Cobb, page 6, paragraph 0047, “The anomaly detection component 322 is configured to compute a probability density function based on the existing clusters in the ART 325 and compute a probability density value for the micro-feature vector”).
and generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects, (Cobb, page 8, paragraph 0064, “At step 476 the anomaly detection component 322 determines a rareness measure for the micro-feature vector. That is, the anomaly detection component 322 estimates a measure of the likelihood of observing the particular micro-feature vector, based on the probability density function and the probability micro-feature vector”).
Casas and Cobb are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas to use the probability density function to determine rareness from Cobb. One of the ordinary skill in the art would have known to apply the known technique of determining rareness in object detection. Therefore, applying Cobb’s technique would yield the predictable result of improving accuracy of the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Casas and Cobb do not teach, but Frtunikj does teach
generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (Frtunikj, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Cobb to create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Cobb’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Regarding Claim 15, Casas, Cobb, and Frtunikj teach the system of claim 14, Casas further teaches:
process the sensor input to generate the intermediate feature map, (Casas, claim 1, “processing the sensor data and the map data with a machine-learned model comprising (i) one or more shared layers that are trained to generate intermediate features”).
process the intermediate feature map to generate the prediction output for the sensor input, wherein the prediction output characterizes one or more of (i) one or more regions of the sensor data or (ii) one or more objects depicted in the one or more regions, (Casas, paragraph, 0020, “In some implementations, for example, corresponding forecasting outputs can include a trajectory indicative of the object's expected path towards a goal location. For instance, the trajectory can be represented by trajectory data comprising a sequence of bounding shapes (e.g., bounding boxes) at a plurality of timestamps. In some implementations, the bounding shapes can be indicative of past and current locations for an object or current and future locations for an object”. Examiner notes that the bounding shapes maps to section (ii) limitation).
Regarding Claim 17, Casas, Cobb, and Frtunikj teach the system of claim 14, Casas further teaches:
the prediction output for the sensor input comprises object detection prediction data for the sensor input that specifies one or more regions of the sensor data that are each predicted to depict a respective object, (Casas, page 4, paragraph 0038, “the computer vision engine is configured to classify each tracked object as being one of a known category of objects using training data that defines a plurality of object types” and “the classification of "other" represents an affirmative assertion that the object is neither a "person" nor a "vehicle." Additionally, the estimator/identifier component may identify characteristics of the tracked object, e.g., for a person, a prediction of gender, an estimation of a pose (e.g., standing or sitting) or an indication of whether the person is carrying an object”).
Regarding Claim 18, Casas, Cobb, and Frtunikj teach the system of claim 14, Casa further teaches:
the prediction output for the sensor input comprises trajectory prediction data for the sensor input that characterizes a predicted future trajectory of a target agent, (Casas, Abstract, “The computing system can receive a jointly determined prediction from the machine-learned intent model for multiple outputs including at least one detection output indicative of one or more objects detected within the surrounding environment of the autonomous vehicle, a first corresponding forecasting output descriptive of a trajectory indicative of an expected path of the one or more objects towards a goal location”).
Regarding Claim 19, Casas, Cobb, and Frtunikj teach the system of claim 14, Casas and Cobb do not teach, but Frtunikj does teach:
for each selected feature vector, generating a training example that includes the sensor input from which the selected feature vector is generated and including the training example in the training data (Frtunikj, paragraph 0005, “The methods then include identifying one or more trends associated with a training dataset (and/or other datasets such as validation dataset/test dataset) that includes a plurality of labeled data logs, and determining a function for assigning an importance score to each of the plurality of unlabeled sensor data logs using the one or more trends. The training dataset is used for training a machine learning model”).
training a downstream neural network on the training data for the downstream task, (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Frtunikj to create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Frtunikj’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Regarding Claim 20:
One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations (Casas, paragraph 004, “One example aspect of the present disclosure is directed to a computing system that includes one or more processors, a machine-learned intent model, and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations”).
Obtaining a sensor input (Casas, paragraph 0015, “a computing system associated with a vehicle (e.g., an autonomous vehicle) can receive sensor data from one or more sensors that generate sensor data relative to the autonomous vehicle. In order to autonomously navigate, the autonomous vehicle can include a plurality of sensors (e.g., a LIDAR system, a RADAR system, cameras, etc.) configured to obtain sensor data associated with the autonomous vehicle's surrounding environment as well as the position and movement of the autonomous vehicle”).
generating one or more feature vectors for the sensor input, comprising processing the sensor input using a prediction neural network to generate a prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network...The operations also include receiving, in response to providing the sensor data and map data as input to the machine-learned convolutional neural network, a jointly determined prediction from the machine-learned convolutional neural network for multiple outputs”. Examiner notes that the convolutional neural network maps to the prediction neural network).
generating the one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the sensor input to generate the prediction output (Casas, paragraph 0005, “The autonomous vehicle also includes a machine-learned convolutional neural network comprising a plurality of shared layers that determine features used to jointly determine multiple outputs of the machine-learned convolutional neural network”. Examiner notes that the shared layer maps to the hidden layer).
Casas does not teach, but Cobb does teach
processing each of the one or more feature vectors using a density estimation model to generate a density score for the feature vector, (Cobb, page 6, paragraph 0047, “The anomaly detection component 322 is configured to compute a probability density function based on the existing clusters in the ART 325 and compute a probability density value for the micro-feature vector”).
and generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a classification of an object depicted in the sensor input is rare relative to other objects, (Cobb, page 8, paragraph 0064, “At step 476 the anomaly detection component 322 determines a rareness measure for the micro-feature vector. That is, the anomaly detection component 322 estimates a measure of the likelihood of observing the particular micro-feature vector, based on the probability density function and the probability micro-feature vector”).
Casas and Cobb are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas to use the probability density function to determine rareness from Cobb. One of the ordinary skill in the art would have known to apply the known technique of determining rareness in object detection. Therefore, applying Cobb’s technique would yield the predictable result of improving accuracy of the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Casas and Cobb do not teach, but Frtunikj does teach
generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Cobb to create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Frtunikj’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas, Cobb, and Frtunikj in view of Mao et al. (3D Object Detection for Autonomous Driving: A Comprehensive Survey) (hereafter referred as Mao).
Casas, Cobb, and Frtunikj teach the method of claim 1, Frtunikj further teaches
the downstream neural network is the same neural network as the prediction neural network, (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”. Examiner notes that further training the machine learning model suggests that the downstream neural network is the same as the prediction neural network).
Casas, Cobb, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Frtunikj to create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Cobb’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Casas, Cobb, and Frtunikj do not teach, but Mao does teach
the downstream task is three-dimensional object detection task, (Mao, paragraph 0002, “To obtain a comprehensive understanding of driving environments, many vision tasks can be involved in a perception system, e.g. object detection and tracking, lane detection, and semantic and instance segmentation. Among these perception tasks, 3D object detection is one of the most indispensable tasks in an automotive perception system”.)
Casas, Cobb, Frtunikj, and Mao are considered analogous because both deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Cobb to have the downstream task be a three-dimensional object detection task. Doing so allows “geometric information predicted by 3D object detection in real-world coordinates [to] be directly utilized to measure the distances between the ego-vehicle and critical objects, and to further help plan driving routes and avoid collisions” (Mao, paragraph 2).
Claim(s) 8 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas, Cobb, and Frtunikj in view of Kuang et al. (Computer Vision and Normalizing Flow-Based Defect Detection) (hereafter referred as Kuang).
Regarding claim 8, Casas, Cobb, and Frtunikj teach the method of claim 1. Casas, Cobb, and Frtunikj do not teach, but Kuang does teach
generating a training set of feature vectors by generating, for each sensor input in a second set of sensor data and using an object detection neural network, a respective feature vector for each of one or more regions in the sensor input that are predicted by the object detection prediction neural network to depict an object (Kuang, page 3, In our defect detection model, we want to focus on the product, and to eliminate the unnecessary information from each frame (such as back ground), we decided to adopt a pre-trained YOLOv5 object detection model to narrow down the defect detection searching window on input images collected from each cameras. The pre-trained YOLOv5 model was further fine-tuned with the ZeroBox dataset” and “For fair and reliable experiment results, ZeroBox Inc. has created a brand new dataset collected from an industrial production line monitoring system. This dataset includes 21 video clips in total which consists of 13 types of products with both good and defective samples... In addition, there are 1381 good product images and 253 defective product images generated through YOLO detection and cropping” (Kuang, page 5)).
training the density estimation model on the training set of feature vectors to maximize an expected log density score of the feature vectors in the training set (Kuang, page 4, “After object detection and background subtraction, the processed images are further resized to the size of 448 by 448 pixels that only contains the product information excluding any background noise. Then the processed images are fed into DifferNet to output a normal distribution by maximum likelihood training. To classify if an input image is anomalous or not, our model uses a scoring function that calculates the average of the negative log-likelihoods using multiple transformations of an image. The result will compare with the threshold value which is learned during training and validation process and is later applied to detect if the image contains an anomaly or not”).
Casas, Cobb, Frtunikj, and Kuang are considered analogous because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Casas, Cobb, and Frtunikj to use DifferNet for the density estimation model. One of the ordinary skill in the art would know doing so is a simple substitution of one known element (density estimation model) for another (DifferNet) to obtain predictable results (generating a density score) (MPEP 2141 (III)(B) Simple substitution of one known element for another to obtain predictable results).
Regarding claim 9, Casas, Cobb, and Frtunikj teach the method of claim 1. Casas, Cobb, and Frtunikj do not teach, but Kuang does teach
the density estimation model is a normalizing flow, (Kuang, page 3, ‘The objective of density estimation is to learn the underlying probability density from a set of independent and identically distributed sample data [18]. In 2020, M. Rudolph et al. [19] proposed a normalizing flow-based model called DifferNet, which utilizes a latent space of normalizing flow to represent normal samples’ feature distribution”)
Casas, Cobb, Frtunikj, and Kuang are considered analogous because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Casas, Cobb, and Frtunikj to use DifferNet for the density estimation model. One of the ordinary skill in the art would know doing so is a simple substitution of one known element (density estimation model) for another (DifferNet) to obtain predictable results of generating a density score (MPEP 2141 (III)(B) Simple substitution of one known element for another to obtain predictable results).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas, Cobb, and Frtunikj in view of Gao et al. (WO 2022226434 A1) (hereafter referred as Gao).
Casas, Cobb, and Frtunikj teach the method of claim 1. Casas, Cobb, and Frtunikj do not teach, but Gao does teach
the rareness score for the feature vector is inversely proportional to the density score for the feature vector (Gao, page 17, paragraph 0067, “The self-driving behavior metric is related to a rareness of an event and a value(s) of the criteria of the self-driving behavior metric are calculated. For each criterion in a particular seif-driving behavior metric, the probability density of this criterion indicates the rareness of the calculated value. The rarer the calculated value, the more unusual (and potentially improper) is the behavior. Also, a smaller criteria value (e.g., smaller minimum distance) may indicate a more improper self-driving behavior. Thus, the self-driving behavior metric for each criterion can be defined to be the inverse of the probability density and the actual criterion value”)
Casas, Cobb, Frtunikj, and Gao are considered analogous to the claimed invention because both deal with retrieving driving data and evaluating rare/improper scenarios. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas, Cobb, and Frtunikj to use the inverse relationship to calculate the rareness score. One of the ordinary skill in the art would have known to apply the known technique of defining a rareness score as the inverse of a density score from Gao. Therefore, applying Gao’s technique would yield the predictable result of determining the rarity of classified objects (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas, Cobb, and Frtunikj in view of Daiki (US 20220156529 A1) (hereafter referred as Daiki).
Casas, Cobb, and Frtunikj teach the method of claim 1. Casas, Cobb, and Frtunikj do not teach, but Daiki does teach
ranking respective feature vectors generated from the plurality of senor inputs by rareness scores, and selecting a proper subset of respective feature vectors having the highest rareness scores according to the ranking (Daiki, page 3, paragraph 0022, “detection program 200 can classify an image as an anomaly utilizing a distance vector based on features of the image” and “ranks and selects a top “k” elements (e.g., from k-means clustering) to obtain a reduced distance vector”)
Casas, Cobb, Frtunikj, and Daiki are considered analogous to the claimed invention because both deal with anomaly detection. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to modify Casas, Cobb, and Frtunikj by using the ranking and selecting method from Daiki. One of the ordinary skill in the art would have known to apply the known technique of ranking and selecting scores from Daiki Therefore, applying Gao’s technique would yield the predictable result of determining the rarity of classified objects, in order to improve training efficiency (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas, Cobb, and Frtunikj in view of Chen et al. (Generating Autonomous Driving Test Scenarios based on OpenSCENARIO) (hereafter referred to as Chen).
Regarding claim 12, Casas, Cobb, and Frtunikj teach the method of claim 1. Casas, Cobb, and Frtunikj do not teach, but Chen does teach
generating, from the sensor input and other sensor inputs, one or more test scripts for a software module, (Chen, Section 2, “A SAM has proposed the Open SCENARIO standard, which developers can easily use to describe test scenarios and clearly describe the dynamic behavior of traffic participants” and “51Sim-One is an autonomous driving system independently developed by 51WORL D that integrates multi-sensor simulation, vehicle dynamics, road and scenario simulation, traffic flow and intelligent body simulation, perception and decision simulation, evaluation indicators, and autonomous driving behavior training”,(Chen, Section 2.2.2) Examiner notes that the 51Sim-One supports and uses the Open SCENARIO standard).
and evaluating a performance of the software module by using the software module to process the one or more test scripts, (Chen, Section 1, “For autonomous vehicles in the simulation world, the autonomous driving system is based on the function under test (e.g., passing through the intersection), and the system under test decides control plan (e.g., local path) based on the semantic data in the simulation platform (e.g., self-vehicle speed), the dynamic model is converted into the corresponding response speed, etc., and fed back to the simulation platform, the user can evaluate the performance of the autonomous driving system through the recorded data in the simulation platform, so as to evaluate the autonomous driving system”)
Casas, Cobb, Frtunikj, and Chen are considered analogous to the claimed invention because they both evaluate determining rareness scores. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas, Cobb, and Frtunikj to also generate testing scripts for evaluation used in Chen. One of the ordinary skill in the art would have known to apply the known technique of creating test scripts and evaluating a performance of the software. Therefore, applying Chen’s technique would yield the predictable result of determining the rarity of classified objects, in order to improve training efficiency (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Casas and Frtunikj in view of Djuric et al. (US 11112796 B2) (hereafter referred to as Djuric).
Regarding claim 13, Casas teaches
Obtaining a sensor input (Casas, paragraph 0015, “a computing system associated with a vehicle (e.g., an autonomous vehicle) can receive sensor data from one or more sensors that generate sensor data relative to the autonomous vehicle. In order to autonomously navigate, the autonomous vehicle can include a plurality of sensors (e.g., a LIDAR system, a RADAR system, cameras, etc.) configured to obtain sensor data associated with the autonomous vehicle's surrounding environment as well as the position and movement of the autonomous vehicle”).
Casas does not teach, but Djuric does teach
processing each of the one or more feature vectors using a density estimation model to generate a density score for the feature vector, wherein the density score represents an estimate of a density of the feature vector in a training set of feature vectors used to train the density estimation model, (Djuric, paragraph 0077, “The negative log-likelihood of the training data set can be minimized, with the potential assumption that all waypoints are independently sampled… assuming a set T of training trajectories” and “the previously described loss function…can be optimized, resulting in a Gaussian Mixture Model or Mixture Density Network” (Djuric, paragraph 0084) Examiner notes that the negative log-likelihood maps to the density estimation model and set T of training trajectories shows that the negative log-likelihood is based off a training set).
generating a rareness score for each of the one or more feature vectors from the density score, wherein the rareness score represents a degree to which a predicted behavior of an agent depicted in the sensor input is rare relative to other objects, (Djuric, paragraph 0081-0082, “the model 136 can determine a trajectory confidence level for each predicted trajectory. A trajectory confidence level can indicate a per-trajectory probability, quantifying how likely an object is to follow a certain trajectory (e.g., the sum of per-trajectory probabilities over all outputted trajectories can equal 1)… The model 136 can be trained to determine a trajectory confidence level (e.g., a per-trajectory probability) by minimizing cross-entropy loss over all per-trajectory probabilities”. Examiner notes that the trajectory confidence level maps to the rareness score).
Casas and Djuric are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas to use the mixed density network to determine rareness from Djuric. One of the ordinary skill in the art would have known to apply the known technique of determining rareness in object detection. Therefore, applying Djuric’s technique would yield the predictable result of improving accuracy of the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Casas and Djuric do not teach, but Frtunikj does teach
generating, from at least the sensor input, a training data set for training a downstream neural network for a downstream task, comprising selecting the sensor input for inclusion in the training data set based on the rareness scores for the one or more feature vectors (Frtunikj, paragraph 0005, “The methods then include using the function for assigning the importance score to each of the plurality of unlabeled sensor data logs, selecting a subset of the plurality of sensor data logs that have an importance score greater than a threshold, and using the subset of the plurality of sensor data logs for further training the machine learning model trained using the training dataset to generate an updated model”).
Casas, Djuric, and Frtunikj are considered analogous to the claimed invention because they deal with object detection. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Casas and Djuric to use create a training dataset to further train a machine learning model from Frtunikj. One of the ordinary skill in the art would have known to apply the known technique of training a downstream model. Therefore, applying Frtunikj’s technique would yield the predictable result of improving data efficiency, cost, and time for the machine learning model (MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predictable results).
Response to Arguments
On page 1, Applicant argues:
Claim 8 stands rejected under 35 U.S.C. § 112 as allegedly indefinite. Applicant has amended claim 8. Therefore, reconsideration and withdrawal of the rejection are respectfully requested.
Regarding the Applicant’s amendment to claim 8, Examiner agrees that claim 8 is no longer indefinite. Therefore, the 35 U.S.C. § 112 rejection has been withdrawn.
On page 1-2, Applicant argues:
Claims 1-20 were rejected under 35 U.S.C. § 101 as allegedly reciting non-patentable subject matter. Applicant respectfully disagrees.
Under step 2A, Prong 1, the claims do not fall into any of the enumerated groupings of ineligible subject matter, and in particular, do not recite a mental process.
Regarding this category, the MPEP states: "i. A Claim With Limitation(s) That Cannot Practically Be Performed In The Human Mind Does Not Recite A Mental Process." MPEP § 2106.04(a)(2)(III)(A) (emphasis in original).
Applicant respectfully submits that the claims as amended include features that cannot practically be performed in the human mind.
Claim 1 as amended for example recites, "processing the sensor input using a prediction neural network to generate a prediction output," "generating the one or more feature vectors for
the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the sensor input to generate the prediction output," and "training a downstream neural network for a downstream task."
Claim 13 as amended for example recites "processing the sensor input using an encoder neural network trained as part of a behavior prediction neural network to generate one or more feature vectors for the sensor input" and "training a downstream neural network for a downstream task."
Applicant respectfully submits that the human mind cannot practically implement a prediction neural network or a behavior prediction neural network. Nor can the human mind practically generate an intermediate feature map by a hidden layer of the prediction neural network. Nor can the human mind practically train a downstream neural network.
Accordingly, claim 1 does not recite features that would fall into any of the enumerated groupings. See also the designated informative PTAB decision Ex Parte Hannun 2018-003323, pp. 9-10 (PTAB) (decided Apr. 1, 2019, designated "informative" on Dec. 11, 2019) (steps for using a trained model to score items "are not steps that can practically be performed mentally" and therefore recite patent-eligible subject matter.)
Regarding the Applicant’s argument that the claims cannot be performed in the human mind and does not recite a mental process, Examiner respectfully disagrees. MPEP 2106.04(a)(2)(III)(C) states “with the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper”. The use of a prediction/behavior neural network does not mean a human cannot mentally process input to generate an output or generate features.
On page 3-4, Applicant argues:
Moreover, Applicant notes that the MPEP now explicitly recognizes that "improvements as to how the machine learning model itself operates, including training a machine learning model" constitutes an improvement in computer functionality. MPEP 2106.04(d) (https://www.uspto.jgov/web/offices/pac/mpep/ANC-Desiardins-Memo-12-5-25.pdf).
The present Specification similarly describes technical improvements that can be
achieved by the claimed subject matter with respect to training a machine learning model (a downstream neural network).
As explained in paragraph 0019 of the Specification, "when incorporated into the training data, these identified rare examples can improve the capability of trained neural networks to handle rare or unusual scenarios. The rare example mining techniques described in this specification can improve overall model performance, and more importantly, the performance with respect to rare data instances, while consuming fewer computing resources and being faster in terms of wall-clock time than the existing techniques. When deployed within an on-board system of a vehicle, processing sensor data using neural networks that have been trained on
training data obtained using the described rare example mining techniques can be used to make autonomous driving decisions for the vehicle with enhanced overall road safety and/or efficiency."
Training neural networks to improve their "performance with respect to rare data
instances, while consuming fewer computing resources and being faster in terms of wall-clock time than the existing techniques" represents "improvements as to how the machine learning model itself operates, including training a machine learning model."
The claims "include the components or steps of the invention that provide the
improvement described in the specification (MPEP § 2106.05(a))." For example, independent claims 1, 14, and 20 each recite:
"generating one or more feature vectors for the sensor input, comprising
processing the sensor input using a prediction neural network to generate a prediction output, and
generating the one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network
when processing the sensor input to generate the prediction output;
processing each of the one or more feature vectors using a density estimation
model to generate a density score for the feature vector;
generating a rareness score for each of the one or more feature vectors from the
density score, wherein the rareness score represents a degree to which a classification of
an object depicted in the sensor input is rare relative to other objects;
generating, from at least the sensor input, a training data set for training a
downstream neural network for a downstream task, comprising selecting the sensor input
for inclusion in the training data set based on the rareness scores for the one or more
feature vectors."
The claims thus recite a specific combination of steps - including "generating one or
more feature vectors," "generating a rareness score for each of the one or more feature vectors," and "selecting the sensor input... based on the rareness scores for the one or more feature vectors" - to generate training data that can be used to train a downstream neural network.
As another example, independent claim 13 recites:
"processing the sensor input using an encoder neural network to generate one or
more feature vectors for the sensor input;
processing each of the one or more feature vectors using a density estimation
model to generate a density score for the feature vector, wherein the density score
represents an estimate of a density of the feature vector in a training set of feature vectors
used to train the density estimation model;
generating a rareness score for each of the one or more feature vectors from the
density score, wherein the rareness score represents a degree to which a predicted
behavior of an agent depicted in the sensor input is rare relative to other objects; and
generating, from at least the sensor input, a training data set for training a
downstream neural network for a downstream task, comprising selecting the sensor input
for inclusion in the training data set based on the rareness scores for the one or more
feature vectors."
Claim 13 thus recites a specific combination of steps - including "processing the sensor input using an encoder neural network," "generating a rareness score for each of the one or more feature vectors," and "selecting the sensor input... based on the rareness scores for the one or more feature vectors" - to generate training data that can be used to train a downstream neural network.
For at least these reasons, independent claim 1, as amended, along with the corresponding dependent claims, are directed to patent-eligible subject matter. Independent claims 13, 14, and 20 have been amended similarly, and together with their corresponding dependent claims, are also directed to patent eligible subject matter for at least analogous reasons.
Regarding the Applicant’s argument that the claims provide an improvement to a technology or technical field, Examiner respectfully disagrees. MPEP 2106.05(a) states the claimed invention provides an improvement can include a discussion in the specification that identifies a technical problem and explains the details of an unconventional technical solution expressed in the claim. The examples of claims 1 and 13 do not express the use of the training dataset to train a downstream neural network, merely the generation of the training data.
However, Examiner agrees that claims 6 and 19 provide an improvement to the technology or technical field. Therefore, the 35 U.S.C. § 101 rejection has been withdrawn for claim 6 and 19.
On page 5-6, Applicant argues:
Claims 1, 14, and 20
Claim 1 as amended recites, "generating one or more feature vectors for the sensor input, wherein the generating comprises processing the sensor input using a prediction neural network to generate a prediction output, and generating the one or more feature vectors for the sensor input based on an intermediate feature map generated by a hidden layer of the prediction neural network when processing the sensor input to generate the prediction output."
Applicant respectfully submits that the cited portion of Cobb does not disclose or suggest these features of amended claim 1.
In particular, the cited portion of Cobb does not disclose or suggest a prediction neural network that generates an intermediate feature map "when processing the sensor input to generate the prediction output." Nor does the cited portions of Cobb disclose or suggest "generating the one or more feature vectors for the sensor input based on [the] intermediate feature map."
Rather, the cited portion of Cobb that merely discloses that "the context processor 220 may be configured to generate a stream of micro-feature vectors corresponding to foreground patches tracked (by tracker component 210)." At paragraph 0037.
Mao, Kuang, Gao, Daiki, and Chen, either alone or in combination, do not alleviate the deficiencies of Cobb in disclosing these features.
Therefore, Applicant respectfully submits that amended claim 1 and its dependent claims are in condition for allowance. Independent claims 14 and 20 and their respective dependent claims are allowable for corresponding reasons.
Claim 13
Claim 13 as amended recites, "processing each of the one or more feature vectors using a density estimation model to generate a density score for the feature vector, wherein the density score represents an estimate of a density of the feature vector in a training set of feature vectors used to train the density estimation model."
Applicant respectfully submits that the cited portion of Cobb does not disclose or suggest these features of amended claim 13.
Cobb teaches the opposite, teaching that "the classification of objects is performed by the micro-feature classifier 221 in the machine learning engine 140 using the micro-feature vectors that are produced by the computer vision engine 135 independent of any training data." At paragraph 0039, emphasis added.
Thus, Cobb fails to disclose setting a density score that "represents an estimate of a density of the feature vector in a training set of feature vectors used to train the density estimation model," as required by claim 13.
Therefore, Applicant respectfully submits that amended claim 13 and its dependent claims are in condition for allowance.
Applicant’s arguments with respect to claim(s) 1, 13, 14, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Examiner respectfully directs the Applicant to the above 103 rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Seow et al. ("UNSUPERVISED LEARNING OF FEATURE ANOMALIES FOR A VIDEO SURVEILLANCE SYSTEM") discloses techniques for analyzing a scene depicted in an input stream of video frames captured by a video camera. Virkar et al. (“Machine Learning Methods and Systems for Identifying Patterns in Data”) discloses methods for training machines to categorize data, and/or recognize patterns in data, and machines and systems so trained.Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN VO whose telephone number is (571)272-9622. The examiner can normally be reached Monday - Friday from 7:00 am - 3:00 pm EST.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.V./Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights