DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because of the following informalities.
View numbers must be preceded by the abbreviation "FIG." See 37 CFR 1.84(u)(1).
Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Title
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Examiner believes that the title of the invention is imprecise. A descriptive title indicative of the invention will help in proper indexing, classifying, searching, etc. See MPEP 606.01. However, the title of the invention should be limited to 500 characters. Examiner suggests including the aspect(s) of the claims which Applicant believes to be novel or nonobvious over the prior art.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
With respect to Claim 1:
Step 1: Claim 1 is a method claim. Therefore, claim 1 is directed to either a process, machine, manufacture or composition of matter.
Step 2A Prong 1:
evaluating first data obtained using one or more first sensors of a first modality with respect to second data obtained using one or more second sensors of a second modality (mental process – user can manually evaluate first data obtained using one or more first sensors of a first modality with respect to second data obtained using one or more second sensors of a second modality)
determining, based at least on the evaluating, that one or more differences corresponding to one or more first points included in the first data and one or more second points included in the second data meet or exceed a threshold (mental process – user can manually determine, based at least on the evaluating, that one or more differences corresponding to one or more first points included in the first data and one or more second points included in the second data meet or exceed a threshold)
based at least on the one or more differences meeting or exceeding the threshold, generating an updated version of the first data based at least on refining at least a portion of the first data that corresponds to the one or more first points (mental process – user can manually based at least on the one or more differences meeting or exceeding the threshold, generate an updated version of the first data based at least on refining at least a portion of the first data that corresponds to the one or more first points)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
at least one of: updating one or more parameters of one or more machine learning models using ground truth data corresponding to the updated version of the first data; or validating the one or more machine learning models using the ground truth data corresponding to the updated version of the first data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f))
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
at least one of: updating one or more parameters of one or more machine learning models using ground truth data corresponding to the updated version of the first data; or validating the one or more machine learning models using the ground truth data corresponding to the updated version of the first data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f))
Conclusion: The claim is not patent eligible.
Regarding Claim 2: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the refining of the portion of the first data comprises updating one or more first values associated with the one or more first points based at least on one or more second values associated with the one or more second points.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 3: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually determining, based at least on the evaluating, that one or more second differences corresponding to one or more first frames of the first data and one or more second frames of the second data meet or exceed a second threshold; and
based at least on the one or more second differences meeting or exceeding the second threshold, causing the one or more first frames to be excluded from the updated version of the first data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 4: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually a first resolution associated with the first data is less than a second resolution associated with the second data, and
the generating of the updated version of the first data comprises increasing the first resolution to a third resolution based at least on updating the first data to include at least a portion of the one or more second points of the second data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 5: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually one or more first values associated with the one or more first points of the first data are representative of one or more first distances between the one or more first sensors and one or more objects in an environment, and
one or more second values associated with the one or more second points of the second data are representative of one or more second distances between the one or more second sensors and the one or more objects.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 6: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the first data comprises LiDAR data obtained using one or more LiDAR sensors and the second data comprises one or more outputs generated using a neural network and based at least on image data obtained using one or more image sensors.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 7: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the evaluating of the first data with respect to the second data comprises evaluating one or more signals representative of one or more metrics associated with at least one of the first data or the second data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
With respect to Claim 8:
Step 1: Claim 8 is a method claim. Therefore, claim 8 is directed to either a process, machine, manufacture or composition of matter.
Step 2A Prong 1:
generate, based at least on the first data and the second data, ground truth data for at least one of training or validating one or more machine learning models (mental process – user can manually generate, based at least on the first data and the second data, ground truth data for at least one of training or validating one or more machine learning models)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
one or more processors (mere instructions to apply the exception using a generic computer component)
obtain first data representing one or more first points associated with an environment, the first data generated based at least on first sensor data obtained using one or more first sensors of a first modality (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))
obtain second data representing one or more second points associated with the environment, the second data generated based at least on second sensor data obtained using one or more second sensors of a second modality (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)))
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
one or more processors (mere instructions to apply the exception using a generic computer component)
obtain first data representing one or more first points associated with an environment, the first data generated based at least on first sensor data obtained using one or more first sensors of a first modality (MPEP 2106.05(d)(II) indicate that merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer)
obtain second data representing one or more second points associated with the environment, the second data generated based at least on second sensor data obtained using one or more second sensors of a second modality (MPEP 2106.05(d)(II) indicate that merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer)
Conclusion: The claim is not patent eligible.
Regarding Claim 9: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually the one or more processors further to generate an updated version of at least one of the first data or the second data to reduce one or more differences between at least a first subset of the one or more first points and a second subset of the one or more second points, wherein the generation of the ground truth data is further based at least on the updated version of the first data or the second data.
The limitation(s) includes the additional elements of one or more processors.
These judicial exceptions are not integrated into a practical application. The additional element(s) of one or more processors are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of one or more processors amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 10: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the ground truth data comprises an updated version of the first data, the updated version of the first data generated based at least on refining at least a first subset of the one or more first points of the first data that correspond to at least a second subset of the one or more second points of the second data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 11: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the updated version of the first data is generated further based at least on excluding one or more frames of the first data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 12: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the ground truth data includes at least a first subset of frames of the first data and a second subset of frames of the second data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 13: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein a first resolution associated with the first data is less than a second resolution associated with the second data, the one or more processors further to update the first resolution associated with the first data to a third resolution using at least a portion of the one or more second points.
The limitation(s) includes the additional elements of one or more processors.
These judicial exceptions are not integrated into a practical application. The additional element(s) of one or more processors are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of one or more processors amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 14: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein:
the one or more first points of the first data are representative of one or more measured distances between one or more objects in the environment and the one or more first sensors,
the one or more second points of the second data are representative of one or more predicted distances between the one or more objects and the one or more second sensors, and
the ground truth data includes one or more third points based at least on the one or more first points and the one or more second points, the one or more third points representative of one or more estimated distances between the one or more objects and at least one of the one or more first sensors or the one or more second sensors.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 15: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually generate one or more signals representative of one or more metrics associated with one or more differences between one or more frames of the first data and one or more corresponding frames of the second data; and
cause to be excluded, from the ground truth data, a subset of the one or more frames based at least on an evaluation of the one or more signals.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 16: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
With respect to Claim 17:
Step 1: Claim 17 is a method claim. Therefore, claims 17 is directed to either a process, machine, manufacture or composition of matter.
Step 2A Prong 1:
the ground truth data generated using first data updated based at least on determining that a difference between one or more portions of first data and one or more corresponding portions of second data meets or exceeds a threshold, the first data determined based at least on first sensor data corresponding to a first sensor modality and the second data determined based at least on second sensor data corresponding to a second sensor modality (mental process – user can manually generate the ground truth data generated using first data updated based at least on determining that a difference between one or more portions of first data and one or more corresponding portions of second data meets or exceeds a threshold, the first data determined based at least on first sensor data corresponding to a first sensor modality and the second data determined based at least on second sensor data corresponding to a second sensor modality)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
one or more circuits to cause performance of one or more operations associated with a machine based at least on one or more outputs of one or more machine learning models (mere instructions to apply the exception using a generic computer component)
wherein the one or more machine learning models are trained using ground truth data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f))
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
one or more circuits to cause performance of one or more operations associated with a machine based at least on one or more outputs of one or more machine learning models (mere instructions to apply the exception using a generic computer component)
wherein the one or more machine learning models are trained using ground truth data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f))
Conclusion: The claim is not patent eligible.
Regarding Claim 18: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the ground truth data includes one or more points having one or more values indicating that the one or more points were missing from the first data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 19: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) includes the additional elements of wherein the second data is determined based at least on applying the second sensor data to at least one of a machine learning model or a vision-based algorithm.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the second data is determined based at least on applying the second sensor data to at least one of a machine learning model or a vision-based algorithm recite merely adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the second data is determined based at least on applying the second sensor data to at least one of a machine learning model or a vision-based algorithm recite adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, the claims are not patent eligible.
Regarding Claim 20: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 8-10, 12-14, 16 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kolych et al. (hereinafter Kolych), U.S. Patent Application Publication 2025/0278463.
Regarding Claim 8, Kolych discloses a system comprising:
one or more processors [“Processing device 802 represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like.” ¶103] to:
obtain first data representing one or more first points associated with an environment, the first data generated based at least on first sensor data obtained using one or more first sensors of a first modality [“receive imaging data from the image capture device 110 and sensor data from the sensor unit 120. The imaging controller 130 can compare the imaging data to the sensor data, to determine whether the imaging data correlates to the sensor data.” ¶26];
obtain second data representing one or more second points associated with the environment, the second data generated based at least on second sensor data obtained using one or more second sensors of a second modality [“receive imaging data from the image capture device 110 and sensor data from the sensor unit 120. The imaging controller 130 can compare the imaging data to the sensor data, to determine whether the imaging data correlates to the sensor data.” ¶26]; and
generate, based at least on the first data and the second data, ground truth data for at least one of training or validating one or more machine learning models [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26].
Regarding Claim 9, Kolych discloses the system of claim 8. Kolych further discloses the one or more processors further to generate an updated version of at least one of the first data or the second data to reduce one or more differences between at least a first subset of the one or more first points and a second subset of the one or more second points, wherein the generation of the ground truth data is further based at least on the updated version of the first data or the second data [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26; “control logic can extract position information from the imaging data (e.g., x-, y-coordinate information)” ¶41; “the user face position satisfies a threshold criterion (e.g., a distance from the image capture device)” ¶41; sensor data can include a set of distances between the sensing element (such as the receiving element 123 of FIG. 1) and the user's face” ¶44].
Regarding Claim 10, Kolych discloses the system of claim 8. Kolych further discloses wherein the ground truth data comprises an updated version of the first data, the updated version of the first data generated based at least on refining at least a first subset of the one or more first points of the first data that correspond to at least a second subset of the one or more second points of the second data [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26; “control logic can extract position information from the imaging data (e.g., x-, y-coordinate information)” ¶41; “the user face position satisfies a threshold criterion (e.g., a distance from the image capture device)” ¶41; sensor data can include a set of distances between the sensing element (such as the receiving element 123 of FIG. 1) and the user's face” ¶44].
Regarding Claim 12, Kolych discloses the system of claim 8. Kolych further discloses wherein the ground truth data includes at least a first subset of frames of the first data and a second subset of frames of the second data [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26].
Regarding Claim 13, Kolych discloses the system of claim 8. Kolych further discloses wherein a first resolution associated with the first data is less than a second resolution associated with the second data [“the image capture device 620 can be capturing imaging data, but at a lower quantity/quality (e.g., lower frames-per-second, a lower resolution, etc.).” ¶82], the one or more processors further to update the first resolution associated with the first data to a third resolution using at least a portion of the one or more second points [“whether the sensor data satisfies a quality criterion. If the sensor data does not satisfy the quality criterion, the control logic can determine a new path for the RADAR unit using new imaging data captured by the image capture device and electronically steer the RADAR unit along the new path to capture another portion of the sensor data using the RADAR unit” ¶94; “other processing techniques can be used to increase sensor data resolution. For example, the object (e.g., face of a human) can be scanned from two or more locations.” ¶93].
Regarding Claim 14, Kolych discloses the system of claim 8. Kolych further discloses wherein:
the one or more first points of the first data are representative of one or more measured distances between one or more objects in the environment and the one or more first sensors [“control logic can extract position information from the imaging data (e.g., x-, y-coordinate information)” ¶41; “the user face position satisfies a threshold criterion (e.g., a distance from the image capture device)” ¶41],
the one or more second points of the second data are representative of one or more predicted distances between the one or more objects and the one or more second sensors [“sensor data can include a set of distances between the sensing element (such as the receiving element 123 of FIG. 1) and the user's face” ¶44], and
the ground truth data includes one or more third points based at least on the one or more first points and the one or more second points, the one or more third points representative of one or more estimated distances between the one or more objects and at least one of the one or more first sensors or the one or more second sensors [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26].
Regarding Claim 16, Kolych discloses the system of claim 8. Kolych further discloses wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources [“The machine can operate in the capacity of a server or a client machine in client-server network environment, as a peer machine in a peer-to-peer ( or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.” ¶100].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-7, 17, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kolych in view of Paluri, U.S. Patent Application Publication 2019/0197667.
Regarding Claim 1, Kolych discloses a method comprising:
evaluating first data obtained using one or more first sensors of a first modality with respect to second data obtained using one or more second sensors of a second modality [“receive imaging data from the image capture device 110 and sensor data from the sensor unit 120. The imaging controller 130 can compare the imaging data to the sensor data, to determine whether the imaging data correlates to the sensor data.” ¶26];
determining, based at least on the evaluating, that one or more differences corresponding to one or more first points included in the first data and one or more second points included in the second data meet or exceed a threshold [“the multiple modes of imaging data can be correlated if data from a first mode of imaging data exceeds a correlation threshold with respect to a second mode of imaging data (e.g., data of the first mode has a sufficient overlap or similarity with data of the second mode).” ¶17];
based at least on the one or more differences meeting or exceeding the threshold, generating an updated version of the first data based at least on refining at least a portion of the first data that corresponds to the one or more first points [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26].
However, Kolych fails to explicitly disclose at least one of:
updating one or more parameters of one or more machine learning models using ground truth data corresponding to the updated version of the first data; or
validating the one or more machine learning models using the ground truth data corresponding to the updated version of the first data.
Paluri discloses at least one of:
updating one or more parameters of one or more machine learning models using ground truth data corresponding to the updated version of the first data [“obtain high resolution depth information to be used as the ground truth for training the machine learning model” ¶22]; or
validating the one or more machine learning models using the ground truth data corresponding to the updated version of the first data.
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych and Paluri before him before the effective filing date of the claimed invention, to modify the method of Kolych to incorporate the model training of Paluri.
Given the advantage of training a model for accurate predictions, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding claim 2, Kolych and Paluri disclose the method of claim 1. Kolych further discloses wherein the refining of the portion of the first data comprises updating one or more first values associated with the one or more first points based at least on one or more second values associated with the one or more second points [“the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26].
Regarding Claim 4, Kolych and Paluri disclose the method of claim 1. Kolych further discloses wherein:
a first resolution associated with the first data is less than a second resolution associated with the second data [“the image capture device 620 can be capturing imaging data, but at a lower quantity/quality (e.g., lower frames-per-second, a lower resolution, etc.).” ¶82], and
the generating of the updated version of the first data comprises increasing the first resolution to a third resolution based at least on updating the first data to include at least a portion of the one or more second points of the second data [“whether the sensor data satisfies a quality criterion. If the sensor data does not satisfy the quality criterion, the control logic can determine a new path for the RADAR unit using new imaging data captured by the image capture device and electronically steer the RADAR unit along the new path to capture another portion of the sensor data using the RADAR unit” ¶94; “other processing techniques can be used to increase sensor data resolution. For example, the object (e.g., face of a human) can be scanned from two or more locations.” ¶93].
Regarding Claim 5, Kolych and Paluri disclose the method of claim 1. Kolych further discloses wherein: one or more first values associated with the one or more first points of the first data are representative of one or more first distances between the one or more first sensors and one or more objects in an environment [“control logic can extract position information from the imaging data (e.g., x-, y-coordinate information)” ¶41; “the user face position satisfies a threshold criterion (e.g., a distance from the image capture device)” ¶41], and
one or more second values associated with the one or more second points of the second data are representative of one or more second distances between the one or more second sensors and the one or more objects [“sensor data can include a set of distances between the sensing element (such as the receiving element 123 of FIG. 1) and the user's face” ¶44].
Regarding Claim 6, Kolych and Paluri disclose the method of claim 1. Kolych further discloses wherein the first data comprises LiDAR data obtained using one or more LiDAR sensors [“using a Light Detection and Ranging (LIDAR) sensor or by using a time-of-flight (TOF) sensor” ¶14] and the second data comprises one or more outputs generated using a neural network and based at least on image data obtained using one or more image sensors [“The imaging data can be provided as input to a machine learning model trained to generate three-dimensional (3D) imaging data based on a two-dimensional (2D) input. In some embodiments, the machine learning model can be a neural network model.” ¶66].
Regarding Claim 7, Kolych and Paluri disclose the method of claim 1. Kolych further discloses wherein the evaluating of the first data with respect to the second data comprises evaluating one or more signals representative of one or more metrics associated with at least one of the first data or the second data [“The control logic can determine a correlation metric between the imaging data and the sensor data.” ¶54].
Regarding Claim 17, Kolych discloses one or more processors comprising:
one or more circuits to cause performance of one or more operations [“Processing device 802 represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like.” ¶103] associated with a machine based at least on one or more outputs of one or more machine learning models, wherein the one or more machine learning models are trained using ground truth data, the ground truth data generated using first data updated based at least on determining that a difference between one or more portions of first data and one or more corresponding portions of second data meets or exceeds a threshold [“the multiple modes of imaging data can be correlated if data from a first mode of imaging data exceeds a correlation threshold with respect to a second mode of imaging data (e.g., data of the first mode has a sufficient overlap or similarity with data of the second mode).” ¶17; “the imaging device 103 can combine the imaging data with the sensor data to obtain multimodal imaging data” ¶26], the first data determined based at least on first sensor data corresponding to a first sensor modality and the second data determined based at least on second sensor data corresponding to a second sensor modality [“receive imaging data from the image capture device 110 and sensor data from the sensor unit 120. The imaging controller 130 can compare the imaging data to the sensor data, to determine whether the imaging data correlates to the sensor data.” ¶26].
However, Kolych fails to explicitly disclose associated with a machine based at least on one or more outputs of one or more machine learning models, wherein the one or more machine learning models are trained using ground truth data,
Paluri discloses associated with a machine based at least on one or more outputs of one or more machine learning models, wherein the one or more machine learning models are trained using ground truth data [“obtain high resolution depth information to be used as the ground truth for training the machine learning model” ¶22],
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych and Paluri before him before the effective filing date of the claimed invention, to modify the method of Kolych to incorporate the model training of Paluri.
Given the advantage of training a model for accurate predictions, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 19, Kolych and Paluri disclose the one or more processors of claim 17. Kolych further discloses wherein the second data is determined based at least on applying the second sensor data to at least one of a machine learning model or a vision-based algorithm [“The imaging data can be provided as input to a machine learning model trained to generate three-dimensional (3D) imaging data based on a two-dimensional (2D) input. In some embodiments, the machine learning model can be a neural network model.” ¶66].
Regarding Claim 20, Kolych and Paluri disclose the one or more processors of claim 17. Kolych further discloses wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources [“The machine can operate in the capacity of a server or a client machine in client-server network environment, as a peer machine in a peer-to-peer ( or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.” ¶100].
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kolych and Paluri in view of Lin et al. (hereinafter Lin), U.S. Patent Application Publication 2022/0067381.
Regarding Claim 3, Kolych and Paluri disclose the method of claim 1.
However, Kolych fails to explicitly disclose further comprising:
determining, based at least on the evaluating, that one or more second differences corresponding to one or more first frames of the first data and one or more second frames of the second data meet or exceed a second threshold; and
based at least on the one or more second differences meeting or exceeding the second threshold, causing the one or more first frames to be excluded from the updated version of the first data.
Lin discloses further comprising:
determining, based at least on the evaluating, that one or more second differences corresponding to one or more first frames of the first data and one or more second frames of the second data meet or exceed a second threshold [“the frame comparison engine 304 determines that the calculated frame difference between the first frame 402(1) and the second frame 402(2) is less than the threshold difference value, the second frame 402(2) may be excluded from inclusion in a frame set” ¶51]; and
based at least on the one or more second differences meeting or exceeding the second threshold, causing the one or more first frames to be excluded from the updated version of the first data [“the frame comparison engine 304 determines that the calculated frame difference between the first frame 402(1) and the second frame 402(2) is less than the threshold difference value, the second frame 402(2) may be excluded from inclusion in a frame set” ¶51].
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych, Paluri, and Lin before him before the effective filing date of the claimed invention, to modify the combination to incorporate comparing frame differences and excluding failed frames to a threshold of Lin.
Given the advantage of ensuring that only correct data is included, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim(s) 11, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kolych and Lin.
Regarding Claim 11, Kolych discloses the system of claim 10.
However, Kolych fails to explicitly disclose wherein the updated version of the first data is generated further based at least on excluding one or more frames of the first data.
Lin discloses wherein the updated version of the first data is generated further based at least on excluding one or more frames of the first data [“the frame comparison engine 304 determines that the calculated frame difference between the first frame 402(1) and the second frame 402(2) is less than the threshold difference value, the second frame 402(2) may be excluded from inclusion in a frame set” ¶51].
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych and Lin before him before the effective filing date of the claimed invention, to modify the system of Kolych to incorporate comparing frame differences and excluding failed frames to a threshold of Lin.
Given the advantage of ensuring that only correct data is included, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 15, Kolych discloses the system of claim 8.
However, Kolych fails to explicitly disclose the one or more processors further to:
generate one or more signals representative of one or more metrics associated with one or more differences between one or more frames of the first data and one or more corresponding frames of the second data; and
cause to be excluded, from the ground truth data, a subset of the one or more frames based at least on an evaluation of the one or more signals.
Lin discloses the one or more processors further to:
generate one or more signals representative of one or more metrics associated with one or more differences between one or more frames of the first data and one or more corresponding frames of the second data [“the frame comparison engine 304 determines that the calculated frame difference between the first frame 402(1) and the second frame 402(2) is less than the threshold difference value, the second frame 402(2) may be excluded from inclusion in a frame set” ¶51]; and
cause to be excluded, from the ground truth data, a subset of the one or more frames based at least on an evaluation of the one or more signals [“the frame comparison engine 304 determines that the calculated frame difference between the first frame 402(1) and the second frame 402(2) is less than the threshold difference value, the second frame 402(2) may be excluded from inclusion in a frame set” ¶51].
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych and Lin before him before the effective filing date of the claimed invention, to modify the system of Kolych to incorporate comparing frame differences and excluding failed frames to a threshold of Lin.
Given the advantage of ensuring that only correct data is included, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kolych and Paluri, in view of Kostrzewa et al. (hereinafter Kostrzewa), U.S. Patent Application Publication 2017/0374305.
Regarding Claim 18, Kolych and Paluri disclose the one or more processors of claim 17.
However, Kolych fails to explicitly disclose wherein the ground truth data includes one or more points having one or more values indicating that the one or more points were missing from the first data.
Kostrzewa discloses wherein the ground truth data includes one or more points having one or more values indicating that the one or more points were missing from the first data [“image frame 300 may be evaluated for spatially anomalous pixels and the results of these determinations may be flagged and stored ( e.g., in frame buffer 640) for further pixel replacement operations or other processing.” ¶102].
It would have been obvious to one having ordinary skill in the art, having the teachings of Kolych, Paluri, and Kostrzewa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the anomalous pixel flagging of Kostrzewa.
Given the advantage of indicated anomalous points to ensure accurate data, one having ordinary skill in the art would have been motivated to make this obvious modification.
Examiner’s Note
The Examiner respectfully requests of the Applicant in preparing responses, to fully consider the entirety of the reference(s) as potentially teaching all or part of the claimed invention. It is noted, REFERENCES ARE RELEVANT AS PRIOR ART FOR ALL THEY CONTAIN. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). A reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art, including non-preferred embodiments (see MPEP 2123). The Examiner has cited particular locations in the reference(s) as applied to the claim(s) above for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim(s), typically other passages and figures will apply as well.
Additionally, any claim amendments for any reason should include remarks indicating clear support in the originally filed specification.
Conclusion
Any prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Applicant is reminded that in amending in response to a rejection of claims, the patentable novelty must be clearly shown in view of the state of the art disclosed by the references cited and the objections made. Applicant must also show how the amendments avoid such references and objections. See 37 CFR §1.111(c). Additionally when amending, in their remarks Applicant should particularly cite to the supporting paragraphs in the original disclosure for the amendments.
Wekel et al. (U.S. Patent Application Publication 2021/003578) discloses a DNN trained using camera-to LiDAR cross injection to generate reliable ground truth data for LiDAR range images.
The following references were found during the examination of this patent application and were found to be relevant to patentability. Applicant is advised to review these references prior to responding to this Office action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT H BEJCEK II whose telephone number is (571)270-3610. The examiner can normally be reached Monday - Friday: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T. Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.B./ Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/ Supervisory Patent Examiner, Art Unit 2148