Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement submitted on October 9, 2024 has been considered by the examiner and made of record in the application file.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially createddoctrine grounded in public policy (a policy reflected in the statute) so as to prevent theunjustified or improper timewise extension of the "right to exclude" granted by a patentand to prevent possible harassment by multiple assignees. A nonstatutoryobviousness-type double patenting rejection is appropriate where the conflicting claimsare not identical, but at least one examined application claim is not patentably distinctfrom the reference claim(s) because the examined application claim is either anticipatedby, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir.1985); In re Van Omum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d)may be used to overcome an actual or provisional rejection based on a nonstatutorydouble patenting ground provided the conflicting application or patent either is shown tobe commonly owned with this application, or claims an invention made as a result ofactivities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign aterminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with37 CFR 3.73(b).
Claims 1-3, 7-13 and 15-20 are rejected on the ground of nonstatutory obvious-type double patenting as being unpatentable over claims 1, 2, 4, 6-11, 13, 14 and 17-20 of U.S. Patent 12,136,269 in view of Reiff et al. (US PGPUB 2017/0139417 A1, hereinafter Reiff). Although the conflicting claims are not identical, they are not patentably distinct from each other as 12,136,269 in view of Reiff reads on the present application.
Please see table below for independent claim 1:
18/910,766
12,136,269
Claim Interpretation
1.A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
storing, in a database, first
metric data associated with a
first region of interest (ROI), the first ROI determined
by a first machine-learned model based at least in part on input sensor data representing an environment in which a vehicle
is operating;
storing, in the database, second metric data associated with a second region of interest (ROI), the second ROI
determined by a second
machine-learned model based at least in part on the input sensor data;
determining, as associated metrics and based at least in part on determining that the first ROI and the second ROI represent associated portions of the input sensor data, metrics that are included in the first metric data with the second metric data;
calculating, as a calculation,
at least one difference between the associated metrics; and
causing presentation, on a graphical user interface, of a first indication of the at least one difference, the first indication associated with a first portion of the calculation.
1.A system comprising:
one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
storing, in a database, first metric data associated with a first bounding box, the
first bounding box determined by a first machine-learned model based at least in part on input sensor data representing an environment in which a vehicle is operating;
storing, in the database, second metric data associated with a second bounding box,
the second bounding box determined by a second machine-learned model based at least in part on the input sensor data, wherein the second machine-learned model is an updated version of the first machine-learned model; determining that the first bounding box and the second bounding box are associated with a same object that is represented in the input sensor data; based at least in part on the first bounding box and the second bounding box being associated with the same object, calculating
multiple differences between associated metrics that are included in the first metric data and the second metric data; causing presentation, on a graphical user interface, of a first visualization associated with a first portion of the multiple differences;
receiving a selection of one or more of the associated metrics; and causing presentation, on the graphical user interface, of a second visualization associated with a second portion of the multiple differences, the second portion of the multiple differences corresponding with the selection.
As can be seen with the side-by-side comparison, the only difference is slight word variations and the present application recites regions of interest (ROIs) vs. 12,136,269 recites bounding boxes (please see Reiff below).
12,136,269 substantially discloses the claimed invention but fails to teach regions of interest (ROIs) (typically bounding boxes are put around ROIs).
However, Reiff teaches regions of interest (ROIs) (paragraph 34, read as determining bounding boxes for identified regions of interest).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have incorporated the teachings of Reiff into the invention of 12,136,260 in order to quickly and efficiently distinguish between specific objects in an environment.
Please see table below for independent claim 8 (and similarly applied to claim 17):
18/910,766
12,136,269
Claim Interpretation
8.A method comprising:
storing, in a database, first
metric data associated with a
first region of interest (ROI) determined by a machine-
learned model,
storing, in the database, second metric data associated with a second ROI determined by an updated version of the machine-learned model;
determining an association between the first ROI and the second ROI;
determining a difference
between the first metric data and the second metric data; and
based at least in part on at least one of the difference or the association, at least one of:
causing presentation of an indication of at least one of the difference or the association on a graphical user interface; or sending the updated version of the machine-learned model to a first vehicle.
6.A method comprising:
storing, in a database, first metric data associated with a first region of interest (ROI) determined by a machine-learned model that is configured for use in a vehicle;
storing, in the database, second metric data associated with a second ROI determined by an updated version of the machine-learned model;
determining an association between the first ROI and the second ROI;
determining a difference between the first metric data and the second metric data
based at least in part on the association; and based at least in part on the difference, at least one of:
causing presentation of a visualization of the difference on a graphical user interface; or sending the updated version of the machine-learned model to a first vehicle.
As can be seen with the side-by-side comparison, the only difference is slight word variations and the present application recites wherein the ROI is based at least in part on sensor data vs. 12,136,269 recites that is configured for use in a vehicle (please see Reiff below).
12,136,269 substantially discloses the claimed invention but fails to teach wherein the ROI is based at least in part on sensor data.
However, Reiff teaches wherein the ROI is based at least in part on sensor data (paragraph 34, read as using neural networks to identify surface markings on the road and other vehicles within sensor viewing ranges and determining bounding boxes for identified regions of interest).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have incorporated the teachings of Reiff into the invention of 12,136,260 in order to quickly and efficiently distinguish between specific objects in an environment.
Please see the following table for the dependent claims:
18/910,766
12,136,269
Claim Interpretation
2.The system of claim 1, the operations further comprising: receiving a selection of one or more of the associated metrics; and
causing presentation, on the graphical user interface, of a second visualization associated with a second portion of the calculation, the second portion of the calculation corresponding with the selection.
From claim 1:
receiving a selection of one or more of the associated metrics; and
causing presentation, on the graphical user interface, of a second visualization associated with a second portion of the multiple differences, the second portion of the multiple differences corresponding with the selection.
No major difference. The only difference is 12,136,269 is concerned with multiple differences where the present application is only concerned with one difference.
3.The system of claim 1, the operations further
comprising:
determining, based at least in part on the calculation,
that the second machine-learned model is an improvement over the first machine-learned model; and sending the second machine-learned model to a first vehicle based at least in part on the improvement.
2.The system of claim 1, the operations further comprising:
determining, based at least in part on the multiple differences,
that the second machine-learned model is an improvement over the first machine-learned model; and sending the second machine-learned model to a first vehicle of a fleet of vehicles based at least in part on the improvement.
Similar to claim 2 of the present application above regarding the multiples differences and also 12,136,269 has the additional feature of the fleet of vehicles.
7.The system of claim 1, wherein the associated
metrics comprise at least one of: yaw metrics; point segmentation metrics;
volume metrics; precision metrics; or recall metrics.
4.The system of claim 2, wherein the associated metrics comprise at least one of: yaw metrics; point segmentation metrics; volume metrics; precision metrics; or recall metrics.
No difference.
9.The method of claim 8, wherein the difference is indicative of an improvement associated with the updated version of the machine-learned model and the
method comprises sending
the updated version of the machine-learned model to the first vehicle based at least in part on the improvement.
7.The method of claim 6, wherein the difference is indicative of an improvement associated with the updated version of the machine-learned model and the method comprises sending the updated version of the machine-learned model to the first vehicle based at least in part on the improvement.
No difference.
Similar analysis for claim 18.
10.The method of claim 8, wherein determining the association between the first ROI and the second ROI is based at least in part on determining that the first ROI and the second ROI are associated with a same object.
8.The method of claim 6, wherein the determining the association is further based at least in part on determining that the first ROI and the second ROI are associated with a same object.
Slight word variation.
11.The method of claim 8, wherein the first ROI is a first bounding box and the second ROI is a second bounding box, the first bounding box and the second bounding box associated with an object in an environment of a
vehicle associated with the sensor data,
the object comprising at least one of another vehicle, a pedestrian, a cyclist, an animal, or a distractor.
9.The method of claim 6, wherein the first ROI is a first bounding box and the second ROI is a second bounding box, the first bounding box and the second bounding box associated with an object in an environment of the vehicle,
the object comprising at least one of another vehicle, a pedestrian, a cyclist, an animal, or a distractor.
The only difference is the sensor data that Reiff teaches above.
12.The method of claim 8, wherein: the first metric data is indicative of a first
difference between the first ROI and a ground truth associated with a same
object; and the second metric data is indicative of a second difference between the
second ROI and the ground truth.
10.The method of claim 6, wherein: the first metric is indicative of a first
difference between the first ROI and a ground truth associated with a same object; and the second metric is indicative of a second difference between the second ROI and the ground truth.
No difference.
Similar analysis for claim 19.
13.The method of claim 8, wherein the first metric data and the second metric data include one or more of: a yaw metric; a point segmentation metric; a recall metric; a noise metric; a classification metric; a precision metric; a volume metric; or a distance metric.
11. The method of claim 6, wherein the first metric data and the second metric data include one or more of: a yaw metric; a point segmentation metric; a volume metric; or a distance metric.
Essentially the same since the claim only requires the metrics to be one of the different metrics in the list.
15.The method of claim 8, wherein the first ROI and the second ROI are determined
by the machine-learned
model and the updated version of the machine-learned model, respectively, based at least in part on input sensor data representing an environment,
the input sensor data being part of a dataset that is associated with evaluating performance of machine-learned models.
13. The method of claim 6, wherein the first ROI and the second ROI are determined by the machine-learned
model and the updated version of the machine-learned model, respectively, based at least in part on input sensor data representing an environment in which the vehicle is operating,
the input sensor data being part of a dataset that is associated with evaluating performance of machine-learned models.
Claim 15 of the present application is broader than claim 13 of 12,136,269.
16.The method of claim 8, wherein: determining the difference comprises determining multiple differences between metrics of the first metric data and
the second metric data, the multiple differences
including at least a first difference between a first metric of the first metric data and a second metric of the second metric data; and the indication of the difference comprises a representation of a portion of the multiple differences, the portion of the multiple differences including the first difference.
14. The method of claim 6, wherein: determining the difference comprises determining multiple differences between metrics of the first metric data and the second metric data, the multiple differences including at least a first difference between a first metric of the first metric data and a second metric of the second metric data; and the visualization of the difference comprises a visualization of a portion of the multiple differences, the portion of the multiple differences including the first difference.
Claim 16 of the present application is broader than claim 13 of 12,136,269.
20.The one or more non-transitory computer-readable media of claim 17, wherein the indication of the difference is indicative of whether the updated version of the machine-learned model improved or retrogressed relative to the machine-learned model.
20.The one or more non-transitory computer-readable media of claim 17, wherein the visualization of the difference is indicative of whether the updated version of the machine-learned model improved or retrogressed relative to the machine-learned model.
No difference.
Allowable Subject Matter
Claims 1-20 are allowed over the prior art.
Applicant’s independent claims 1, 8, and 17 each recites a particular combination of elements, which is neither taught nor suggested by the prior art. Oosake, Reiff and a thorough search in the art disclose various aspects and features of applicant's claimed invention. However, Oosake, Reiff and a thorough search in the art do not disclose or suggest the specific combination of storing, in a database, first metric data associated with a first region of interest (ROI), the first ROI determined by a first machine-learned model based at least in part on input sensor data representing an environment in which a vehicle is operating; storing, in the database, second metric data associated with a second region of interest (ROI), the second ROI determined by a second machine-learned model based at least in part on the input sensor data; determining, as associated metrics and based at least in part on determining that the first ROI and the second ROI represent associated portions of the input sensor data, metrics that are included in the first metric data with the second metric data; calculating, as a calculation, at least one difference between the associated metrics; and causing presentation, on a graphical user interface, of a first indication of the at least one difference, the first indication associated with a first portion of the calculation. In addition, Oosake, Reiff and a thorough search in the art do not disclose or suggest the specific combination of storing, in a database, first metric data associated with a first region of interest (ROI) determined by a machine-learned model, wherein the ROI is based at least in part on sensor data; storing, in the database, second metric data associated with a second ROI determined by an updated version of the machine-learned model; determining an association between the first ROI and the second ROI; determining a difference between the first metric data and the second metric data; and based at least in part on at least one of the difference or the association, at least one of: causing presentation of an indication of at least one of the difference or the association on a graphical user interface; or sending the updated version of the machine-learned model to a first vehicle. Moreover, one of ordinary skill in the art would not have been motivated to arrive at applicant's claimed invention unless one was using applicant's claims and specification as a roadmap, thus using impermissible hindsight.
As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER M BRANDT whose telephone number is (571)270-1098. The examiner can normally be reached Mon - Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anthony Addy can be reached at 571-272-7795. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER M BRANDT/Primary Examiner, Art Unit 2645 July 24, 2026
Prosecution Insights