DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 15-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Popov et al. (US 20210156960).
Regarding claim 15, Popov et al. discloses one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors (“Now referring to FIGS. 10-12, each block of methods 1000, 1100, and 1200, described herein, comprises a computing process that may be performed using any combination of hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory” at paragraph 0100, line 1), perform operations comprising:
determining, based at least in part on discretized values associated with an object in an environment, a plurality of proposed candidate detection center values (“Generally, the instance decoder 140 may identify candidate bounding boxes (or other bounding shapes) (e.g., for each object class) based on object instance data (e.g., location, size, and/or orientation data) from the corresponding channels of an instance regression tensor 112 and/or the confidence map from a corresponding channel of a class confidence tensor 110 for that class. More specifically, a predicted confidence map and predicted object instance data may specify information about detected object instances, such as where the object is located, object height, object width, object orientation, and/or the like. This information may be used to identify candidate object detections (e.g., candidates having a unique center point, object height, object width, object orientation, and/or the like). he result may be a set of candidate bounding boxes (or other bounding shapes) for each object class.” at paragraph 0054, line 7);
determining, based at least in part on the plurality of proposed candidate detection center values, a candidate detection representing the object (“Various types of filtering and/or clustering 420 may be applied to remove duplication and/or noise from the candidate bounding boxes (or other bounding shapes) for each object class” at paragraph 0055, line 1; “For example, each candidate bounding box (or other bounding shape) may be associated with a corresponding confidence/probability value associated with one or more corresponding pixels from a corresponding channel of the class confidence tensor 110 for the class being evaluated (e.g., using the confidence/probability value of a representative pixel such as a center pixel, using an averaged or some other composite value computed over the candidate region, etc.). Thus, candidate bounding shapes that have a confidence/probability of being a member of the object class less than some threshold (e.g., 50%) may be filtered out” at paragraph 0056, line 1); and
determining, based at least in part on the candidate detection, an output detection (“In such examples, the confidence value that is the highest for the object instance may be used to determine which candidate bounding box to use for that object instance, and non-maximum suppression may be used to remove, or suppress, the other candidates’ at paragraph 0055, last sentence) for use in controlling a vehicle (“Once the locations, size, and/or orientations of the object instances have been determined, 2D pixel coordinates defining the object instances may be converted to 3D world coordinates for use by the autonomous vehicle in performing one or more operations (e.g., obstacle avoidance, lane keeping, lane changing, path planning, mapping, etc.)” at paragraph 0059, line 1).
Regarding claim 16, Popov et al. discloses a medium wherein the operations further generating, based at least in part on the output detection, multichannel output data representing the object (“As such, the DNN may predict a multi-channel class confidence tensor and/or a multi-channel instance regression tensor from a given RADAR data tensor” at paragraph 0029, last sentence).
Regarding claim 17, Popov et al. discloses a medium wherein a channel of the multichannel output data comprises one or more confidence values for the discretized values (“As such, the DNN may predict a multi-channel class confidence tensor and/or a multi-channel instance regression tensor from a given RADAR data tensor” at paragraph 0029, last sentence).
Regarding claim 18, Popov et al. discloses a medium wherein the plurality of proposed candidate detection center values are determined further based at least in part on machine-learned model output (“Generally, the instance decoder 140 may identify candidate bounding boxes (or other bounding shapes) (e.g., for each object class) based on object instance data (e.g., location, size, and/or orientation data) from the corresponding channels of an instance regression tensor 112 and/or the confidence map from a corresponding channel of a class confidence tensor 110 for that class” at paragraph 0054, line 7).
Regarding claim 19, Popov et al. discloses a medium wherein the discretized values are associated with pixel data associated with the object (“For example, class confidence head 320 may include a channel (e.g., a stream of layers plus a classifier) for each class of object to be detected (e.g., vehicles, cars, trucks, vulnerable road users, pedestrians, cyclists, motorbikes, etc.), such that class confidence head 320 serves to predict classification data—such as a confidence map—in the form of a multi-channel tensor (e.g., class confidence tensor 110). Each channel may be thought of as a heat map with confidence/probability values that each pixel belongs to the class corresponding to the channel.” at paragraph 0050, line 5).
Regarding claim 20, Popov et al. discloses a medium wherein the operations further comprise receiving the pixel data from a sensor configured at the vehicle (“At a high level, the process 100 may include one or more machine learning models 108 configured to detect objects such as instances of obstacles from sensor data 102 such as RADAR detections generated from RADAR sensors 101” at paragraph 0038, line 1).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-3, 7, 8, 10, 14-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 7-20 of U.S. Patent No. 12,080,074. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are anticipated or rendered obvious by the claims of ‘074.
Regarding claim 1, ‘074 discloses a system comprising:
one or more processors (col. 20, lines 31-32); and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations (col. 20, lines 30-32) comprising:
determining unimodal confidence values for discretized values associated with an object in an environment (col. 20, lines 38-40);
determining, based at least in part on the unimodal confidence values, a plurality of proposed candidate detection box center values from the discretized values (col. 20, lines 41-44);
determining, based at least in part on the plurality of proposed candidate detection box center values, a candidate detection box representing the object (col. 20, lines 45-48); and
determining, based at least in part on the candidate detection box, an output detection box for use in controlling a vehicle (col. 20, lines 49-52).
Regarding claim 7, ‘074 discloses a method comprising:
determining unimodal confidence values for discretized values associated with an object in an environment (col. 19, lines 50-52);
determining, based at least in part on the unimodal confidence values, a plurality of proposed candidate detection box center values from the discretized values (col. 19, lines 53-56);
determining, based at least in part on the plurality of proposed candidate detection box center values, a candidate detection box representing the object (col. 19, lines 57-60); and
determining, based at least in part on the candidate detection box, an output detection box for use in controlling a vehicle (col. 19, lines 61-64).
Regarding claim 15, ‘074 discloses one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors (col. 20, lines 30-32), perform operations comprising:
determining, based at least in part on discretized values associated with an object in an environment, a plurality of proposed candidate detection center values (col. 20, lines 41-44);
determining, based at least in part on the plurality of proposed candidate detection center values, a candidate detection representing the object (col. 20, lines 45-48); and
determining, based at least in part on the candidate detection, an output detection for use in controlling a vehicle (col. 20, lines 49-52).
Claims 4-6, 9 and 11-13 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 7-20 of U.S. Patent No. 12,080,074 in view of Popov et al.
Regarding claim 4, ‘074 discloses a system wherein the operations further comprise executing a machine-learned model to generate the machine-learned model output (col. 20, lines 35-37).
‘074 does not explicitly disclose that the machine-learned model is trained based at least in part on a parameter associated with a center pixel of an object detection box.
Popov et al. teaches a system in the same field of endeavor of autonomous vehicle navigation with object detection, wherein the machine-learned model is trained based at least in part on a parameter associated with a center pixel of an object detection box (“Additionally or alternatively, the location, size, orientation, and/or class of each of the (remaining) LIDAR labels may be used to generate object instance data matching the size and dimensionality of instance regression tensor 112. For example, for each pixel contained with the LIDAR label, the LIDAR label may be used to compute corresponding location, size, and/or orientation information (e.g., where the object is located—such as the object center—relative to each pixel, object height, object width, object orientation (e.g., rotation angles relative to the orientation of the projection image), and/or the like). The computed object instance data may be stored in a corresponding channel of a ground truth instance regression tensor” at paragraph 0098, line 12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the training as taught by Popov et al. to train the machine learning model of ‘074 to provide a supervised learning to ensure proper training of the object detection system.
Regarding claim 5, Popov et al. discloses a system wherein the operations further comprise training the machine-learned model by: determining a loss based at least in part on the center pixel; and backpropagating the loss at the machine-learned model (see paragraph 0099).
Regarding claim 6, ‘074 discloses a system wherein the loss comprises one or more of a focal loss (col. 20, lines 53-56), a propagation loss, or a classification loss.
Regarding claim 11, ‘074 discloses the elements of claim 10.
‘074 does not explicitly disclose training the machine-learned model using a parameter associated with a center pixel of a detection box representing a second object and ground truth data associated with the second object.
Popov et al. teaches a method in the same field of endeavor of autonomous vehicle navigation with object detection comprising training the machine-learned model using a parameter associated with a center pixel of a detection box representing a second object and ground truth data associated with the second object (“Additionally or alternatively, the location, size, orientation, and/or class of each of the (remaining) LIDAR labels may be used to generate object instance data matching the size and dimensionality of instance regression tensor 112. For example, for each pixel contained with the LIDAR label, the LIDAR label may be used to compute corresponding location, size, and/or orientation information (e.g., where the object is located—such as the object center—relative to each pixel, object height, object width, object orientation (e.g., rotation angles relative to the orientation of the projection image), and/or the like). The computed object instance data may be stored in a corresponding channel of a ground truth instance regression tensor” at paragraph 0098, line 12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the training as taught by Popov et al. to train the machine learning model of ‘074 to provide a supervised learning to ensure proper training of the object detection system.
Regarding claim 12, Popov et al. discloses a method wherein training the machine-learned model comprises: determining a loss based at least in part on the center pixel; and backpropagating the loss at the machine-learned model (see paragraph 0099).
The following is a mapping of the claims of the instant application to the claims of ‘074:
Claims of Instant Application
Claims of ‘074
1
15
2
19
3
15
4
15 + Popov et al.
5
15 + Popov et al.
6
16
7
7
8
12
9
12 (threshold implied by NMS)
10
7
11
7 + Popov et al.
12
7 + Popov et al.
13
7 + Popov et al. + obviousness for mask
14
8
15
7 or 15
16
9
17
10
18
7
19
7 (implied for pixels)
20
7
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672
Prosecution Insights