DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/04/2023; 12/13/2024; 04/03/2025; 05/05/2025; 06/25/2025; 08/11/2025; 10/08/2025; 12/04/2025; 04/03/2026; 06/10/2026; 08/07/2026 have been considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 11, and 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 12/072,443.
Regarding claims 1 and 11, Claim 1 of the U.S. Patent No. 12/072,443 discloses
generate a representation of one or more bounding shapes of one or more elements in an environment based at least on a neural network processing a projected representation of one or more classified detections (Claim 1: generating a representation of a range image; classification data representing one or more extracted classifications of elements in the environment; generating one or more bounding shapes of the elements based at least on the labeled geometry data); and
cause performance of one or more operations corresponding to a machine based at least on the representation of the one or more bounding shapes (Claim 1: providing data representing the one or more bounding shapes to a control component of an autonomous vehicle).
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12/164,059.
Regarding claims 1 and 11, Claim 1 of the U.S. Patent No. 12/164,059 discloses
generate a representation of one or more bounding shapes of one or more elements in an environment based at least on a neural network processing a projected representation of one or more classified detections (Claim 1: generating one or more bounding shapes of the one or more elements based at least on the class confidence data wherein classifications corresponding to one or more pixels to one or more Neural Networks (NNs), class confidence data representing one or more classifications of one or more elements in the environment); and
cause performance of one or more operations corresponding to a machine based at least on the representation of the one or more bounding shapes (Claim 1: providing data representing the one or more bounding shapes to a control component of an autonomous vehicle).
Regarding claims 2 and 12, Claim 2 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on transforming a segmented range image into a top-down representation (Claim 2: generating one or more height maps that represent the LiDAR point cloud from a top-down view).
Regarding claims 3 and 13, Claim 3 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on projecting one or more classified points in a point cloud onto a plane (Claim 3: the one or more height maps having pixels storing minimum or maximum height values using one or more columns of the one or more points in the LiDAR point cloud).
Regarding claims 4 and 14, Claim 4 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on associating one or more sensor detections with one or more corresponding classification values from one or more predicted confidence maps (Claim 4: generating first class confidence data representing one or more initial classifications of an image of a first two-dimensional (2D) view of the environment).
Regarding claims 5 and 15, Claim 5 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on filtering out at least one of the one or more classified detections that is above a threshold height (Claim 5: extracting of the class confidence data using the one or more NNs comprises applying the representation of the height data and the one or more view transformed per-pixel classifications to separate input channels of the one or more NNs).
Regarding claims 6 and 16, Claim 5 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on storing, in at least one individual pixel of one or more pixels of the projected representation, extracted classification data associated with at least one of the one or more classified detections that project onto the at least one individual pixel (Claim 5: extracting of the class confidence data using the one or more NNs comprises applying the representation of the height data and the one or more view transformed per-pixel classifications to separate input channels of the one or more NNs).
Regarding claims 7 and 17, Claim 7 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on transforming at least one of one or more confidence maps or one or more segmentation masks into a different two-dimensional view (Claim 7: the representation of the height data and the one or more view transformed per-pixel classifications representing one or more initial classifications transformed into a top-down view of the environment).
Regarding claims 8 and 18, Claim 5 of the U.S. Patent No. 12/164,059 discloses generate the one or more classified detections based at least on un-projecting classified image data into a three-dimensional space (Claim 5).
Regarding claim 9, Claim 5 of the U.S. Patent No. 12/164,059 discloses generate the projected representation of the one or more classified detections based at least on projecting one or more detected three-dimensional locations labeled with one or more classifications (Claim 5).
Regarding claims 10, 19, and 21, Claim 20 of the U.S. Patent No. 12/164,059 discloses the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (Claim 20).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ziyaee et al. (US Patent No. 10,860,034) in view of Banerjee et al. (US 2020/0301013; Cited in IDS).
Regarding claims 1 and 11, Ziyaee discloses one or more processors and a system, the processor comprising processing circuitry to:
generate a representation of one or more bounding shapes of one or more elements in an environment based at least on a neural network (Col. 5 lines 4-10; Col. 6 lines 14-20; Col. 6 lines 42-49; e.g., one or more objects that have classified as static may correspond to barriers (e.g., walls, fences, medians, guard rails, or curbs) and these barriers may be correlated with boundaries in or other features in navigation map) and (Col. 3 lines 5-10; Col. 3 lines 15-20; Col. 4 lines 35-45; e.g., a birds-eye-view map may be input to a machine learning system (e.g., a convolutional neural network) to obtain a classified map that labels objects detected in the space with classifications (e.g., static or dynamic)); and
cause performance of one or more operations corresponding to a machine based at least on the representation of the one or more bounding shapes (Col. 3 lines 24-30; e.g., outputting commands to the actuators ot manuever o a vehicle in operation 180); It is noted that commands are based on the classified map and object moving from object tracker.
Ziyaee fails to specifically disclose a neural network processing a projected representation of one or more classified detections.
However, Banerjee discloses a neural network processing a projected representation of one or more classified detections (paragraphs [0006], [0019]; e.g., each data type of fused hybrid data is fed into an independent CNN network to learn features and then the learned features are concatenated and fed through fully connected layers via a CNN to detect and classify object (class score) and predict the bounding box for the object ).
Therefore, taking the teachings of Ziyaee in combination of Banerjee as a whole, it would have been obvious to one having ordinary skill in the art at the time of the invention by applicant to processing a projected representation of one or more classified detections in order to detect or classify objects in the scene based on the image data and the encoded projected depth data (Banerjee: paragraph [0004]).
Regarding claims 2 and 12, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on transforming a segmented range image into a top-down representation (Ziyaee: Col. 1 lines 54-55; see birds-eye-view maps).
Regarding claims 3 and 13, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on projecting one or more classified points in a point cloud onto a plane (Ziyaee: Col. 2 lines 50-57; Col. 4 lines 10-15).
Regarding claims 4 and 14, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on associating one or more sensor detections with one or more corresponding classification values from one or more predicted confidence maps (Ziyaee: Col. 6 lines 5-11; lines 26-40).
Regarding claims 5 and 15, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on filtering out at least one of the one or more classified detections that is above a threshold height (Ziyaee: Col. 6 lines 5-11; lines 26-40) and (Banerjee: paragraphs [0024], [0119]).
Regarding claims 6 and 16, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on storing, in at least one individual pixel of one or more pixels of the projected representation, extracted classification data associated with at least one of the one or more classified detections that project onto the at least one individual pixel (Ziyaee: Col. 2 lines 50-57; Col. 4 lines 10-16; Col. 6 lines 5-11; Col. 6 lines 26-40).
Regarding claims 7 and 17, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on transforming at least one of one or more confidence maps or one or more segmentation masks into a different two-dimensional view (Ziyaee: Col. 6 lines 5-11; lines 26-40) and (Banerjee: paragraph [0144], [0146]).
Regarding claims 8 and 18, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the processing circuitry is further to generate the one or more classified detections based at least on un-projecting classified image data into a three-dimensional space (Ziyaee: Col. 3 lines 5-10; Col. 3 lines 60-65).
Regarding claim 9, Ziyaee in combination with Banerjee discloses the one or more processors of claim 1, wherein the processing circuitry is further to generate the projected representation of the one or more classified detections based at least on projecting one or more detected three-dimensional locations labeled with one or more classifications (Ziyaee: Col. 3 lines 5-10; Col. 3 lines 60-65) and (Banerjee: paragraph [0119]).
Regarding claims 10 and 19, Ziyaee in combination with Banerjee discloses the one or more processors and the system of claims 1 and 11, wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (Ziyaee: Col. 2 lines 60-65; see a machine learning system classifies objects detected) and (Banerjee: paragraphs [0074]; [0131]; e.g., see Lidar sensor in the autonomous car).
Claims 20 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Ziyaee in combination with Banerjee, in view of Douillard et al. (US Patent No. 10,593,042; Cited in IDS)
Regarding claim 20, Ziyaee discloses a method comprising:
generating a representation of one or more bounding shapes of one or more elements in the environment based at least on a neural network in the environment (Col. 5 lines 4-10; Col. 6 lines 14-20; Col. 6 lines 42-49; e.g., one or more objects that have classified as static may correspond to barriers (e.g., walls, fences, medians, guard rails, or curbs) and these barriers may be correlated with boundaries in or other features in navigation map) and (Col. 3 lines 5-10; Col. 3 lines 15-20; Col. 4 lines 35-45; e.g., a birds-eye-view map may be input to a machine learning system (e.g., a convolutional neural network) to obtain a classified map that labels objects detected in the space with classifications (e.g., static or dynamic)); and
causing performance of one or more operations associated with the machine based at least on the one or more bounding shapes (Col. 3 lines 24-30; e.g., outputting commands to the actuators ot manuever o a vehicle in operation 180); It is noted that commands are based on the classified map and object moving from object tracker.
Ziyaee fails to specifically disclose a neural network processing a projected representation of one or more classified detections and generating, within a simulation that is rendered using one or more ray-tracing operations, a visualization of at least a portion of a simulated environment around a simulated ego-machine.
However, Banerjee discloses a neural network processing a projected representation of one or more classified detections (paragraphs [0006], [0019]; e.g., each data type of fused hybrid data is fed into an independent CNN network to learn features and then the learned features are concatenated and fed through fully connected layers via a CNN to detect and classify object (class score) and predict the bounding box for the object ).
Therefore, taking the teachings of Ziyaee in combination of Banerjee as a whole, it would have been obvious to one having ordinary skill in the art at the time of the invention by applicant to processing a projected representation of one or more classified detections in order to detect or classify objects in the scene based on the image data and the encoded projected depth data (Banerjee: paragraph [0004]).
Ziyaee in combination of Banerjee fails to specifically disclose generating, within a simulation that is rendered using one or more ray-tracing operations, a visualization of at least a portion of a simulated environment around a simulated ego-machine.
However, Douillard discloses generating, within a simulation that is rendered using one or more ray-tracing operations, a visualization of at least a portion of a simulated environment around a simulated ego-machine (Col. 5 lines 5-8; Col. 7 lines 11-15; Col. 10 lines 5-7).
Therefore, taking the teachings of Ziyaee in combination of Banerjee and Douillard as a whole, it would have been obvious to one having ordinary skill in the art at the time of the invention by applicant to generating, within a simulation that is rendered using one or more ray-tracing operations, a visualization of at least a portion of a simulated environment around a simulated ego-machine in order to performing object classification (Douillard: Col. 1 lines 35-36).
Regarding claim 21, Ziyaee in combination with Banerjee and Douillard discloses the method of claim 20, wherein the one or more operations include at least one of: performing one or more control operations associated with the simulated ego-machine within the simulated environment; or testing or validating the neural network (Ziyaee: Col. 6 lines 42-50; Col. 7 lines 20-25).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY X PHAM whose telephone number is (571)270-7115. The examiner can normally be reached Mon-Fri: 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Resha Desai can be reached at 571-270-7792. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIMOTHY X PHAM/Primary Examiner, Art Unit 3648
