VIEWPOINT-ADAPTIVE PERCEPTION FOR AUTONOMOUS MACHINES AND APPLICATIONS USING SIMULATED SENSOR DATA

§102 §103 §112 §DOUBLEPATENT

DETAILED ACTION 1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . 2. This communication is in response to the Applicant’s submission filed 31 May 2024, where: Claims 1-20 are pending. Claims 1-20 are rejected. Information Disclosure Statement 3. Information disclosure statements were submitted on 05 June 2024 (qty 2). The submissions comply with the provisions of 37 CFR 1.97. Accordingly, the Examiner considered the information disclosure statements. Drawings 4. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: Figure 2, reference “245” Figure 3, reference “360” Figure 3, reference “365” Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance. Specification 5. The use of the terms TensorFlow, PyTorch, Keras, MXNet, Caffe, Microsoft Cognitive Toolkit (CNTK), Fast.ai, or Theano., (see, e.g., Specification 0030), which are a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term. Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks. Double Patenting 6. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer. 7. Claims 1, 2, and 6-10 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 5-10 of copending Application No. 18680378 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other as described below in detail. Appl’n S/N 18680454 [Instant Appl’n] Co-pending Appl’n S/N 18680378 [‘378 Appl’n] Claim 1 of Instant Appl’n Claim 1 of ‘378 Appl’n 1. One or more processors comprising processing circuitry to perform one or more operations with an ego-machine using a three-dimensional (3D) perception network that is updated based at least on applying simulated two-dimensional (2D) sensor data to at least a portion of the 3D perception network, the simulated 2D sensor data being generated based at least on simulating a source sensor rig of an ego-machine and a target sensor rig. 1. One or more processors comprising processing circuitry to: apply two-dimensional (2D) sensor data generated using a source sensor rig of an ego-machine to at least a portion of a 3D perception network; apply simulated 2D sensor data to at least the portion of the 3D perception network, the simulated 2D sensor data being simulated based at least on a simulated source sensor rig and a simulated target sensor rig; update a three-dimensional (3D) perception network based at least on the 2D sensor data and the simulated 2D sensor data; and perform one or more operations with the ego-machine using the updated 3D perception network. Claim 1 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 1, as set out above. Claim 2 of Instant Appl’n Claim 5 of ‘378 Appl’n 2. The one or processors of claim 1, wherein the processing circuitry is further to update the 3D perception network based at least on updating a viewpoint adjustment network comprising one or more layers of the 3D perception network replicated as at least a portion of multiple channels of the viewpoint adjustment network. 5. The one or processors of claim 1, wherein the processing circuitry is further to update the 3D perception network based at least on updating a viewpoint adjustment network comprising one or more layers of the 3D perception network shared across one or more first paths of the viewpoint adjustment network that process the real 2D sensor data and one or more second paths of the viewpoint adjustment network that process the simulated 2D sensor data. Claim 5 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 2, as set out above. Claim 6 of Instant Appl’n Claim 6 of ‘378 Appl’n 6. The one or processors of claim 1, wherein the source sensor rig represents a first sensor configuration that differs from a second sensor configuration of the target sensor rig. 6. The one or processors of claim 1, wherein the source sensor rig and the simulated source sensor rig represent a first sensor configuration of the ego-machine that differs from a second sensor configuration of the simulated target sensor rig. Claim 6 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 6, as set out above. Claim 7 of Instant Appl’n Claim 7 of ‘378 Appl’n 7. The one or processors of claim 1, wherein the processing circuitry is further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more simulated sensors of the source sensor rig and at least one corresponding sensor of the target sensor rig, and simulated ground truth data. 7. The one or processors of claim 1, wherein the processing circuitry is further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more sensors of the simulated source sensor rig and at least one corresponding sensor of the simulated target sensor rig and simulated ground truth data. Claim 7 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 7, as set out above. Claim 8 of Instant Appl’n Claim 8 of ‘378 Appl’n 8. The one or processors of claim 1, wherein the processing circuitry is further to update a first viewpoint adjustment network comprising one or more layers of the 3D perception network based at least on applying real 2D sensor data and the simulated 2D sensor data to the first viewpoint adjustment network, prior to updating a second viewpoint adjustment network comprising the one or more layers of the 3D perception network based at least on applying the simulated 2D sensor data to the second viewpoint adjustment network. 8. The one or processors of claim 1, wherein the processing circuitry is further to update a first viewpoint adjustment network comprising one or more layers of the 3D perception network based at least on applying the real 2D sensor data and the simulated 2D sensor data to the first viewpoint adjustment network, prior to updating a second viewpoint adjustment network comprising the one or more layers of the 3D perception network based at least on applying the simulated 2D sensor data to the second viewpoint adjustment network. Claim 8 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 8, as set out above. Claim 9 of Instant Appl’n Claim 9 of ‘378 Appl’n 9. The one or processors of claim 1, wherein the 3D perception network is updated to perform at least one of a 2D-to-3D transformation or 2D-3D fusion. 9. The one or processors of claim 1, wherein the 3D perception network is updated to perform at least one of a 2D-to-3D transformation or 2D-3D fusion. Claim 9 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 9, as set out above. Claim 10 of Instant Appl’n Claim 10 of ‘378 Appl’n 10. The one or more processors of claim 1, wherein at least one of the 3D perception network or the one or more processors is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system for generating synthetic data using generative AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. 10. The one or more processors of claim 1, wherein at least one of the 3D perception network or the one or more processors is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. Claim 10 of Co-Pending Application S/N 18680378 teaches all of the limitations of instant claim 10, as set out above. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Claim Rejections - 35 U.S.C. § 112 8. The following is a quotation of 35 U.S.C. § 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. 9. Claims 1-10 are rejected under 35 U.S.C. § 112(b) or 35 U.S.C. § 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. § 112, the applicant), regards as the invention. Claim 1, line 5, recites “an ego-machine,” while also initially reciting “an ego-machine” at line 2. It is unclear whether the second occurrence of the element is intended to draw antecedence from the initial occurrence of the element at line 2, or whether it is intended to be another occurrence of the element. Thus, the claim is indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. Claims 2-10 depend directly or indirectly from claim 1, and are rejected as depending from a rejected claim; further, the claims fail to cure the deficiencies of claim 1. Claim Rejections - 35 U.S.C. § 102 10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 11. Claims 1-3, 6, 9, and 10 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by US Published Application 20200074266 to Peake et al. [hereinafter Peake]. Regarding claim 1, Peake teaches [o]ne or more processors (Peake ¶ 0007 teaches “a non-transitory computer-readable medium, storing thereon instructions executable by one or more processors”) comprising processing circuitry to perform one or more operations with an ego-machine (Peake, Fig. 7A, teaches a vehicle with a source sensor rig [Examiner annotations in dashed-line text boxes]: PNG media_image1.png 653 870 media_image1.png Greyscale Peake ¶ 0069 teaches “that each of vehicle 700 and vehicle 760 may represent either virtual vehicles in a virtual environment or real-world vehicles in a real-world environment [(that is, an ego-machine)]”; Peake ¶ 0151 teaches a “point cloud, information from the point cloud, or other image may be provided to a vehicle controller 722 via a corresponding electrical, optical, or radio link 720. The vehicle controller 722 may include one or more CPUs, GPUs, and a non-transitory memory with persistent components (e.g., flash memory, an optical disk) and/or non-persistent components (e.g., RAM) [(that is, processing circuitry to perform one or more operations with an ego-machine)]”; [Examiner notes that the plain and ordinary meaning of “ego-machine” is the vehicle itself from whose perspective the system perceives and reasons about the environment; accordingly, the term “ego-machine” covers the teaches of the vehicle of Peake, which is not inconsistent with the Applicant’s disclosure. (MPEP § 2111; Specification ¶ 0022 (“Although the present disclosure may be described with respect to an example autonomous or semi-autonomous vehicle or machine 700 (alternatively referred to herein as “vehicle 700” or “ego-machine 700,” an example of which is described with respect to FIGS. 7A-7D), this is not intended to be limiting.”))]) using a three-dimensional (3D) perception network (Peake, Fig. 5, teaches a self driving control architecture (SDCA) using one or more machine learning model(s) [Examiner annotations in dashed-line text boxes]: PNG media_image2.png 643 996 media_image2.png Greyscale Peake ¶ 0112 teaches “the feature training dataset, as generated by automated training dataset generator 100, may be associated with training a machine learning model to detect, classify and/or track ( e.g., via perception component 506 of FIG. 5 [(that is, “perception component” is using a three-dimensional (3D) perception network)]) one or more objects within the virtual environment or the real-world environment”; also sensor data 502 includes 3D data, where Peake ¶ 0152 teaches that “each of the sensor heads 712 includes a controller or processor that constructs a point cloud or other image (e.g., 2D, 3D, and/or RGB image) for a portion of the 360-degree horizontal view around the vehicle and provides the respective point cloud to the vehicle controller 722 [(that is, “3D point clouds” pertain to using a three-dimensional (3D) perception network)]”) that is updated based at least on applying simulated two-dimensional (2D) sensor data (Peake, Fig. 1, teaches an automated training dataset generator 100 [Examiner annotations in dashed-line text boxes]: PNG media_image3.png 698 875 media_image3.png Greyscale Peake ¶ 0034 teaches the “automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data [(that is, at least on applying simulated two-dimensional (2d) sensor data)]), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment”; Peake ¶ 0066 teaches that, “[b]ased on the reward, the automated training dataset generator 100 may adjust vehicle or driving parameters to maximize reward/increase performance of predictions (e.g., update weights of a machine model to correspond to a higher margin of safety in order to cause the autonomous vehicle to operate more closely with the predetermined ground truth route [(that is, updated based at least on applying simulated two-dimensional (2D) sensor data)]”) to at least a portion of the 3D perception network (Peake ¶ 0118 teaches “a machine learning model for prediction component 520 may be trained using virtual sensor data [(that is, to at least a portion of the 3D perception network)]”), the simulated 2D sensor data being generated based at least on simulating a source sensor rig of an ego-machine and a target sensor rig (Peake ¶ 0043 teaches “correspondences between actions (e.g., driving forward in a certain setting) and safety-related outcomes (e.g., avoiding collision) can be expressed as a ground truth [(that is, “ground truth” is a target sensor rig)] and used in generating training dataset(s) or other data as described herein”; Peake ¶ 0127 teaches “parameter settings for use by virtual or real-world sensors may be determined, via sensor parameter optimizer 112, by one or more machine learning models or self-driving control architectures, where, for example, a number of various parameter settings [(that is, the “parameter settings” pertaining to at least on simulating a source sensor rig)] are tested against operation of a vehicle (e.g., any of vehicles 401, 451, 700, and/or 760) in a real or virtual environment to determine parameters that cause the vehicle to operate in a desired manner (e.g., operate in a safe manner or operate in accordance with a ground truth [(that is, at least on . . . a target sensor rig)])”; see also Peake ¶ 0133, which teaches an “optimal parameter setting may be determined, by sensor parameter optimizer 112, via evolutionary learning based on vehicle operation data captured when an autonomous vehicle . . . interacts with one or more objects or surfaces . . . with a virtual environment . . . .”). Regarding claim 2, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein the processing circuitry is further to update the 3D perception network based at least on updating a viewpoint adjustment network (Peake, Fig. 5, teaches perception signals 508 of the perception component 506 include indicating object positions [Examiner annotations in dashed-line text boxes]: PNG media_image4.png 633 993 media_image4.png Greyscale Peake ¶ 0119 & Fig. 5 teaches “perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc. [(that is, viewpoints)]) within an overhead view [(bird’s-eye-viewpoint)] of the autonomous vehicle's environment [(that is, “object positions, boundaries, orientations” which is update the 3D perception network based at least on updating a viewpoint adjustment network )]”) comprising one or more layers of the 3D perception network replicated as at least a portion of multiple channels of the viewpoint adjustment network (Peake, Fig. 6A, teaches an example occupancy grid generator 600 of the perception component [Examiner annotations in dashed-line text boxes]: PNG media_image5.png 648 939 media_image5.png Greyscale Peake ¶ 0134 & Fig. teaches “an occupancy grid may be encoded as multi-channel image (e.g., an RGB image). In addition, an occupancy grid may include one or more layers at different levels or channels [(that is, replicated)], where each layer may define different information. Each of the layers may be read, or used, by a computing device, machine learning model, SDCA, etc., in a highly efficient manner, because the layers of the occupancy grid may represent simplified information of a virtual or real-world environment or scene [(that is, comprising one or more layers of the 3D perception network replicated as at least a portion of multiple channels of the viewpoint adjustment network)]”). Regarding claim 3, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein the processing circuitry is further to update the 3D perception network (Peake ¶ 0066 teaches that, “[b]ased on the reward, the automated training dataset generator 100 may adjust vehicle or driving parameters to maximize reward/increase performance of predictions (e.g., update weights of a machine model to correspond to a higher margin of safety in order to cause the autonomous vehicle to operate more closely with the predetermined ground truth route [(that is, update the 3D perception network)]”) based at least on applying the simulated 2D sensor data simulating the source sensor rig to one or more first paths of at least the portion of the 3D perception network (Peake ¶ 0133 teaches an “optimal parameter setting may be determined, by sensor parameter optimizer 112, via evolutionary learning based on vehicle operation data captured [(that is, “captured” is simulating)] when an autonomous vehicle . . . interacts with one or more objects or surfaces . . . with a virtual environment . . . [(that is, applying the simulated 2D sensor data simulating the source sensor rig to one or more first paths of at least the portion of the 3D perception network)]”) and applying the simulated 2D sensor data simulating the target sensor rig to one or more second paths of at least the portion of the 3D perception network (Peake ¶ 0043 teaches “[u]nlike in the real-world, ground truth correspondences may be learned by simulating alternative virtual realities relative to any given starting point/scenario [(that is, simulating the target sensor rig)]. . . . . The ground truth data may be used for various types of training [(that is, applying the simulated 2D sensor data simulating the target sensor rig to one or more second paths of at least the portion of the 3D perception network)]”; [Examiner notes that the broadest reasonable interpretation of “path” is defined by the training data on a path, where captured in a virtual environment is “one or more first paths” and where captured ground truth data based on a virtual reality is a “one or more second paths,” as taught by Peake, which is not inconsistent with the Applicant’s specification. (MPEP § 2111; Specification ¶ 0036 (“alternates training the path for real source right data 210 and the paths for the simulated source rig data 215”))]). Regarding claim 6, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein the source sensor rig represents a first sensor configuration that differs from a second sensor configuration of the target sensor rig (Peake ¶ 0042 teaches “configuration file manager may be used to accept a predefined configuration that defines the configurable parameters”; Peake ¶ 0079 teaches “configuration information may include a weight of a particular object or surface (e.g., vehicles 412 and/or 414), a number of sensors associated with a virtual vehicle (e.g., vehicle 401), or a location of sensors placed on the virtual vehicle (e.g., sensors associated with vehicle 401)”; Peake ¶ 0129 teaches “[t]he parameter setting may include settings of multiple devices, e.g., such that the sensor parameter optimizer 112 would be able to experiment with a lesser ranging sensor facing backwards but a longer ranging sensor facing forwards, where the sensors are installed in a virtual vehicle (e.g., as depicted and described for FIGS. 7A and 7B [(that is, a first sensor configuration differs from a second sensor configuration)])”; Peake ¶ 0133 teaches an “evolutionary learning technique may be, at least in some embodiments, a reinforcement learning technique as described herein. In some embodiments, the optimal parameter may be determined while a sensor or autonomous vehicle is operating within the virtual environment, or the optimal parameter may be determined at a later time after data for the sensor or autonomous vehicle operating within the virtual environment has been collected. . . . . Alternatively, multiple different parameter settings may be applied to a sensor operating in a virtual environment [(that is, “multiple different parameter settings” is the source sensor rig represents a first sensor configuration that differs from a second sensor configuration of the target sensor rig)]”). Regarding claim 9, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein the 3D perception network is updated to perform at least one of a 2D-to-3D transformation or 2D-3D fusion (Peake ¶ 0064 teaches “The 3D image may then be rasterized, converted, or otherwise transformed into a 2D image, e.g., having RGB pixel data. Such RGB pixel data may be used as training data, datasets(s), or as otherwise described herein. In addition, the 3D and/or 2D image may also be converted or otherwise transformed into point cloud data and/or simulated point cloud data, e.g., as described with respect to FIG. 2B, or otherwise herein [(that is, wherein the 3D perception network is updated to perform at least one of a 2D-to-3d transformation . . . )].”). Regarding claim 10, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein at least one of the 3D perception network or the one or more processors is comprised in at least one of: a control system for an autonomous or semi-autonomous machine (Peake ¶ 0153 teaches “the vehicle 700 may be an autonomous vehicle where the vehicle controller 722 provides control signals to various components 730 within the vehicle 700 to maneuver and otherwise control operation of the vehicle 700 [(that is, wherein at least one of the 3D perception network or the one or more processors is comprised in at least one of: a control system an autonomous or semi-autonomous machine . . . . )].”); a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system for generating synthetic data using generative AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. Claim Rejections – 35 U.S.C. § 103 12. The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 13. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 14. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention. 15. Claims 4, 5, and 8 are rejected under 35 U.S.C. § 103 as being unpatentable over Peake et al. [hereinafter Peake] in view of US Published Application 20210156960 to Popov et al. [hereinafter Popov ‘960]. Regarding claim 4, Peake teaches all of the limitations of claim 1, as described above in detail. Though Peake teaches an imaging engine that transforms 3D sensor data to 2D sensor data and vice versa, Peake, however, does not explicitly teach – wherein the 3D perception network comprises a view transformation that generates transformed features, wherein the processing circuitry is further to update the 3D perception network based at least on a loss that minimizes differences between the transformed features of one or more first paths and of one or more second paths of at least the portion of the 3D perception network. But Popov ‘960 teaches - wherein the 3D perception network comprises a view transformation that generates transformed features (Popov ‘960 ¶ 0052 teaches “the output of the first DNN featurizer may be transformed to a second view of the environment (e.g., a top down view) [(that is, the 3D perception network comprises a view transformation that generates transformed features)], which may be fed into a second DNN featurizer (e.g., that predicts object instance data)”), wherein the processing circuitry is further to update the 3D perception network based at least on a loss that minimizes differences between the transformed features of one or more first paths and of one or more second paths of at least the portion of the 3D perception network (Popov ‘960 ¶ 0099 teaches “the training set of RADAR data tensors may be input into machine learning model(s) 108, and the ground truth class segmentation and/or instance regression tensors may be used to train machine learning model(s) 108. For example, one or more loss functions (e.g., a single loss function, a loss function for each output type, etc.) may be used to compare the accuracy of the output(s) of machine learning model(s) 108 to ground truth [(that is, update the 3D perception network based at least on a loss that minimizes differences between the transformed features of one or mor first paths and of one or more second paths of at least the portion of the 3D perception network)], and the parameters of the machine learning model(s) 108 may be updated (e.g., using backward passes, backpropagation, forward passed, etc.) until the accuracy reaches an optimal or acceptable level. In some embodiments in which machine learning model(s) 108 includes multiple heads, the multiple heads may be co-trained together with a common trunk, or may be trained separately. In embodiments involving multi-view scenarios with separate DNN featurizers, the DNN featurizers may be co-trained together or separately. These and other variations are contemplated within the present disclosure”). Peake and Popov ‘960 are from the same or similar field of endeavor. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘960 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Peake pertaining to automated training dataset generators with the viewpoint transformers of Popov ‘960. The motivation to do is to “the sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, rotating, cropping, and/or extracting at least one region of interest (ROI) from images corresponding to the sensor data to a field of view of a different physical or virtual sensor. . . . As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor. This process increases the scalability of the system while removing vehicle specific dependencies to generate a machine learning model that is deployable in any number of different vehicles.” (Popov ‘960 ¶ 0005). Regarding claim 5, Peake teaches all of the limitations of claim 1, as described above in detail. Though Peake teaches automated training dataset generates to generate feature training datasets for use in real-world autonomous driving applications based on virtual environments, Peake, however, does not explicitly teach – update the 3D perception network based at least on a loss that minimizes differences between view-transformed features of one or more first paths that process the simulated 2D sensor data corresponding to the source sensor rig and one or more second paths that process the simulated 2D sensor data corresponding to the target sensor rig. But Popov ‘960 teaches - wherein the processing circuitry is further to update the 3D perception network based at least on a loss that minimizes differences between view-transformed features (Popov ‘960 ¶ 0099 teaches “training set of RADAR data tensors may be input into machine learning model(s) 108, and the ground truth class segmentation and/or instance regression tensors may be used to train machine learning model(s) 108. For example, one or more loss functions (e.g., a single loss function, a loss function for each output type, etc.) may be used to compare the accuracy of the output(s) of machine learning model(s) 108 to ground truth, and the parameters of the machine learning model(s) 108 may be updated (e.g., using backward passes, backpropagation, forward passed, etc.) until the accuracy reaches an optimal or acceptable level [(that is, “acceptable level” is update the 3D perception network based at least on a loss that minimizes differences between view-transformed features)]. In some embodiments in which machine learning model(s) 108 includes multiple heads, the multiple heads may be co-trained together with a common trunk, or may be trained separately. In embodiments involving multi-view scenarios with separate DNN featurizers, the DNN featurizers may be co-trained together or separately. These and other variations are contemplated within the present disclosure”) of one or more first paths that process the simulated 2D sensor data corresponding to the source sensor rig (Peake ¶ 0133 teaches an “optimal parameter setting may be determined, by sensor parameter optimizer 112, via evolutionary learning based on vehicle operation data captured [(that is, “captured” is simulating)] when an autonomous vehicle . . . interacts with one or more objects or surfaces . . . with a virtual environment . . . [(that is, of one or more first paths that process the simulated 2D sensor data corresponding to the source sensor rig)]”) and one or more second paths that process the simulated 2D sensor data corresponding to the target sensor rig (Peake ¶ 0043 teaches “[u]nlike in the real-world, ground truth correspondences may be learned by simulating alternative virtual realities relative to any given starting point/scenario [(that is, simulating the target sensor rig)]. . . . . The ground truth data may be used for various types of training [(that is, one or more second paths that process the simulated 2D sensor data corresponding to the target sensor rig)]”; [Examiner notes that the broadest reasonable interpretation of “path” is defined by the training data on a path, where captured in a virtual environment is “one or more first paths” and where captured ground truth data based on a virtual reality is a “one or more second paths,” as taught by Peake, which is not inconsistent with the Applicant’s specification. (MPEP § 2111; Specification ¶ 0036 (“alternates training the path for real source right data 210 and the paths for the simulated source rig data 215”))]). Peake and Popov ‘960 are from the same or similar field of endeavor. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘960 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Peake pertaining to automated training dataset generators with the viewpoint transformers of Popov ‘960. The motivation to do is to “the sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, rotating, cropping, and/or extracting at least one region of interest (ROI) from images corresponding to the sensor data to a field of view of a different physical or virtual sensor. . . . As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor. This process increases the scalability of the system while removing vehicle specific dependencies to generate a machine learning model that is deployable in any number of different vehicles.” (Popov ‘960 ¶ 0005). Regarding claim 8, Peake teaches all of the limitations of claim 1, as described above in detail. Peake teaches - wherein the processing circuitry is further to update a first viewpoint adjustment network (Peake, Fig. 5, teaches perception signals 508 of the perception component 506 include indicating object positions [Examiner annotations in dashed-line text boxes]: PNG media_image4.png 633 993 media_image4.png Greyscale Peake ¶ 0119 & Fig. 5 teaches “perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc. [(that is, viewpoints)]) within an overhead view [(bird’s-eye-viewpoint)] of the autonomous vehicle's environment [(that is, “object positions, boundaries, orientations” which is update the 3D perception network based at least on updating a viewpoint adjustment network )]”) comprising . . . . Though Peake teaches automated training dataset generates to generate feature training datasets for use in real-world autonomous driving applications based on virtual environments, Peake, however, does not explicitly teach – * * * one or more layers of the 3D perception network based at least on applying real 2D sensor data and the simulated 2D sensor data to the first viewpoint adjustment network, prior to updating a second viewpoint adjustment network comprising the one or more layers of the 3D perception network based at least on applying the simulated 2D sensor data to the second viewpoint adjustment network. But Popov ‘960 teaches – * * * one or more layers of the 3D perception network based at least on applying real 2D sensor data and the simulated 2D sensor data (Popov ‘960 ¶ 0032 teaches “[o]nce the object instances have been determined, the 2D pixel coordinates [(that is, applying real 2D sensor data and the simulated 2D sensor data)] defining the object instances may be converted to 3D world coordinates (e.g., by reprojecting detection object instances from the 2D orthographic projection back to 3D world coordinates”; Popov ‘960 ¶ 0220 teaches “training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine) [(that is, real 2d sensor data and the simulated 2D sensor data)]”) to the first viewpoint adjustment network, prior to updating a second viewpoint adjustment network comprising the one or more layers of the 3D perception network based at least on applying the simulated 2D sensor data to the second viewpoint adjustment network (Popov ‘960 ¶ 0052 teaches “sequential DNN featurizers may be chained. In this example, a first projection image may be generated with a first view of the environment (e.g., a perspective view) p(that is, the first viewpoint adjustment network)], the first projection image may be fed into a first DNN featurizer (e.g., that predicts classification data), the output of the first DNN featurizer may be transformed to a second view of the environment (e.g., a top down view), which may be fed into a second DNN featurizer (e.g., that predicts object instance data) [(that is, the simulated 2D sensor data to the first viewpoint adjustment network prior to updating a second viewpoint adjustment network)]”). Peake and Popov ‘960 are from the same or similar field of endeavor. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘960 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Peake pertaining to automated training dataset generators with the viewpoint transformers of Popov ‘960. The motivation to do is to “the sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, rotating, cropping, and/or extracting at least one region of interest (ROI) from images corresponding to the sensor data to a field of view of a different physical or virtual sensor. . . . As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor. This process increases the scalability of the system while removing vehicle specific dependencies to generate a machine learning model that is deployable in any number of different vehicles.” (Popov ‘960 ¶ 0005). 16. Claim 7 is rejected under 35 U.S.C. § 103 as being unpatentable over Peake et al. [hereinafter Peake] in view of US Published Application 20210156963 to Popov et al. [hereinafter Popov ‘963]. Regarding claim 7, Peake teaches all of the limitations of claim 1, as described above in detail. Though Peake teaches a segmentation module configured to analyze a point cloud or other data frames to identify subsets of points within each frame that correspond to probable physical objects or surfaces in the environment, Peake, however, does not explicitly teach - wherein the processing circuitry is further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more simulated sensors of the source sensor rig and at least one corresponding sensor of the target sensor rig, and simulated ground truth data. But Popov ‘963 teaches - wherein the processing circuitry is further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more simulated sensors of the source sensor rig and at least one corresponding sensor of the target sensor rig, and simulated ground truth data (Popov ‘963 ¶ 0033 teaches “To train the DNN, training data may be generated using the pre-processing technique described above. However, given how sparse RADAR data may be, it is often challenging to distinguish objects such as vehicles in the RADAR data alone. As such, in some embodiments, ground truth data may be generated from LIDAR data or other sources of 3D information [(that is, generate . . . simulated ground truth data)] . . . . More specifically, a scene may be observed with RADAR and LIDAR sensors to collect a frame of RADAR data and LIDAR data for a particular time slice [(that is, generate, for at least one time slice of one or more time slices)]. The RADAR data may be used to generate an input RADAR data tensor, and the LIDAR data associated with the same or closest time slice as the RADAR data may be used to generate ground truth labels, which may be used to generate ground truth class segmentation and/or instance regression tensors”; Popov ‘963 ¶ 0094 teaches “LIDAR data associated with the same or closest time slice as the RADAR data may be used to generate ground truth labels, which may be used to generate ground truth classification data and/or object instance data corresponding to the output of machine learning model(s) 108 (e.g., class confidence tensor 110 and instance regression tensor 112 of FIG. 1)”;) Peake and Popov ‘963 are from the same or similar field of endeavor. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘963 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Peake pertaining to automated training dataset generators with the time slices of Popov ‘963. The motivation to do is because “conventional sensing techniques are generally unreliable in inclement weather conditions, and the underlying sensors are often prohibitively expensive. Moreover, because the output signal from these conventional systems requires heavy post-processing in order to extract accurate three-dimensional (3D) information, the run-time of these conventional systems is generally higher and requires additional computational and processing demands, thereby reducing the efficiency of these conventional systems.” (Popov ‘963 ¶ 0003). 17. Claims 11-13, 16, and 18 are rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20210383616 to Rong et al. [hereinafter Rong] in view of US Published Application 20200074266 to Peake et al. [hereinafter Peake]. Regarding claim 11, Rong teaches [a] system comprising one or more processors to perform one or more operations with an ego machine (Rong ¶ 0032 teaches “store 3D object reconstruction data including but not limited to 3D meshes, captured and/or predicted images, and/or object poses in ego-vehicle-centric coordinates [(that is, an ego machine)]"; Rong ¶ 0115 teaches “computing system 1405 can perform data collection and reconstruction to generate an object bank of real world objects; Rong ¶ 0116 teaches “computing system 1405 can include one or more processors 1410 and a memory 1415 [that is, a system comprising one or more processors to perform one or more operations with an ego machine)]”) using a three-dimensional (3D) reconstruction network (Rong ¶ 0015 teaches a “machine-learned object reconstruction model”; Rong ¶ 0024 teaches “systems and methods that build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data [(that is, using a three-dimensional (3D) reconstruction network)]”) that is updated based at least on applying simulated two-dimensional (2D) sensor data to at least a portion of the 3D reconstruction network (Rong ¶ 0027 teaches “autonomous vehicle safety testing can be aided by the generation of novel simulated imagery. An application that utilizes environment data such as camera-captured imagery [(that is, “camera-captured imagery” is two-dimensional sensor data)] and/or three-dimensional imaging and Light Detection and Ranging (LiDAR) data to generate synthetic data for autonomous vehicle testing [(that is, “synthetic data” is simulated . . . sensor data)] can provide a larger database of road data to test the safety features of autonomous vehicle software.; Rong ¶ 0070 teaches “one or more sensors can be located on various parts of the vehicle 105 including a front side, rear side, left side, right side, top, and/or bottom of the vehicle 105. The sensor data 155 can include image data (e.g., 2D camera data, video data, etc.), RADAR data, LIDAR data (e.g., 3D point cloud data, etc.), audio data, and/or other types of data. The vehicle 105 can also include other sensors configured to acquire data associated with the vehicle 105”; Rong ¶ 0053 teaches the “model can be updated (e.g., parameter values adjusted) in response to the evaluation of the loss function(s). [(that is, a reconstruction network that is updated)]”), . . . . Though Rong teaches a trained system may be used to compile 3D assets to aid in generating photorealistic image simulations, Rong, however, does not explicitly teach – . . . the simulated 2D sensor data being generated based at least on simulating a source sensor rig of the ego-machine and a target sensor rig. But Peake teaches - . . . the simulated 2D sensor data being generated based at least on simulating a source sensor rig of the ego-machine and a target sensor rig (Peake ¶ 0043 teaches “correspondences between actions (e.g., driving forward in a certain setting) and safety-related outcomes (e.g., avoiding collision) can be expressed as a ground truth [(that is, “ground truth” is a target sensor rig)] and used in generating training dataset(s) or other data as described herein”; Peake ¶ 0127 teaches “parameter settings for use by virtual or real-world sensors may be determined, via sensor parameter optimizer 112, by one or more machine learning models or self-driving control architectures, where, for example, a number of various parameter settings [(that is, the “parameter settings” pertaining to at least on simulating a source sensor rig)] are tested against operation of a vehicle (e.g., any of vehicles 401, 451, 700, and/or 760) in a real or virtual environment to determine parameters that cause the vehicle to operate in a desired manner (e.g., operate in a safe manner or operate in accordance with a ground truth [(that is, at least on . . . a target sensor rig)])”; see also Peake ¶ 0133, which teaches an “optimal parameter setting may be determined, by sensor parameter optimizer 112, via evolutionary learning based on vehicle operation data captured when an autonomous vehicle . . . interacts with one or more objects or surfaces . . . with a virtual environment . . . .”). Rong and Peake are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Rong pertaining to training a machine-learned objection reconstruction model with the source and target sensor rigs of Peake. The motivation to do so is to “generally extremely costly, burdensome, dangerous, and/or impracticable to collect sufficient amounts of training datasets for real-world driving and/or autonomous vehicle activities or purposes.” (Peake ¶ 0004). Regarding claim 12, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Rong teaches - wherein the one or more processors are further to update the 3D reconstruction network based at least on updating a viewpoint adjustment network (Rong ¶ 0046 teaches “selected object data sets may be refined based on viewpoints and distance [(that is, update the 3D reconstruction network based at least on updating a viewpoint adjustment network)]”) . . . . Though Rong teaches viewpoint refinement, Rong, however, does not explicitly teach – . . . comprising one or more layers of the 3D reconstruction network replicated as at least a portion of multiple channels of the viewpoint adjustment network. But Peake teaches - . . . comprising one or more layers of the 3D reconstruction network replicated as at least a portion of multiple channels of the viewpoint adjustment network (Peake, Fig. 6A, teaches an example occupancy grid generator 600 of the perception component [Examiner annotations in dashed-line text boxes]: PNG media_image5.png 648 939 media_image5.png Greyscale Peake ¶ 0134 & Fig. teaches “an occupancy grid may be encoded as multi-channel image (e.g., an RGB image). In addition, an occupancy grid may include one or more layers at different levels or channels [(that is, replicated)], where each layer may define different information. Each of the layers may be read, or used, by a computing device, machine learning model, SDCA, etc., in a highly efficient manner, because the layers of the occupancy grid may represent simplified information of a virtual or real-world environment or scene [(that is, comprising one or more layers of the 3D reconstruction network replicated as at least a portion of multiple channels of the viewpoint adjustment network)]”). Rong and Peake are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Rong pertaining to training a machine-learned objection reconstruction model with the source and target sensor rigs of Peake. The motivation to do so is to “generally extremely costly, burdensome, dangerous, and/or impracticable to collect sufficient amounts of training datasets for real-world driving and/or autonomous vehicle activities or purposes.” (Peake ¶ 0004). Regarding claim 13, the combination of Rong and Peake teaches all of the limitations of claim 11, as described above in detail. Peake teaches - wherein the one or more processors are further to update the 3D reconstruction network based at least on applying the simulated 2D sensor data simulating the source sensor rig to one or more first paths of at least the portion of the 3D reconstruction network (Peake ¶ 0133 teaches an “optimal parameter setting may be determined, by sensor parameter optimizer 112, via evolutionary learning based on vehicle operation data captured [(that is, “captured” is simulating)] when an autonomous vehicle . . . interacts with one or more objects or surfaces . . . with a virtual environment . . . [(that is, applying the simulated 2D sensor data simulating the source sensor rig to one or more first paths of at least the portion of the 3D reconstruction network)]”) and applying the simulated 2D sensor data simulating the target sensor rig to one or more second paths of at least the portion of the 3D reconstruction network (Peake ¶ 0043 teaches “[u]nlike in the real-world, ground truth correspondences may be learned by simulating alternative virtual realities relative to any given starting point/scenario [(that is, simulating the target sensor rig)]. . . . . The ground truth data may be used for various types of training [(that is, applying the simulated 2D sensor data simulating the target sensor rig to one or more second paths of at least the portion of the 3D reconstruction network )]”; [Examiner notes that the broadest reasonable interpretation of “path” is defined by the training data on a path, where captured in a virtual environment is “one or more first paths” and where captured ground truth data based on a virtual reality is a “one or more second paths,” as taught by Peake, which is not inconsistent with the Applicant’s specification. (MPEP § 2111; Specification ¶ 0036 (“alternates training the path for real source right data 210 and the paths for the simulated source rig data 215”))]). Regarding claim 16, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Peake teaches - wherein the source sensor rig represents a first sensor configuration that differs from a second sensor configuration of the target sensor rig (Peake ¶ 0042 teaches “configuration file manager may be used to accept a predefined configuration that defines the configurable parameters”; Peake ¶ 0079 teaches “configuration information may include a weight of a particular object or surface (e.g., vehicles 412 and/or 414), a number of sensors associated with a virtual vehicle (e.g., vehicle 401), or a location of sensors placed on the virtual vehicle (e.g., sensors associated with vehicle 401)”; Peake ¶ 0129 teaches “[t]he parameter setting may include settings of multiple devices, e.g., such that the sensor parameter optimizer 112 would be able to experiment with a lesser ranging sensor facing backwards but a longer ranging sensor facing forwards, where the sensors are installed in a virtual vehicle (e.g., as depicted and described for FIGS. 7A and 7B [(that is, a first sensor configuration differs from a second sensor configuration)])”; Peake ¶ 0133 teaches an “evolutionary learning technique may be, at least in some embodiments, a reinforcement learning technique as described herein. In some embodiments, the optimal parameter may be determined while a sensor or autonomous vehicle is operating within the virtual environment, or the optimal parameter may be determined at a later time after data for the sensor or autonomous vehicle operating within the virtual environment has been collected. . . . . Alternatively, multiple different parameter settings may be applied to a sensor operating in a virtual environment [(that is, “multiple different parameter settings” is the source sensor rig represents a first sensor configuration that differs from a second sensor configuration of the target sensor rig)]”). Regarding claim 18, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Rong teaches - wherein at least one of the system or the 3D reconstruction network is comprised in at least one of: a control system for an autonomous or semi-autonomous machine (Rong ¶ 0061 teaches “The vehicle 105 can be configured to operate in a plurality of operating modes. The vehicle 105 can be configured to operate in a fully autonomous (e.g., self-driving) operating mode in which the vehicle 105 is controllable without user input (e.g., can drive and navigate with no input from a human operator present in the vehicle 105 and/or remote from the vehicle 105) [(that is, a control system for an autonomous or semi-autonomous machine)]”); a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system for generating synthetic data using generative AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. 18. Claims 14 and 15 are rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20210383616 to Rong et al. [hereinafter Rong] in view of US Published Application 20200074266 to Peake et al. [hereinafter Peake] and US Published Application 20210156960 to Popov et al. [hereinafter Popov ‘960]. Regarding claim 14, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Though Rong and Peake teach an imaging engine that transforms 3D sensor data to 2D sensor data and vice versa, the combination of Rong and Peake, however, does not explicitly teach – wherein the 3D reconstruction network is updated to perform a view transformation that generates transformed features, wherein the one or more processors are further to update the 3D reconstruction network based at least on a loss that minimizes differences between the transformed features of one or more first paths and of one or more second paths of at least the portion of the 3D reconstruction network. But Popov ‘960 teaches - wherein the 3D reconstruction network is updated to perform a view transformation that generates transformed features (Popov ‘960 ¶ 0052 teaches “the output of the first DNN featurizer may be transformed to a second view of the environment (e.g., a top down view) [(that is, the 3D perception network comprises a view transformation that generates transformed features)], which may be fed into a second DNN featurizer (e.g., that predicts object instance data)”), wherein the one or more processors are further to update the 3D reconstruction network based at least on a loss that minimizes differences between the transformed features of one or more first paths and of one or more second paths of at least the portion of the 3D reconstruction network (Popov ‘960 ¶ 0099 teaches “the training set of RADAR data tensors may be input into machine learning model(s) 108, and the ground truth class segmentation and/or instance regression tensors may be used to train machine learning model(s) 108. For example, one or more loss functions (e.g., a single loss function, a loss function for each output type, etc.) may be used to compare the accuracy of the output(s) of machine learning model(s) 108 to ground truth [(that is, update the 3D perception network based at least on a loss that minimizes differences between the transformed features of one or mor first paths and of one or more second paths of at least the portion of the 3D perception network)], and the parameters of the machine learning model(s) 108 may be updated (e.g., using backward passes, backpropagation, forward passed, etc.) until the accuracy reaches an optimal or acceptable level. In some embodiments in which machine learning model(s) 108 includes multiple heads, the multiple heads may be co-trained together with a common trunk, or may be trained separately. In embodiments involving multi-view scenarios with separate DNN featurizers, the DNN featurizers may be co-trained together or separately. These and other variations are contemplated within the present disclosure”). Rong, Peake and Popov ‘960 are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘960 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Rong and Peake pertaining to automated training dataset generators with the viewpoint transformers of Popov ‘960. The motivation to do is to “the sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, rotating, cropping, and/or extracting at least one region of interest (ROI) from images corresponding to the sensor data to a field of view of a different physical or virtual sensor. . . . As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor. This process increases the scalability of the system while removing vehicle specific dependencies to generate a machine learning model that is deployable in any number of different vehicles.” (Popov ‘960 ¶ 0005). Regarding claim 15, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Rong teaches - wherein the one or more processors are further to update the 3D reconstruction network based at least on a loss (Rong ¶ 0037 teaches “first, second, and/or third embedding models may be trained by comparing predicted data to ground truth data with one or more loss functions [(that is, update the 3D reconstruction network based at least on a loss)]”) . . . . Though Rong and Peake teach updating embedding models by comparing predicted data to ground truth data with one or more loss functions, the combination of Rong and Peake, however, does not explicitly teach - . . . that minimizes differences between view-transformed features of one or more first paths that process the simulated 2D sensor data corresponding to the source sensor rig and one or more second paths that process the simulated 2D sensor data corresponding to the target sensor rig. But Popov ‘960 teaches – . . . that minimizes differences between view-transformed features of one or more first paths that process the simulated 2D sensor data corresponding to the source sensor rig and one or more second paths that process the simulated 2D sensor data corresponding to the target sensor rig (Popov ‘960 ¶ 0099 teaches “the training set of RADAR data tensors may be input into machine learning model(s) 108, and the ground truth class segmentation and/or instance regression tensors may be used to train machine learning model(s) 108. For example, one or more loss functions (e.g., a single loss function, a loss function for each output type, etc.) may be used to compare the accuracy of the output(s) of machine learning model(s) 108 to ground truth [(that is, update the 3D perception network based at least on a loss that minimizes differences between the transformed features of one or mor first paths and of one or more second paths of at least the portion of the 3D reconstruction network)], and the parameters of the machine learning model(s) 108 may be updated (e.g., using backward passes, backpropagation, forward passed, etc.) until the accuracy reaches an optimal or acceptable level. In some embodiments in which machine learning model(s) 108 includes multiple heads, the multiple heads may be co-trained together with a common trunk, or may be trained separately. In embodiments involving multi-view scenarios with separate DNN featurizers, the DNN featurizers may be co-trained together or separately. These and other variations are contemplated within the present disclosure”) Rong, Peake and Popov ‘960 are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘960 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify the combination of Rong and Peake pertaining to automated training dataset generators with the loss minimization of Popov ‘960. The motivation to do is to “the sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, rotating, cropping, and/or extracting at least one region of interest (ROI) from images corresponding to the sensor data to a field of view of a different physical or virtual sensor. . . . As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor. This process increases the scalability of the system while removing vehicle specific dependencies to generate a machine learning model that is deployable in any number of different vehicles.” (Popov ‘960 ¶ 0005). 19. Claim 17 is rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20210383616 to Rong et al. [hereinafter Rong] in view of US Published Application 20200074266 to Peake et al. [hereinafter Peake] and US Published Application 20210156963 to Popov et al. [hereinafter Popov ‘963]. Regarding claim 17, the combination of Rong and Peake teach all of the limitations of claim 11, as described above in detail. Though Rong and Peake teach segmentation module configured to analyze a point cloud or other data frames to identify subsets of points within each frame that correspond to probable physical objects or surfaces in the environment, the combination of Rong and Peake, however, does not explicitly teach - wherein the one or more processors are further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more simulated sensors of the source sensor rig and at least one corresponding sensor of the target sensor rig, and simulated ground truth data. But Popov ‘963 teaches - wherein the processing circuitry is further to generate, for at least one time slice of one or more time slices of a simulation, a simulated frame of 2D sensor data for at least one sensor of one or more simulated sensors of the source sensor rig and at least one corresponding sensor of the target sensor rig, and simulated ground truth data (Popov ‘963 ¶ 0033 teaches “To train the DNN, training data may be generated using the pre-processing technique described above. However, given how sparse RADAR data may be, it is often challenging to distinguish objects such as vehicles in the RADAR data alone. As such, in some embodiments, ground truth data may be generated from LIDAR data or other sources of 3D information [(that is, generate . . . simulated ground truth data)] . . . . More specifically, a scene may be observed with RADAR and LIDAR sensors to collect a frame of RADAR data and LIDAR data for a particular time slice [(that is, generate, for at least one time slice of one or more time slices)]. The RADAR data may be used to generate an input RADAR data tensor, and the LIDAR data associated with the same or closest time slice as the RADAR data may be used to generate ground truth labels, which may be used to generate ground truth class segmentation and/or instance regression tensors”; Popov ‘963 ¶ 0094 teaches “LIDAR data associated with the same or closest time slice as the RADAR data may be used to generate ground truth labels, which may be used to generate ground truth classification data and/or object instance data corresponding to the output of machine learning model(s) 108 (e.g., class confidence tensor 110 and instance regression tensor 112 of FIG. 1)”;) Rong, Peake and Popov ‘963 are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Peake teaches automated training dataset generators that generate feature training datasets for use in real-world autonomous driving applications based on virtual environments. Popov ‘963 teaches deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Peake pertaining to automated training dataset generators with the viewpoint transformers of Popov ‘960. The motivation to do is to “conventional sensing techniques are generally unreliable in inclement weather conditions, and the underlying sensors are often prohibitively expensive. Moreover, because the output signal from these conventional systems requires heavy post-processing in order to extract accurate three-dimensional (3D) information, the run-time of these conventional systems is generally higher and requires additional computational and processing demands, thereby reducing the efficiency of these conventional systems.” (Popov ‘963 ¶ 0003). 20. Claims 19 and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over US Published Application 20210383616 to Rong et al. [hereinafter Rong] in view of US Published Application 20210347387 to Avedisov et al. [hereinafter Avedisov]. Regarding claim 19, Rong teaches [a] method comprising: applying simulated two-dimensional (2D) sensor data to at least a portion of the 3D reconstruction network (Rong ¶ 0027 teaches “autonomous vehicle safety testing can be aided by the generation of novel simulated imagery. An application that utilizes environment data such as camera-captured imagery [(that is, “camera-captured imagery” is two-dimensional sensor data)] and/or three-dimensional imaging and Light Detection and Ranging (LiDAR) data to generate synthetic data for autonomous vehicle testing [(that is, “synthetic data” is simulated . . . sensor data)] can provide a larger database of road data to test the safety features of autonomous vehicle software.; Rong ¶ 0070 teaches “one or more sensors can be located on various parts of the vehicle 105 including a front side, rear side, left side, right side, top, and/or bottom of the vehicle 105. The sensor data 155 can include image data (e.g., 2D camera data, video data, etc.), RADAR data, LIDAR data (e.g., 3D point cloud data, etc.), audio data, and/or other types of data. The vehicle 105 can also include other sensors configured to acquire data associated with the vehicle 105”; Rong ¶ 0053 teaches the “model can be updated (e.g., parameter values adjusted) in response to the evaluation of the loss function(s). [(that is, a reconstruction network that is updated)]”), . . . ; updating a three-dimensional (3D) reconstruction network based at least on the simulated 2D sensor data (Rong ¶ 0167 teaches the “[d]ynamics determination can complement the object sampling process by utilizing an Intelligent Driver Model (IDM) fitted to a kinematic model. The models can update the simulated object's state for realistic interactions with surrounding traffic [(that is, a reconstruction network that is updated)]”; and performing one or more operations with the second ego-machine using the 3D reconstruction network (Rong ¶ 0164 teaches “new objects can be interacting with other dynamic agents [(that is, “other dynamic agents” is the second ego-machine)]. In some implementations, the system or method may first select the placement of objects onto the HD map so that the new image reflects the traffic and 3D layout in a realistic manner. The system may then select objects from the reconstructed 3D asset bank based on the determined possible placements for insertion. The system can then use novel-view rendering with 3D occlusion reasoning with respect to all elements in the scene, to create the appearance of the new object in the new image [(that is, performing one or more operations with the second ego-machine using the 3D reconstruction network)]”). Though Rong teaches a trained system may be used to compile 3D assets to aid in generating photorealistic image simulations, Rong, however, does not explicitly teach - . . . the simulated 2D sensor data being generated based at least on simulating a first sensor rig of a first ego-machine and a second sensor rig of a second ego-machine; * * * But Avedisov teaches - . . . the simulated 2D sensor data being generated based at least on simulating a first sensor rig of a first ego-machine and a second sensor rig of a second ego-machine (Avedisov ¶ 0047 teaches “The driving behavior data 126 can include any suitable dynamic model(s) and/or simulator(s) for interpolating and/or extrapolating the behavior of a vehicle based on known information [(that is, simulated 2D sensor data)]”; Avedisov ¶ 0052 teaches “[t]he sensor fusion module 131 can receive all relevant sensory and/or wireless data from one or more elements of the traffic reconstruction system 100. The sensor fusion module 131 can collect all data relevant to a certain time interval over which the traffic reconstruction is to occur. Alternatively, the sensor fusion module can select a subset of the data received from all sources. The subset can be less than all of the data received. In one or more arrangements, the subset can be temporal-based. The sensor fusion module(s) 131 can identify the critical objects (vehicles) in the received data”; Avedisov ¶ 0058 teaches “The ego vehicle 710 [(that is, a first ego-machine)], the connected vehicles 721, 723, and the vehicles 720, 722 can be located in the first travel lane 704. The vehicles 730, 731, 732 can be located in the second travel lane 706; Avedisov ¶ 0059 teaches ”the connected vehicle 723 can sense the driving environment 700 [(that is, a second sensor rig of a second ego-machine)]. The connected vehicle 723 can detect the vehicles 731, 732. More particularly, the connected vehicle 723 can detect the position and speed information about the vehicles 731, 732 at the first moment in time t1. The connected vehicle 723 can send a cooperative perception message 740 to the ego vehicle 710 [(that is, a first sensor rig of a first ego-machine)]”); * * * Rong and Avedisov are from the same or similar field of endeavor. Rong teaches generating photorealistic image simulation data with geometry-aware composition for testing autonomous vehicles. Avedisov teaches traffic reconstruction can be generated use cooperative perception messages and other data received from a plurality of different remote sources, such as connected vehicles and/or connected infrastructure devices, at different times. Thus, it would have been obvious to a person having ordinary skill in the art as of the effective filing date of the Applicant’s invention to modify Rong pertaining to image simulation data for autonomous vehicles with the multiple sensor rigs and ego vehicles of the simulations of Avedisov. The motivation to do so is because “The fusion of the data from the various remote vehicles can be done not only in space but in time. Dynamic models can be used in generating the traffic reconstruction. In this way, the driving environment can be cooperatively perceived by the vehicle and/or an infrastructure device. The perception can be considered to be “cooperative” in that the receiving device (e.g., the vehicle and/or the infrastructure device) can use driving environment data from various remote sources (e.g., vehicles, infrastructure, etc.) to attain a holistic picture of the driving environment. Of course, the receiving device can also use data captured by its own sensors.” (Avedisov ¶ 0029). Regarding claim 20, the combination of Rong and Avedisov teaches all of the limitations of claim 19, as described above in detail. Rong teaches - wherein at least one of the system or the 3D reconstruction network is comprised in at least one of: a control system for an autonomous or semi-autonomous machine (Rong ¶ 0061 teaches “The vehicle 105 can be configured to operate in a plurality of operating modes. The vehicle 105 can be configured to operate in a fully autonomous (e.g., self-driving) operating mode in which the vehicle 105 is controllable without user input (e.g., can drive and navigate with no input from a human operator present in the vehicle 105 and/or remote from the vehicle 105) [(that is, a control system for an autonomous or semi-autonomous machine)]”); a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system for generating synthetic data using generative AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. Conclusion 21. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: (US Published Application 20210300389 to McGill et al.) teaches raw sensor data (e.g., image data, LiDAR data, and the like) is inputted into a sensor data neural network that outputs a sensor data intermediate space (i.e., latent space), which may then be inputted into a sensor data reconstruction neural network that recreates the input sensor data. A loss function may be applied to calculate the loss between the input sensor data and the reconstructed sensor data to minimize the loss therebetween. (US Published Application 20220269937 to Kim et al.) teaches inference and/or training logic used to perform inferencing and/or training operations associated with one or more embodiments. Details regarding inference and/or training logic. (Muzahid et al., “Deep Reinforcement Learning-Based Driving Strategy for Avoidance of Chain Collisions and Its Safety Efficiency Analysis in Autonomous Vehicles,” IEEE (2022)) teaches A reward function is being developed to deal with the challenge of multiple vehicle collision avoidance. A perception network structure based on formation and on actor-critic methodologies is employed to enhance the decision-making process. Finally, in the safety efficiency analysis phase, we investigated the safety efficiency performance of the agent vehicle in both single-agent and multi-agent autonomous driving environments. Three state-of-the-art contemporary actor-critic algorithms are used to create an extensive simulation in Unity3D. (Han et al., Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges,” arXiv (Aug 2023)) teaches a brief overview of collaboration schemes. After that, we systematically summarize the collaborative perception methods for ideal scenarios and real-world issues. The former focuses on collaboration modules and efficiency, and the latter is devoted to addressing the problems in actual application. Furthermore, we present largescale public datasets and summarize quantitative results on these benchmarks. Finally, we highlight gaps and overlook challenges between current academic research and real-world applications. 22. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEVIN L. SMITH whose telephone number is (571) 272-5964. Normally, the Examiner is available on Monday-Thursday 0730-1730. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, KAKALI CHAKI can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /K.L.S./ Examiner, Art Unit 2122 /KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122