CTFR 18/469,213 CTFR 97705 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 12-151 AIA 26-51 12-51 Status of Claims This communication is in response to the remarks and amendments filed 03/06/2026. Claims 1-30 are pending. Response to Remarks Applicant’s arguments with respect to independent claims 1, 12, 23 and 30 have been carefully and respectfully considered in light of the instant amendment, but are not persuasive. Accordingly, this action has been made FINAL . Claim Rejections - 35 USC § 103 On page 9 of the remarks filed 03/06/2026, Applicant argues “[t]he cited references do not teach or suggest ‘training the feature detection model using the DAR frame and the interpolated image frame’” and describes Ozbligin reference to indicate that “Ozbilgin does not train any feature detection model”. It is noted that independent claim(s) 1, 12, 23 and 30 were rejected as a 103 combination rejection of Ozbilgin and Singh. In response to Applicant’s arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. In re Keller , 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co. , 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Furthermore, on pages 9-10 of the remarks, Applicant argues “Singh does not disclose ‘training the feature detection model using the DAR frame and the interpolated frame’” and cites Office Action at Singh ¶13 to state “[t]his passage discloses that DNNs can be trained on various types of sensor data individually. It does not teach or suggest training a model using a DAR frame paired with a temporally-interpolated image frame, as claimed”. As noted above, the combination of Ozbilgin and Singh are used for the rejection of the independent claim(s). Specifically, ¶13 of Singh is explicit in teaching that a DNN (i.e., a feature detection model) can be trained with inputs such as image data from a camera, range data from a lidar or radar data from a radar sensor, to label and locate objects in the image data (i.e., feature detection). This passage alone and in combination with the teachings of Ozbilgin are enough to disclose the claimed limitations of “training the feature detection model using the DAR frame and the interpolated frame”. More specifically, Singh teaches that a DNN (or feature detection model) may be trained with inputs including DAR frames (lidar/radar frames) and interpolated image frames (i.e., synthetic images generated from camera frames, or image data). While Singh discloses image data from a camera (which may include interpolated image data and/or synthetic image data) as an input to a DNN, Ozbilgin further teaches synthesizing an interpolated frame at a temporally synchronized time of a point-cloud frame (i.e., DAR frame) at ¶25. Next, Applicant argues “Singh’s ¶47 describes spatial super-resolution interpolation, i.e., upscaling pixel resolution within a single image. This is categorically different from the claimed temporal interpolation between two camera frames captures at different times to create a new frame at an intermediate time” and cites the instant application’s PGPUB ¶32 which describes the image interpolation “can be designed to generate, as interpolated image frame 414, an image frame that represents a prediction of what a hypothetical image frame captured by camera 102 —at a nominal capture time between the actual respective capture times of the image frames 404A and 404B would look like”. As described above, Singh’s ¶47 describes a type of interpolation such that a DNN is capable of being trained using interpolated frames (in Singh’s case, synthetic images) with a DAR frame (e.g., radar, lidar, point cloud). However, because both Singh and Ozbilgin are directed to a sensor fusion system to detect features (e.g., objects in an environment), and Ozbilgin additionally includes teaching an interpolated frame at a temporally synchronized time of a point-cloud, the combination of Ozbilgin and Singh teach the claimed limitations as described. Finally, Applicant argues “Singh at ¶35 and 36 ‘synthetic images’ are computer generated simulated images enhanced by a GAN to appear more photorealistic... In contrast, the claimed ‘interpolated image frame’ is produced by temporally interpolating between two real-world camera frames captured at different times to synthesize a frame time-aligned with a DAR frame. Singh’s synthetic images originate from a simulation engine. In contrast, the claimed interpolated image frames originate from real-world camera captures”. As shown above, Singh at ¶35 and 36 describe “synthetic images” which are a form of interpolated images as they both originate from a source different from the original captured image(s). That is, Singh discloses the use of “synthetic images” to train a DNN and with the combination of Ozbilgin to teach an interpolated frame at a temporally synchronized time of a point-cloud obtained from a camera, radar and/or lidar units, the combination of Ozbilgin and Singh teach the claimed limitations as described. On page 11 of the remarks, Applicant argues “the Office Action relies on two different concepts in Singh and treats both as equivalent to the claimed interpolation [and] submits that the Office Action conflates categorically different operations”. The Examiner respectfully disagrees. As stated above, Singh’s describes a type of interpolation such that a DNN is capable of being trained using interpolated frames (in Singh’s case, synthetic images) with a DAR frame and the addition of Ozbilgin further teaching an interpolated frame at a temporally synchronized time (interpolated image frame) of a point-cloud (i.e., a DAR frame), the combination of Ozbilgin in view of Singh disclose the claimed limitations of the independent claim(s) Therefore, the argued limitations were written broad such that they read upon the cited references or are shown explicitly by the references. As a result, the claims stand rejected as follows. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1-3, 6-7, 9-14, 17-18, 20-25, 27-28 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Ozbilgin et al. (US 20220390957 A1) in view of Singh et al. (US 20220388535 A1) . Regarding claim 1, Ozbilgin discloses a method for multi-sensor training of a feature detection model, comprising: obtaining a detection-and-ranging (DAR) frame captured by a DAR sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present” Ozbilgin, [0019]) ; obtaining a first image frame captured at a first time by a camera and a second image frame captured at a second time by the camera (“The system 10 includes a first-sensor 20, for example, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The first-sensor 20 is configured to output a first-frame 22 of data and a subsequent-frame 26 of data indicative of objects 18 present in a first-field-of-view 36... The first-frame 22 is characterized by a first-time-stamp 24 (T1 in FIG. 2A), and the subsequent-frame 26 of data characterized by a subsequent-time-stamp 28 (Ts in FIG. 2B) that is different from the first-time-stamp 24.” Ozbilgin, [0018]) , wherein at a capture time of the DAR frame, a field-of-view (FOV) of the DAR sensor overlaps an FOV of the camera (“The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present in a second-field-of-view 38 that overlaps (partially or fully, covering an area larger than or smaller than) the first-field-of-view 36.” Ozbilgin, [0019]) , and wherein the capture time of the DAR frame is between the first time and the second time (“The second-frame 32 is characterized by a second-time-stamp 34 (T2 in FIG. 3B) that is temporally located between the first-time-stamp 24 and the subsequent-time-stamp 28” Ozbilgin, [0020]; “For example, the second-time-stamp 34 may correspond to an instant in time that is after the first-time-stamp 24 and before the subsequent-time-stamp 28.” Ozbilgin, [0021]) ; interpolating based on the first image frame and the second image frame to create an interpolated image frame, wherein a nominal capture time of the interpolated image frame corresponds to the capture time of the DAR frame (“synthesize the interpolated-frame 52 from the first-frame 22 and the subsequent-frame 26. The interpolated-frame 52 characterized by the interpolated-time-stamp 54 that corresponds to the second-time-stamp 34. I.e., the interpolated-time-stamp 54 and the second-time-stamp 34 are essentially or approximately or exactly equal so that image of the interpolated-frame 52 is temporally synchronized with the point-cloud of the second-frame 32.” Ozbilgin, [0025]) . Ozbilgin discloses all of the subject matter as described above except for specifically teaching training the feature detection model using the DAR frame and the interpolated image frame . However, Singh in the same field of endeavor teaches training the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; where a DNN may include super resolution training by interpolating image details between pixels of an input image, see Singh, [0047] “inputting the image to a deep neural network trained to increase image resolution by interpolating image details between the pixels of an input image.”; Additionally, Singh at [0035] includes the use of using synthetic images (i.e., interpolated image frames) to train the DNN, “A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302. Photorealistic features are portions of an image that make the image look more like it was acquired using a real-world camera viewing a real-world scene.”; and [0036]). Therefore, it would have been obvious to one of ordinary skill in the art to combine Ozbilgin and Singh before the effective filing date of the claimed invention. The motivation for this combination of references would have been to increase realism using the synthetic images to be used to train a DNN to detect objects reliably in real world images (Singh, [0036]) thereby reducing sensor motion data between detections. This motivation for the combination of Ozbilgin and Singh is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III). Regarding claim 2, Ozbilgin and Singh disclose the method of claim 1, wherein the interpolating based on the first image frame and the second image frame includes interpolating using a generative machine-learning model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 3, Ozbilgin and Singh disclose the method of claim 2, wherein the generative machine-learning model is a generative adversarial network (GAN) model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 6, Ozbilgin and Singh disclose the method of claim 1, wherein the feature detection model is a DAR-based detection model, and training the feature detection model using the DAR frame and the interpolated image frame includes (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) : detecting a feature in the interpolated image frame using an image-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; annotating the interpolated image frame to indicate a location of the feature in the interpolated image frame (“ DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]; and [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) ; and training the DAR-based detection model based on the DAR frame and the annotated interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 7, Ozbilgin and Singh disclose the method of claim 1, wherein the feature detection model is an image-based detection model, and training the feature detection model using the DAR frame and the interpolated image frame includes (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013], [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) : detecting a feature in the DAR frame using a DAR-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; annotating the DAR frame to indicate a location of the feature in the DAR frame (“ DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]) ; and training the image-based detection model based on the interpolated image frame and the annotated DAR frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013], [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 9, Ozbilgin and Singh disclose the method of claim 1, wherein the DAR sensor is a radio detection and ranging (radar) sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 10, Ozbilgin and Singh disclose the method of claim 1, wherein the DAR sensor is a light detection and ranging (lidar) sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 11, Ozbilgin and Singh disclose the method of claim 1, wherein the camera and the DAR sensor are sensors of a vehicle (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 12, Ozbilgin and Singh disclose an apparatus for multi-sensor training of a feature detection model, comprising: at least one processor (“a processor 42 such as one or more instances of a microprocessor or other control circuitry such as analog and/or digital control circuitry including an application specific integrated circuit (ASIC)” Ozbilgin, [0022]) ; and at least one memory communicatively coupled with the at least one processor and storing processor-readable code that, when executed by the at least one processor, is configured to (Ozbilgin, [0022]) : obtain a detection-and-ranging (DAR) frame captured by a DAR sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present” Ozbilgin, [0019]) ; obtain a first image frame captured at a first time by a camera and a second image frame captured at a second time by the camera (“The system 10 includes a first-sensor 20, for example, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The first-sensor 20 is configured to output a first-frame 22 of data and a subsequent-frame 26 of data indicative of objects 18 present in a first-field-of-view 36... The first-frame 22 is characterized by a first-time-stamp 24 (T1 in FIG. 2A), and the subsequent-frame 26 of data characterized by a subsequent-time-stamp 28 (Ts in FIG. 2B) that is different from the first-time-stamp 24.” Ozbilgin, [0018]) , wherein at a capture time of the DAR frame, a field-of-view (FOV) of the DAR sensor overlaps an FOV of the camera (“The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present in a second-field-of-view 38 that overlaps (partially or fully, covering an area larger than or smaller than) the first-field-of-view 36.” Ozbilgin, [0019]) , and wherein the capture time of the DAR frame is between the first time and the second time (“The second-frame 32 is characterized by a second-time-stamp 34 (T2 in FIG. 3B) that is temporally located between the first-time-stamp 24 and the subsequent-time-stamp 28” Ozbilgin, [0020]; “For example, the second-time-stamp 34 may correspond to an instant in time that is after the first-time-stamp 24 and before the subsequent-time-stamp 28.” Ozbilgin, [0021]) ; interpolate based on the first image frame and the second image frame to create an interpolated image frame, wherein a nominal capture time of the interpolated image frame corresponds to the capture time of the DAR frame (“synthesize the interpolated-frame 52 from the first-frame 22 and the subsequent-frame 26. The interpolated-frame 52 characterized by the interpolated-time-stamp 54 that corresponds to the second-time-stamp 34. I.e., the interpolated-time-stamp 54 and the second-time-stamp 34 are essentially or approximately or exactly equal so that image of the interpolated-frame 52 is temporally synchronized with the point-cloud of the second-frame 32.” Ozbilgin, [0025]) ; and train the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; where a DNN may include super resolution training by interpolating image details between pixels of an input image, see Singh, [0047] “inputting the image to a deep neural network trained to increase image resolution by interpolating image details between the pixels of an input image.”; Additionally, Sign at [0035] includes the use of using synthetic images (i.e., interpolated image frames) to train the DNN, “A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302. Photorealistic features are portions of an image that make the image look more like it was acquired using a real-world camera viewing a real-world scene.”; and [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 13, Ozbilgin and Singh disclose the apparatus of claim 12, wherein to interpolate based on the first image frame and the second image frame, the processor-readable code is, when executed by the at least one processor, configured to interpolate using a generative machine-learning model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 14, Ozbilgin and Singh disclose the apparatus of claim 13, wherein the generative machine-learning model is a generative adversarial network (GAN) model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 17, Ozbilgin and Singh disclose the apparatus of claim 12, wherein the feature detection model is a DAR-based detection model, and wherein to train the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) , the processor-readable code is, when executed by the at least one processor, configured to: detect a feature in the interpolated image frame using an image-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; annotate the interpolated image frame to indicate a location of the feature in the interpolated image frame (“ DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]; and [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) ; and train the DAR-based detection model based on the DAR frame and the annotated interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 18, Ozbilgin and Singh disclose the apparatus of claim 12, wherein the feature detection model is an image-based detection model, and wherein to train the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013], [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) , the processor-readable code is, when executed by the at least one processor, configured to: detect a feature in the DAR frame using a DAR-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; annotate the DAR frame to indicate a location of the feature in the DAR frame (“ DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]) ; and train the image-based detection model based on the interpolated image frame and the annotated DAR frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013], [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 20, Ozbilgin and Singh disclose the apparatus of claim 12, wherein the DAR sensor is a radio detection and ranging (radar) sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 21, Ozbilgin and Singh disclose the apparatus of claim 12, wherein the DAR sensor is a light detection and ranging (lidar) sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 22, Ozbilgin and Singh disclose the apparatus of claim 12, wherein the camera and the DAR sensor are sensors of a vehicle (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12.” Ozbilgin, [0019]) . Regarding claim 23, Ozbilgin and Singh disclose an apparatus for multi-sensor training of a feature detection model, comprising: means for obtaining a detection-and-ranging (DAR) frame captured by a DAR sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present” Ozbilgin, [0019]) ; means for obtaining a first image frame captured at a first time by a camera and a second image frame captured at a second time by the camera (“The system 10 includes a first-sensor 20, for example, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The first-sensor 20 is configured to output a first-frame 22 of data and a subsequent-frame 26 of data indicative of objects 18 present in a first-field-of-view 36... The first-frame 22 is characterized by a first-time-stamp 24 (T1 in FIG. 2A), and the subsequent-frame 26 of data characterized by a subsequent-time-stamp 28 (Ts in FIG. 2B) that is different from the first-time-stamp 24.” Ozbilgin, [0018]) , wherein at a capture time of the DAR frame, a field-of-view (FOV) of the DAR sensor overlaps an FOV of the camera (“The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present in a second-field-of-view 38 that overlaps (partially or fully, covering an area larger than or smaller than) the first-field-of-view 36.” Ozbilgin, [0019]) , and wherein the capture time of the DAR frame is between the first time and the second time (“The second-frame 32 is characterized by a second-time-stamp 34 (T2 in FIG. 3B) that is temporally located between the first-time-stamp 24 and the subsequent-time-stamp 28” Ozbilgin, [0020]; “For example, the second-time-stamp 34 may correspond to an instant in time that is after the first-time-stamp 24 and before the subsequent-time-stamp 28.” Ozbilgin, [0021]) ; means for interpolating based on the first image frame and the second image frame to create an interpolated image frame, wherein a nominal capture time of the interpolated image frame corresponds to the capture time of the DAR frame (“synthesize the interpolated-frame 52 from the first-frame 22 and the subsequent-frame 26. The interpolated-frame 52 characterized by the interpolated-time-stamp 54 that corresponds to the second-time-stamp 34. I.e., the interpolated-time-stamp 54 and the second-time-stamp 34 are essentially or approximately or exactly equal so that image of the interpolated-frame 52 is temporally synchronized with the point-cloud of the second-frame 32.” Ozbilgin, [0025]) ; and means for training the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; where a DNN may include super resolution training by interpolating image details between pixels of an input image, see Singh, [0047] “inputting the image to a deep neural network trained to increase image resolution by interpolating image details between the pixels of an input image.”; Additionally, Sign at [0035] includes the use of using synthetic images (i.e., interpolated image frames) to train the DNN, “A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302. Photorealistic features are portions of an image that make the image look more like it was acquired using a real-world camera viewing a real-world scene.”; and [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 24, Ozbilgin and Singh disclose the apparatus of claim 23, wherein the means for interpolating based on the first image frame and the second image frame includes means for interpolating using a generative machine-learning model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 25, Ozbilgin and Singh disclose the apparatus of claim 24, wherein the generative machine-learning model is a generative adversarial network (GAN) model (“A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302.” Singh, [0035]; “The increased realism included in the synthetic image 310 can permit the synthetic images 310 to be used to train a DNN 200 to detect objects reliably in real world images.” Singh, [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 27, Ozbilgin and Singh disclose the apparatus of claim 23, wherein the feature detection model is a DAR-based detection model, and the means for training the feature detection model using the DAR frame and the interpolated image frame includes (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) : means for detecting a feature in the interpolated image frame using an image-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; means for annotating the interpolated image frame to indicate a location of the feature in the interpolated image frame (“DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]; and [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) ; and means for training the DAR-based detection model based on the DAR frame and the annotated interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 28, Ozbilgin and Singh disclose the apparatus of claim 23, wherein the feature detection model is an image-based detection model, and the means for training the feature detection model using the DAR frame and the interpolated image frame includes (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; [0036]; [0064] “The synthetic images 310 can be added to the training dataset to provide additional images for training a DNN 200”) : means for detecting a feature in the DAR frame using a DAR-based detection model (The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]) ; means for annotating the DAR frame to indicate a location of the feature in the DAR frame (“ DNN 200 can be trained using a training dataset that includes images and corresponding ground truth. Training datasets for a DNN 200 can include thousands or millions of images and corresponding annotations or ground truth.” Singh, [0033], [0036]) ; and means for training the image-based detection model based on the interpolated image frame and the annotated DAR frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013], [0036]; [0064]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1. Regarding claim 30, Ozbilgin and Singh disclose a non-transitory computer-readable medium (“non-transitory computer-readable storage-medium” Ozbilgin, [0022]) storing instructions for multi-sensor training of a feature detection model, the instructions including code to: obtain a detection-and-ranging (DAR) frame captured by a DAR sensor (“The system 10 also includes a second-sensor 30, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present” Ozbilgin, [0019]) ; obtain a first image frame captured at a first time by a camera and a second image frame captured at a second time by the camera (“The system 10 includes a first-sensor 20, for example, e.g. a camera, a radar-unit or a lidar-unit, that is mounted on the host-vehicle 12. The first-sensor 20 is configured to output a first-frame 22 of data and a subsequent-frame 26 of data indicative of objects 18 present in a first-field-of-view 36... The first-frame 22 is characterized by a first-time-stamp 24 (T1 in FIG. 2A), and the subsequent-frame 26 of data characterized by a subsequent-time-stamp 28 (Ts in FIG. 2B) that is different from the first-time-stamp 24.” Ozbilgin, [0018]) , wherein at a capture time of the DAR frame, a field-of-view (FOV) of the DAR sensor overlaps an FOV of the camera (“The second-sensor 30 is configured to output a second-frame 32 of data indicative of objects 18 present in a second-field-of-view 38 that overlaps (partially or fully, covering an area larger than or smaller than) the first-field-of-view 36.” Ozbilgin, [0019]) , and wherein the capture time of the DAR frame is between the first time and the second time (“The second-frame 32 is characterized by a second-time-stamp 34 (T2 in FIG. 3B) that is temporally located between the first-time-stamp 24 and the subsequent-time-stamp 28” Ozbilgin, [0020]; “For example, the second-time-stamp 34 may correspond to an instant in time that is after the first-time-stamp 24 and before the subsequent-time-stamp 28.” Ozbilgin, [0021]) ; interpolate based on the first image frame and the second image frame to create an interpolated image frame, wherein a nominal capture time of the interpolated image frame corresponds to the capture time of the DAR frame (“synthesize the interpolated-frame 52 from the first-frame 22 and the subsequent-frame 26. The interpolated-frame 52 characterized by the interpolated-time-stamp 54 that corresponds to the second-time-stamp 34. I.e., the interpolated-time-stamp 54 and the second-time-stamp 34 are essentially or approximately or exactly equal so that image of the interpolated-frame 52 is temporally synchronized with the point-cloud of the second-frame 32.” Ozbilgin, [0025]) ; and train the feature detection model using the DAR frame and the interpolated image frame (“acquire image data regarding the external environment of a vehicle and detect objects in the image data using a DNN. The data can include image data acquired from a still or video camera, range data acquired from a lidar sensor or radar data acquired from a radar sensor. A DNN can be trained to label and locate objects in the image data, range data, or radar data.” Singh, [0013]; where a DNN may include super resolution training by interpolating image details between pixels of an input image, see Singh, [0047] “inputting the image to a deep neural network trained to increase image resolution by interpolating image details between the pixels of an input image.”; Additionally, Sign at [0035] includes the use of using synthetic images (i.e., interpolated image frames) to train the DNN, “A typically more efficient technique for increasing the photorealism of a simulated image 302 that yields synthetic images 310 that are include sufficient photorealism to be used to train a DNN 200 is to use a GAN 300 to add photorealistic features to simulated images 302. Photorealistic features are portions of an image that make the image look more like it was acquired using a real-world camera viewing a real-world scene.”; and [0036]) . Therefore, combining Ozbilgin and Singh would meet the claim limitations for the same reasons as previously discussed in claim 1 . 07-21-aia AIA Claim (s) 4-5, 8, 15-16, 19, 26 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Ozbilgin et al. in view of Singh et al. in further view of Wisth et al. (“Unified multi-modal landmark tracking for tightly coupled lidar-visual-inertial odometry”, 2021) . Regarding claim 4 , the combination of Ozbilgin and Singh as whole does not expressly disclose an image capture timing gradient across a dimension in the interpolated image frame based on a DAR capture timing gradient associated with the dimension in the DAR frame . However, Wisth in the same field of endeavor teaches wherein the interpolating based on the first image frame and the second image frame includes implementing an image capture timing gradient across a dimension in the interpolated image frame based on a DAR capture timing gradient associated with the dimension in the DAR frame (“An efficient method for extracting lidar features, which are then optimized as landmarks. Both lidar and visual features share a unified representation, as the landmarks are all treated as n-dimensional parametric manifolds (i.e., points, lines and planes). This compact representation allows us to process all the lidar scans at nominal framerate” Wisth, pg. 2 Col 1. under B. Contribution ; Fig. 6) . Therefore, it would have been obvious to one of ordinary skill in the art to combine Ozbilgin , Singh and Wisth before the effective filing date of the claimed invention. The motivation for this combination of references would have been to obtain different output frequences of sensors and synchronization between them such that an optimization of features can be executed for all sensors at once (Wisth, pg. 5 Col 2). This motivation for the combination of Ozbilgin , Singh and Wisth is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III). Regarding claim 5, Ozbilgin , Singh and Wisth disclose the method of claim 4, wherein the DAR capture timing gradient is associated with a motion rate of the FOV of the DAR sensor with respect to the dimension (“To limit drift and factor graph growth when the platform is stationary, we add zero velocity constraints to the graph when updates from two out of three modalities (camera, lidar, IMU) report no motion.” Wisth, pg. 6 Col 2) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 8, Ozbilgin , Singh and Wisth disclose the method of claim 1, further comprising detecting a feature in a joint FOV of the camera and the DAR sensor based on the DAR frame and the interpolated image frame, using a multi-sensor fusion (MSF) feature detection model (Multi-modal sensor fusion, includes fusion with sensor such as lidar and camera, Wisth pg. 1 Col 1-2; pg. 7 under C. Multi-Sensor Fusion ) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 15, Ozbilgin , Singh and Wisth disclose the apparatus of claim 12, wherein to interpolate based on the first image frame and the second image frame, the processor-readable code is, when executed by the at least one processor, configured to implement an image capture timing gradient across a dimension in the interpolated image frame based on a DAR capture timing gradient associated with the dimension in the DAR frame (“An efficient method for extracting lidar features, which are then optimized as landmarks. Both lidar and visual features share a unified representation, as the landmarks are all treated as n-dimensional parametric manifolds (i.e., points, lines and planes). This compact representation allows us to process all the lidar scans at nominal framerate” Wisth, pg. 2 Col 1. under B. Contribution ; Fig. 6) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 16, Ozbilgin , Singh and Wisth disclose the apparatus of claim 15, wherein the DAR capture timing gradient is associated with a motion rate of the FOV of the DAR sensor with respect to the dimension (“To limit drift and factor graph growth when the platform is stationary, we add zero velocity constraints to the graph when updates from two out of three modalities (camera, lidar, IMU) report no motion.” Wisth, pg. 6 Col 2) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 19, Ozbilgin , Singh and Wisth disclose the apparatus of claim 12, wherein the processor-readable code is, when executed by the at least one processor, further configured to detect a feature in a joint FOV of the camera and the DAR sensor based on the DAR frame and the interpolated image frame, using a multi-sensor fusion (MSF) feature detection model (Multi-modal sensor fusion, includes fusion with sensor such as lidar and camera, Wisth pg. 1 Col 1-2; pg. 7 under C. Multi-Sensor Fusion ) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 26, Ozbilgin , Singh and Wisth disclose the apparatus of claim 23, wherein the means for interpolating based on the first image frame and the second image frame includes means for implementing an image capture timing gradient across a dimension in the interpolated image frame based on a DAR capture timing gradient associated with the dimension in the DAR frame (“An efficient method for extracting lidar features, which are then optimized as landmarks. Both lidar and visual features share a unified representation, as the landmarks are all treated as n-dimensional parametric manifolds (i.e., points, lines and planes). This compact representation allows us to process all the lidar scans at nominal framerate” Wisth, pg. 2 Col 1. under B. Contribution ; Fig. 6) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Regarding claim 29, Ozbilgin , Singh and Wisth disclose the apparatus of claim 23, further comprising means for detecting a feature in a joint FOV of the camera and the DAR sensor based on the DAR frame and the interpolated image frame, using a multi-sensor fusion (MSF) feature detection model (Multi-modal sensor fusion, includes fusion with sensor such as lidar and camera, Wisth pg. 1 Col 1-2; pg. 7 under C. Multi-Sensor Fusion ) . Therefore, combining Ozbilgin, Singh and Wisth would meet the claim limitations for the same reasons as previously discussed in claim 4. Conclusion 07-39 AIA THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Inquiries Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMANUEL SILVA-AVINA whose telephone number is (571)270-0729. The examiner can normally be reached Monday - Friday 11 AM - 8 PM EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /EMMANUEL SILVA-AVINA/Examiner, Art Unit 2673 /CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673 Application/Control Number: 18/469,213 Page 2 Art Unit: 2673 Application/Control Number: 18/469,213 Page 3 Art Unit: 2673 Application/Control Number: 18/469,213 Page 4 Art Unit: 2673
Prosecution Insights