DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is responsive to Applicant’s remarks received on May 08, 2026. Claims 1-20 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7, 8, 11-14 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Anumula et al. (US 11,533,484) and Scheunert et al. (“Generalized Grid Framework for Multi Sensor Data Fusion”).
Regarding claim 1, Anumula et al. discloses a system, comprising:
a processor coupled to a memory that stores instructions executable by the processor (“In its most basic configuration, the operating environment 1200 typically includes at least one processing unit 1202 and memory 1204. Depending on the exact configuration and type of computing device, memory 1204 (instructions for encoding and/or optimizing as disclosed herein)” at col. 12, line 51) to:
obtain a camera image to include a scene (“data from the camera sensor” at col. 3, line 39);
identify, from non-camera sensor data, an area of interest in the scene (“The vehicle 110 may further comprise a processor configured to receive data from the multiple sensors 120 and to process the data before encoding the data” at col. 3, line 36; “The set of multiple sensors 120 may include a camera, a LIDAR, a radar, a time-of-flight device and other sensors and devices that may be used for observing the environment of the vehicle 110” at col. 3, line 31);
identify, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest and a second portion of the camera image that excludes the area of interest (“Each of the detectors 310, or alternatively a subset of the detectors, may generate an ROI probability map, for example, at pixel level. These probability maps may be converted, by a ROI adaption module 320, into a single ROI-id map at pixel-block level” at col. 5, line 9; areas not identified as part of an ROI is therefore the second portion); and
apply a first compression rule to the first portion of the camera image and a second compression rule to the second portion of the camera image, wherein the first compression rule is less lossy than the second compression rule (“For example, a pixel block within an ROI may be compressed less aggressively than a pixel block that is not within the ROI” at col. 5, line 49; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI” at col. 8, line 19).
Anumula et al. does not explicitly disclose that the area of interest is determined by forming a point group based on the non-camera sensor data, the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors, and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data.
Scheunert et al. teaches a system in the same field of endeavor of object detection, comprising:
a processor coupled to a memory that stores instructions executable by the processor (implied that there is a programmed processor in the vehicle to perform the method) to:
identify, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest, wherein the area of interest is determined by forming a point group based on the non-camera sensor data (“As the measurement grid is typically sensor specific, an alignment of the grid is necessary to allow the usage of a standardized update operation” at page 815, left column, second full paragraph, line 5; “The accumulation grid A stores the relative position vxA.” at page 815, right column, line 7), the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors (“As figure 4 shows the detection cells of radar and laser are determined by the polar coordinate system” at page 815, left column, third full paragraph, line 8), and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data (as demonstrated in Figures 3 and 4, the aligned measurement grid takes all of the sensor grids to align them into the same coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the grid alignment as taught by Scheunert et al. in defining the ROI of Anumula et al. to incorporate resolution and sensor uncertainty in determination of the object detection model (see Scheunert et al. at page 815, left column, second and third full paragraphs).
Regarding claim 12, Anumula et al. discloses a method, comprising:
obtaining a camera image to include a scene (“data from the camera sensor” at col. 3, line 39);
identifying, from non-camera sensor data, an area of interest in the scene (“The vehicle 110 may further comprise a processor configured to receive data from the multiple sensors 120 and to process the data before encoding the data” at col. 3, line 36; “The set of multiple sensors 120 may include a camera, a LIDAR, a radar, a time-of-flight device and other sensors and devices that may be used for observing the environment of the vehicle 110” at col. 3, line 31);
identifying, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest and a second portion of the camera image that excludes the area of interest (“Each of the detectors 310, or alternatively a subset of the detectors, may generate an ROI probability map, for example, at pixel level. These probability maps may be converted, by a ROI adaption module 320, into a single ROI-id map at pixel-block level” at col. 5, line 9; areas not identified as part of an ROI is therefore the second portion); and
applying a first compression rule to the first portion of the camera image and a second compression rule to the second portion of the camera image, wherein the first compression rule is less lossy than the second compression rule (“For example, a pixel block within an ROI may be compressed less aggressively than a pixel block that is not within the ROI” at col. 5, line 49; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI” at col. 8, line 19).
Anumula et al. does not explicitly disclose that the area of interest is determined by forming a point group based on the non-camera sensor data, the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors, and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data.
Scheunert et al. teaches a method in the same field of endeavor of object detection, comprising:
identifying, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest, wherein the area of interest is determined by forming a point group based on the non-camera sensor data (“As the measurement grid is typically sensor specific, an alignment of the grid is necessary to allow the usage of a standardized update operation” at page 815, left column, second full paragraph, line 5; “The accumulation grid A stores the relative position vxA.” at page 815, right column, line 7), the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors (“As figure 4 shows the detection cells of radar and laser are determined by the polar coordinate system” at page 815, left column, third full paragraph, line 8), and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data (as demonstrated in Figures 3 and 4, the aligned measurement grid takes all of the sensor grids to align them into the same coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the grid alignment as taught by Scheunert et al. in defining the ROI of Anumula et al. to incorporate resolution and sensor uncertainty in determination of the object detection model (see Scheunert et al. at page 815, left column, second and third full paragraphs).
Regarding claim 17, Anumula et al. discloses an article, comprising:
a non-transitory computer-readable media having instructions encoded thereon which, when executed by a processor coupled to at least one memory are operable (“In its most basic configuration, the operating environment 1200 typically includes at least one processing unit 1202 and memory 1204. Depending on the exact configuration and type of computing device, memory 1204 (instructions for encoding and/or optimizing as disclosed herein)” at col. 12, line 51) to:
obtain a camera image to include a scene (“data from the camera sensor” at col. 3, line 39);
identify, from non-camera sensor data, an area of interest in the scene (“The vehicle 110 may further comprise a processor configured to receive data from the multiple sensors 120 and to process the data before encoding the data” at col. 3, line 36; “The set of multiple sensors 120 may include a camera, a LIDAR, a radar, a time-of-flight device and other sensors and devices that may be used for observing the environment of the vehicle 110” at col. 3, line 31);
identify, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest and a second portion of the camera image that excludes the area of interest (“Each of the detectors 310, or alternatively a subset of the detectors, may generate an ROI probability map, for example, at pixel level. These probability maps may be converted, by a ROI adaption module 320, into a single ROI-id map at pixel-block level” at col. 5, line 9; areas not identified as part of an ROI is therefore the second portion); and
apply a first compression rule to the first portion of the camera image and a second compression rule to the second portion of the camera image, wherein the first compression rule is less lossy than the second compression rule (“For example, a pixel block within an ROI may be compressed less aggressively than a pixel block that is not within the ROI” at col. 5, line 49; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI” at col. 8, line 19).
Anumula et al. does not explicitly disclose that the area of interest is determined by forming a point group based on the non-camera sensor data, the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors, and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data.
Scheunert et al. teaches an article in the same field of endeavor of object detection, comprising:
a non-transitory computer-readable media having instructions encoded thereon which, when executed by a processor coupled to at least one memory are operable (implied that there is a programmed processor in the vehicle to perform the method) to:
identify, based on the non-camera sensor data, a first portion of the camera image that includes the area of interest, wherein the area of interest is determined by forming a point group based on the non-camera sensor data (“As the measurement grid is typically sensor specific, an alignment of the grid is necessary to allow the usage of a standardized update operation” at page 815, left column, second full paragraph, line 5; “The accumulation grid A stores the relative position vxA.” at page 815, right column, line 7), the point group including points that are determined by respective range and/or velocity measurements from one or more non-camera sensors (“As figure 4 shows the detection cells of radar and laser are determined by the polar coordinate system” at page 815, left column, third full paragraph, line 8), and that correspond to respective pixel coordinates utilizing a coordinate system that is common to non-camera sensors providing the non-camera sensor data and the camera providing the image data (as demonstrated in Figures 3 and 4, the aligned measurement grid takes all of the sensor grids to align them into the same coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the grid alignment as taught by Scheunert et al. in defining the ROI of Anumula et al. to incorporate resolution and sensor uncertainty in determination of the object detection model (see Scheunert et al. at page 815, left column, second and third full paragraphs).
Regarding claim 2, Anumula et al. discloses a system wherein first compression rule operates to:
decrease compression of the first portion of the camera image (“For example, a pixel block within an ROI may be compressed less aggressively than a pixel block that is not within the ROI” at col. 5, line 49; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI” at col. 8, line 19).
Regarding claim 3, the Anumula et al. and Scheunert et al. combination discloses a system wherein first compression rule operates to:
decrease compression of the first portion of the camera image (“For example, a pixel block within an ROI may be compressed less aggressively than a pixel block that is not within the ROI” at col. 5, line 49; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI” Anumula et al. at col. 8, line 19).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose decreasing compression nearby the first portion of the camera image.
However, using Figure 2 of Anumula et al. as reference, if there are two ROI objects of importance adjacent to each other, such as two vehicles 204 in adjacent lanes ahead of the vehicle 110 of Anumula et al., both of these ROIs would be subject to lossless compression such as described in conjunction with claim 13 below.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to apply the same compression rule to both ROIs to ensure that both objects of interest are not subject to loss in quality.
Regarding claim 7, Anumula et al. discloses a system wherein the first compression rule is to apply zero or negligible compression responsive to determination that the area of interest includes a moving vehicle, a moving pedestrian, a moving bicyclist (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles” at col. 4, line 45), a moving motorcycle, a moving natural object, or a moving animal.
Regarding claim 8, Anumula et al. discloses a system wherein the first compression rule is to apply non-zero or non-negligible compression responsive to a determination that the area of interest includes a stationary vehicle, a stationary pedestrian, a stationary bicyclist, a stationary natural object (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles, lane/road markers, etc. in a video stream may be more important than objects like sky, vegetation or buildings etc. Therefore, not all of the objects in a video frame are of same importance and different compression/quality factors (depending upon the importance of the objects) may be chosen while compressing the video frame” at col. 4, line 45), or a stationary animal.
Regarding claim 11, Anumula et al. discloses a system wherein the second compression rule operates to:
apply lossy compression in the second portion of the camera image that includes free space (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles, lane/road markers, etc. in a video stream may be more important than objects like sky, vegetation or buildings etc. Therefore, not all of the objects in a video frame are of same importance and different compression/quality factors (depending upon the importance of the objects) may be chosen while compressing the video frame” at col. 4, line 45; “In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI. In other words, a quantization parameter may be computed for each pixel-block to preserve target quality in the ROI regions and aggressive compression elsewhere to meet the target bitrate.” at col. 8, line 19; non-ROI areas include background areas where there are no objects of interest, such as the sky region at numeral 207 in Figure 2).
Regarding claim 13, Anumula et al. discloses a method further comprising:
applying the first compression rule to include zero or negligible compression of the first portion of the camera image (“In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI. In other words, a quantization parameter may be computed for each pixel-block to preserve target quality in the ROI regions and aggressive compression elsewhere to meet the target bitrate. For almost lossless compression, a high value is chosen for the quantization parameter, resulting in a smaller quantization step size” at col. 8, line 19).
Regarding claim 14, the Anumula et al. and Scheunert et al. combination discloses a method as described in claim 12 above.
The Anumula et al. and Scheunert et al. combination does not explicitly disclose applying the first compression rule to bring about zero or negligible compression of an area nearby the first portion of the camera image.
However, using Figure 2 of Anumula et al. as reference, if there are two ROI objects of importance adjacent to each other, such as two vehicles 204 in adjacent lanes ahead of the vehicle 110 of Anumula et al., both of these ROIs would be subject to lossless compression such as described in conjunction with claim 13 above.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to apply the same compression rule to both ROIs to ensure that both objects of interest are not subject to loss in quality.
Claim(s) 4-6, 15, 16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Anumula et al. and Scheunert et al. and further in view of Yonezawa (US 11,281,927).
Regarding claims 4, 15 and 18, the Anumula et al. and Scheunert et al. combination discloses the elements of claims 1, 12 and 17 as described above.
The Anumula et al. and Scheunert et al. combination does not explicitly disclose that the encoded instructions are additionally to: determine whether the first portion of the camera image includes a moving object or includes a stationary object.
Yonezawa teaches a system, method and article in the same field of endeavor of ROI correlated variable quantization, wherein the encoded instructions are additionally to:
determine whether the first portion of the camera image (“In step S530, processing regarding setting of a specific object ROI (addition or deletion of a specific object ROI) is performed” at col. 7, line 39) includes a moving object or includes a stationary object (“In step S540, the second detection unit 214 detects a moving object from the selected frame image. Detection of a moving object from the frame image may be performed for each frame, or may be performed every few frames. A moving object may be detected from a range using the entire selected frame image as the range, or may be detected from a range using the specific object ROI set by the region generation unit 213 as the range” at col. 9, line 51).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a moving object estimation as taught by Yonezawa in the system of the Anumula et al. and Scheunert et al. combination to prioritize particular motion objects as objects of interest (see Yonezawa at col. 11, lines 30-36 that states moving objects for the ROI have a lower QP compression QP). Therefore, non-motion objects can still be objects of interest but designated less important in accordance with the above cited disclosure of Anumula et al.
Regarding claim 5, the Anumula et al. and Scheunert et al. combination discloses a system wherein the first compression rule is to apply a first level of decreased compression responsive to a determination that the first portion of the camera image indicates presence of an object, and wherein the first compression rule is to apply a second level of decreased compression responsive to a determination that the first portion of the camera image indicates presence of another object (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles, lane/road markers, etc. in a video stream may be more important than objects like sky, vegetation or buildings etc. Therefore, not all of the objects in a video frame are of same importance and different compression/quality factors (depending upon the importance of the objects) may be chosen while compressing the video frame” Anumula et al. at col. 4, line 45).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose determining presence of a moving object and presence of a stationary object to determine compression level.
Yonezawa teaches a system in the same field of endeavor of ROI correlated variable quantization, wherein the encoded instructions are additionally to:
determine whether the first portion of the camera image (“In step S530, processing regarding setting of a specific object ROI (addition or deletion of a specific object ROI) is performed” at col. 7, line 39) includes a moving object or includes a stationary object (“In step S540, the second detection unit 214 detects a moving object from the selected frame image. Detection of a moving object from the frame image may be performed for each frame, or may be performed every few frames. A moving object may be detected from a range using the entire selected frame image as the range, or may be detected from a range using the specific object ROI set by the region generation unit 213 as the range” at col. 9, line 51).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a moving object estimation as taught by Yonezawa in the system of the Anumula et al. and Scheunert et al. combination to prioritize particular motion objects as the most important objects of interest (see Yonezawa at col. 11, lines 30-36 that states moving objects for the ROI have a lower QP compression QP). Therefore, non-motion objects can still be objects of interest but designated less important in accordance with the above cited disclosure of Anumula et al.
Regarding claims 6 and 19, the Anumula et al. and Scheunert et al. combination discloses a system and article wherein the encoded instructions are additionally to:
apply zero or negligible compression to the first portion of the camera image responsive to a determination that the area of interest indicates presence of an object (“In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI. In other words, a quantization parameter may be computed for each pixel-block to preserve target quality in the ROI regions and aggressive compression elsewhere to meet the target bitrate. For almost lossless compression, a high value is chosen for the quantization parameter, resulting in a smaller quantization step size” Anumula et al. at col. 8, line 19).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose determining presence of a moving object.
Yonezawa teaches a system and article in the same field of endeavor of ROI correlated variable quantization, wherein the encoded instructions are additionally to:
apply zero or negligible compression to the first portion of the camera image (“In step S560, for the specific object ROI including part or all of the moving object region, the compression encoding unit 215 sets “35” as “the qP value inside the ROI” at col. 11, line 30) responsive to a determination that the area of interest indicates presence of a moving object (“In step S540, the second detection unit 214 detects a moving object from the selected frame image. Detection of a moving object from the frame image may be performed for each frame, or may be performed every few frames. A moving object may be detected from a range using the entire selected frame image as the range, or may be detected from a range using the specific object ROI set by the region generation unit 213 as the range” at col. 9, line 51).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a moving object estimation as taught by Yonezawa in the system of the Anumula et al. and Scheunert et al. combination to prioritize particular motion objects as the most important objects of interest (see Yonezawa at col. 11, lines 30-36 that states moving objects for the ROI have a lower QP compression QP).
Regarding claim 16, the Anumula et al. and Scheunert et al. combination discloses a method comprising:
applying the first compression rule to include zero or negligible compression to the first portion of the camera image based on the first portion of the camera image including an object (“In step S550, a quantization parameter for the grouped pixels is determined. The quantization parameter for the grouped pixels of the ROI yields less compression than a quantization parameter of a non-ROI. In other words, a quantization parameter may be computed for each pixel-block to preserve target quality in the ROI regions and aggressive compression elsewhere to meet the target bitrate. For almost lossless compression, a high value is chosen for the quantization parameter, resulting in a smaller quantization step size” Anumula et al. at col. 8, line 19); and
applying non-zero or non-negligible compression based on the first portion of the camera image including another object (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles, lane/road markers, etc. in a video stream may be more important than objects like sky, vegetation or buildings etc. Therefore, not all of the objects in a video frame are of same importance and different compression/quality factors (depending upon the importance of the objects) may be chosen while compressing the video frame” Anumula et al. at col. 4, line 45).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose determining presence of a moving object and presence of a stationary object to determine compression level.
Yonezawa teaches a method in the same field of endeavor of ROI correlated variable quantization, wherein the encoded instructions are additionally to:
determine whether the first portion of the camera image (“In step S530, processing regarding setting of a specific object ROI (addition or deletion of a specific object ROI) is performed” at col. 7, line 39) includes a moving object or includes a stationary object (“In step S540, the second detection unit 214 detects a moving object from the selected frame image. Detection of a moving object from the frame image may be performed for each frame, or may be performed every few frames. A moving object may be detected from a range using the entire selected frame image as the range, or may be detected from a range using the specific object ROI set by the region generation unit 213 as the range” at col. 9, line 51).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a moving object estimation as taught by Yonezawa in the system of the Anumula et al. and Scheunert et al. combination to prioritize particular motion objects as the most important objects of interest (see Yonezawa at col. 11, lines 30-36 that states moving objects for the ROI have a lower QP compression QP). Therefore, non-motion objects can still be objects of interest but designated less important in accordance with the above cited disclosure of Anumula et al.
Regarding claim 20, the Anumula et al. and Scheunert et al. combination discloses an article wherein the encoded instructions are additionally operable to:
apply non-zero or non-negligible compression to the first portion of the camera image responsive to determining that the first portion of the camera image includes an object (“Notably, detection of objects-of-interest, e.g. other vehicles, traffic lights, pedestrians, bicycles, lane/road markers, etc. in a video stream may be more important than objects like sky, vegetation or buildings etc. Therefore, not all of the objects in a video frame are of same importance and different compression/quality factors (depending upon the importance of the objects) may be chosen while compressing the video frame” Anumula et al. at col. 4, line 45; certain objects are therefore compressed at a relatively higher rate to important objects of interest).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose determining presence of a stationary object to determine compression level.
Yonezawa teaches a method in the same field of endeavor of ROI correlated variable quantization, wherein the encoded instructions are additionally to:
determine whether the first portion of the camera image (“In step S530, processing regarding setting of a specific object ROI (addition or deletion of a specific object ROI) is performed” at col. 7, line 39) includes a moving object or includes a stationary object (“In step S540, the second detection unit 214 detects a moving object from the selected frame image. Detection of a moving object from the frame image may be performed for each frame, or may be performed every few frames. A moving object may be detected from a range using the entire selected frame image as the range, or may be detected from a range using the specific object ROI set by the region generation unit 213 as the range” at col. 9, line 51).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a moving object estimation as taught by Yonezawa in the system of the Anumula et al. and Scheunert et al. combination to prioritize particular motion objects as the most important objects of interest (see Yonezawa at col. 11, lines 30-36 that states moving objects for the ROI have a lower QP compression QP). Therefore, non-motion objects can still be objects of interest but designated less important in accordance with the above cited disclosure of Anumula et al.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Anumula et al. and Scheunert et al. and further in view of Chondro et al. (US 2020/0111225).
The Anumula et al. and Scheunert et al. combination disclose a system wherein a camera to generate the camera image and sensors to generate the non-camera sensor data are mounted on a common platform (“FIG. 1 illustrates a system 100 including a vehicle 110, a set of multiple sensors 120 of the vehicle 110, another vehicle 130, and a cloud environment 140. The set of multiple sensors 120 may include a camera, a LIDAR, a radar, a time-of-flight device and other sensors and devices that may be used for observing the environment of the vehicle 110” Anumula et al. at col. 3, line 29).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose that the camera and sensors having at least partially overlapping fields-of-view.
Chondro et al. teaches a system in the same field of endeavor of exterior vehicle monitoring, wherein a camera to generate the camera image and sensors to generate the non-camera sensor data are mounted on a common platform having at least partially overlapping fields-of-view (“With reference to FIG. 5 and FIG. 6A˜6F, the framework to be described would include a depth estimation apparatus that utilizes multiple types of sensing devices (e.g. a RGB camera and a LiDAR transducer illustrated in FIG. 6A) to perform depth estimations by using multiple algorithms for each type of sensing devices over the overlapping FOVs (as illustrated in FIG. 6B), wherein the FOV distance of the RGB camera is 100 meters and the FOV degree of the LiDAR sensor is 360 degrees. The multiple types of sensing devices may include a first type of sensor (e.g. RGB camera array 501 illustrated in FIG. 5) and a second type of sensor (e.g. LiDAR transducer array 502 illustrated in FIG. 5)” at paragraph 0041, line 8).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a configuration as taught by Condro et al. for the sensors of the Anumula et al. and Scheunert et al. combination to ensure that the ROI data from each sensor system is properly correlated for object detection.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Anumula et al. and Scheunert et al. and further in view of Gayaka et al. (US 2022/0300001).
The Anumula et al. and Scheunert et al. combination discloses a system wherein the non-camera sensor data comprises data from at least one of a LIDAR sensor and a radar sensor (“FIG. 1 illustrates a system 100 including a vehicle 110, a set of multiple sensors 120 of the vehicle 110, another vehicle 130, and a cloud environment 140. The set of multiple sensors 120 may include a camera, a LIDAR, a radar, a time-of-flight device and other sensors and devices that may be used for observing the environment of the vehicle 110” Anumula et al. at col. 3, line 29).
The Anumula et al. and Scheunert et al. combination does not explicitly disclose a infrared sensor and an ultrasonic sensor.
Gayaka et al. teaches a system in the same field of endeavor of obstacle detection, wherein the non-camera sensor data comprises data from a LIDAR sensor, a radar sensor, an infrared sensor, and an ultrasonic sensor (“Depth sensors such as ultrasonic sensors, optical sensors such as a TOF depth camera, LIDAR, radar, and so forth may provide depth data that is indicative of the presence or absence of objects in the physical space 102 within the FOV 110 of the depth sensor. For example, a sensor 134 such as a TOF depth camera may emit a pulse of infrared light and use a detected return time for reflected light to determine a distance between the sensor and the object that reflected the light” at paragraph 0067, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a configuration as taught by Gayaka et al. for the sensors of the Anumula et al. and Scheunert et al. combination to provide various depth information at particular resolutions for detection of the object data.
Response to Arguments
Summary of Remarks (@ response page labeled 9): “In short, Delp at most associates non-point cloud data with point cloud data, but Delp's disclosure of associating additional attributes (e.g., RGB values or semantic information) with points in a point cloud does not include camera data and non- camera sensor data in a common coordinate system.”
Examiner’s Response: This argument is moot in view of the newly cited Scheunert et al. reference.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672
Prosecution Insights