Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 - 3, 5, & 7 - 19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7, 12, 14, & 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1).
Regarding Claim 1:
Seo discloses: A system comprising: (Seo discloses in at least Paragraph 0020 a system for sensor blindness detection in autonomous driving applications, including the usability analysis of the collected sensor data)
at least one sensing device; (Seo discloses in at least Paragraph 0023 wherein a plurality of sensors, including cameras, LIDAR sensors, radar sensors, and the like, may be disposed on a vehicle and used to generate sensor data for evaluation by the sensor blindness detection and classification system [i.e. the system comprises at least one sensing device])
a processor in electronic communication with the at least one sensing device; and a memory having instructions thereon, wherein the instructions when executed by the processor, cause the processor to: (Seo discloses in at least Paragraphs 0050, 0085, & 0086 wherein the vehicle may include one or more controllers, comprising processors and associated memory, which may receive data from sensor inputs and process said data [i.e. a processor and a memory having instructions thereon in electronic communication with the at least one sensing device])
monitor image data via the at least one sensing device; and (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare, broken lens, light, mud, paper, person, and the like [i.e. monitor image data via the at least one sensing device])
detect a low visibility or obstructed condition based, at least in part on direct analysis of the image data to identify one or more images that are washed out due to excessive light, and (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare [i.e. excessive light], broken lens, light, mud, paper, person, and the like [i.e. wherein the low visibility or obstructed condition is determined based on direct analysis of the image data to identify one or more images that are washed out due to excessive light])
in response to detecting the low visibility or obstructed condition, trigger a corrective operation in relation to the at least one sensing device (Seo discloses in at least Paragraphs 0043 & 0055 wherein upon the detection of sensor blindness [i.e. a low visibility or obstructed condition] a control decision may be made for the vehicle based on the determined sensor blindness, including disengaging automatic control of the vehicle, or ignoring certain instances of the sensor data that is determined to be subject to blindness, as well as the predictions stemming from said sensor data [i.e. trigger a corrective operation in relation to the at least one sensing device]. This may include, as disclosed in at least Paragraph 0048, attributing different regions of the collected sensor data, and determining the usability of each sensor data region based on the blindness classifications and attributions [i.e. a corrective operation in relation to the at least one sensing device])
Seo however appears to be silent regarding:
a predictive output based at least in part on a direction of travel, time of day, and weather condition:
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device.
However Creusot teaches wherein based on the weather, date, time, and vehicle orientation, perception degradation in vehicle sensors may be predicted and the vehicle may be maneuvered to reposition the sensor to mitigate the perception degradation.
a predictive output based at least in part on a direction of travel, time of day, and weather condition: (However Creusot teaches in at least Paragraphs 0010 – 0012 wherein perception degradation of a vehicle sensor system may be projected based on locations and orientations of a vehicle sensor system relative to the sun according to the date and time of day [i.e. a predictive output based at least in part on a direction of travel and time of day]. At least Paragraphs 0006 & 0042 of Creusot further teaches wherein the perception degradation data may further be based on a current weather condition, such that a route having a particular location and orientation is not predicted to exceed a threshold level of perception degradation when current weather conditions for the location and orientation are identified as cloudy, however will when the weather is sunny [i.e. a predictive output based at least in part on a weather condition])
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device. (However Creusot teaches in at least Paragraphs 0010, 0035, & 0036 wherein the vehicle computing system may, responsive to determining that perception degradation of a sensor system has exceeded a threshold, cause the autonomous vehicle to execute a maneuver that reduces perception degradation of the sensor system, which may comprise repositioning the sensor [i.e. wherein the corrective operation comprises changing a position of the at least one sensing device])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of sensor system degradation based on the weather, date, time, and vehicle orientation, and the correction of sensor position based on such as taught by Creusot.
The motivation to do so is that, as acknowledged by Creusot in at least Paragraphs 0035 & 0036, the vehicle may be maneuvered to avoid perception degradation based on predicting the positions of the vehicle likely to cause perception degradation, improving the mitigation of perception loss in the vehicle sensors due to the position of the sun.
Regarding Claim 7:
The system of claim 1, wherein direct analysis of the image data comprises periodically sampling one or more frames of the image data.
Seo discloses in at least Paragraph 0045 wherein the motion-based sensor blindness algorithm may be implemented periodically at an interval [i.e. periodically sampling one or more frames of the image data].
Regarding Claim 12:
The system of claim 11, wherein the image data is utilized for machine vision operations.
Seo discloses in at least Paragraphs 0118 – 0124 wherein the vehicle processors may be configured to execute computer vision algorithms for object detection and autonomous vehicle operation [i.e. the image data is utilized for machine vision operations].
Regarding Claim 14:
The system of claim 1, wherein the system is embodied as a fully autonomous vehicle, a semi-autonomous vehicle, surgical robot, or care-giving robot.
Seo discloses in at least Paragraph 0020 a system for sensor blindness detection in autonomous driving applications, which may include fully or semi-autonomous vehicles [i.e. wherein the system is embodied as a fully autonomous vehicle or a semi-autonomous vehicle].
Regarding Claim 15:
The system of claim 1, wherein the system comprises a machine vision system.
Seo discloses in at least Paragraphs 0118 – 0124 wherein the vehicle processors may be configured to execute computer vision algorithms for object detection and autonomous vehicle operation [i.e. the system comprises a machine vision system].
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) as applied to claim 1 above, and further in view of Wan (CN 114841910 A).
Regarding Claim 2:
The system of claim 1, wherein the instructions when executed by the processor cause the processor to further: determine a confidence measure in relation to the detected low visibility or obstructed condition; and trigger the corrective operation in an instance in which the confidence measure satisfies a predetermined threshold.
Seo does not appear to specifically disclose wherein a corrective operation is triggered based on a low visibility/obstruction confidence measure exceeding a threshold.
However Wan teaches in at least Paragraphs 0029, 0188, & 0238 wherein an occlusion confidence degree may be determined, said occlusion confidence degree indicating the blurring degree of a candidate occlusion area of a captured image, such that the higher the occlusion confidence, the more blurred the candidate occlusion area is. At least Paragraphs 0216 & 0238 of Wan further teach wherein this confidence degree may be compared to a threshold, to determine if the vehicle sensor is blocked [i.e. an instance in which the confidence measure satisfies a predetermined threshold]. At least Paragraphs 0004 & 0221 of Wan further teach wherein upon a determination that the sensor is partially or completely blocked, measures such as triggering an alarm or disabling an autonomous driving system may be taken [i.e. trigger the corrective operation in an instance in which the confidence measure satisfies a predetermined threshold].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of a confidence measure in relation to the detected low visibility or obstructed condition as taught by Wan.
The motivation to do so is that, as acknowledged by Wan in at least Paragraphs 0029 & 0188, a probability of sensor blockage may be determined based on the image analysis, and used to assess if the sensor is blocked or not, improving the assessment of degree of sensor occlusion.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) and Wan (CN 114841910 A) as applied to claim 2 above, and further in view of Krunic (US 2020/0105139 A1).
Regarding Claim 3:
The system of claim 2, wherein the confidence measure is determined based at least in part on data obtained from one or more other apparatuses or computing devices.
Seo does not appear to specifically disclose wherein the confidence measure is determined based at least in part on data obtained from one or more other apparatuses or computing devices.
However Krunic teaches in at least Paragraphs 0006, 0058, 0068, & 0072 wherein historical data stored in a glare detection system, which may be acquired from other vehicles or third party sources as taught in at least Paragraphs 0056 & 0057, may be analyzed to predict anticipated levels of glare for an area of travel, said data including historical glare exposure, weather, traffic, geography, and the like [i.e. wherein the confidence measure is determined based at least in part on data obtained from one or more other apparatuses or computing devices].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of a glare condition confidence based on historical data acquired form other sources as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraphs 0006 & 0058, an upcoming glare condition may be anticipated along a route given the present and historical conditions, improving the routing of the vehicle to avoid glare where possible.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) as applied to claim 1 above, and further in view of Krunic (US 2020/0105139 A1).
Regarding Claim 5:
The system of claim 1, wherein the instructions when executed by the processor cause the processor to further: transmit an indication of the low visibility or obstructed condition to another apparatus or another self-driving car that is within a predetermined range of the at least one sensing device or to a central server.
Seo does not appear to specifically disclose transmitting an indication of the low visibility or obstructed condition to another location.
However Krunic teaches in at least Paragraph 0052 wherein glare information [i.e. an indication of the low visibility or obstructed condition] may be transmitted from a vehicle travelling along a road segment to one or more vehicles trailing the vehicle along the same road segment or generally in the vicinity [i.e. the indication is transmitted to another apparatus or another self-driving car that is within a predetermined range of the at least one sensing device]. At least Paragraph 0056 of Krunic further teaches wherein said information may further be transmitted to a glare detection server [i.e. to a central server].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the transmission of low-visibility information to other vehicles or a server as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraph 0052, surrounding vehicles may be informed regarding the glare information, allowing the surrounding vehicles to take preemptive action to mitigate the glare condition, improving the safety of surrounding vehicles.
Regarding Claim 9:
The system of claim 1, wherein the low visibility or obstructed condition is determined based at least in part on real-time or historical visibility information/data corresponding with a geographic location of the at least one sensing device.
Seo does not appear to specifically disclose wherein the low visibility or obstructed condition is determined based at least in part on real-time or historical visibility information/data corresponding with a geographic location of the at least one sensing device.
However Krunic teaches in at least Paragraphs 0006, 0058, 0068, & 0072 wherein historical data stored in a glare detection system may be analyzed to predict anticipated levels of glare for an area of travel, said data including historical glare exposure, weather, traffic, geography, and the like [i.e. the low visibility or obstructed condition is determined based at least in part on real-time or historical visibility information/data corresponding with a geographic location of the at least one sensing device].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of a glare condition based on historical data as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraphs 0006 & 0058, an upcoming glare condition may be anticipated along a route given the present and historical conditions, improving the routing of the vehicle to avoid glare where possible.
Regarding Claim 10:
The system of claim 9, wherein the real-time or historical visibility information/data comprises at least one of longitude, latitude, time of day, direction, and weather conditions.
Seo does not appear to specifically disclose wherein the real-time or historical visibility information/data comprises at least one of longitude, latitude, time of day, direction, and weather conditions.
However Krunic teaches in at least Paragraphs 0006, 0058, 0068, & 0072 wherein historical data stored in a glare detection system may be analyzed to predict anticipated levels of glare for an area of travel, said data including historical glare exposure, weather, traffic, geography, and the like [i.e. the historical visibility information/data comprises at least weather conditions].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of a glare condition based on historical data such as weather information as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraphs 0006 & 0058, an upcoming glare condition may be anticipated along a route given the present and historical conditions, improving the routing of the vehicle to avoid glare where possible.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) as applied to claim 1 above, and further in view of Tryndin (US 2023/0186593 A1).
Regarding Claim 8:
The system of claim 1, wherein the low visibility or obstructed condition is detected based at least in part on a measure of brightness or contrast with respect to one or more frames of the image data that meets or exceeds a predetermined threshold.
Seo does not appear to specifically disclose wherein the low visibility or obstructed condition is detected based at least in part on a measure of brightness or contrast with respect to one or more frames of the image data meeting or exceeding a predetermined threshold.
However Tryndin teaches in at least Paragraphs 0022 & 0023 wherein a vehicle system may acquire surrounding image data, which may be processed to determine contrast values for the individual pixels of the acquired image. At least Paragraphs 0028 & 0030 of Trynden further teach wherein the contrast values may be compared to one or more threshold values to determine a quantity of high and low contrast pixels in the image, which may be utilized to determine if glare is detected that should be mitigated as taught in at least Paragraphs 0031 & 0032 of Tryndin [i.e. the low visibility or obstructed condition is detected based at least in part on a measure of brightness or contrast with respect to one or more frames of the image data that meets or exceeds a predetermined threshold].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of glare in a detected image based on contrast values in comparison to a threshold as taught by Tryndin.
The motivation to do so is that, as acknowledged by Tryndin in at least Paragraphs 0028 – 0031, glare in an image may be more reliably detected and mitigating actions may take place accordingly, improving the glare mitigation with respect to sensor systems of a vehicle.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) as applied to claim 1 above, and further in view of Patel (US 2024/0159891 A1).
Regarding Claim 11:
The system of claim 1, wherein triggering the corrective operation comprises at least one of: adjusting one or more image processing parameters of image processing software utilized by the system, modifying at least one operational parameter of the at least one sensing device, and applying a polarizing filter.
Seo appears to be silent regarding wherein triggering the corrective operation comprises at least one of: adjusting one or more image processing parameters of image processing software utilized by the system, modifying at least one operational parameter of the at least one sensing device, and applying a polarizing filter.
However Patel teaches in at least Paragraphs 0014 & 0024 wherein upon determining that an occlusion is blocking the field of view of a sensor disposed on a vehicle, the vehicle controller may send instructions to reposition the affected sensor to a different position through the use of an actuator [i.e. modifying at least one operational parameter of the at least one sensing device].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the adjustment of position of a sensing device in response to the determination of an occlusion as taught by Patel.
The motivation to do so is that, as acknowledged by Patel in at least Paragraph 0014, the sensor may be enabled to better view the occluded area, improving the capability of the vehicle to sense its surroundings in response to occlusion.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Creusot (US 2020/0183386 A1) as applied to claim 1 above, and further in view of Chen (US 2022/0391621 A1).
Regarding Claim 13:
The system of claim 1, wherein the instructions when executed by the processor cause the processor to further: detect the low visibility or obstructed condition and/or determine an appropriate corrective operation based at least in part on the low visibility or obstructed condition using a neural network model that is configured to determine object permanence with respect to objects in the image data.
Seo does not appear to specifically disclose wherein the low visibility/obstruction condition or an appropriate corrective operation is based on using a neural network model that is configured to determine object permanence with respect to objects in the image data.
However Chen teaches in at least Paragraphs 0042 – 0048 wherein a target object position may be estimated through the use of a neural network based on a trajectory of the object estimated from previously tracked frames of the object, which may be utilized to detect, predict, or estimate an occlusion through a loss of tracking with an object as taught in at least Paragraphs 0050 – 0052 of Chen [i.e. detect the low visibility or obstructed condition using a neural network model that is configured to determine object permanence with respect to objects in the image data].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of obstruction based on a loss of tracking of an object as taught by Chen.
The motivation to do so is that, as acknowledged by Chen in at least Paragraph 0049, by tracking the trajectories of objects in the environment, occlusions of the sensors may be better predicted, improving the vehicle monitoring of the surroundings.
Claim(s) 16 - 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Krunic (US 2020/0105139 A1) and Creusot (US 2020/0183386 A1).
Regarding Claim 16:
Seo discloses: A cooperative driving or robotic system comprising: (Seo discloses in at least Paragraph 0020 a system for sensor blindness detection in autonomous driving applications, including the usability analysis of the collected sensor data)
at least one sensing device; (Seo discloses in at least Paragraph 0023 wherein a plurality of sensors, including cameras, LIDAR sensors, radar sensors, and the like, may be disposed on a vehicle and used to generate sensor data for evaluation by the sensor blindness detection and classification system [i.e. the system comprises at least one sensing device])
a processor in electronic communication with the at least one sensing device; and a memory having instructions thereon, wherein the instructions when executed by the processor, cause the processor to: (Seo discloses in at least Paragraphs 0050, 0085, & 0086 wherein the vehicle may include one or more controllers, comprising processors and associated memory, which may receive data from sensor inputs and process said data [i.e. a processor and a memory having instructions thereon in electronic communication with the at least one sensing device])
monitor image data via the at least one sensing device; (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare, broken lens, light, mud, paper, person, and the like [i.e. monitor image data via the at least one sensing device])
detect a low visibility or obstructed condition based, at least in part on direct analysis of the image data to identify one or more images that are washed out due to excessive light, and (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare [i.e. excessive light], broken lens, light, mud, paper, person, and the like [i.e. wherein the low visibility or obstructed condition is determined based on direct analysis of the image data to identify one or more images that are washed out due to excessive light])
in response to detecting the low visibility or obstructed condition, trigger a corrective operation in relation to the at least one sensing device, (Seo discloses in at least Paragraphs 0043 & 0055 wherein upon the detection of sensor blindness [i.e. a low visibility or obstructed condition] a control decision may be made for the vehicle based on the determined sensor blindness, including disengaging automatic control of the vehicle, or ignoring certain instances of the sensor data, as well as the predictions stemming from said sensor data [i.e. trigger a corrective operation in relation to the at least one sensing device]. This may include, as disclosed in at least Paragraph 0048, attributing different regions of the collected sensor data, and determining the usability of each sensor data region based on the blindness classifications and attributions)
Seo however appears to be silent regarding:
a plurality of vehicles or robots in electronic communication with one another, each vehicle or robot comprising:
a predictive output based at least in part on a direction of travel, time of day, and weather condition;
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device,
and wherein each of the plurality of vehicles or robots is configured to transmit an indication of detected low visibility or obstructed conditions to at least another vehicle and/or trigger corrective operations in relation to the at least another vehicle.
However Krunic teaches wherein determined glare conditions may be transmitted to other vehicles in the surroundings when said conditions are detected.
a plurality of vehicles or robots in electronic communication with one another, each vehicle or robot comprising: (However Krunic teaches in at least Paragraphs 0032, 0040, & 0052 wherein a plurality of vehicles may be equipped with sensing and communication systems, and may communicate with one another to share data regarding upcoming conditions, such as glare, on a road [i.e. a plurality of vehicles in electronic communication with one another])
wherein each of the plurality of vehicles or robots is configured to transmit an indication of detected low visibility or obstructed conditions to at least another vehicle and/or trigger corrective operations in relation to the at least another vehicle. (However Krunic teaches in at least Paragraph 0052 wherein glare information [i.e. an indication of the low visibility or obstructed condition] may be transmitted from a vehicle travelling along a road segment to one or more vehicles trailing the vehicle along the same road segment or generally in the vicinity [i.e. the indication is transmitted to another apparatus or another self-driving car that is within a predetermined range of the at least one sensing device])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the transmission of low-visibility information to other vehicles as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraph 0052, surrounding vehicles may be informed regarding the glare information, allowing the surrounding vehicles to take preemptive action to mitigate the glare condition, improving the safety of surrounding vehicles.
However Creusot teaches wherein based on the weather, date, time, and vehicle orientation, perception degradation in vehicle sensors may be predicted and the vehicle may be maneuvered to reposition the sensor to mitigate the perception degradation.
a predictive output based at least in part on a direction of travel, time of day, and weather condition; (However Creusot teaches in at least Paragraphs 0010 – 0012 wherein perception degradation of a vehicle sensor system may be projected based on locations and orientations of a vehicle sensor system relative to the sun according to the date and time of day [i.e. a predictive output based at least in part on a direction of travel and time of day]. At least Paragraphs 0006 & 0042 of Creusot further teaches wherein the perception degradation data may further be based on a current weather condition, such that a route having a particular location and orientation is not predicted to exceed a threshold level of perception degradation when current weather conditions for the location and orientation are identified as cloudy, however will when the weather is sunny [i.e. a predictive output based at least in part on a weather condition])
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device, (However Creusot teaches in at least Paragraphs 0010, 0035, & 0036 wherein the vehicle computing system may, responsive to determining that perception degradation of a sensor system has exceeded a threshold, cause the autonomous vehicle to execute a maneuver that reduces perception degradation of the sensor system, which may comprise repositioning the sensor [i.e. wherein the corrective operation comprises changing a position of the at least one sensing device])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of sensor system degradation based on the weather, date, time, and vehicle orientation, and the correction of sensor position based on such as taught by Creusot.
The motivation to do so is that, as acknowledged by Creusot in at least Paragraphs 0035 & 0036, the vehicle may be maneuvered to avoid perception degradation based on predicting the positions of the vehicle likely to cause perception degradation, improving the mitigation of perception loss in the vehicle sensors due to the position of the sun.
Regarding Claim 17:
The cooperative driving or robotic system of claim 16, wherein each of the plurality of vehicles or robots is configured to transmit the indication of detected low visibility or obstructed conditions to at least another vehicle or robot and/or trigger corrective operations in relation to the at least another vehicle or robot when it is within a predetermined range.
Seo does not appear to specifically disclose transmitting an indication of the low visibility or obstructed condition to another location.
However Krunic teaches in at least Paragraph 0052 wherein glare information [i.e. an indication of the low visibility or obstructed condition] may be transmitted from a vehicle travelling along a road segment to one or more vehicles trailing the vehicle along the same road segment or generally in the vicinity [i.e. the indication is transmitted to another apparatus or another self-driving car that is within a predetermined range of the at least one sensing device].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the transmission of low-visibility information to other vehicles as taught by Krunic.
The motivation to do so is that, as acknowledged by Krunic in at least Paragraph 0052, surrounding vehicles may be informed regarding the glare information, allowing the surrounding vehicles to take preemptive action to mitigate the glare condition, improving the safety of surrounding vehicles.
Regarding Claim 18:
The cooperative driving or robotic system of claim 16, wherein each vehicle is an autonomous or semi-autonomous vehicle.
Seo discloses in at least Paragraph 0020 a system for sensor blindness detection in autonomous driving applications, which may include fully or semi-autonomous vehicles [i.e. wherein the system is embodied as a fully autonomous vehicle or a semi-autonomous vehicle].
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo (US 2020/0090322 A1) in view of Wan (CN 114841910 A) and Creusot (US 2020/0183386 A1).
Regarding Claim 19:
Seo discloses: A computer-implemented method for identifying and correcting low visibility or obstructed conditions, the computer-implemented method comprising: (Seo discloses in at least Paragraphs 0020 & 0050 a method for sensor blindness detection in autonomous driving applications, including the usability analysis of the collected sensor data, which may be implemented using one or more computing devices [i.e. a computer-implemented method for identifying and correcting low visibility or obstructed conditions])
monitoring image data via at least one sensing device; (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare, broken lens, light, mud, paper, person, and the like [i.e. monitor image data via the at least one sensing device])
detecting a low visibility or obstructed condition based, at least in part on direct analysis of the image data to identify one or more images that are washed out due to excessive light, and (Seo discloses in at least Paragraphs 0022 & 0029 wherein image data representing a video obtained from corresponding sensors may be analyzed to determine features of the image, which may include a classification of sensor blindness with a classification of associated causes. At least Paragraphs 0034 – 0036 & 0044 of Seo discloses wherein blindness regions in an image may be evaluated, defined by pixels or coordinates, and classified to indicate a blocked or blurred status, along with attributes such as the cause of the sensor blindness, such as rain, glare [i.e. excessive light], broken lens, light, mud, paper, person, and the like [i.e. wherein the low visibility or obstructed condition is determined based on direct analysis of the image data to identify one or more images that are washed out due to excessive light])
triggering a corrective operation in relation to the at least one sensing device in an instance (Seo discloses in at least Paragraphs 0043 & 0055 wherein upon the detection of sensor blindness [i.e. a low visibility or obstructed condition] a control decision may be made for the vehicle based on the determined sensor blindness, including disengaging automatic control of the vehicle, or ignoring certain instances of the sensor data, as well as the predictions stemming from said sensor data [i.e. trigger a corrective operation in relation to the at least one sensing device]. This may include, as disclosed in at least Paragraph 0048, attributing different regions of the collected sensor data, and determining the usability of each sensor data region based on the blindness classifications and attributions)
Seo however appears to be silent regarding:
a predictive output based at least in part on a direction of travel, time of day, and weather condition;
in response to detecting the low visibility or obstructed condition, determine a confidence measure in relation to the detected low visibility or obstructed condition; and
triggering a corrective operation in relation to the at least one sensing device in an instance in which the determined confidence measure satisfies a predetermined threshold
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device.
However Wan teaches wherein an occlusion confidence degree may be determined and used to establish if a sensor of a vehicle is blocked, to take countermeasures accordingly.
in response to detecting a low visibility or obstructed condition, determine a confidence measure in relation to the detected low visibility or obstructed condition; and (However Wan teaches in at least Paragraphs 0029, 0188, & 0238 wherein an occlusion confidence degree may be determined, said occlusion confidence degree indicating the blurring degree of a candidate occlusion area of a captured image, such that the higher the occlusion confidence, the more blurred the candidate occlusion area is [i.e. in response to detecting a low visibility or obstructed condition, determine a confidence measure in relation to the detected low visibility or obstructed condition])
triggering a corrective operation in relation to the at least one sensing device in an instance in which the determined confidence measure satisfies a predetermined threshold. (However Wan teaches in at least Paragraphs 0216 & 0238 wherein the computed confidence degree may be compared to a threshold, to determine if the vehicle sensor is blocked [i.e. an instance in which the confidence measure satisfies a predetermined threshold]. At least Paragraphs 0004 & 0221 of Wan further teach wherein upon a determination that the sensor is partially or completely blocked, measures such as triggering an alarm or disabling an autonomous driving system may be taken [i.e. trigger the corrective operation in an instance in which the confidence measure satisfies a predetermined threshold])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of a confidence measure in relation to the detected low visibility or obstructed condition as taught by Wan.
The motivation to do so is that, as acknowledged by Wan in at least Paragraphs 0029 & 0188, a probability of sensor blockage may be determined based on the image analysis, and used to assess if the sensor is blocked or not, improving the assessment of degree of sensor occlusion.
However Creusot teaches wherein based on the weather, date, time, and vehicle orientation, perception degradation in vehicle sensors may be predicted and the vehicle may be maneuvered to reposition the sensor to mitigate the perception degradation.
a predictive output based at least in part on a direction of travel, time of day, and weather condition; (However Creusot teaches in at least Paragraphs 0010 – 0012 wherein perception degradation of a vehicle sensor system may be projected based on locations and orientations of a vehicle sensor system relative to the sun according to the date and time of day [i.e. a predictive output based at least in part on a direction of travel and time of day]. At least Paragraphs 0006 & 0042 of Creusot further teaches wherein the perception degradation data may further be based on a current weather condition, such that a route having a particular location and orientation is not predicted to exceed a threshold level of perception degradation when current weather conditions for the location and orientation are identified as cloudy, however will when the weather is sunny [i.e. a predictive output based at least in part on a weather condition])
wherein the corrective operation comprises changing a position or angle of incidence of the at least one sensing device. (However Creusot teaches in at least Paragraphs 0010, 0035, & 0036 wherein the vehicle computing system may, responsive to determining that perception degradation of a sensor system has exceeded a threshold, cause the autonomous vehicle to execute a maneuver that reduces perception degradation of the sensor system, which may comprise repositioning the sensor [i.e. wherein the corrective operation comprises changing a position of the at least one sensing device])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present claimed invention to have modified the disclosure of Seo by incorporating the determination of sensor system degradation based on the weather, date, time, and vehicle orientation, and the correction of sensor position based on such as taught by Creusot.
The motivation to do so is that, as acknowledged by Creusot in at least Paragraphs 0035 & 0036, the vehicle may be maneuvered to avoid perception degradation based on predicting the positions of the vehicle likely to cause perception degradation, improving the mitigation of perception loss in the vehicle sensors due to the position of the sun.
Conclusion
The following prior art made of record but not relied upon is considered pertinent to the Applicant’s disclosure:
Dingli (US 2020/0331435 A1): Dingli recites a system for managing the sensors of an autonomous vehicle, including the assessment of future light position information with respect to the vehicle, and the cleaning of sensors based on the determined upcoming conditions.
Lu (US 2020/0213581 A1): Lu recites a method for identifying defects in optical detection systems based on stray light effects in the detected image. A bright object may be detected, and stray light is further determined from said object. The stray light is compared to a threshold to determine if any sensor system defects are present.
Li (US 2021/0185207 A1): Li recites a system for glare control with respect to the sensors of an autonomous vehicle. A negative impact from environmental glare is determined for a sensor, and a glare blocking mechanism is controlled to shield the sensor from the glare by positioning itself in the path of the glare.
Patel (US 2024/0159891 A1): Patel recites wherein upon determining that an occlusion is blocking the field of view of a sensor disposed on a vehicle, the vehicle controller may send instructions to reposition the affected sensor to a different position through the use of an actuator.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER RYAN CARDIMINO whose telephone number is (571)272-2759. The examiner can normally be reached M-Th 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramya Burgess can be reached at (571)272-6011. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER R CARDIMINO/Examiner, Art Unit 3661
/MATTHIAS S WEISFELD/Examiner, Art Unit 3661
Prosecution Insights