Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/25/2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/17/2025 and 03/31/2026 were filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 5, 8, 13, 14, and 17 are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter that the inventor or a joint inventor regards as the invention.
Claims 4, 5, 13, and 14
Claims 4 and 13 respectively recite:
“wherein the one or more sensors of the vehicle include any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors.”
Claims 1 and 10 recite sensors serving two different roles: (1) “one or more sensors” used for monitoring the vehicle and measuring sensor-calibration triggers; and (2) “one or more sensors of the vehicle that are affected by the detected sensor calibration trigger,” which are identified and commanded to recalibrate.
It is unclear whether “the one or more sensors of the vehicle” in claims 4 and 13 refers to the sensors used for monitoring the vehicle, the sensors identified as affected by the trigger, or both sets. These interpretations produce different claim scopes because the listed sensor types may limit the monitoring sensors, the affected sensors, or both. Accordingly, the metes and bounds of claims 4 and 13 are unclear.
Claims 5 and 14 depend from claims 4 and 13, respectively, and incorporate the indefinite limitation. Although claims 5 and 14 further refer to “the one or more sensors for monitoring the vehicle,” that additional language does not clarify which sensor set is limited by claims 4 and 13.
For purposes of the prior-art rejections, the Office interprets “the one or more sensors of the vehicle” in claims 4 and 13 as referring to the sensors used for monitoring the vehicle. This interpretation permits prior-art examination but does not resolve the ambiguity in the claims. See MPEP §§ 2173.05(e) and 2173.06.
The ambiguity may be resolved by amending claims 4 and 13 to specify either “the one or more sensors for monitoring the vehicle” or “the one or more sensors identified as being affected by the detected sensor calibration trigger,” consistent with Applicant’s intended scope and the original disclosure.
Claims 8 and 17
Claims 8 and 17 respectively recite:
“wherein automatically performing sensor calibration includes detecting a severity level of the sensor calibration trigger, the sensor calibration trigger being identified based at least in part on the severity level.”
Claims 1 and 10 first require detecting a sensor-calibration trigger and then, “in response to detecting the sensor calibration trigger,” automatically performing sensor calibration. Claims 8 and 17, however, state that automatically performing sensor calibration itself includes detecting the severity level upon which the sensor-calibration trigger is identified.
The resulting temporal and functional relationship is unclear. The language may reasonably be interpreted as requiring the severity level to be detected during the initial trigger-detection operation, before sensor calibration begins. Alternatively, it may be interpreted as requiring the trigger to be detected first, sensor calibration to begin in response to that detection, and the severity level then to be detected as part of the calibration operation. The claims also do not clarify whether the trigger “being identified” is the same operation as the earlier detection of the trigger or a separate subsequent determination.
Because the claims permit more than one reasonable interpretation concerning when the severity level is detected and how that severity level is used to identify an already-detected trigger, the sequence and scope of the claimed operations are unclear.
For purposes of the prior-art rejections, the Office interprets the limitation as requiring the overall trigger-detection operation to include determining that the measured event falls within a categorical severity level. This interpretation permits prior-art examination but does not resolve the ambiguity. See MPEP §§ 2173.02 and 2173.06.
The ambiguity may be resolved, if consistent with Applicant’s intended scope and the original disclosure, by reciting that “detecting the sensor calibration trigger includes detecting a severity level of the sensor calibration trigger and identifying the sensor calibration trigger based at least in part on the severity level,” or by otherwise expressly stating the intended sequence.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3–6, 8–10, 12–15, and 17–19 are rejected under 35 U.S.C. § 103 as being unpatentable over Levinson (US 2018/0190046 A1) in view of Prokhorov (US 2016/0161602 A1).
Claims 2, 11, and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Levinson in view of Prokhorov (US 2016/0161602 A1) and further in view of Wang (US 2022/0236729 A1).
Regarding Claim 1,
Disclosure by Levinson
Levinson discloses:
A computing system for a vehicle
See at least:
“A miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
“[T]he autonomous vehicle system 3602 (“AV system”) may include a perception system 3702 that includes an intrinsic sensor calibration module 3704, an extrinsic sensor calibration module 3706, a data transform module 3708, a generative model module 3710, a heuristics engine module 3712, and a sensor drift tracking module 3714.” (Levinson [0160])
Rationale:
Perception system 3702 and its calibration modules reside onboard AV system 3602 and perform the vehicle’s sensor-calibration functions. They are therefore a computing system for a vehicle.
that operates autonomously or semi-autonomously,
See at least:
“An autonomous vehicle system 3602 may be operating and driving past various structures in a typical driving scenario.” (Levinson [0142])
Rationale:
Levinson identifies vehicle system 3602 as an autonomous vehicle system and describes it operating and driving. The claim recites autonomous or semi-autonomous operation in the alternative.
the computing system comprising:
See at least:
“[T]he autonomous vehicle system 3602 (“AV system”) may include a perception system 3702 that includes an intrinsic sensor calibration module 3704, an extrinsic sensor calibration module 3706, a data transform module 3708, a generative model module 3710, a heuristics engine module 3712, and a sensor drift tracking module 3714.” (Levinson [0160])
Rationale:
Levinson introduces perception system 3702 and its constituent calibration modules as components included within AV system 3602.
one or more processors;
See at least:
“Processor 4006 or 4106 can be implemented with one or more central processing units (“CPUs”) . . . or one or more virtual processors, as well as any combination of CPUs and virtual processors.” (Levinson [0183])
Rationale:
Levinson discloses one or more processors, and at [0151] locates the processors performing the calibration onboard AV system 3602.
a memory storing instructions
See at least:
“In the example shown, system memory 4010 or 4110 can include various modules that include executable instructions to implement functionalities described herein.” (Levinson [0187])
“System memory 4010 or 4110 may include an operating system (“O/S”) 4030 or 4130, as well as an application 4032 or 4132 and/or logic module 4050 or 4150.” (Levinson [0187])
Rationale:
System memory 4110 of FIG. 41 stores executable instruction modules, including logic module 4150, which contains perception system 3702 and its calibration modules. The rejection relies on system memory 4110.
that, when executed by the one or more processors,
See at least:
“According to some examples, computing platform 4000 or 4100 performs specific operations by processor 4006 or 4106 executing one or more sequences of one or more instructions stored in system memory 4010 or 4110.” (Levinson [0184])
Rationale:
Levinson discloses processor 4106 executing the instructions stored in system memory 4110.
cause the computing system to:
See at least:
“One or more of the modules included in memory 4010 or 4110 can be configured to provide or consume outputs to implement one or more functions described herein.” (Levinson [0187])
Rationale:
Execution of the stored instructions causes the onboard platform and its calibration modules to perform the disclosed functions.
when the vehicle is operating autonomously or semi-autonomously on a road network,
See at least:
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment, such as static objects like lane markings and curbs as well as dynamic objects like moving vehicles in real-time and continuously.” (Levinson [0147])
Rationale:
Lane markings, curbs, and other moving vehicles are features of a road network. Levinson performs these functions in real time and continuously while the vehicle is operating.
monitor the vehicle,
See at least:
“In one embodiment, the autonomous vehicle system 3602 may track the drift of each sensor over time.” (Levinson [0146])
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment . . . in real-time and continuously.” (Levinson [0147])
Rationale:
Tracking each sensor’s drift over time and continuously determining the vehicle’s location and environment is repeated evaluation of the operating vehicle, not an isolated calibration measurement.
using one or more sensors,
See at least:
“[T]he autonomous vehicle system 3602 may include many types of sensors or any quantity of sensors to facilitate perception, including image capture sensors, audio capture sensors, LIDAR, RADAR, SONAR, GPS, and IMU.” (Levinson [0142])
Rationale:
The vehicle’s onboard sensors supply the measurements on which perception system 3702 performs the monitoring identified above.
based on the monitoring,
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
The indication arises from sensor measurements taken while the vehicle is operating. Calibration detection module 3730 acts on the result of that monitoring.
detect a sensor calibration trigger
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
Calibration detection module 3730 registers a condition indicating that a sensor requires calibration and initiates the calibration sequence. That is the detection of a trigger for sensor calibration. In the combination, the condition so registered is a Prokhorov trigger type mapped below.
and in response to detecting the sensor calibration trigger,
See at least:
“Once a sensor is identified as potentially miscalibrated, log file data may be retrieved from a log file store 3716 to assist in calibrating the identified sensor.” (Levinson [0162])
Rationale:
Levinson’s “[o]nce a sensor is identified” establishes that the calibration sequence commences only upon, and because of, the detection.
automatically performing sensor calibration
See at least:
“By aligning detected edges from laser returns of LIDAR sensors with the same edges of objects within captured images, a miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
“In this way, the AV system 3602 has been self-calibrated while in operation, without having to stop and interrupt the user experience.” (Levinson [0164])
“[T]he AV system 3602 may automatically identify a course of action based on the confirmed sensor miscalibration.” (Levinson [0178])
Rationale:
Levinson performs the calibration by onboard processors, on the fly, while the vehicle remains in operation and without operator action.
Levinson’s optional offline calibration at [0169] and optional teleoperator consultation at [0171] are alternative embodiments and do not negate the disclosure relied upon. See MPEP 2123.
and (ii) transmitting a recalibration command
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
“Planner 364 may transmit steering and propulsion commands . . . to motion controller 362. Motion controller 362 subsequently may convert any of the commands . . . into control signals . . . to implement changes.” (Levinson [0065])
“Diagram 400 depicts an autonomous vehicle controller (“AV”) 447 disposed in an autonomous vehicle 430, which, in turn, includes a number of sensors 470 coupled to autonomous vehicle controller 447.” (Levinson [0068])
“In some examples, computing platforms 4000 and 4100 may be used to implement computer programs, applications, methods, processes, algorithms, or other software to perform the above-described techniques.” (Levinson [0181])
“[T]he structures and constituent elements above, as well as their functionality, may be aggregated with one or more other structures or elements.” (Levinson [0188])
Rationale:
Levinson discloses vehicle sensors coupled to the onboard controller at [0068], onboard processors that determine intrinsic calibration parameters for the identified sensor at [0163] and cause that sensor to focus and adjust its lens properties at [0151], and a control architecture in which processor-generated commands are transmitted and converted into signals that implement change at [0065].
Levinson does not expressly recite transmitting a command to the sensor. Given those teachings, it would nonetheless have been obvious to one of ordinary skill in the art to implement Levinson’s calibration by transmitting to the identified sensor a command conveying or invoking the computed intrinsic calibration parameters. The implementation uses Levinson’s disclosed command-and-control approach for its established purpose and predictably produces the sensor adjustment Levinson discloses.
Reliance on the controller architecture of FIGS. 3A and 4 together with the calibration architecture of FIGS. 36–37 and 41 is supported by the reference itself: Levinson at [0181] directs that computing platforms 4000 and 4100 implement software “to perform the above-described techniques,” and at [0188] contemplates that its disclosed structures and their functionality may be aggregated.
to each of the one or more sensors,
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
Rationale:
Levinson determines calibration parameters for an individually identified sensor and at [0164] stores the resulting correction in association with that sensor. Where the detection process identifies more than one affected sensor, directing a corresponding command to each identified sensor is the ordinary iterative application of that process.
to trigger the one or more sensors
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
Rationale:
Levinson identifies the sensor, not the processor, as the component that focuses and adjusts its lens properties. In the implementation described above, the transmitted command initiates that sensor-side adjustment.
to intrinsically recalibrate
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor. For example, a LIDAR sensor may require an intrinsic calibration of reflectivity values captured by laser returns.” (Levinson [0163])
“Cameras, on the other hand, may have other intrinsic calibration parameters, such as color mapping, focal length, image positioning, scaling/skew factors, and lens distortion that may affect the imaging process.” (Levinson [0163])
Rationale:
Levinson identifies its calibration as intrinsic and identifies the internal sensor characteristics adjusted: lidar reflectivity, and camera color mapping, focal length, image positioning, scaling, skew, and lens distortion. Levinson assigns intrinsic and extrinsic calibration to separate modules, 3704 and 3706.
in accordance with the recalibration command.
See at least:
“The intrinsic sensor calibration module 3704 may operate in conjunction with a data transform module 3708 and a generative model module 3710 to perform the computations necessary to converge on the optimal intrinsic calibration parameters for the miscalibrated sensor.” (Levinson [0163])
Rationale:
The parameters are computed for the particular identified sensor. It would have been obvious for the command to convey or invoke those parameters so that the sensor’s adjustment conforms to them.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following complete relational limitations:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
of one of the multiple types of sensor calibration triggers;
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
Disclosure by Prokhorov
Prokhorov discloses:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
See at least:
“[T]he installed applications 112 includ[e] programs or apps that permit the CPU 102 to implement the autonomous features of the vehicle 200 as well as the auto-calibration features.” (Prokhorov [0019])
“Among other information measurable by the sensors 130, the sensors 130 can detect vehicle speed, vehicle direction, vehicle acceleration, vehicle rotation, vehicle location, environmental weather conditions, traffic conditions, and road conditions.” (Prokhorov [0022])
“If the sensors 130 capture data for a dead-reckoning system, data relating to wheel revolution speeds, travel distance, steering angle, and steering angular rate of change can be captured.” (Prokhorov [0026])
“The notifications or alerts can be issued periodically or at certain triggers, for example, based on . . . a defined number of miles driven since the last calibration. Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov discloses two calibration-trigger types, each corresponding to an event or scenario the vehicle experiences on the road and measures with its own sensors: accumulated mileage since the last calibration, measured through the travel-distance and wheel-revolution data Prokhorov’s sensors capture at [0026]; and a large jolt arising when the vehicle hits a severe bump or pothole, measured by an onboard IMU. Prokhorov’s sensors likewise measure traffic and road conditions at [0022], and its applications implement the vehicle’s autonomous features at [0019].
A person of ordinary skill in the art would have understood the accumulated mileage of [0031] to be derived from the travel-distance and wheel-revolution data that [0026] identifies as captured by the vehicle’s sensors. Levinson at [0142] and [0147] supplies the autonomous on-road operation during which the combined system monitors for and detects the trigger.
These are the trigger types for which Levinson’s onboard system monitors in the combination. Prokhorov’s elapsed-time trigger is not relied upon.
of one of the multiple types of sensor calibration triggers;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov discloses IMU detection of the large-jolt trigger, one of the two trigger types identified above. In the combination, that trigger is the condition supplied to Levinson’s calibration detection module 3730.
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle . . . that may have affected the alignment of the sensor 130. IMUs 132 can be installed very near or incorporated into the sensors 130 to more accurately detect or predict whether such sensors 130 have become misaligned.” (Prokhorov [0031])
Rationale:
For a detected large-jolt trigger, Prokhorov uses IMUs installed near or incorporated into respective sensors to determine or predict whether those sensors became misaligned, confining the determination to “such sensors 130.” Which sensors are evaluated thus follows from which type of trigger was detected: a jolt-type trigger implicates the sensors whose alignment a mechanical shock disturbs, evaluated through the co-located IMUs. The claim requires identification “based on,” rather than exclusively based on, the trigger type.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to configure Levinson’s onboard autonomous-vehicle calibration system to monitor for Prokhorov’s mileage-accumulation and large-jolt calibration-trigger types and, upon detection of a large-jolt trigger, to use Prokhorov’s sensor-associated IMUs to identify the sensors affected by the jolt before applying Levinson’s onboard calibration process to those sensors.
The references address the same recognized problem. Levinson discloses at [0148] that a sensor may become out of alignment while the vehicle is in operation, and Prokhorov discloses at [0024] that sensors may become misaligned due to normal use or sudden jolts. Levinson supplies the onboard architecture that detects a calibration condition, identifies the miscalibrated sensor, determines intrinsic calibration parameters, and calibrates the sensor while the vehicle remains in operation. Prokhorov supplies the event-monitoring techniques that recognize when an operating event warrants calibration and which sensors that event affected. That Prokhorov uses the detected trigger to alert a user does not limit its teaching of the trigger itself; the question is what the combined teachings would have suggested, and Levinson supplies the automatic calibration the detected trigger initiates. See MPEP 2145(III).
The modification employs known components for their established functions and requires no unconventional hardware. Levinson already relies at [0146] on GPS, IMU, RADAR, SONAR, and cameras during operation, and includes odometry sensors 3616 among the monitored sensors at [0161]. Prokhorov’s mileage and jolt monitors would supply trigger indications to Levinson’s calibration detection module 3730, and Prokhorov’s sensor-associated IMUs would identify the sensors potentially affected by a detected jolt.
A person of ordinary skill would have had a reasonable expectation of success, as both references employ onboard vehicle sensors, processors, memory, IMUs, and sensor-calibration functionality, and the integration requires only conventional exchange of sensor data and trigger indications within an autonomous-vehicle computing system.
The modification would predictably:
detect event-induced sensor miscalibration promptly, rather than after it manifests in degraded perception data;
avoid continued reliance on a sensor affected by a detected event, which Levinson identifies at [0170] as forcing the vehicle into a sub-optimal mode of operation;
avoid unnecessary calibration of unaffected sensors, conserving the processing time and computational effort Levinson identifies at [0169]; and
maintain perception accuracy and safe autonomous operation without a controlled calibration facility, consistent with Levinson’s statement at [0147] that its determinations are undertaken for safety reasons and operational efficiency.
The combination applies a known event-monitoring and affected-sensor-identification technique to a known autonomous-vehicle calibration system ready for improvement, producing the predictable result of initiating onboard calibration for the sensors affected by a measurable operating event. See KSR International Co. v. Teleflex Inc., 550 U.S. 398, 416–18 (2007); MPEP 2143(I)(A), (D).
Regarding Claim 2,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 2.
Claim Limitations Not Explicitly Disclosed by the Combination of Levinson and Prokhorov
After combining the teachings of Levinson and Prokhorov, the following claim limitation is not explicitly disclosed:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
Levinson quantifies the effect of a miscalibration at [0170] and differentiates its calibration response at [0164] and [0169]. Prokhorov discloses the mileage-accumulation and large-jolt trigger types established with respect to Claim 1 and distinguishes normal use from sudden jolts at [0024]. Neither reference associates each such trigger type with a corresponding severity level.
Disclosure by Wang
Levinson, Prokhorov, and Wang render obvious:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
See at least:
“If the diagnostics service determines that sensor calibration values exceed an acceptable range determined by the server in the back end . . . then the criticality level is also classified as high . . . . However, if the diagnostics service does not detect the above, but does detect that there are non-critical warnings produced by hardware within the autonomous vehicle . . . then the criticality level is classified as medium . . . . If not, and the diagnostics service or remote back end server indicates that the autonomous vehicle is due for preventative maintenance . . . then the criticality level is classified as low.” (Wang [0063]–[0065])
“Depending on how far the current calibration values deviate from the expected range, a criticality of the calibration issue may be determined by the system.” (Wang [0062])
Rationale:
Wang assigns a corresponding severity level to each of several distinct detected vehicle-condition types, labeling the resulting levels in FIG. 3B as “Severity is ‘High’” (354), “Severity is ‘Medium’” (360), and “Severity is ‘Low’” (364). Wang determines the severity of a sensor-calibration issue from the extent to which calibration values deviate from the expected range, and uses severity to select the maintenance response, dispatching vehicles “to needed facilities based on the severity of the issue(s)” at [0038].
Wang does not classify Prokhorov’s mileage-accumulation and large-jolt trigger types. This limitation is rendered obvious by the combination: Wang supplies a known framework for assigning a corresponding severity level to each of several detected condition types, and applying that framework to each of the trigger types on which the combined system monitors the vehicle would have been obvious to one of ordinary skill in the art.
In the combination, a person of ordinary skill in the art would have applied Wang’s severity framework to both operative trigger types, classifying mileage accumulation according to maintenance urgency and classifying a jolt according to its measured magnitude and resulting calibration risk. Each type of sensor calibration trigger is thereby associated with a corresponding severity level.
Motivation to Combine Levinson, Prokhorov, and Wang
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson, Prokhorov, and Wang before them, to apply Wang’s high, medium, and low severity classification to each of Prokhorov’s mileage-accumulation and large-jolt sensor-calibration-trigger types. Wang classifies detected vehicle conditions by severity in FIG. 3B, determines the criticality of a sensor-calibration issue based on the extent of calibration-value deviation at [0062], and uses severity to determine the appropriate maintenance response at [0038]. Applying that known classification framework to Prokhorov’s known calibration-trigger types would have predictably permitted the combined system to select and prioritize the calibration response according to the seriousness of the detected condition.
Regarding Claim 3,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 3.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors for monitoring the vehicle comprise one or more inertial measurement units (IMUs).
See at least:
“Inertial measurement units (IMUs) may also be used to help calibrate LIDAR sensors and cameras. . . . Thus, in a situation where a sensor measurement may not be expected, the current velocity may be captured to identify other sensor data previously captured by the same sensor and other sensors to aid in detecting whether a sensor needs to be calibrated.” (Levinson [0152])
Rationale:
Levinson employs inertial measurement units among the sensors that determine whether a sensor needs to be calibrated. The IMUs are therefore among the sensors that perform the recited monitoring.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to monitor the vehicle for Prokhorov’s trigger types using the inertial measurement units Levinson already carries and already employs at [0152] to aid in detecting whether a sensor needs calibration. Prokhorov’s jolt trigger is itself measured by an IMU at [0031], so the two references assign the same sensing function to the same known component.
Claim Limitations Not Explicitly Disclosed by the Combination of All References
After combining the teachings of Levinson and Prokhorov, all limitations of Claim 3 are disclosed or rendered obvious.
Regarding Claim 4,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 4.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors of the vehicle include any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors.
See at least:
“[T]he autonomous vehicle system 3602 may include many types of sensors or any quantity of sensors to facilitate perception, including image capture sensors, audio capture sensors, LIDAR, RADAR, SONAR, GPS, and IMU.” (Levinson [0142])
Rationale:
Levinson’s LIDAR sensors, image capture sensors, and RADAR sensors are within the recited alternatives. The limitation recites its alternatives in the disjunctive, and disclosure of any one satisfies it.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to apply the combined trigger-based calibration process to the LIDAR, image, and radar sensors Levinson carries at [0142]. Levinson states at [0146] that each sensor may experience drift or miscalibration over time, and Prokhorov at [0031] identifies the jolt as an event that may have affected the alignment of a sensor, so the sensors the combination calibrates are the sensors both references identify as subject to miscalibration.
Regarding Claim 5,
The combination of Levinson and Prokhorov establishes the computing system of Claim 4, which is the basis for Claim 5.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors for monitoring the vehicle are included in a sensor suite of the vehicle.
See at least:
“[S]ensors 3610 included in the AV system 3602, which may include LIDAR sensors 3604, RADAR sensors 3620, other sensors 3660, IMUS 3612, cameras 3614, odometry sensors 3616, GPS 3618, and SONAR sensors 3622 . . .” (Levinson [0161])
Rationale:
Levinson designates the vehicle’s sensors collectively as sensors 3610 carried on AV system 3602 and enumerates the constituent sensor types. A collectively designated group of sensors carried on the vehicle is a sensor suite of the vehicle, and Levinson’s use of different terminology does not avoid the limitation.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to draw the monitoring measurements from the collectively carried sensors 3610 Levinson identifies at [0161]. Levinson fuses data across those sensors to detect a calibration condition, and Prokhorov’s triggers are measured by sensors of the same kinds, so monitoring for Prokhorov’s triggers within Levinson’s existing sensor group uses known components for their established sensing function and avoids duplicating sensing hardware already present on the vehicle.
Regarding Claim 6,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 6.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitation:
wherein the multiple types of sensor calibration triggers comprise at least one of: windshield breakage, windshield replacement, a shock or vibration experienced by the vehicle, an over-the-air update to software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.
Levinson recognizes at [0148] and [0164] that a sensor may be displaced by a bird strike or other impact while the vehicle is in operation. Those conditions are not relied upon here, however, because the antecedent phrase “the multiple types of sensor calibration triggers” refers to the trigger set established with respect to Claim 1, which is Prokhorov’s mileage-accumulation and large-jolt trigger set. Levinson’s conditions are cumulative to that set and do not establish this limitation.
Disclosure by Prokhorov
Prokhorov discloses:
wherein the multiple types of sensor calibration triggers comprise at least one of: windshield breakage, windshield replacement, a shock or vibration experienced by the vehicle, an over-the-air update to software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
A large jolt arising when the vehicle hits a severe bump or pothole is a shock experienced by the vehicle, and it is one of the two trigger types established with respect to Claim 1. The limitation recites its alternatives in the disjunctive, and disclosure of one satisfies it.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to include among the monitored trigger types the shock Prokhorov detects when the vehicle strikes a severe bump or pothole, for the reasons stated in the motivation to combine set forth with respect to Claim 1, and because Levinson independently recognizes at [0148] that mechanical disturbance of a sensor while the vehicle is in operation produces the misalignment its calibration system exists to correct. Monitoring for the disturbance itself, as Prokhorov teaches, permits that misalignment to be addressed when it arises rather than after it degrades the perception data on which Levinson’s vehicle navigates.
Regarding Claim 8,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 8.
For purposes of this rejection, the limitation is interpreted as requiring the trigger-detection operation to include determining that the measured event falls within a categorical severity level.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitations:
wherein automatically performing sensor calibration includes detecting a severity level of the sensor calibration trigger,
the sensor calibration trigger being identified based at least in part on the severity level.
Levinson quantifies the magnitude of a detected miscalibration, determining at [0164] that a sensor was “horizontally rotated by 9 degrees and vertically rotated by 1 degree” and determining at [0170] a quantifiable measure of how the perception system is affected. Those determinations measure the severity of the resulting miscalibration rather than the severity of the trigger, and are not relied upon for these limitations.
Disclosure by Prokhorov
Levinson and Prokhorov render obvious:
wherein automatically performing sensor calibration includes detecting a severity level of the sensor calibration trigger,
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov identifies a “large jolt” as a condition indicating possible sensor misalignment, and its IMU measures the acceleration the vehicle experiences at [0026]. Prokhorov does not use the term “severity level.” This limitation is rendered obvious for the following reason: classifying a measured jolt as “large” assigns a categorical magnitude to the trigger, and that classification is the detected severity level of the trigger.
the sensor calibration trigger being identified based at least in part on the severity level.
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle . . . that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
In the combined system, detection of that severity level identifies the jolt as the sensor calibration trigger and initiates Levinson’s automatic calibration process.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to treat the large jolt Prokhorov’s IMU detects as the sensor calibration trigger that initiates Levinson’s onboard calibration process, so that the process is initiated for disturbances significant enough to affect sensor alignment and not for ordinary road inputs. Doing so avoids expending the processing time and computational effort Levinson identifies at [0169], while still avoiding the sub-optimal mode of operation Levinson identifies at [0170] as the consequence of continuing to operate with a miscalibrated sensor.
Regarding Claim 9,
The combination of Levinson and Prokhorov establishes the computing system of Claim 1, which is the basis for Claim 9.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitation:
wherein the executed instructions cause the computing system to output a calibration alert, either on a display screen of the vehicle or by wirelessly transmitting the calibration alert to a display screen of a computing device of a user.
Levinson at [0171] presents information concerning a calibration issue to a teleoperator system through an interface on that system. Levinson does not disclose outputting a calibration alert on a display screen of the vehicle, and its statement at [0174] that network 3804 “can include links using technologies such as Ethernet, 802.11, worldwide interoperability for microwave access (WiMAX), 3G, 4G, CDMA, digital subscriber line (DSL), etc.” does not establish that the communication is wireless. Levinson is accordingly not relied upon for this limitation.
Disclosure by Prokhorov
Prokhorov discloses:
wherein the executed instructions cause the computing system to output a calibration alert, either on a display screen of the vehicle or by wirelessly transmitting the calibration alert to a display screen of a computing device of a user.
See at least:
“The vehicle systems 116 can include one or more vehicle interfaces 118 that can allow the driver to communicate with the computing device 100 or receive information from the computing device 100. The vehicle interfaces 118 can include, for example, one or more interactive displays . . .” (Prokhorov [0020])
“Users can be reminded to calibrate the sensors 130 using a notification or alert (for example, using a vehicle interface 118 such as an interactive display or audio system).” (Prokhorov [0031])
Rationale:
A notification informing the user that the sensors require calibration, output on the vehicle’s interactive display, is a calibration alert output on a display screen of the vehicle. The limitation recites its two alternatives in the disjunctive, and disclosure of the first satisfies it.
Prokhorov’s installed applications permit CPU 102 to implement the auto-calibration features at [0019], and computing device 100 outputs information to the user through vehicle interface 118, including an interactive display. The executed instructions therefore cause the computing system to output the recited calibration alert.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to output Prokhorov’s calibration notification on the vehicle’s interactive display upon detection of one of Prokhorov’s trigger types. The modification would inform the user that the vehicle experienced an event warranting sensor calibration. Prokhorov discloses the notification for that purpose in the same paragraph that discloses the trigger types, and Levinson at [0171] independently reports a calibration issue outside the calibration operation itself, so reporting the condition to the user adds a known and compatible function to the combined system.
Regarding Claim 10,
Disclosure by Levinson
Levinson discloses:
A non-transitory computer readable medium storing instructions
See at least:
“The term “computer readable medium” refers to any tangible medium that participates in providing instructions to processor 4006 or 4106 for execution. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media.” (Levinson [0184])
“Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CDROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.” (Levinson [0185])
Rationale:
Levinson’s computer readable medium is a tangible medium, and the enumerated forms — disks, magnetic tape, CDROM, RAM, PROM, EPROM, FLASH-EPROM, and memory chips or cartridges — are physical storage articles rather than transitory signals. The medium holds the instructions it provides to the processor for execution, and is therefore a non-transitory computer readable medium storing instructions.
that, when executed by one or more processors of a computing system of a vehicle,
See at least:
“[A] miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
Rationale:
Levinson executes the calibration instructions on processors carried onboard AV system 3602, and discloses at [0184] that the platform performs its operations by the processor executing instruction sequences stored in system memory. The processors are therefore processors of a computing system of a vehicle.
cause the computing system to:
See at least:
“One or more of the modules included in memory 4010 or 4110 can be configured to provide or consume outputs to implement one or more functions described herein.” (Levinson [0187])
Rationale:
Execution of the stored instruction modules causes the onboard computing system to perform the disclosed calibration functions.
when the vehicle is operating autonomously or semi-autonomously on a road network,
See at least:
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment, such as static objects like lane markings and curbs as well as dynamic objects like moving vehicles in real-time and continuously.” (Levinson [0147])
Rationale:
Lane markings, curbs, and other moving vehicles are features of a road network. Levinson performs these functions in real time and continuously while the vehicle is operating.
monitor the vehicle,
See at least:
“In one embodiment, the autonomous vehicle system 3602 may track the drift of each sensor over time.” (Levinson [0146])
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment . . . in real-time and continuously.” (Levinson [0147])
Rationale:
Tracking each sensor’s drift over time and continuously determining the vehicle’s location and environment is repeated evaluation of the operating vehicle, not an isolated calibration measurement.
using one or more sensors,
See at least:
“[T]he autonomous vehicle system 3602 may include many types of sensors or any quantity of sensors to facilitate perception, including image capture sensors, audio capture sensors, LIDAR, RADAR, SONAR, GPS, and IMU.” (Levinson [0142])
Rationale:
The vehicle’s onboard sensors supply the measurements on which perception system 3702 performs the monitoring identified above.
based on the monitoring,
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
The indication arises from sensor measurements taken while the vehicle is operating. Calibration detection module 3730 acts on the result of that monitoring.
detect a sensor calibration trigger
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
Calibration detection module 3730 registers a condition indicating that a sensor requires calibration and initiates the calibration sequence. That is the detection of a trigger for sensor calibration. In the combination, the condition so registered is a Prokhorov trigger type mapped below.
and in response to detecting the sensor calibration trigger,
See at least:
“Once a sensor is identified as potentially miscalibrated, log file data may be retrieved from a log file store 3716 to assist in calibrating the identified sensor.” (Levinson [0162])
Rationale:
Levinson’s “[o]nce a sensor is identified” establishes that the calibration sequence commences only upon, and because of, the detection.
automatically performing sensor calibration
See at least:
“By aligning detected edges from laser returns of LIDAR sensors with the same edges of objects within captured images, a miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
“In this way, the AV system 3602 has been self-calibrated while in operation, without having to stop and interrupt the user experience.” (Levinson [0164])
“[T]he AV system 3602 may automatically identify a course of action based on the confirmed sensor miscalibration.” (Levinson [0178])
Rationale:
Levinson performs the calibration by onboard processors, on the fly, while the vehicle remains in operation and without operator action.
Levinson’s optional offline calibration at [0169] and optional teleoperator consultation at [0171] are alternative embodiments and do not negate the disclosure relied upon. See MPEP 2123.
and (ii) transmitting a recalibration command
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
“Planner 364 may transmit steering and propulsion commands . . . to motion controller 362. Motion controller 362 subsequently may convert any of the commands . . . into control signals . . . to implement changes.” (Levinson [0065])
“Diagram 400 depicts an autonomous vehicle controller (“AV”) 447 disposed in an autonomous vehicle 430, which, in turn, includes a number of sensors 470 coupled to autonomous vehicle controller 447.” (Levinson [0068])
“In some examples, computing platforms 4000 and 4100 may be used to implement computer programs, applications, methods, processes, algorithms, or other software to perform the above-described techniques.” (Levinson [0181])
“[T]he structures and constituent elements above, as well as their functionality, may be aggregated with one or more other structures or elements.” (Levinson [0188])
Rationale:
Levinson discloses vehicle sensors coupled to the onboard controller at [0068], onboard processors that determine intrinsic calibration parameters for the identified sensor at [0163] and cause that sensor to focus and adjust its lens properties at [0151], and a control architecture in which processor-generated commands are transmitted and converted into signals that implement change at [0065].
Levinson does not expressly recite transmitting a command to the sensor. Given those teachings, it would nonetheless have been obvious to one of ordinary skill in the art to implement Levinson’s calibration by transmitting to the identified sensor a command conveying or invoking the computed intrinsic calibration parameters. The implementation uses Levinson’s disclosed command-and-control approach for its established purpose and predictably produces the sensor adjustment Levinson discloses.
Reliance on the controller architecture of FIGS. 3A and 4 together with the calibration architecture of FIGS. 36–37 and 41 is supported by the reference itself: Levinson at [0181] directs that computing platforms 4000 and 4100 implement software “to perform the above-described techniques,” and at [0188] contemplates that its disclosed structures and their functionality may be aggregated.
to each of the one or more sensors,
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
Rationale:
Levinson determines calibration parameters for an individually identified sensor and at [0164] stores the resulting correction in association with that sensor. Where the detection process identifies more than one affected sensor, directing a corresponding command to each identified sensor is the ordinary iterative application of that process.
to trigger the one or more sensors
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
Rationale:
Levinson identifies the sensor, not the processor, as the component that focuses and adjusts its lens properties. In the implementation described above, the transmitted command initiates that sensor-side adjustment.
to intrinsically recalibrate
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor. For example, a LIDAR sensor may require an intrinsic calibration of reflectivity values captured by laser returns.” (Levinson [0163])
“Cameras, on the other hand, may have other intrinsic calibration parameters, such as color mapping, focal length, image positioning, scaling/skew factors, and lens distortion that may affect the imaging process.” (Levinson [0163])
Rationale:
Levinson identifies its calibration as intrinsic and identifies the internal sensor characteristics adjusted: lidar reflectivity, and camera color mapping, focal length, image positioning, scaling, skew, and lens distortion. Levinson assigns intrinsic and extrinsic calibration to separate modules, 3704 and 3706.
in accordance with the recalibration command.
See at least:
“The intrinsic sensor calibration module 3704 may operate in conjunction with a data transform module 3708 and a generative model module 3710 to perform the computations necessary to converge on the optimal intrinsic calibration parameters for the miscalibrated sensor.” (Levinson [0163])
Rationale:
The parameters are computed for the particular identified sensor. It would have been obvious for the command to convey or invoke those parameters so that the sensor’s adjustment conforms to them.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitations:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
of one of the multiple types of sensor calibration triggers;
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
Disclosure by Prokhorov
Prokhorov discloses:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
See at least:
“[T]he installed applications 112 includ[e] programs or apps that permit the CPU 102 to implement the autonomous features of the vehicle 200 as well as the auto-calibration features.” (Prokhorov [0019])
“Among other information measurable by the sensors 130, the sensors 130 can detect vehicle speed, vehicle direction, vehicle acceleration, vehicle rotation, vehicle location, environmental weather conditions, traffic conditions, and road conditions.” (Prokhorov [0022])
“If the sensors 130 capture data for a dead-reckoning system, data relating to wheel revolution speeds, travel distance, steering angle, and steering angular rate of change can be captured.” (Prokhorov [0026])
“The notifications or alerts can be issued periodically or at certain triggers, for example, based on . . . a defined number of miles driven since the last calibration. Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov discloses two calibration-trigger types, each corresponding to an event or scenario the vehicle experiences on the road and measures with its own sensors: accumulated mileage since the last calibration, measured through the travel-distance and wheel-revolution data Prokhorov’s sensors capture at [0026]; and a large jolt arising when the vehicle hits a severe bump or pothole, measured by an onboard IMU. Prokhorov’s sensors likewise measure traffic and road conditions at [0022], and its applications implement the vehicle’s autonomous features at [0019].
A person of ordinary skill in the art would have understood the accumulated mileage of [0031] to be derived from the travel-distance and wheel-revolution data that [0026] identifies as captured by the vehicle’s sensors. Levinson at [0142] and [0147] supplies the autonomous on-road operation during which the combined system monitors for and detects the trigger.
These are the trigger types for which Levinson’s onboard system monitors in the combination. Prokhorov’s elapsed-time trigger is not relied upon.
of one of the multiple types of sensor calibration triggers;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov discloses IMU detection of the large-jolt trigger, one of the two trigger types identified above. In the combination, that trigger is the condition supplied to Levinson’s calibration detection module 3730.
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle . . . that may have affected the alignment of the sensor 130. IMUs 132 can be installed very near or incorporated into the sensors 130 to more accurately detect or predict whether such sensors 130 have become misaligned.” (Prokhorov [0031])
Rationale:
For a detected large-jolt trigger, Prokhorov uses IMUs installed near or incorporated into respective sensors to determine or predict whether those sensors became misaligned, confining the determination to “such sensors 130.” Which sensors are evaluated thus follows from which type of trigger was detected: a jolt-type trigger implicates the sensors whose alignment a mechanical shock disturbs, evaluated through the co-located IMUs. The claim requires identification “based on,” rather than exclusively based on, the trigger type.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to configure Levinson’s onboard autonomous-vehicle calibration system to monitor for Prokhorov’s mileage-accumulation and large-jolt calibration-trigger types and, upon detection of a large-jolt trigger, to use Prokhorov’s sensor-associated IMUs to identify the sensors affected by the jolt before applying Levinson’s onboard calibration process to those sensors.
The references address the same recognized problem. Levinson discloses at [0148] that a sensor may become out of alignment while the vehicle is in operation, and Prokhorov discloses at [0024] that sensors may become misaligned due to normal use or sudden jolts. Levinson supplies the onboard architecture that detects a calibration condition, identifies the miscalibrated sensor, determines intrinsic calibration parameters, and calibrates the sensor while the vehicle remains in operation. Prokhorov supplies the event-monitoring techniques that recognize when an operating event warrants calibration and which sensors that event affected. That Prokhorov uses the detected trigger to alert a user does not limit its teaching of the trigger itself; the question is what the combined teachings would have suggested, and Levinson supplies the automatic calibration the detected trigger initiates. See MPEP 2145(III).
The modification employs known components for their established functions and requires no unconventional hardware. Levinson already relies at [0146] on GPS, IMU, RADAR, SONAR, and cameras during operation, and includes odometry sensors 3616 among the monitored sensors at [0161]. Prokhorov’s mileage and jolt monitors would supply trigger indications to Levinson’s calibration detection module 3730, and Prokhorov’s sensor-associated IMUs would identify the sensors potentially affected by a detected jolt.
A person of ordinary skill would have had a reasonable expectation of success, as both references employ onboard vehicle sensors, processors, memory, IMUs, and sensor-calibration functionality, and the integration requires only conventional exchange of sensor data and trigger indications within an autonomous-vehicle computing system.
The modification would predictably:
detect event-induced sensor miscalibration promptly, rather than after it manifests in degraded perception data;
avoid continued reliance on a sensor affected by a detected event, which Levinson identifies at [0170] as forcing the vehicle into a sub-optimal mode of operation;
avoid unnecessary calibration of unaffected sensors, conserving the processing time and computational effort Levinson identifies at [0169]; and
maintain perception accuracy and safe autonomous operation without a controlled calibration facility, consistent with Levinson’s statement at [0147] that its determinations are undertaken for safety reasons and operational efficiency.
The combination applies a known event-monitoring and affected-sensor-identification technique to a known autonomous-vehicle calibration system ready for improvement, producing the predictable result of initiating onboard calibration for the sensors affected by a measurable operating event. See KSR International Co. v. Teleflex Inc., 550 U.S. 398, 416–18 (2007); MPEP 2143(I)(A), (D).
Regarding Claim 11,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 11.
Claim Limitations Not Explicitly Disclosed by the Combination of Levinson and Prokhorov
After combining the teachings of Levinson and Prokhorov, the following claim limitation is not explicitly disclosed:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
Levinson quantifies the effect of a miscalibration at [0170] and differentiates its calibration response at [0164] and [0169]. Prokhorov discloses the mileage-accumulation and large-jolt trigger types established with respect to Claim 10 and distinguishes normal use from sudden jolts at [0024]. Neither reference associates each such trigger type with a corresponding severity level.
Disclosure by Wang
Levinson, Prokhorov, and Wang render obvious:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
See at least:
“If the diagnostics service determines that sensor calibration values exceed an acceptable range determined by the server in the back end . . . then the criticality level is also classified as high . . . . However, if the diagnostics service does not detect the above, but does detect that there are non-critical warnings produced by hardware within the autonomous vehicle . . . then the criticality level is classified as medium . . . . If not, and the diagnostics service or remote back end server indicates that the autonomous vehicle is due for preventative maintenance . . . then the criticality level is classified as low.” (Wang [0063]–[0065])
“Depending on how far the current calibration values deviate from the expected range, a criticality of the calibration issue may be determined by the system.” (Wang [0062])
Rationale:
Wang assigns a corresponding severity level to each of several distinct detected vehicle-condition types, labeling the resulting levels in FIG. 3B as “Severity is ‘High’” (354), “Severity is ‘Medium’” (360), and “Severity is ‘Low’” (364). Wang determines the severity of a sensor-calibration issue from the extent to which calibration values deviate from the expected range, and uses severity to select the maintenance response, dispatching vehicles “to needed facilities based on the severity of the issue(s)” at [0038].
Wang does not classify Prokhorov’s mileage-accumulation and large-jolt trigger types. This limitation is rendered obvious by the combination: Wang supplies a known framework for assigning a corresponding severity level to each of several detected condition types, and applying that framework to each of the trigger types on which the combined system monitors the vehicle would have been obvious to one of ordinary skill in the art.
In the combination, a person of ordinary skill in the art would have applied Wang’s severity framework to both operative trigger types, classifying mileage accumulation according to maintenance urgency and classifying a jolt according to its measured magnitude and resulting calibration risk. Each type of sensor calibration trigger is thereby associated with a corresponding severity level.
Motivation to Combine Levinson, Prokhorov, and Wang
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson, Prokhorov, and Wang before them, to apply Wang’s high, medium, and low severity classification to each of Prokhorov’s mileage-accumulation and large-jolt sensor-calibration-trigger types. Wang classifies detected vehicle conditions by severity in FIG. 3B, determines the criticality of a sensor-calibration issue based on the extent of calibration-value deviation at [0062], and uses severity to determine the appropriate maintenance response at [0038]. Applying that known classification framework to Prokhorov’s known calibration-trigger types would have predictably permitted the combined system to select and prioritize the calibration response according to the seriousness of the detected condition.
Regarding Claim 12,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 12.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors for monitoring the vehicle comprise one or more inertial measurement units (IMUs).
See at least:
“Inertial measurement units (IMUs) may also be used to help calibrate LIDAR sensors and cameras. . . . Thus, in a situation where a sensor measurement may not be expected, the current velocity may be captured to identify other sensor data previously captured by the same sensor and other sensors to aid in detecting whether a sensor needs to be calibrated.” (Levinson [0152])
Rationale:
Levinson employs inertial measurement units among the sensors that determine whether a sensor needs to be calibrated. The IMUs are therefore among the sensors that perform the recited monitoring.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to monitor the vehicle for Prokhorov’s trigger types using the inertial measurement units Levinson already carries and already employs at [0152] to aid in detecting whether a sensor needs calibration. Prokhorov’s jolt trigger is itself measured by an IMU at [0031], so the two references assign the same sensing function to the same known component.
Regarding Claim 13,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 13.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors of the vehicle include any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors.
See at least:
“[T]he autonomous vehicle system 3602 may include many types of sensors or any quantity of sensors to facilitate perception, including image capture sensors, audio capture sensors, LIDAR, RADAR, SONAR, GPS, and IMU.” (Levinson [0142])
Rationale:
Levinson’s LIDAR sensors, image capture sensors, and RADAR sensors are within the recited alternatives. The limitation recites its alternatives in the disjunctive, and disclosure of any one satisfies it.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to apply the combined trigger-based calibration process to the LIDAR, image, and radar sensors Levinson carries at [0142]. Levinson states at [0146] that each sensor may experience drift or miscalibration over time, and Prokhorov at [0031] identifies the jolt as an event that may have affected the alignment of a sensor, so the sensors the combination calibrates are the sensors both references identify as subject to miscalibration.
Regarding Claim 14,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 13, which is the basis for Claim 14.
Disclosure by Levinson
Levinson discloses:
wherein the one or more sensors for monitoring the vehicle are included in a sensor suite of the vehicle.
See at least:
“[S]ensors 3610 included in the AV system 3602, which may include LIDAR sensors 3604, RADAR sensors 3620, other sensors 3660, IMUS 3612, cameras 3614, odometry sensors 3616, GPS 3618, and SONAR sensors 3622 . . .” (Levinson [0161])
Rationale:
Levinson designates the vehicle’s sensors collectively as sensors 3610 carried on AV system 3602 and enumerates the constituent sensor types. A collectively designated group of sensors carried on the vehicle is a sensor suite of the vehicle, and Levinson’s use of different terminology does not avoid the limitation.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to draw the monitoring measurements from the collectively carried sensors 3610 Levinson identifies at [0161]. Levinson fuses data across those sensors to detect a calibration condition, and Prokhorov’s triggers are measured by sensors of the same kinds, so monitoring for Prokhorov’s triggers within Levinson’s existing sensor group uses known components for their established sensing function and avoids duplicating sensing hardware already present on the vehicle.
Regarding Claim 15,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 15.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitation:
wherein the multiple types of sensor calibration triggers comprise: at least one of: windshield breakage, windshield replacement, a shock or vibration experienced by the vehicle, an over-the-air update to software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.
Levinson recognizes at [0148] and [0164] that a sensor may be displaced by a bird strike or other impact while the vehicle is in operation. Those conditions are not relied upon here, however, because the antecedent phrase “the multiple types of sensor calibration triggers” refers to the trigger set established with respect to Claim 10, which is Prokhorov’s mileage-accumulation and large-jolt trigger set. Levinson’s conditions are cumulative to that set and do not establish this limitation.
Disclosure by Prokhorov
Prokhorov discloses:
wherein the multiple types of sensor calibration triggers comprise: at least one of: windshield breakage, windshield replacement, a shock or vibration experienced by the vehicle, an over-the-air update to software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
A large jolt arising when the vehicle hits a severe bump or pothole is a shock experienced by the vehicle, and it is one of the two trigger types established with respect to Claim 10. The limitation recites its alternatives in the disjunctive, and disclosure of one satisfies it.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to include among the monitored trigger types the shock Prokhorov detects when the vehicle strikes a severe bump or pothole, for the reasons stated in the motivation to combine set forth with respect to Claim 10, and because Levinson independently recognizes at [0148] that mechanical disturbance of a sensor while the vehicle is in operation produces the misalignment its calibration system exists to correct. Monitoring for the disturbance itself, as Prokhorov teaches, permits that misalignment to be addressed when it arises rather than after it degrades the perception data on which Levinson’s vehicle navigates.
Regarding Claim 17,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 17.
For purposes of this rejection, the limitation is interpreted as requiring the trigger-detection operation to include determining that the measured event falls within a categorical severity level.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitations:
wherein automatically performing sensor calibration includes detecting a severity level of the sensor calibration trigger,
the sensor calibration trigger being identified based at least in part on the severity level.
Levinson quantifies the magnitude of a detected miscalibration, determining at [0164] that a sensor was “horizontally rotated by 9 degrees and vertically rotated by 1 degree” and determining at [0170] a quantifiable measure of how the perception system is affected. Those determinations measure the severity of the resulting miscalibration rather than the severity of the trigger, and are not relied upon for these limitations.
Disclosure by Prokhorov
Levinson and Prokhorov render obvious:
wherein automatically performing sensor calibration includes detecting a severity level of the sensor calibration trigger,
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov identifies a “large jolt” as a condition indicating possible sensor misalignment, and its IMU measures the acceleration the vehicle experiences at [0026]. Prokhorov does not use the term “severity level.” This limitation is rendered obvious for the following reason: classifying a measured jolt as “large” assigns a categorical magnitude to the trigger, and that classification is the detected severity level of the trigger.
the sensor calibration trigger being identified based at least in part on the severity level.
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle . . . that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
In the combined system, detection of that severity level identifies the jolt as the sensor calibration trigger and initiates Levinson’s automatic calibration process.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to treat the large jolt Prokhorov’s IMU detects as the sensor calibration trigger that initiates Levinson’s onboard calibration process, so that the process is initiated for disturbances significant enough to affect sensor alignment and not for ordinary road inputs. Doing so avoids expending the processing time and computational effort Levinson identifies at [0169], while still avoiding the sub-optimal mode of operation Levinson identifies at [0170] as the consequence of continuing to operate with a miscalibrated sensor.
Regarding Claim 18,
The combination of Levinson and Prokhorov establishes the non-transitory computer readable medium of Claim 10, which is the basis for Claim 18.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly disclose the following claim limitation:
wherein the executed instructions cause the computing system to output a calibration alert, either on a display screen of the vehicle or by wirelessly transmitting a calibration alert to a display screen of the vehicle.
Levinson at [0171] presents information concerning a calibration issue to a teleoperator system through an interface on that system. Levinson does not disclose outputting a calibration alert on a display screen of the vehicle, and is accordingly not relied upon for this limitation.
Disclosure by Prokhorov
Prokhorov discloses:
wherein the executed instructions cause the computing system to output a calibration alert, either on a display screen of the vehicle or by wirelessly transmitting a calibration alert to a display screen of the vehicle.
See at least:
“The vehicle systems 116 can include one or more vehicle interfaces 118 that can allow the driver to communicate with the computing device 100 or receive information from the computing device 100. The vehicle interfaces 118 can include, for example, one or more interactive displays . . .” (Prokhorov [0020])
“Users can be reminded to calibrate the sensors 130 using a notification or alert (for example, using a vehicle interface 118 such as an interactive display or audio system).” (Prokhorov [0031])
Rationale:
A notification informing the user that the sensors require calibration, output on the vehicle’s interactive display, is a calibration alert output on a display screen of the vehicle. The limitation recites its two alternatives in the disjunctive, and disclosure of the first satisfies it.
Prokhorov’s installed applications permit CPU 102 to implement the auto-calibration features at [0019], and computing device 100 outputs information to the user through vehicle interface 118, including an interactive display. The executed instructions therefore cause the computing system to output the recited calibration alert.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to output Prokhorov’s calibration notification on the vehicle’s interactive display upon detection of one of Prokhorov’s trigger types. The modification would inform the user that the vehicle experienced an event warranting sensor calibration. Prokhorov discloses the notification for that purpose in the same paragraph that discloses the trigger types, and Levinson at [0171] independently reports a calibration issue outside the calibration operation itself, so reporting the condition to the user adds a known and compatible function to the combined system.
Regarding Claim 19,
Disclosure by Levinson
Levinson teaches:
A computer-implemented method for monitoring a vehicle for sensor recalibration,
See at least:
“FIG. 39 is a high-level flow diagram illustrating a process for calibration for autonomous vehicle operation, according to some examples. Data associated with a sensor measurement of a perceived object is received 3900. . . . A calibration parameter associated with the sensor measurement is determined 3906 based on the retrieved log file data.” (Levinson [0180])
Rationale:
Levinson sets out a process for calibrating the sensors of an autonomous vehicle, carried out by the vehicle’s computing system on received sensor measurements. Levinson further tracks the drift of each sensor over time at [0146]. The taught process is therefore a computer-implemented method for monitoring a vehicle for sensor recalibration.
the method being performed by one or more processors of the vehicle
See at least:
“[A] miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
Rationale:
Levinson performs the calibration process on processors carried onboard AV system 3602, rather than at a remote or service-facility system.
and comprising: when the vehicle is operating autonomously or semi-autonomously on a road network,
See at least:
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment, such as static objects like lane markings and curbs as well as dynamic objects like moving vehicles in real-time and continuously.” (Levinson [0147])
Rationale:
Lane markings, curbs, and other moving vehicles are features of a road network. Levinson performs these functions in real time and continuously while the vehicle is operating.
monitoring the vehicle,
See at least:
“In one embodiment, the autonomous vehicle system 3602 may track the drift of each sensor over time.” (Levinson [0146])
“[T]he autonomous vehicle system 3602 determines its location and the surrounding environment . . . in real-time and continuously.” (Levinson [0147])
Rationale:
Tracking each sensor’s drift over time and continuously determining the vehicle’s location and environment is repeated evaluation of the operating vehicle, not an isolated calibration measurement.
using one or more sensors,
See at least:
“[T]he autonomous vehicle system 3602 may include many types of sensors or any quantity of sensors to facilitate perception, including image capture sensors, audio capture sensors, LIDAR, RADAR, SONAR, GPS, and IMU.” (Levinson [0142])
Rationale:
The vehicle’s onboard sensors supply the measurements on which perception system 3702 performs the monitoring identified above.
based on the monitoring,
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
The indication arises from sensor measurements taken while the vehicle is operating. Calibration detection module 3730 acts on the result of that monitoring.
detecting a sensor calibration trigger
See at least:
“Upon receiving an indication of an anomaly in a sensor measurement from a sensor of the sensors 3610 included in the AV system 3602 . . . a perception system 3702 may utilize a calibration detection module 3730 to process the anomalous sensor measurements.” (Levinson [0161])
Rationale:
Calibration detection module 3730 registers a condition indicating that a sensor requires calibration and initiates the calibration sequence. That is the detection of a trigger for sensor calibration. In the combination, the condition so registered is a Prokhorov trigger type mapped below.
and in response to detecting the sensor calibration trigger,
See at least:
“Once a sensor is identified as potentially miscalibrated, log file data may be retrieved from a log file store 3716 to assist in calibrating the identified sensor.” (Levinson [0162])
Rationale:
Levinson’s “[o]nce a sensor is identified” establishes that the calibration sequence commences only upon, and because of, the detection.
automatically performing sensor calibration
See at least:
“By aligning detected edges from laser returns of LIDAR sensors with the same edges of objects within captured images, a miscalibrated camera may be calibrated, on the fly in real time by onboard processors on the AV system 3602.” (Levinson [0151])
“In this way, the AV system 3602 has been self-calibrated while in operation, without having to stop and interrupt the user experience.” (Levinson [0164])
“[T]he AV system 3602 may automatically identify a course of action based on the confirmed sensor miscalibration.” (Levinson [0178])
Rationale:
Levinson performs the calibration by onboard processors, on the fly, while the vehicle remains in operation and without operator action.
Levinson’s optional offline calibration at [0169] and optional teleoperator consultation at [0171] are alternative embodiments and do not negate the teaching relied upon. See MPEP 2123.
and (ii) transmitting a recalibration command
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
“Planner 364 may transmit steering and propulsion commands . . . to motion controller 362. Motion controller 362 subsequently may convert any of the commands . . . into control signals . . . to implement changes.” (Levinson [0065])
“Diagram 400 depicts an autonomous vehicle controller (“AV”) 447 disposed in an autonomous vehicle 430, which, in turn, includes a number of sensors 470 coupled to autonomous vehicle controller 447.” (Levinson [0068])
“In some examples, computing platforms 4000 and 4100 may be used to implement computer programs, applications, methods, processes, algorithms, or other software to perform the above-described techniques.” (Levinson [0181])
“[T]he structures and constituent elements above, as well as their functionality, may be aggregated with one or more other structures or elements.” (Levinson [0188])
Rationale:
Levinson teaches vehicle sensors coupled to the onboard controller at [0068], onboard processors that determine intrinsic calibration parameters for the identified sensor at [0163] and cause that sensor to focus and adjust its lens properties at [0151], and a control architecture in which processor-generated commands are transmitted and converted into signals that implement change at [0065].
Levinson does not expressly recite transmitting a command to the sensor. Given those teachings, it would nonetheless have been obvious to one of ordinary skill in the art to implement Levinson’s calibration by transmitting to the identified sensor a command conveying or invoking the computed intrinsic calibration parameters. The implementation uses Levinson’s taught command-and-control approach for its established purpose and predictably produces the sensor adjustment Levinson teaches.
Reliance on the controller architecture of FIGS. 3A and 4 together with the calibration architecture of FIGS. 36–37 and 41 is supported by the reference itself: Levinson at [0181] directs that computing platforms 4000 and 4100 implement software “to perform the above-described techniques,” and at [0188] contemplates that its taught structures and their functionality may be aggregated.
to each of the one or more sensors,
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor.” (Levinson [0163])
Rationale:
Levinson determines calibration parameters for an individually identified sensor and at [0164] stores the resulting correction in association with that sensor. Where the detection process identifies more than one affected sensor, directing a corresponding command to each identified sensor is the ordinary iterative application of that process.
to trigger the one or more sensors
See at least:
“A miscalibrated camera may be calibrated by using LIDAR data to help identify edges of objects, thus enabling the camera to focus and adjust lens properties to sharpen images.” (Levinson [0151])
Rationale:
Levinson identifies the sensor, not the processor, as the component that focuses and adjusts its lens properties. In the implementation described above, the transmitted command initiates that sensor-side adjustment.
to intrinsically recalibrate
See at least:
“An intrinsic sensor calibration module 3704 may be used to determine intrinsic calibration parameters for a sensor. For example, a LIDAR sensor may require an intrinsic calibration of reflectivity values captured by laser returns.” (Levinson [0163])
“Cameras, on the other hand, may have other intrinsic calibration parameters, such as color mapping, focal length, image positioning, scaling/skew factors, and lens distortion that may affect the imaging process.” (Levinson [0163])
Rationale:
Levinson identifies its calibration as intrinsic and identifies the internal sensor characteristics adjusted: lidar reflectivity, and camera color mapping, focal length, image positioning, scaling, skew, and lens distortion. Levinson assigns intrinsic and extrinsic calibration to separate modules, 3704 and 3706.
in accordance with the recalibration command.
See at least:
“The intrinsic sensor calibration module 3704 may operate in conjunction with a data transform module 3708 and a generative model module 3710 to perform the computations necessary to converge on the optimal intrinsic calibration parameters for the miscalibrated sensor.” (Levinson [0163])
Rationale:
The parameters are computed for the particular identified sensor. It would have been obvious for the command to convey or invoke those parameters so that the sensor’s adjustment conforms to them.
Claim Limitations Not Explicitly Disclosed by Levinson
Levinson does not explicitly teach the following claim limitations:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
of one of the multiple types of sensor calibration triggers;
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
Disclosure by Prokhorov
Prokhorov teaches:
for multiple types of sensor calibration triggers, each type of sensor calibration trigger corresponding to an event or scenario experienced by the vehicle operating autonomously or semi-autonomously on the road network and measured by the one or more sensors of the vehicle;
See at least:
“[T]he installed applications 112 includ[e] programs or apps that permit the CPU 102 to implement the autonomous features of the vehicle 200 as well as the auto-calibration features.” (Prokhorov [0019])
“Among other information measurable by the sensors 130, the sensors 130 can detect vehicle speed, vehicle direction, vehicle acceleration, vehicle rotation, vehicle location, environmental weather conditions, traffic conditions, and road conditions.” (Prokhorov [0022])
“If the sensors 130 capture data for a dead-reckoning system, data relating to wheel revolution speeds, travel distance, steering angle, and steering angular rate of change can be captured.” (Prokhorov [0026])
“The notifications or alerts can be issued periodically or at certain triggers, for example, based on . . . a defined number of miles driven since the last calibration. Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov teaches two calibration-trigger types, each corresponding to an event or scenario the vehicle experiences on the road and measures with its own sensors: accumulated mileage since the last calibration, measured through the travel-distance and wheel-revolution data Prokhorov’s sensors capture at [0026]; and a large jolt arising when the vehicle hits a severe bump or pothole, measured by an onboard IMU. Prokhorov’s sensors likewise measure traffic and road conditions at [0022], and its applications implement the vehicle’s autonomous features at [0019].
A person of ordinary skill in the art would have understood the accumulated mileage of [0031] to be derived from the travel-distance and wheel-revolution data that [0026] identifies as captured by the vehicle’s sensors. Levinson at [0142] and [0147] supplies the autonomous on-road operation during which the combined system monitors for and detects the trigger.
These are the trigger types for which Levinson’s onboard system monitors in the combination. Prokhorov’s elapsed-time trigger is not relied upon.
of one of the multiple types of sensor calibration triggers;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle (which may arise if the vehicle hits a severe bump or pothole, for example) that may have affected the alignment of the sensor 130.” (Prokhorov [0031])
Rationale:
Prokhorov teaches IMU detection of the large-jolt trigger, one of the two trigger types identified above. In the combination, that trigger is the condition supplied to Levinson’s calibration detection module 3730.
by (i) identifying, based on the type of the detected sensor calibration trigger, one or more sensors of the vehicle that are affected by the detected sensor calibration trigger;
See at least:
“Additionally, IMUs 132 in the vehicle 200 can be configured to detect a large jolt to the vehicle . . . that may have affected the alignment of the sensor 130. IMUs 132 can be installed very near or incorporated into the sensors 130 to more accurately detect or predict whether such sensors 130 have become misaligned.” (Prokhorov [0031])
Rationale:
For a detected large-jolt trigger, Prokhorov uses IMUs installed near or incorporated into respective sensors to determine or predict whether those sensors became misaligned, confining the determination to “such sensors 130.” Which sensors are evaluated thus follows from which type of trigger was detected: a jolt-type trigger implicates the sensors whose alignment a mechanical shock disturbs, evaluated through the co-located IMUs. The claim requires identification “based on,” rather than exclusively based on, the trigger type.
Motivation to Combine Levinson and Prokhorov
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson and Prokhorov before them, to configure Levinson’s onboard autonomous-vehicle calibration system to monitor for Prokhorov’s mileage-accumulation and large-jolt calibration-trigger types and, upon detection of a large-jolt trigger, to use Prokhorov’s sensor-associated IMUs to identify the sensors affected by the jolt before applying Levinson’s onboard calibration process to those sensors.
The references address the same recognized problem. Levinson teaches at [0148] that a sensor may become out of alignment while the vehicle is in operation, and Prokhorov teaches at [0024] that sensors may become misaligned due to normal use or sudden jolts. Levinson supplies the onboard architecture that detects a calibration condition, identifies the miscalibrated sensor, determines intrinsic calibration parameters, and calibrates the sensor while the vehicle remains in operation. Prokhorov supplies the event-monitoring techniques that recognize when an operating event warrants calibration and which sensors that event affected. That Prokhorov uses the detected trigger to alert a user does not limit its teaching of the trigger itself; the question is what the combined teachings would have suggested, and Levinson supplies the automatic calibration the detected trigger initiates. See MPEP 2145(III).
The modification employs known components for their established functions and requires no unconventional hardware. Levinson already relies at [0146] on GPS, IMU, RADAR, SONAR, and cameras during operation, and includes odometry sensors 3616 among the monitored sensors at [0161]. Prokhorov’s mileage and jolt monitors would supply trigger indications to Levinson’s calibration detection module 3730, and Prokhorov’s sensor-associated IMUs would identify the sensors potentially affected by a detected jolt.
A person of ordinary skill would have had a reasonable expectation of success, as both references employ onboard vehicle sensors, processors, memory, IMUs, and sensor-calibration functionality, and the integration requires only conventional exchange of sensor data and trigger indications within an autonomous-vehicle computing system.
The modification would predictably:
detect event-induced sensor miscalibration promptly, rather than after it manifests in degraded perception data;
avoid continued reliance on a sensor affected by a detected event, which Levinson identifies at [0170] as forcing the vehicle into a sub-optimal mode of operation;
avoid unnecessary calibration of unaffected sensors, conserving the processing time and computational effort Levinson identifies at [0169]; and
maintain perception accuracy and safe autonomous operation without a controlled calibration facility, consistent with Levinson’s statement at [0147] that its determinations are undertaken for safety reasons and operational efficiency.
The combination applies a known event-monitoring and affected-sensor-identification technique to a known autonomous-vehicle calibration system ready for improvement, producing the predictable result of initiating onboard calibration for the sensors affected by a measurable operating event. See KSR International Co. v. Teleflex Inc., 550 U.S. 398, 416–18 (2007); MPEP 2143(I)(A), (D).
Regarding Claim 20,
The combination of Levinson and Prokhorov establishes the method of Claim 19, which is the basis for Claim 20.
Claim Limitations Not Explicitly Disclosed by the Combination of Levinson and Prokhorov
After combining the teachings of Levinson and Prokhorov, the following claim limitation is not explicitly taught:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
Levinson quantifies the effect of a miscalibration at [0170] and differentiates its calibration response at [0164] and [0169]. Prokhorov teaches the mileage-accumulation and large-jolt trigger types established with respect to Claim 19 and distinguishes normal use from sudden jolts at [0024]. Neither reference associates each such trigger type with a corresponding severity level.
Disclosure by Wang
Levinson, Prokhorov, and Wang render obvious:
wherein each type of sensor calibration trigger of the multiple types of sensor calibration triggers is associated with a corresponding severity level.
See at least:
“If the diagnostics service determines that sensor calibration values exceed an acceptable range determined by the server in the back end . . . then the criticality level is also classified as high . . . . However, if the diagnostics service does not detect the above, but does detect that there are non-critical warnings produced by hardware within the autonomous vehicle . . . then the criticality level is classified as medium . . . . If not, and the diagnostics service or remote back end server indicates that the autonomous vehicle is due for preventative maintenance . . . then the criticality level is classified as low.” (Wang [0063]–[0065])
“Depending on how far the current calibration values deviate from the expected range, a criticality of the calibration issue may be determined by the system.” (Wang [0062])
Rationale:
Wang assigns a corresponding severity level to each of several distinct detected vehicle-condition types, labeling the resulting levels in FIG. 3B as “Severity is ‘High’” (354), “Severity is ‘Medium’” (360), and “Severity is ‘Low’” (364). Wang determines the severity of a sensor-calibration issue from the extent to which calibration values deviate from the expected range, and uses severity to select the maintenance response, dispatching vehicles “to needed facilities based on the severity of the issue(s)” at [0038].
Wang does not classify Prokhorov’s mileage-accumulation and large-jolt trigger types. This limitation is rendered obvious by the combination: Wang supplies a known framework for assigning a corresponding severity level to each of several detected condition types, and applying that framework to each of the trigger types on which the combined system monitors the vehicle would have been obvious to one of ordinary skill in the art.
In the combination, a person of ordinary skill in the art would have applied Wang’s severity framework to both operative trigger types, classifying mileage accumulation according to maintenance urgency and classifying a jolt according to its measured magnitude and resulting calibration risk. Each type of sensor calibration trigger is thereby associated with a corresponding severity level.
Motivation to Combine Levinson, Prokhorov, and Wang
Therefore, given the teachings as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having Levinson, Prokhorov, and Wang before them, to apply Wang’s high, medium, and low severity classification to each of Prokhorov’s mileage-accumulation and large-jolt sensor-calibration-trigger types. Wang classifies detected vehicle conditions by severity in FIG. 3B, determines the criticality of a sensor-calibration issue based on the extent of calibration-value deviation at [0062], and uses severity to determine the appropriate maintenance response at [0038]. Applying that known classification framework to Prokhorov’s known calibration-trigger types would have predictably permitted the combined system to select and prioritize the calibration response according to the seriousness of the detected condition.
Response to Arguments
Applicant’s arguments filed on 03/25/2026 have been fully considered. The arguments are moot as to the present rejection because the claims submitted with the Request for Continued Examination (“RCE”) materially changed the limitations at issue and necessitated a new ground of rejection based on Levinson in view of Prokhorov. See MPEP Form Paragraph 7.38.
Effect of the RCE Amendments
The RCE claims differ materially from the claims addressed in the prior final Office action.
First, the prior claims required monitoring generally “when the vehicle is operating.” The RCE independent claims now require monitoring while the vehicle is operating autonomously or semi-autonomously on a road network. They further require each trigger type to correspond to an event or scenario experienced during that on-road autonomous or semi-autonomous operation and measured by vehicle sensors.
Second, the prior independent claims required identifying and performing one of several broadly recited “recalibration actions.” The RCE independent claims instead require:
identifying, based on the detected trigger type, the sensors affected by the trigger; and
transmitting a recalibration command to each affected sensor so that the sensor intrinsically recalibrates in accordance with the command.
Third, prior Claims 8 and 17 recited intrinsic calibration as a dependent limitation. The RCE amendments moved intrinsic recalibration into independent Claims 1, 10, and 19 and amended Claims 8 and 17 to require detecting a severity level and identifying the trigger based at least in part on that severity level.
These amendments required additional searching and the application of Levinson, which directly addresses real-time onboard calibration, identification of miscalibrated sensors, and intrinsic sensor calibration during autonomous vehicle operation.
The present grounds are:
Claims 1, 3–6, 8–10, 12–15, and 17–19 are rejected over Levinson in view of Prokhorov; and
Claims 2, 11, and 20 are rejected over Levinson in view of Prokhorov and further in view of Wang.
Prokhorov’s Calibration-Object Procedure
Applicant argues that Prokhorov’s paragraphs [0037]–[0039] require a user to position a calibration object before automatic calibration begins and that this procedure does not occur while the vehicle operates on a road network.
Those arguments are moot because the present rejection does not rely on Prokhorov’s calibration object, projected footprint, or associated user procedure.
Instead, Prokhorov is relied upon for two sensor-measured calibration-trigger types:
a defined number of miles driven since the preceding calibration; and
a large jolt caused by a severe bump or pothole.
Prokhorov [0031]. Prokhorov further discloses sensors that capture wheel-revolution and travel-distance data, Prokhorov [0026], and an onboard IMU that detects the large jolt, Prokhorov [0031]. Thus, accumulated mileage and a road-induced jolt are operating conditions experienced by the vehicle and measured by its sensors. Prokhorov’s elapsed-time condition is not relied upon.
Levinson supplies the required autonomous on-road operation. Levinson discloses an autonomous vehicle driving in a typical driving scenario and continuously determining its location and surrounding road environment, including lane markings, curbs, and moving vehicles. Levinson [0142], [0147]. In the combination, Levinson’s autonomous vehicle monitors for Prokhorov’s mileage and jolt trigger types while operating on the road network.
Detection and Identification of Affected Sensors
Levinson discloses receiving an indication of an anomalous sensor measurement and processing that indication using calibration detection module 3730. Levinson [0161]. The module identifies the potentially miscalibrated sensor. Levinson [0162].
Prokhorov further discloses IMUs installed near or incorporated into respective sensors to determine or predict whether “such sensors” became misaligned following a detected jolt. Prokhorov [0031]. Thus, detection of the jolt trigger causes the system to determine which sensors were affected by that type of event.
The claim requires that the identification be “based on” the trigger type, not based exclusively on the trigger type. Using measurements from the sensor-associated IMUs to determine which sensors were affected remains an identification based on the detected jolt-type trigger.
Automatic Sensor Calibration
Applicant’s argument that Prokhorov’s calibration-object procedure involves preliminary user participation does not address the present rejection because Levinson supplies the automatic calibration.
Levinson discloses:
calibrating a miscalibrated camera “on the fly in real time by onboard processors,” Levinson [0151];
self-calibrating the autonomous vehicle system “while in operation, without having to stop,” Levinson [0164]; and
automatically determining whether the vehicle should self-calibrate following confirmed sensor miscalibration, Levinson [0178].
In the combination, Prokhorov’s sensor-measured trigger initiates Levinson’s onboard calibration process for the sensors identified as affected by the trigger.
Levinson’s optional offline-calibration and teleoperator embodiments do not negate its express automatic-calibration embodiments. A reference is considered for everything it reasonably discloses, including alternative and nonpreferred embodiments. MPEP § 2123.
Recalibration Command and Intrinsic Recalibration
Applicant argues that Prokhorov and Wang do not disclose transmitting a recalibration command to each affected sensor so that the sensor intrinsically recalibrates. That argument does not address the present ground. Wang is not applied to this limitation, and Prokhorov is not relied upon for the intrinsic-calibration operation.
Levinson discloses intrinsic sensor calibration module 3704, which determines intrinsic calibration parameters for an identified sensor. These parameters include lidar reflectivity and camera color mapping, focal length, image positioning, scaling/skew, and lens distortion. Levinson [0163]. Levinson also discloses causing a camera to focus and adjust its lens properties. Levinson [0151].
Levinson further discloses a vehicle-control architecture in which transmitted commands are converted into control signals that implement a commanded change, Levinson [0065], and vehicle sensors coupled to the onboard controller, Levinson [0068]. Levinson expressly permits its computing platforms to implement the previously described techniques and permits the disclosed structures and functions to be combined. Levinson [0181], [0188].
Levinson does not expressly use the phrase “transmitting a recalibration command to the sensor.” Nevertheless, it would have been obvious to implement Levinson’s sensor calibration by transmitting to the identified sensor a command conveying or invoking the intrinsic calibration parameters computed for that sensor. This applies Levinson’s command-and-control architecture for its established purpose and predictably causes the sensor to implement the intrinsic adjustment disclosed by Levinson.
Because “one or more sensors” encompasses one sensor, transmitting the command to one identified sensor satisfies transmission to “each” identified sensor. If multiple sensors are identified, transmitting the corresponding sensor-specific command to each is the predictable repetition of the same operation.
This is an articulated obviousness rationale, not a finding that Levinson expressly recites the claimed command. See MPEP § 2143.
Reason to Combine
Levinson and Prokhorov address the same problem: vehicle sensors becoming misaligned during operation. Levinson [0148]; Prokhorov [0024].
Levinson provides an onboard system that detects miscalibration, identifies the miscalibrated sensor, determines intrinsic calibration parameters, and calibrates the sensor while the vehicle remains in operation. Prokhorov provides sensor-measured events indicating when calibration may be required and sensor-associated IMUs for determining which sensors were affected by a detected jolt.
It would have been obvious to configure Levinson’s calibration-detection module to receive Prokhorov’s mileage and jolt trigger indications and, following detection of a jolt, use Prokhorov’s sensor-associated IMUs to identify the affected sensors before applying Levinson’s onboard calibration process. The combination would predictably provide prompt calibration of affected sensors, maintain perception accuracy, and avoid unnecessary calibration of unaffected sensors.
A person of ordinary skill would have had a reasonable expectation of success because both references employ onboard sensors, IMUs, processors, memory, and sensor-calibration functionality. The combination requires only the conventional exchange of sensor measurements and trigger indications within the vehicle computing system.
The rejection does not bodily incorporate Prokhorov’s calibration-object procedure into Levinson. The relevant inquiry is what the combined teachings would have suggested to a person of ordinary skill. In re Keller, 642 F.2d 413, 425 (CCPA 1981); MPEP § 2145.
Accordingly, the RCE amendments do not distinguish Claim 1 from Levinson in view of Prokhorov.
Independent Claims 10 and 19
Claims 10 and 19 recite the same substantive monitoring, trigger-detection, affected-sensor-identification, recalibration-command, and intrinsic-recalibration limitations in computer-readable-medium and method form, respectively. The findings and reasoning for Claim 1 therefore apply equally to Claims 10 and 19.
Levinson additionally discloses processors executing instructions stored on computer-readable media to perform the disclosed functions. Levinson [0184]–[0185]. Applicant presents no separate argument directed to the statutory form of Claims 10 or 19.
Dependent Claims
Applicant’s assertion that the dependent claims are allowable because their respective independent claims are allowable is not persuasive because the independent claims are rendered obvious by the present combination.
For Claims 8 and 17, Prokhorov’s IMU detects a “large jolt.” Prokhorov [0031]. Under the interpretation stated in the rejection, determining that a measured jolt falls within the “large” category constitutes detecting a categorical severity level, and the jolt is identified as a calibration trigger based at least in part on that classification.
For Claims 2, 11, and 20, Wang provides a known framework for classifying detected vehicle conditions by severity. Wang classifies conditions as high, medium, or low, Wang FIG. 3B, [0063]–[0065], and determines the criticality of a calibration issue from the extent of calibration-value deviation, Wang [0062]. Applying that framework to Prokhorov’s mileage and jolt trigger types would have predictably allowed the system to prioritize its response according to the seriousness of the detected condition.
Applicant presents no separate substantive argument addressing the additional limitations of Claims 3–6, 9, 12–15, or 18.
Conclusion
Applicant’s arguments concerning Prokhorov’s user-assisted calibration-object procedure and Wang’s failure to supply a sensor-directed recalibration command addressed the different claims and combination considered in the prior final Office action. Those arguments are moot in view of the materially different RCE amendments and the new grounds presented here.
Levinson supplies the automatic onboard calibration, affected-sensor identification, intrinsic-calibration, and control teachings. Prokhorov supplies the sensor-measured calibration triggers and sensor-associated IMUs. Wang supplies the severity-classification framework applied to Claims 2, 11, and 20.
Accordingly, the amendments submitted with the RCE do not overcome the prior art, and Claims 1–6, 8–15, and 17–20 remain unpatentable under 35 U.S.C. § 103 for the reasons stated in this action.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWABUSAYO ADEBANJO AWORUNSE whose telephone number is (571)272-4311. The examiner can normally be reached M - F (8:30AM - 5PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571) 270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWABUSAYO ADEBANJO AWORUNSE/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights