DETAILED ACTION
Response to Arguments
Applicant’s amendments to claims submitted 1/26/2026 have been recorded. As such Examiner’s previous 35 U.S.C. 112(b) of claims 9 and 15-19, and 35 U.S.C. 101 rejection of claim 14 have been withdrawn.
Applicant has amended claims 1, 4, 5, 7, 9-10, 13-20; added claims 21; and canceled claims 8; claims 1-7 and 9-21 are currently pending.
Applicant's arguments filed 01/26/2026 have been fully considered but they are not fully persuasive. Applicant’s representative asserts that Bradski does not disclose or suggest the limitation “in response to determining that the predicted physical distance satisfies the difference threshold, adjusting, using the timing offset, subsequent data from the first sensor or the second sensor.” Examiner respectfully disagrees. In Bradski ¶56; iterative calibration is disclosed. The calibration is to the correct velocity of the rotating target object. It is discussed that the target object should not complete a three-hundred-and-sixty-degree rotation (threshold rotational distance) between the two different sensor captures. The iterative calibration is used to make sure that the measured difference between the two captures informs the amount of time elapsed between the two captures. Further, this calibration suggests that timing offsets would not be applied until calibration is met. Therefore the limitation is considered to be taught. Although the limitation is considered to be taught, it is not necessarily in the same embodiment. Therefore the 35 U.S.C. 102(a)(1) has been withdrawn, and a new grounds of rejection is made under 35 U.S.C. 103, as is detailed below.
Applicant’s arguments with respect to claim(s) 13-14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments with respect to claim(s) 10 and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Therefore this action is made FINAL.
Claim Objections
Claims 15-15-19 and 21 objected to because of the following informalities: “The media” should be “The one or more non-transitory computer storage media” to match the claim language of independent claim 14. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bradski (Pub. No. US20190014310A1).
Regarding claim 1, Bradski discloses A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: (Bradski ¶110; implementing any disclosed embodiments on a processor with computer-readable instructions stored on a computer-readable medium is disclosed.) obtaining (i) first sensor data from a first sensor and (ii) second sensor data from a second sensor; (Bradski ¶53; captures generated by multiple sensors of the same object are disclosed.) detecting a representation of an object in both the first sensor data and the second sensor data; (Although not necessarily in the same embodiment, Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed, therefore the synchronization target is represented in the multiple sensor data.) determining a predicted physical distance between a first representation of the object in the first sensor data and a second representation of the object in the second sensor data; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed.) determining whether the predicted physical distance between the first representation of the object and the second representation of the object satisfies a difference threshold; (Bradski ¶56; iterative calibration is disclosed. The calibration is to the correct velocity of the rotating target object. It is discussed that the target object should not complete a three-hundred-and-sixty-degree rotation (threshold rotational distance) between the sensor captures.) determining a timing offset between the different sensors using the predicted physical distance between the first representation of the object and the second representation of the object; and (Bradski ¶53; characterization of target position in the sensor captures can be used to calculate time point offsets.) in response to determining that the predicted physical distance satisfies the difference threshold, adjusting, using the timing offset, subsequent data from the first sensor or the second sensor. (Bradski ¶40; difference is used to adjust timing based on the offset. ¶56 iterative calibration of the velocity of the target object is disclosed. The velocity needs to be at a rate the sensors can perform captures before the object rotates over three-hundred-and-sixty-degrees. Otherwise determining time elapsed between captures is impossible and therefore updating timing offset would not be accurate. Therefore once it is determine that the rotational distance is less than three-hundred-and-sixty-degrees and is iteratively calibrated )
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to use the rotating calibration board as disclosed in Bradski in order to provide a calibration board the can stay in the field of view of the cameras to be calculated without the cameras needing to be reoriented (Bradski ¶55).
Regarding claim 2, Bradski discloses the claim limitations with regards to claim 1, as described above. Bradski further discloses wherein detecting the representation of the object in both the first sensor data and the second sensor data comprises: detecting a rotating object with at least one first feature detectable by the first sensor and at least one second feature detectable by the second sensor. (Bradski ¶51-52; calibration boards/ synchronization targets can be rotating objects. ¶53; differences in target position perceived by the multiple sensors is disclosed, therefore the synchronization target is represented in the multiple sensor data.)
Regarding claim 3, Bradski discloses the claim limitations with regards to claim 2, as described above. Bradski further discloses wherein: the first sensor is an infrared camera and the at least one first feature comprises an infrared marker; and the second sensor is a depth sensor that captures depth data of the rotation of the object. (Bradski ¶80; sensors can be depth sensors. IR patterns can be projected and then measured using IR sensitive cameras.)
Regarding claim 4, Bradski discloses the claim limitations with regards to claim 2, as described above. Although not necessarily in the same embodiment, Bradski further discloses wherein determining the predicted physical distance between the first representation of the object in the first sensor data and the second representation of the object in the second sensor data comprises: determining an angular distance between (i) a first line indicating a position of the object in the first sensor data and (ii) a second line indicating a position of the object in the second sensor data. (Bradski Fig. 4 see 401 and ¶60-61; the use of AprilTag pattern (note the lines in the pattern that can be utilized) allows for measurement of an angle of rotation with respect to a reference angle. This is then used to find a relative off-set between sensors.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to add an AprilTag pattern to the rotating calibration board as disclosed in Bradski because they allow for efficient processing and determination of position and orientation of the pattern (Bradski ¶58).
Regarding claim 5, Bradski discloses the claim limitations with regards to claim 1, as described above. Although not necessarily in the same embodiment, Bradski further discloses prior to determining the timing offset between the different sensors using the predicted physical distance between the first representation of the object and the second representation of the object, wherein the operations comprise: determining a velocity of the object using the first sensor data and the second sensor data; (Bradski ¶54; if variable velocity is used an iterative approach can be used to calculate the timing offset, meaning the velocity is calculated since it is not known.) and subsequent to determining the velocity, determining the timing offset using the velocity. (Bradski ¶54 use of velocity to calculate timing offset is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to calculate the velocity to expand the usefulness to cases where the velocity is unknown as in Bradski ¶54.
Regarding claim 6, Bradski discloses the claim limitations with regards to claim 1, as described above. Bradski further discloses comprising using the subsequent data from the first sensor or the second sensor to perform one or more of the following: navigation, localization, or environmental adjustments. (Bradski ¶3; data is used to create a navigable model of a captured scene (environmental adjustments).)
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bradski (Pub. No. US20190014310A1) in view of Greer (Pub. No. US20190302883A1).
Regarding claim 7, Bradski discloses the claim limitations with regards to claim 1, as described above. Bradski further discloses wherein determining the timing offset comprises: providing the predicted physical distance between the first representation of the object in the first sensor data and the second representation of the object in the second sensor data (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed.) to a the subsequent data using the timing offset. (Bradski ¶53; characterization of target position in the sensor captures can be used to calculate time point offsets.)
Bradski does not explicitly disclose a proportional-integral-derivative (PID) controller.
Greer, however, discloses proportional-integral-derivative (PID) controller (Greer ¶60; a PID controller is used to control image sensors).
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski by using a PID controller in order to correct error over time and reduce overshoot and undershoot (Greer ¶60).
Claim(s) 10-11 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bradski (Pub. No. US20190014310A1) in view of Inakura (Pub. No. US20200213525A1).
Regarding claim 10, Bradski discloses the claim limitations with regards to claim 1, as disclosed above. Bradski further discloses wherein adjusting the timing offset used to adjust the subsequent data using a difference between the first change and the second change is responsive to: detecting a representation of an object in both the first sensor data and the second sensor data; (Bradski ¶53; temporal offsets for the first sensor and second sensor based on first and second images from first and second sensors are disclosed. Offset between the cameras is discussed meaning the two sensors have different amounts of change.) determining a predicted physical distance between a first representation of the object in the first sensor data and a second representation of the object in the second sensor data; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed.) and determining the timing offset between the different sensors using the predicted physical distance between the first representation of the object and the second representation of the object. (Bradski ¶53; characterization of target position in the sensor captures can be used to calculate time point offsets.)
Bradski does not explicitly disclose wherein the operations comprise: detecting, from a plurality of processes that the system can perform and that can cause a first change for processing data from the first sensor or a second change for processing data from the second sensor, a process performed by the system at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause the first change for processing data from the first sensor to be a different change than the second change for processing data from the second sensor.
Inakura, however, discloses wherein the operations comprise: detecting, from a plurality of processes that the system can perform and that can cause a first change for processing data from the first sensor or a second change for processing data from the second sensor, a process performed by the system at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause the first change for processing data from the first sensor to be a different change than the second change for processing data from the second sensor (Inakura ¶88; sending communication time delays between sensors is disclosed. These values can be updated when values are larger than the stored values. Claim 3 these timing delays are used to update the delay times and synchronize captures from multiple cameras. Changes in the delay changes the processed synchronization time.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski with the teachings of Inakura by including periodic updates of communication delay times in order to know when timing needs to be adjusted (Inakura ¶88).
Regarding claim 11, the combination of Bradski and Inakura disclose the claim limitations with regards to claim 10, as described above. They further disclose wherein one of the first change or the second change for processing data indicates no change. (Bradski Fig. 4; it can be seen for sensor 1, element 204, from 400 to 402 that no offset was calculated. While for sensor 2, element 205, from 401 to 403 an offset was calculated.)
Regarding claim 20, Bradski discloses A computer-implemented method comprising: (Bradski ¶110; implementing any disclosed embodiments on a processor with computer-readable instructions stored on a computer-readable medium is disclosed.) obtaining, for a device that includes a first sensor and a second sensor, (i) first sensor data from the first sensor and (ii) second sensor data from the second sensor; (Bradski ¶53; captures generated by multiple sensors of the same object are disclosed.) detecting a representation of an object in both the first sensor data and the second sensor data; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed, therefore the synchronization target is represented in the multiple sensor data.) determining a predicted physical distance between a first representation of the object in the first sensor data and a second representation of the object in the second sensor data; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed.) determining a timing offset between the different sensors using the predicted physical distance between the first representation of the object and the second representation of the object; and (Bradski ¶53; characterization of target position in the sensor captures can be used to calculate time point offsets.) adjusting, using the timing offset, subsequent data from the first sensor or the second sensor. (Bradski ¶40; difference is used to adjust timing based on the offset)
Bradski does not explicitly disclose detecting, from a plurality of processes that can affect timing data for captured sensor data, a process executed by the device at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause a discrepancy in a prior offset for timestamps for the first and second sensor or in response to detecting the process executed by the device at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause a discrepancy in the prior offset for timestamps for the first sensor and the second sensor, adjusting, using the timing offset, subsequent data from the first sensor or the second sensor.
Inakura detecting, from a plurality of processes that can affect timing data for captured sensor data, a process executed by the device at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause a discrepancy in a prior offset for timestamps for the first and second sensor (Inakura ¶88; sending communication time delays between sensors is disclosed. These values can be updated when values are larger than the stored values. Claim 3 these timing delays are used to update the delay times and synchronize captures from multiple cameras. Changes in the delay changes the processed synchronization time.) in response to detecting the process executed by the device at least partially concurrently with processing of the subsequent data from the first sensor or the second sensor and that will cause a discrepancy in the prior offset for timestamps for the first sensor and the second sensor, adjusting, using the timing offset, subsequent data from the first sensor or the second sensor. (Inakura ¶104, 110, common timing is calculated using the client management table (from ¶88 is updated when values change). Common timing is utilized to find the correct delay so that the cameras can perform exposure at the same time.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski with the teachings of Inakura by including periodic updates of communication delay times in order to know when timing needs to be adjusted (Inakura ¶88).
Claim(s) 9 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bradski (Pub. No. US20190014310A1) in view of Takahashi (Pub. No. 20180107613A1).
Regarding claim 9, Bradski discloses the claim limitations with regards to claim 1, as described above.
Bradski does not explicitly disclose , wherein the operations comprise: performing processing operations that simulate a processing load, wherein the processing operations affects processing of the first sensor data or the second sensor data.
Takahashi, however, discloses , wherein the operations comprise: performing processing operations that simulate a processing load, wherein the processing affects processing of the first sensor data or the second sensor data. (Takahashi ¶98; required bandwidth is estimated for image processing processes)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski by including a calculated processing bandwidth for image processing processes in order to allocate resources to different processes and suppress or avoid processing delays (Takahashi ¶98).
Regarding claim 12, Bradski discloses the claim limitations with regards to claim 1, as described above.
Bradski does not explicitly disclose wherein the operations comprise: providing current processing bandwidth to a model configured to determine a timing offset using the current processing bandwidth; and determining the timing offset using output from the model.
Takahashi, however, discloses wherein the operations comprise: providing current processing bandwidth to a model configured to determine a timing offset using the current processing bandwidth; and determining the timing offset using output from the model. (Takahashi ¶98; required bandwidth is estimated for image processing processes. Timing delays are also determined.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski by including a calculated processing bandwidth for image processing processes and timing delays in order to allocate resources to different processes and suppress or avoid processing delays (Takahashi ¶98).
Claim(s) 13-19, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bradski (Pub. No. US20190014310A1) in view of Lu (Pub. No. US20140098229A1).
Regarding claim 13, Bradski discloses the claim limitations of claim 1 as described above. Bradski further discloses calibration by obtaining (i) the first sensor data from the first sensor and (ii) the second sensor data from the second sensor ; (Bradski ¶53; captures generated by multiple sensors of the same object are disclosed.)
Bradski discloses calibration by obtaining first and second sensor data does not explicitly disclose wherein in response to determining a change in processing firmware or hardware of the system, obtaining (i) the first sensor data from the first sensor and (ii) the second sensor data from the second sensor is responsive to detecting a change in the processing firmware or hardware of a device that includes the first sensor and the second sensor.
Lu, however, discloses wherein in response to determining a change in processing firmware or hardware of the system, obtaining (i) the first sensor data from the first sensor and (ii) the second sensor data from the second sensor is responsive to detecting a change in the processing firmware or hardware of a device that includes the first sensor and the second sensor. (Lu ¶64; when a camera is replaced in the system the cameras need to be recalibrated. Automatic calibration process can be performed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski with the teachings of Lu by including automatic recalibration when a damaged camera is replaced in order provide proper performance of the multicamera system (Lu ¶64).
Regarding claim 14, Bradski discloses One or more non-transitory computer storage media encoded with instructions that, when executed by a system comprising one or more computers, cause the system comprising the one or more computers to perform operations comprising: (Bradski ¶110; implementing any disclosed embodiments on a processor with computer-readable instructions stored on a computer-readable medium is disclosed.) obtaining (i) first sensor data from the first sensor and (ii) second sensor data from the second sensor; (Bradski ¶53; captures generated by multiple sensors of the same object are disclosed. ) detecting a representation of an object in both the first sensor data and the second sensor data; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed, therefore the synchronization target is represented in the multiple sensor data.) determining a predicted physical distance between a first representation of the object and a second representation of the object; (Bradski ¶53; differences in target position perceived by the multiple sensors is disclosed.) determining a timing offset between the different sensors using the predicted physical distance between the first representation of the object and the second representation of the object; and (Bradski ¶53; characterization of target position in the sensor captures can be used to calculate time point offsets.) adjusting, using the timing offset, subsequent data from the first sensor or the second sensor. (Bradski ¶40; difference is used to adjust timing based on the offset)
Bradski does not explicitly disclose detecting a change in processing firmware or hardware of the system; in response to detecting the change in the processing firmware or hardware of the system, obtaining (i) first sensor data from a first sensor and (ii) second sensor data from a second sensor.
Lu, however, discloses detecting a change in processing firmware or hardware of the system; in response to detecting the change in the processing firmware or hardware of the system, obtaining (i) first sensor data from a first sensor and (ii) second sensor data from a second sensor. (Lu ¶64; when a camera is replaced in the system the cameras need to be recalibrated. Automatic calibration process can be performed. Therefore a change in how the sensors are processed is detected.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the system of Bradski with the teachings of Lu by including automatic recalibration (including capturing sensor data) when a damaged camera is replaced (hardware change) in order provide proper performance of the multicamera system (Lu ¶64).
Regarding claim 15-19, the combination of Bradski and Lu disclose the claim limitations with respect to claim 14, as described above. Additional limitations correspond to claim 2-6 respectively and are rejected for similar reasons.
Regarding claim 21, the combination of Bradski and Lu disclose the claim limitations of claim 14, as described above. They further disclose wherein: the system comprises a device that includes the first sensor and the second sensor; and (Bradski Fig. 2 and ¶48 that two sensors are utilized in the same device (see also ¶48). detecting the change in the processing firmware or hardware of the system comprises detecting the change in the processing firmware or hardware of the device. (Lu ¶64; when a camera is replaced in the system the cameras need to be recalibrated. Automatic calibration process can be performed. Therefore a change in how the sensors are processed is detected. Wherein it would have been obvious to include automatic recalibration (including capturing sensor data) when a damaged camera is replaced (hardware change) in order provide proper performance of the multicamera system).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5. Examiner’s email is Meredith.taylor@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights