DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/14/2026 has been entered.
Response to Amendment
The Amendment filed 04/02/2026 has been entered. Claims 16-20 have been added. No claims have been cancelled. Claims 1, 9, 11, and 12 have been amended. Claims 1-7 and 9-20 remain pending in the application.
Response to Arguments
Applicant’s arguments, see pages 6-9, filed 04/02/2026, with respect to the 103 rejection of claim 1 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of a newly found reference, specifically NPL reference Hofbauer presented below.
Examiner concurs the previously presented prior art fails to disclose or suggest the amended claim language. An updated search resulted in the Hofbauer reference which discloses a multiplexer and an encoder sending encoded combined camera video data to a decoder and a demultiplexer providing a plurality of camera video data streams. Further, amended claim language introduces indefiniteness as presented in the 112 rejection section below. New dependent claims 16-20 have been rejected accordingly. Therefore, amended claim 1 is now rejected over the combination of Hu and Hofbauer with dependent claims 2-7 and 9-20 rejected accordingly.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-7 and 9-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent claim 1 simultaneously recites performing vehicle cabin sensing using combined sensor data and at least one of the extracted respective sensor data streams after extracting sensor data streams from the combined sensor data. Further, the vehicle cabin sensing using combined sensor data occurs after the separation of the respective sensor data from the combined data. The claim is considered indefinite because there is a question or doubt as to how the system performs vehicle cabin sensing using combined sensor data after the combined sensor data has been separated and extracted and used to perform vehicle cabin sensing. Dependent claims 2-7 and 9-20 fall together accordingly.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5-7, 10-12, and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Hu et al. US 2021/0056306 A1, hereafter Hu in view of Hofbauer et al. “Adaptive Multi-View Live Video Streaming for Teledriving Using a Single Hardware Encoder”, hereafter Hofbauer.
Regarding claim 1, Hu discloses a vehicle cabin sensing system (systems…sensors used to detect those occupants) [abstract] comprising:
an electronic control unit (autonomous vehicle) [0089];
a camera system comprising:
multiple image sensors (one or more cameras (72, 73, 74, 75) deployed around the vehicle) [0082], wherein each image sensor provides respective sensor data (provide coverage of the driver and other occupants) [0082];
a combination circuit (MSCM 904)) [FIG. 9] configured to combine the sensor data provided by the image sensors into combined sensor data (see video data and fsync lines into serializer 906 from 912 and 943) [FIG. 9]; and
an interface configured to transmit the combined sensor data to the electronic control unit (see transmission GMSL link from 906 to 901) [FIG. 9]; and
a processor of the electronic control unit (AI Supercomputers (800), (900); autonomous vehicle may include one or more multi-sensor camera modules (MSCM)) [FIG. 9; 0089] is configured to:
separate the combined sensor data (see GMSL deserializer of AI supercomputer 900 separating combined video data from serializer 906) [FIG. 9]
perform vehicle cabin sensing using one or more algorithms for a vehicle cabin sensing application wherein the electronic control unit is configured to receive the combined sensor data and to perform vehicle cabin sensing using the combined sensor data (advanced SoC 902; advanced SoC or platform used to execute an intelligent assistant software stack (IX) that conducts risks assessments and provides the notifications, warning and autonomously control the vehicle) [FIG. 9; 0080].
However, Hu fails to explicitly disclose separate the combined sensor data into the respective sensor data provided by the respective image sensors; extract, from the combined sensor data, respective sensor data streams corresponding to the respective image sensors; and
Hofbauer, in an analogous environment, discloses separate the combined sensor data into the respective sensor data provided by the respective image sensors (see decoder and demultiplexer producing sensor data streams of camera 1…camera N) [Fig. 2];
extract, from the combined sensor data, respective sensor data streams corresponding to the respective image sensors (see decoder and demultiplexer producing sensor data streams of camera 1…camera N) [Fig. 2].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to extract individual sensor data streams from combined sensor data, as disclosed by Hofbauer, to perform the vehicle cabin sensing of the invention disclosed by Hu, the motivation being smaller total bitrate [abstract].
Regarding claim 2, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the image sensors are operable to capture image data in different electromagnetic spectra (IR Image Sensor (912), RGB Image Sensor (918)) [0091].
Regarding claim 3, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the image sensors include at least two of a near-infrared image sensor, a longwave infrared or thermal image sensor, a depth image sensor and a colour image sensor (IR Image Sensor (912), RGB Image Sensor (918)) [0091].
Regarding claim 5, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the interface is configured to transmit the combined sensor data stream to the electronic control unit via serial data communication (serializer; see GMSL link and GSML deserializer) [0094].
Regarding claim 6, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further disclose the interface comprises a Serializer/Deserializer and is configured to transmit the combined sensor data stream to the electronic control unit using the Serializer/Deserializer (serializer; see GMSL link and GSML deserializer) [0094].
Regarding claim 7, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses performing vehicle cabin sensing comprises performing detection of at least one of the detection of the presence of persons or objects or both in the vehicle cabin, and the detection of activities in the vehicle cabin (the neural networks preferably are trained to detect a number of different features and events, including: the presence of a face (5001)) [0083].
Regarding claim 10, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses a housing in which the multiple image sensors are arranged (single housing) [0089].
Regarding claim 11, Hu and Hofbauer address all of the features with respect to claim 10 as outlined above.
Hu further discloses the camera system comprises one or more optical systems for the image sensors, wherein the one or more optical systems are arranged in the housing (multiple sensors in a single housing…IR+RGB) [0089].
Regarding claim 12, Hu and Hofbauer address all of the features with respect to claim 10 as outlined above.
Hu further discloses a respective system for each image sensor, wherein, for each image sensor the respective optical system is arranged in the housing (multiple sensors in a single housing…IR+RGB) [0089].
Regarding claim 16, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the combined sensor data comprises a single data stream transmitted to the electronic control unit via a single connection or link (see single data stream from serializer 906 to GMSL deserializer) [FIG. 9].
Regarding claim 17, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the interface is configured to transmit the combined sensor data to the electronic control unit via serial data communication (see single data stream from serializer 906 to GMSL deserializer) [FIG. 9].
Regarding claim 18, Hu and Hofbauer address all of the features with respect to claim 17 as outlined above.
Hu further discloses the interface comprises a serializer/deserializer, and wherein the combined sensor data is transmitted over a serializer/deserializer serial link (see single data stream from serializer 906 to GMSL deserializer) [FIG. 9].
Regarding claim 19, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
Hu further discloses the processor is configured to process different ones of the extracted respective sensor data streams using different algorithms for different vehicle cabin sensing applications (see 5000 DNNs of FIG. 7) [FIG. 7].
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Hu and Hofbauer further in view of Kim KR 10-2020-0069585 (machine translation provided by Espace.net), hereafter Kim.
Regarding claim 4, Hu and Hofbauer addresses all of the features with respect to claim 1 as outlined above.
However, the combination fails to explicitly disclose the image sensors include all of a near-infrared image sensor, a longwave infrared or thermal image sensor, a depth image sensor and a colour image sensor.
Kim, in an analogous environment, discloses the image sensors include all of a near-infrared image sensor, a longwave infrared or thermal image sensor, a depth image sensor and a colour image sensor (the image acquisition 110 may acquire one or more image information in a vehicle. At this time, the image information acquisition 110 may include an RGB camera, a Depth sensor and a ThermalIR sensor. However, various other types of camera, for example,…an infrared camera) [0025].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to replace the multiple image sensors of Hu and Hofbauer with the sensors of Kim, obtaining the predictable results more/different image sensor type combos.
Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Hu and Hofbauer further in view of Golston et al. US 20180251122 A1, hereafter Golston.
Regarding claim 9, Hu and Hofbauer addresses all of the features with respect to claim 1 as outlined above.
However, while the combination discloses a serializer to combine data, the combination fails to explicitly disclose the processor is configured to perform the vehicle cabin sensing using data fusion of the sensor data.
Golston, in an analogous environment, discloses the processor is configured to perform the vehicle cabin sensing using data fusion of the sensor data (sensor fuser 142 may fuse multiple data types) [0067].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to fuse the sensor data, as disclosed by Golston, with the invention disclosed by Hu, the motivation being using multiple sensors to determine occupant status [0067].
Regarding claim 20, Hu and Hofbauer addresses all of the features with respect to claim 1 as outlined above.
However, while the combination discloses a serializer to combine data, the combination fails to explicitly disclose the processor is configured to perform the vehicle cabin sensing using data fusion of the respective sensor data provided by at least two of the image sensors.
Golston, in an analogous environment, discloses the processor is configured to perform the vehicle cabin sensing using data fusion of the respective sensor data provided by at least two of the image sensors (sensor fuser 142 may fuse multiple data types) [0067].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to fuse the sensor data, as disclosed by Golston, with the invention disclosed by Hu, the motivation being using multiple sensors to determine occupant status [0067].
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Hu and Hofbauer further in view of Kang et al. US 2022/0114754 A1, hereafter Kang
Regarding claim 13, Hu and Hofbauer addresses all of the features with respect to claim 1 as outlined above.
However, the combination fails to explicitly disclose the image sensors have different resolutions.
Kang, in an analogous environment, discloses the image sensors have different resolutions (a resolution of the color image 510 may be higher than a resolution of the IR image 515) [0100].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the use the different resolutions, as disclosed by Kang, with the invention disclosed by Hu and Hofbauer, yielding predictable results of sensors with different capture capability.
Regarding claim 14, Hu and Hofbauer address all of the features with respect to claim 1 as outlined above.
However, the combination fails to explicitly disclose the multiple image sensors comprise a thermal image sensor and a colour image sensor, wherein the resolution of the colour image sensor is higher than the resolution of the thermal image sensor.
Kang, in an analogous environment, discloses the multiple image sensors comprise a thermal image sensor and a colour image sensor, wherein the resolution of the colour image sensor is higher than the resolution of the thermal image sensor (a resolution of the color image 510 may be higher than a resolution of the IR image 515) [0100].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the use the different resolutions, as disclosed by Kang, with the invention disclosed by Hu and Hofbauer, yielding predictable results of sensors with different capture capability.
Regarding claim 15, Hu and Hofbauer addresses all of the features with respect to claim 1 as outlined above.
However, the combination fails to explicitly disclose the resolution of the colour image sensor is at least double or at least four times higher than the resolution of the thermal image sensor.
Kang, in an analogous environment, discloses the resolution of the colour image sensor is at least double or at least four times higher than the resolution of the thermal image sensor (a resolution of the color image 510 may be higher than a resolution of the IR image 515) [0100].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the use the different resolutions, as disclosed by Kang, with the invention disclosed by Hu and Hofbauer, yielding predictable results of sensors with different capture capability.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEFAN GADOMSKI whose telephone number is (571)270-5701. The examiner can normally be reached Monday - Friday, 12-8PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at 571-272-2988. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
STEFAN GADOMSKI
Primary Examiner
Art Unit 2485
/STEFAN GADOMSKI/Primary Examiner, Art Unit 2485
Prosecution Insights