Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114.
RESPONSE TO ARGUMENTS
Applicant’s amendment and remarks filed on 2/4/26 have been fully considered and the rejections withdrawn to Claims 11-18. The rejections to Claims 1-2, 5-10, and 19-20 stand after further consideration.
Regarding Applicants amendment and remarks filed on 2/4/26, the amendment has been fully considered and the rejections withdrawn to Claims 11-18. The rejections to Claims 1-2, 5-10, and 19-20 stand after further consideration whereby Examiner has provided additional citation to the prior art of record where applicable to read upon the newly amended language. Accordingly, the rejections stand.
Claim Rejections - 35 USC § 112(d)
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 3-4 are rejected under 35 U.S.C. 112(d) as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1-2, 9-10, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over HUNG (Pub. No: US 2023-0166661) in view of LUNDGREN et al. (Pub. No: US 2020-0322581).
As per Claim 1 HUNG discloses One or more hardware processors comprising processing circuitry to (Figs. 1 system 1 and processor 30 [Abstract]): generate a first frame of RGB image data based at least on operating an image processing pipeline in an RGB imaging mode (Figs. 1-3 start with RGB mode image capture [0021, 0025-0027, 0070]); and generate a second frame of infrared (IR) image data based at least on operating the image processing pipeline in an IR imaging mode (Figs. 1-3 in IR mode image capture – increased light intensity level - capture sequences going forward [0021, 0025-0027, 0070]) and at least one of:
based at least on detecting that a first amount of light intensity (Figs. 1-3 in RGB mode image capture – lower light intensity level - capture mode and capture sequences going forward [0021, 0025-0027, 0070]) is less than a first intensity threshold, switch from operating the image processing pipeline in the RGB imaging mode to the IR imaging mode (Figs. 1-3 in RGB mode image capture – lower light intensity level - capture sequences going forward [0021, 0025-0027, 0070])
HUNG does not disclose but LUNDGREN discloses light intensity that quantifies a first amount of green light in associated with the first frame of RGB image data (Figs. 1-2 [0025] [0029] [0031-0032, 0037]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include light intensity that quantifies a first amount of green light in associated with the first frame of RGB image data as taught by LUNDGREN into the system of HUNG because of the benefit taught by LUNDGREN to include further processing techniques and considerations for multi-spectral images to include further green light intensities consideration to achieve color filter alignment whereby HUNG is directed towards green light intensity considerations for system image processing requirements and would benefit from the inclusion of related green-light-processing advancements.
(either or) based at least on detecting that a second amount of light intensity that quantifies a second amount of green light in the second frame of IR image data exceeds a second intensity threshold, switch from operating the image processing pipeline in the IR imaging mode to the RGB imaging mode (either or)
As per Claim 2 HUNG discloses The one or more hardware processors of claim 1, wherein the first intensity threshold for switching from the RGB imaging mode to the IR imaging mode is lower in intensity (Figs. 1-3 in IR mode image capture – increased light intensity level switch to RGB capture mode and capture sequences going forward [0021, 0025-0027, 0070]) than the second intensity threshold for switching from the IR imaging mode to the RGB imaging mode (Figs. 1-3 in IR mode image capture – increased light intensity level switch to RGB capture mode and capture sequences going forward [0021, 0025-0027, 0070]).
As per Claim 9 HUNG discloses The one or more hardware processors of claim 1, wherein the processing circuitry is further to control one or more RGB/IR cameras in an occupant monitoring system (Figs. 1-3 vehicle occupant -monitor the vehicle exterior for the occupant [0021, 0070]).
As per Claim 10 HUNG discloses The one or more hardware processors of claim 1, wherein the one or more hardware processors are comprised in at least one of (See said analysis for Claim 1): a control system for an autonomous or semi-autonomous machine (Figs. 1-3 driver assistance autonomous IR/RGB switching – semi-autonomous overall driving [0022, 0067, 0070]);
(at least one of ) a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
As per Claim 19 HUNG discloses A method comprising (Figs. 1 [Abstract]): generating a first frame of RGB image data based at least on operating an image processing pipeline in an RGB imaging mode (Figs. 1-3 start with RGB mode image capture [0021, 0025-0027, 0070]); generating a second frame of infrared (IR) image data based at least on operating the image processing pipeline in an IR imaging mode (See said analysis for Claim 1) and at least one of (See said analysis for Claim 1):
based at least on detecting that a first amount of light intensity (Figs. 1-3 in RGB mode image capture – lower light intensity level - capture mode and capture sequences going forward [0021, 0025-0027, 0070]) is less than a first intensity threshold, switch from operating the image processing pipeline in the RGB imaging mode to the IR imaging mode (See said analysis for Claim 1)
HUNG does not disclose but LUNDGREN discloses light intensity that quantifies a first amount of green light in associated with the first frame of RGB image data (See said analysis for Claim 1).
(either or) based at least on detecting that a second amount of light intensity that quantifies a second amount of green light in the second frame of IR image data exceeds a second intensity threshold, switch from operating the image processing pipeline in the IR imaging mode to the RGB imaging mode (either or)
As per Claim 20 HUNG discloses The method of claim 19, wherein the method is performed by at least one of (See said analysis for Claim 19): a control system for an autonomous or semi-autonomous machine (See said analysis for Claim 10);
(at least one of ) a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (See said analysis for Claim 10).
Claim 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over HUNG (Pub. No: US 2023-0166661) in view of LUNDGREN et al. et al. (Pub. No: US 2020-0322581), as applied in Claims 1-2, 9-10, 19-20, and further in view of SAKAMOTO et al. (US Pub. No.: 2020-0112662).
As per Claim 5 HUNG discloses The one or more hardware processors of claim 1, wherein the processing circuitry is further to switch to operating the image processing pipeline in the RGB imaging mode based at least on detecting that the second amount of light intensity exceeds the second intensity threshold (Figs. 1-3 in IR mode image capture – increased light intensity level switch to RGB capture mode – exceed the predetermined value [0021, 0025-0027, 0070])
HUNG and LUNDGREN do not disclose but SAKAMOTO discloses detecting that a ratio of an amount of IR light in the second frame of IR image data to the second amount of light intensity is less than a third threshold (Figs. 1-2, 21-26, 32 – plurality of image data - detecting IR light amount as a ratio of amount included in environmental light with considerations of high/low thresholds [0084] [0206-0207, 0209]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include detecting that a ratio of an amount of IR light in the second frame of IR image data to the second amount of light intensity is less than a third threshold as taught by SAKAMOTO into the system of HUNG and LUNDGREN because of the benefit taught by SAKAMOTO to include additional measurements and processing for IR image data which would extend and enhance said systems which are directed towards IR/color intensity detection in images that leads to additional processing requirements and would benefit from the additional ability for ratio of IR light and intensity considerations to enhance said systems.
Claim 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over HUNG (Pub. No: US 2023-0166661) in view of LUNDGREN et al. (Pub. No: US 2020-0322581), as applied in Claims 1-2, 9-10, 19-20, and further in view of JONES et al. (US Pub. No.: 2022-0133241).
As per Claim 6 HUNG discloses The one or more hardware processors of claim 1, wherein the processing circuitry is further to determine one of the first amount of light intensity or the second amount of light intensity (See said analysis for Claim 1)
HUNG and LUNDGREN do not disclose but JONES discloses amount of light intensity based at least on averaging intensity values in a designated subset of the first frame or the second frame corresponding to a monitored region (Figs. 4, 6-7 averaging of pixel intensities where observed/measured [0071] [0213, 0215])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include amount of light intensity based at least on averaging intensity values in a designated subset of the first frame or the second frame corresponding to a monitored region as taught by JONES into the system of HUNG and LUNDGREN because of the benefit taught by JONES to include additional mechanisms for determining light intensity values whereby said systems directed towards measuring determined amounts of light intensity which is used to switch imaging modes and would benefit from additional means to measure and determine light intensity to assist with system automation.
Claims 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over HUNG (Pub. No: US 2023-0166661) in view of LUNDGREN et al. et al. (Pub. No: US 2020-0322581), as applied in Claims 1-2, 9-10, 19-20, in view of WU (US Pub. No.: 2019-0287272).
As per Claim 7 HUNG discloses The one or more hardware processors of claim 1, wherein the processing circuitry is further to switch to operating of the image processing pipeline in the RGB imaging mode based on (See said analysis for Claim 1)
HUNG and LUNDGREN do not disclose but WU discloses based at least on turning off an IR emitter and subtracting IR intensities from one or more color channels (Figs. 1-13 IR light source 14 [0017] [0065] with source 14 turned off, generate subtraction IR components [0037, 0042-0043, 0079-0081])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include based at least on turning off an IR emitter and subtracting IR intensities from one or more color channels as taught by WU into the system of HUNG and LUNDGREN because of the benefit taught by WU to include IR light sources for control of IR light intensity which is a critical feature of said systems which requires changes in light intensity to drive system changes in said systems which would thereby benefit from the ability to control IR light values for control of driving systems changes.
As per Claim 8 HUNG discloses The one or more hardware processors of claim 1, wherein the operating the image processing pipeline in the IR imaging mode comprises(See said analysis for Claim 1)
HUNG and LUNDGREN do not disclose but WU discloses activating an IR illumination emitter and rendering in greyscale (Figs. 1-13 [0070]) (The motivation that applied in Claim 7 applies equally to Claim 8).
Allowable Subject Matter
REASONS FOR ALLOWANCE
The following is an Examiner’s statement of reasons for allowance: the closest prior art obtained from an Examiner’s search (HUNG, Pub. No: US 2023-0166661; LUNDGREN, Pub. No: US 2020-0322581; WU, US Pub. No.: 2019-0287272; JONES, US Pub. No.: 2022-0133241) does not teach nor suggest in detail the limitations:
“A system comprising one or more hardware processors to toggle, by an image processing pipeline, between an infrared (IR) imaging mode and an RGB imaging mode in response to one or more thresholds of detected green light intensity that depend on a toggling state: and at least one of: switch operation of the image processing pipeline from the RGB imaging mode to the IR imaging mode based at least on detecting that a first amount of green light in a frame of RGB image data generated in the RGB imaging mode is less than a first threshold of the one or more thresholds of detected green light intensity; or switch operation of the image processing pipeline from the IR imaging mode to the RGB imaging mode based at least on detecting that a second amount of green light in a frame of IR image data generated in the IR imaging mode exceeds a second threshold of the one or more thresholds of detected green light intensity”
as well as the combination of all the limitations within the independent claims and the enabling portions of the specification.
The closest prior art of record HUNG does not teach or suggest in detail at least in responding to one or more thresholds of detected green light intensity, or detecting that a first amount of green light in a frame of RGB image data generated in the RGB imaging mode is less than a first threshold of the one or more thresholds of detected green light intensity as amended and presented by the Applicant.
HUNG only discloses generating a first frame of RGB image data based at least on operating an image processing pipeline in an RGB imaging mode and based at least on detecting that a first amount of light intensity associated with the first frame of RGB image data is less than a first intensity threshold, switch to operating the image processing pipeline in an IR imaging mode to generate a second frame of infrared (IR) image data. The prior art also discloses that based at least on detecting that a second amount of light intensity associated with the second frame of IR image data exceeds a second intensity threshold, switching to operating the image processing pipeline in the RGB imaging mode.
Whereas, as stated above, Applicant’s claimed invention includes a system comprising one or more hardware processors to toggle, by an image processing pipeline, between an infrared (IR) imaging mode and an RGB imaging mode in response to one or more thresholds of detected green light intensity that depend on a toggling state. The claims further recite one of switching operation of the image processing pipeline from the RGB imaging mode to the IR imaging mode based at least on detecting that a first amount of green light in a frame of RGB image data generated in the RGB imaging mode is less than a first threshold of the one or more thresholds of detected green light intensity; or switching operation of the image processing pipeline from the IR imaging mode to the RGB imaging mode based at least on detecting that a second amount of green light in a frame of IR image data generated in the IR imaging mode exceeds a second threshold of the one or more thresholds of detected green light intensity.
So as indicated by the above statements, Applicant’s arguments and amendment have been considered persuasive, in light of the claim limitations as well as the enabling portions of the specification.
The dependent claims further limit the independent claims and are considered allowable on the same basis as the independent claims as well as for the further limitations set forth.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Claims 11-18 are allowed.
Claims 3-4 is/are objected to as being dependent upon the rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and but for any outstanding rejections.
Claims 3-4 is/are allowed, but for the outstanding rejections. The following is an examiner’s statement of reasons for allowance:
As per Claim 3 the prior art of record either alone or in reasonable combination fails to teach or suggest “The one or more hardware processors of claim 1, wherein the processing circuitry is further to switch to operating the image processing pipeline in the IR imaging mode based at least on detecting that the first amount of green light in the first frame of RGB image data is less than the first intensity threshold" These limitations in combination with the other limitations of the independent claim are thus deemed allowable.
As per Claim 4 the prior art of record either alone or in reasonable combination fails to teach or suggest “The one or more hardware processors of claim 1, wherein the processing circuitry is further to switch to operating the image processing pipeline in the RGB imaging mode based at least on detecting that the second amount of green light in the second frame of IR image data exceeds the second intensity threshold" These limitations in combination with the other limitations of the independent claim are thus deemed allowable.
The closest prior art of record HUNG (Pub. No: US 2023-0166661) for Claims 3-4 does not teach all the elements in combination with the other limitations of the independent claim. HUNG only discloses generating a first frame of RGB image data based at least on operating an image processing pipeline in an RGB imaging mode and based at least on detecting that a first amount of light intensity associated with the first frame of RGB image data is less than a first intensity threshold, switch to operating the image processing pipeline in an IR imaging mode to generate a second frame of infrared (IR) image data. The prior art also discloses that based at least on detecting that a second amount of light intensity associated with the second frame of IR image data exceeds a second intensity threshold, switching to operating the image processing pipeline in the RGB imaging mode.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EILEEN M ADAMS whose telephone number is 571-270-3688. The examiner can normally be reached on Monday-Friday from 7:00-4:00. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, William Vaughn can be reached on (571) 272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-270-4688.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EILEEN M ADAMS/Primary Examiner, Art Unit 2481