DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 10; 2, 8, 11 and 17 is/are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Iyer et al. U.S. Pub. No. 2021/0397751.
Re: claims 1 and 10 (which are rejected under the same rationale), Iyer teaches
1. (Original) A system, comprising: a display; (display 221; Iyer, [0032], Fig. 2)
a first processor; (application processor 602; Iyer, Fig. 6)
a second processor; (sensor processor 612; Iyer, Fig. 6)
(“The system 600 includes an application processor coupled with system memory 604. The application processor also couples with the sensor processor 612”; Iyer, [0055], Fig. 6)
Fig. 6 illustrates an application processor (first processor) and a sensor processor (second processor).
an image sensor configured to output image sensor data to the first processor or the second processor; (camera 123; Iyer, Fig. 6)
(“... the software initiated camera and microphone indicator can be enabled whenever multimedia 622 (e.g., audio, images, video) is configured to be received at the application processor 602... ”; Iyer, [0056])
Fig. 6 illustrates an camera (image sensor) that outputs image sensor data to, for example, the application processor.
an ambient light sensor configured to output ambient light data to the first processor and/or the second processor; (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
A camera or image sensor is used to detect ambient light level (ambient light sensor configured to output ambient light data). Environmental metadata includes camera metadata as an ambient light level.
(“The software can continue the suppression of the camera and microphone indicator when metadata 624 is transmitted to the application processor 602 and/or system memory 604 without a concurrent transmission multimedia 622.”; Iyer, [0056])
The metadata, which includes the ambient light level, is transmitted to the application processor (output ambient light data to the first processor and/or the second processor).
and a memory storing instructions executable by the first processor to: determine that the image sensor is not in use by an application executed at the first processor; and on condition that the image sensor is not in use by the application: block the image sensor data from the first processor; (“Embodiments described herein present a software camera and microphone indicator that indicates unambiguously to a user when audio or video/image data from a camera is being provided to either 1) the operating system, 2) first party applications, or 3) third party applications.”; Iyer, [0022])
The system includes a software camera indicator that indicates when the video/image data from the camera is being provided. Thus a determination is made as to whether the camera (image sensor) is in use.
(“The software can continue the suppression of the camera and microphone indicator when metadata 624 is transmitted to the application processor 602 and/or system memory 604 without a concurrent transmission multimedia 622. When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor.”; Iyer, [0056])
The software suppresses (block the image sensor data from the first processor) the camera/indicator when metadata is transmitted to the application processor (determine that the image sensor is not in use by an application executed at the first processor). The camera is not in use to provide images/video to the application processor.
and route the image sensor data to the second processor to thereby enable the second processor to execute a color adjustment algorithm configured to use at least the image sensor data and the ambient light data to adjust one or more color parameters of content displayed on the display, (“When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor During the processing, the data from the camera 123 and microphone 121 may be stored in sensor processor memory.”; Iyer, [0056])
When the camera is not in use to provide images/video to the application processor, data from the camera (image sensor data) is processed by the sensor processor (route the image sensor data to the second processor) to determine, for example, metadata values.
(“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to perform color space adjustments (use at least the image sensor data the ambient light data to adjust one or more of color parameters of content displayed on the display).
and execute a brightness adjustment algorithm configured to use at least the image sensor data and the ambient light data to adjust a luminance of the display. (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to adjust screen brightness (use at least the image sensor data the ambient light data to adjust a luminance of the display).
Re: claims 2 and 11(which are rejected under the same rationale), Iyer teaches
2. (Original) The system of claim 1, wherein the instructions are further executable to: determine that the image sensor is in use by the application executed at the first processor; and on condition that the image sensor is in use by the application: route the image sensor data to the first processor; (“... the software initiated camera and microphone indicator can be enabled whenever multimedia 622 (e.g., audio, images, video) is configured to be received at the application processor 602 and/or stored in system memory 604.”; Iyer, [0056], Fig. 6)
The software enables camera/indicator when images/video are to be received at the application processor (determine that the image sensor is in use by the application executed at the first processor).
cause execution of the color adjustment algorithm at the first processor to use at least the image sensor data and the ambient light data to adjust one or more color parameters of content displayed on the display; (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to perform color space adjustments (use at least the image sensor data the ambient light data to adjust one or more of color parameters of content displayed on the display).
and cause execution of the brightness adjustment algorithm at the first processor to use at least the image sensor data and the ambient light data to adjust a luminance of the display. (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to adjust screen brightness (use at least the image sensor data the ambient light data to adjust a luminance of the display).
Re: claims 8 and 17 (which are rejected under the same rationale), Iyer teaches
8. (Previously Presented) The system of claim 1, further comprising instructions executable by the second processor to: receive environmental context data; and use the environmental context data to adjust the one or more color parameters and/or adjust the luminance of the display. (“Environmental metadata also includes camera or image sensor metadata such as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect an ambient light level, which is then used to adjust screen brightness or perform color-space adjustments... Analysis of such metadata may be performed by sensor processors, secure processors, or low-power processors of the system without providing any image or video data to the operating system or applications.”; Iyer, [0037])
Environmental metadata (environmental context data) includes camera or image sensor data such as ambient light level. A camera or image sensor is used to detect an ambient light level (receive environmental context data), which is then used to adjust screen brightness or perform color space adjustments (use the environmental context data to adjust the one or more color parameters and/or adjust the luminance of the display).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer et al. U.S. Pub. No. 2021/039775 in view of Roberts et al. U.S. Pub. No. 2016/0286619, Dahlkamp et al. U.S. Pub. No. 2021/0383100 and Park et al. U.S. Pub. No. 2023/0281885.
Re: claim 19, Iyer teaches
19. (Original) A computing device, comprising: a display; (display 221; Iyer, [0032], Fig. 2)
a first processor; (application processor 602; Iyer, Fig. 6)
a second processor; (sensor processor 612; Iyer, Fig. 6)
(“The system 600 includes an application processor coupled with system memory 604. The application processor also couples with the sensor processor 612”; Iyer, [0055], Fig. 6)
Fig. 6 illustrates an application processor (first processor) and a sensor processor (second processor).
an image sensor configured to output image sensor data to the first processor or the second processor; (camera 123; Iyer, Fig. 6)
(“... the software initiated camera and microphone indicator can be enabled whenever multimedia 622 (e.g., audio, images, video) is configured to be received at the application processor 602... ”; Iyer, [0056])
Fig. 6 illustrates an camera (image sensor) that outputs image sensor data to, for example, the application processor.
... an ambient light sensor configured to output ambient light data to the first processor and/or the second processor,
(“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
A camera or image sensor is used to detect ambient light level (ambient light sensor configured to output ambient light data). Environmental metadata includes camera metadata as an ambient light level.
(“The software can continue the suppression of the camera and microphone indicator when metadata 624 is transmitted to the application processor 602 and/or system memory 604 without a concurrent transmission multimedia 622.”; Iyer, [0056])
The metadata, which includes the ambient light level, is transmitted to the application processor (output ambient light data to the first processor and/or the second processor).
Iyer is silent regarding the image sensor data has a first image resolution, and wherein the ambient light data has a second image resolution that is less than the first image resolution, however Roberts teaches
wherein the image sensor data has a first image resolution; wherein the ambient light data has a second image resolution less than the first image resolution, (“image data may be provided directly to the driver circuitry 40, communications circuitry 28, or both. For example, low resolution image data for ambient light or occupancy determination may be provided to the driver circuitry 40 for processing. High resolution image data could be sent to the communications circuitry 28 for delivery to a security center so that security personnel can monitor high resolution images.”; Roberts, [0084], Fig. 8)
Fig. 8 illustrates that image sensor provides image data to the driver circuitry at a low resolution (second image resolution) for ambient light (ambient light data has a second image resolution that is less than the first image resolution) and provides image data to the communication circuitry at a high resolution (first resolution) for delivery to the security. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of the image sensor data has a first image resolution, and wherein the ambient light data has a second image resolution that is less than the first image resolution, in order to add intelligence to lighting fixtures that can be leveraged to employ lighting control based on environmental conditions such as ambient light, as taught by Roberts ([0004]).
Iyer teaches and a memory storing instructions executable by the first processor to, detect whether the image sensor is in use by an application executed at the first processor, on condition that the image sensor is in use by the application, route the image sensor data to the first processor, (“... the software initiated camera and microphone indicator can be enabled whenever multimedia 622 (e.g., audio, images, video) is configured to be received at the application processor 602 and/or stored in system memory 604.”; Iyer, [0056], Fig. 6)
The software enables camera/indicator when images/video are to be received at the application processor (detect whether the image sensor is in use by an application executed at the processor, on condition that the image sensor is in use by the application, route the image sensor data to the first processor).
cause execution of a color adjustment algorithm configured to use at least the image sensor data and the ambient light data to adjust one or more color parameters of content displayed on the display, (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to perform color space adjustments (use at least the image sensor data the ambient light data to adjust one or more of color parameters of content displayed on the display).
and cause execution of a brightness adjustment algorithm configured to use at least the image sensor data and the ambient light data to adjust a luminance of the display; (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to adjust screen brightness (use at least the image sensor data the ambient light data to adjust a luminance of the display).
and wherein the second image resolution is less than a threshold resolution that enables facial recognition; (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as facial recognition-based device unlock. A camera or image sensor is used to detect user attention and facial recognition-based device unlock. Environmental metadata includes camera metadata as facial recognition-based device unlock.
(“The software can continue the suppression of the camera and microphone indicator when metadata 624 is transmitted to the application processor 602 and/or system memory 604 without a concurrent transmission multimedia 622. When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor.”; Iyer, [0056])
The software suppresses the camera/indicator when metadata is transmitted to the application processor (on condition that the image sensor is not in use by the application). The camera is not in use to provide images/video to the application processor (first processor).
(“When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor During the processing, the data from the camera 123 and microphone 121 may be stored in sensor processor memory.”; Iyer, [0056])
When the camera is not in use to provide images/video to the application processor, data from the camera (image sensor data) is processed by the sensor processor to determine, for example, metadata values for facial recognition. Iyer is silent regarding the received image sensor data being at a resolution that is less than a threshold that enables facial recognition, however Dahlkamp teaches
(“... the image captured by the observation camera may have a second level of image quality... the second level of image quality may be different from (e.g., less than) the first level of image quality of the user device camera. For example, the observation camera 403 may record images at a lower resolution and/or contrast than the user device camera.”; Dahlkamp, [0048], Fig. 4)
Image captured by the observation camera are at a lower resolution (resolution that is less than a threshold resolution) than the user device camera.
(“... the system may perform facial recognition using images that may have different camera sources and/or different levels of quality. For example, a set of reference images may be primarily captured by a user’s mobile phone. These reference images may typically have a higher level of quality than images captured by an observation camera. Embodiments of the present disclosure effectively “bridge the gap” by allowing the system to recognize faces in images of lower quality by comparing them with faces in images of higher quality.”; Dahlkamp, [0055])
The system performs facial recognition using images on the lower resolution images (receive the image sensor data at a resolution that is less than a threshold resolution that enables facial recognition) by comparing them with faces in the higher resolution images. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of the second image resolution is less than a threshold resolution that enables facial recognition, in order to perform facial recognition using images that may have different camera sources and/or differing levels of quality by allowing the systems to recognize faces in images of lower quality by comparing with faces in images of higher quality, as taught by Dahlkamp ([0055]).
Iyer teaches and on condition that the image sensor is not in use by the application, block the image sensor data from the first processor, and route the image sensor data to the second processor at a third image resolution that is less than the threshold resolution to thereby enable the second processor to use at least the image sensor data and the ambient light data to execute the color adjustment algorithm and the brightness adjustment algorithm. (“When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor During the processing, the data from the camera 123 and microphone 121 may be stored in sensor processor memory.”; Iyer, [0056])
When the camera is not in use to provide images/video to the application processor, data from the camera (image sensor data) is processed by the sensor processor (route the image sensor data to the second processor) to determine, for example, metadata values.
(“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to perform color space adjustments (use at least the image sensor data the ambient light data to adjust one or more of color parameters of content displayed on the display). Iyer is silent regarding a third image resolution that is less than the threshold resolution, however, Park teaches this limitation.
(“To generate the modified image data 242, the first image processing engine 240 modifies the image data 232... to decrease image resolution of the image data 232 from a first resolution to a second resolution to reduce bandwidth usage... to adjust brightness, to adjust contrast, to adjust gamma, to adjust tone, to adjust luminance, to adjust color saturation... ”; Park, [0064])
The resolution of the image date is reduced from a first resolution to a second resolution (third resolution) and the brightness/luminance is adjusted and the tone/color saturation is adjusted. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of the image sensor data has a first image resolution, a third image resolution that is less than the threshold resolution, in order to provide additional functionality for video conferencing, allowing a user to see where other users are looking at a given moment, which provides helpful context for conversations conducted over the video conference, as taught by Park ([0041]).
Iyer teaches and execute a brightness adjustment algorithm configured to use at least the image sensor data and the ambient light data to adjust a luminance of the display. (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as an ambient light level, which is used to adjust screen brightness (use at least the image sensor data the ambient light data to adjust a luminance of the display).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer, Roberts, Dahlkamp and Park as applied to claim 19 above, and further in view of Greenebaum et al. U.S. Pub. No. 2020/0105225.
Claim 20 is a device analogous to the system of claim 1, is similar in scope and is rejected under the same rationale.
Claim(s) 3 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer as applied to claims 1 and 10 above, and further in view of Roberts.
Re: claims 3 and 12 (which are rejected under the same rationale), Iyer is silent regarding the image sensor data has a first image resolution, and wherein the ambient light data has a second image resolution that is less than the first image resolution, however Roberts teaches
3. (Previously Presented) The system of claim 1,wherein the image sensor data has a first image resolution, and wherein the ambient light data has a second image resolution that is less than the first image resolution. (“image data may be provided directly to the driver circuitry 40, communications circuitry 28, or both. For example, low resolution image data for ambient light or occupancy determination may be provided to the driver circuitry 40 for processing. High resolution image data could be sent to the communications circuitry 28 for delivery to a security center so that security personnel can monitor high resolution images.”; Roberts, [0084], Fig. 8)
Fig. 8 illustrates that image sensor provides image data to the driver circuitry at a low resolution (second image resolution) for ambient light (ambient light data has a second image resolution that is less than the first image resolution) and provides image data to the communication circuitry at a high resolution (first resolution) for delivery to the security. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of the image sensor data has a first image resolution, and wherein the ambient light data has a second image resolution that is less than the first image resolution, in order to add intelligence to lighting fixtures that can be leveraged to employ lighting control based on environmental conditions such as ambient light, , as taught by Roberts ([0004]).
Claim(s) 9 and18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer as applied to claims 1 and 10 above, and further in view of Dahlkamp and Fleming U.S. Pub. No. 2021/0109585.
Re: claims 9 and 18 (which are rejected under the same rationale), Iyer teaches
9. (Currently Amended) The system of claim 1, further comprising instructions executable by the second processor to, on condition that the image sensor is not in use by the application, downgrade performance of the image sensor to reduce a resolution of the image sensor data below a threshold resolution that enables facial recognition. (“Environmental metadata also includes camera or image sensor metadata as an ambient light level, user presence, user attention, or facial recognition-based device unlock. For example, one or more cameras or image sensors may be used to detect ambient light level which is then used to adjust screen brightness or perform color-space adjustments.”; Iyer, [0037])
Environmental metadata includes image sensor metadata as facial recognition-based device unlock. A camera or image sensor is used to detect user attention and facial recognition-based device unlock. Environmental metadata includes camera metadata as facial recognition-based device unlock.
(“The software can continue the suppression of the camera and microphone indicator when metadata 624 is transmitted to the application processor 602 and/or system memory 604 without a concurrent transmission multimedia 622. When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor.”; Iyer, [0056])
The software suppresses the camera/indicator when metadata is transmitted (instead of multimedia data) to the application processor (on condition that the image sensor is not in use by the application). The camera is not in use to provide images/video/multimedia data to the application processor (first processor).
(“When metadata 624 is transmitted, values for the metadata can be determined by processing data from the camera 123 and microphone 121 using software or firmware logic that executes on the sensor processor During the processing, the data from the camera 123 and microphone 121 may be stored in sensor processor memory.”; Iyer, [0056])
When the camera is not in use to provide images/video to the application processor, data from the camera (image sensor data) is processed by the sensor processor to determine, for example, metadata values for facial recognition. Iyer is silent regarding the received image sensor data being at a resolution that is less than a threshold that enables facial recognition, however, Dahlkamp and Fleming teach
(“... the image captured by the observation camera may have a second level of image quality... the second level of image quality may be different from (e.g., less than) the first level of image quality of the user device camera. For example, the observation camera 403 may record images at a lower resolution and/or contrast than the user device camera.”; Dahlkamp, [0048], Fig. 4)
Image captured by the observation camera are at a lower resolution (resolution that is below a threshold resolution) than the user device camera.
(“... the system may perform facial recognition using images that may have different camera sources and/or different levels of quality. For example, a set of reference images may be primarily captured by a user’s mobile phone. These reference images may typically have a higher level of quality than images captured by an observation camera. Embodiments of the present disclosure effectively “bridge the gap” by allowing the system to recognize faces in images of lower quality by comparing them with faces in images of higher quality.”; Dahlkamp, [0055])
The system performs facial recognition using images on the lower resolution images (the image sensor data at a resolution that is below a threshold resolution that enables facial recognition) by comparing them with faces in the higher resolution images.
(“Further, to reduce power during the streaming mode, the image sensor 122 captures image data at a relatively low resolution that is sufficient to enable the low power operations controller 130 to perform facial recognition and/or gaze direction analysis of a user within the field of view of the image sensor.”; Fleming, [0038])
To reduce power (downgrade performance), the image sensor captures image data at a low resolution (below a threshold) that is sufficient to enable the low power operations controller to perform facial recognition. Fleming and Dahlkamp are combined with Iyer such that the system of Iyer includes the function of the image sensor capturing image data at a lower resolutions of Fleming and Dahlkamp, when power is reduced. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of downgrade performance of the image sensor to reduce a resolution of the image sensor data below a threshold resolution that enables facial recognition, in order to perform facial recognition using images that may have different camera sources and/or differing levels of quality by allowing the systems to recognize faces in images of lower quality by comparing with faces in images of higher quality, as taught by Dahlkamp ([0055]) and in order to enable the low power operations controller 130 to identify changes in the presence and/or engagement of a user and respond with changes to the operation of the computing device without delay that would be noticeable to the user, as taught by Fleming ([0038]).
Claim(s) 4, 5, 7, 13, 14 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer as applied to claims 1 and 10 above, and further in view of Greenebaum.
Re: claims 4, 13 and 20 (which are rejected under the same rationale), Iyer is silent regarding segment background pixels from the image sensor data; and use the background pixels to execute the color adjustment algorithm and the brightness adjustment algorithm, however Greenbaum teaches
4. (Previously Presented) The system of claim 1, further comprising instructions executable by the second processor to: segment background pixels from the image sensor data; and use the background pixels to execute the color adjustment algorithm and the brightness adjustment algorithm. (“ For example, where the unintended light desaturates two adjacent colors, e.g., in the case of orange lettering on a red background, such that the cumulative driven and unintended light of the two adjacent intended colors are indistinguishable from one another... and the orange lettering is indistinguishable from the red background, the partial adjustment may optimize color contrast in order to recreate the intended color contrast including both colors, while, for example, allowing brightness or another parameter to vary from the source author's original intent.”; Greenbaum, [0031])
The system determines that image includes orange lettering on a red background (segment background pixels from the image sensor data). When unintended light desaturates these two adjacent colors such that the orange lettering is indistinguishable from the red background, a partial adjustment optimizes the color contrast while varying the brightness, to recreate the intended color contrast using both colors (use the background pixels to execute the color adjustment algorithm and brightness adjustment algorithm. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of segment background pixels from the image sensor data; and use the background pixels to execute the color adjustment algorithm and the brightness adjustment algorithm, in order to continuously and dynamically adjust display 340 to compensate for changing unintended light, and modulate the adjustments, e.g., at a rate commensurate with a viewer's ability to perceive the adjustments, such that the adaptation appears seamlessly to the viewer, as taught by Greenebaum ([0029]).
Re: claims 5 and 14 (which are rejected under the same rationale), Iyer is silent regarding segment pixels that correspond to a reflective object from the image sensor data; and use the pixels that correspond to the reflective object to account for light reflected by the reflective object, however, Greenebaum teaches
5. (Previously Presented) The system of claim 1, further comprising instructions executable by the second processor to: segment pixels that correspond to a reflective object from the image sensor data; and use the pixels that correspond to the reflective object to account for light reflected by the reflective object. (“... when source content 304 is determined to need additional adjustment in regions of pixels having a particular color(s) or at a particular place(s) on the display screen, e.g., if source content 304 included a person wearing a red sweater standing against a white background, the pixels making up the red sweater portion of the displayed image may require a greater degree of ambient resaturation adjustment than, say, the pixels making up the white background portion of the displayed image.”; Greenebaum, [0029])
The system determines that the source content includes a person wearing a red sweater (reflective object) standing against a white background (segment pixels that correspond to a reflective object from the image sensor). The pixels making up the red sweater portion of the image requires a greater degree of ambient resaturation adjustment (use the pixels that correspond to the reflective object to account for light reflected by the reflective object). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of segment pixels that correspond to a reflective object from the image sensor data; and use the pixels that correspond to the reflective object to account for light reflected by the reflective object, in order to continuously and dynamically adjust display 340 to compensate for changing unintended light, and modulate the adjustments, e.g., at a rate commensurate with a viewer's ability to perceive the adjustments, such that the adaptation appears seamlessly to the viewer, as taught by Greenebaum ([0029]).
Re: claims 7 and 16 (which are rejected under the same rationale), Iyer is silent regarding segment pixels that correspond to a light source from the image sensor data; and use at least the pixels that correspond to the light source to execute the color adjustment algorithm and the brightness adjustment algorithm, however, Greenebaum teaches
7. (Previously Presented) The system of claim 1, further comprising instructions executable by the second processor to: segment pixels that correspond to a light source from the image sensor data; and use at least the pixels that correspond to the light source to execute the color adjustment algorithm and the brightness adjustment algorithm. (“... modulator 330 may be used to dynamically compensate for unintended light. Dynamically compensating for unintended light may be based, e.g., on a calculation received from saturation model 320 about unintended light being added to light driven display 340. This may mean adjusting the light driven by a small group of pixels of the display device to compensate for a localized effect from unintended light or adjusting the light driven by all pixels of the display to compensate for a more global effect from unintended light.”; Greenebaum, [0029], Fig. 3)
A modulator is used to compensate for unintended light by adjusting the light driven by a small group of pixels (segment pixels that correspond to a light source from the image sensor data) to compensate for a localized effect from unintended light (use at least the pixels that correspond to the light source to execute the color adjustment algorithm and the brightness adjustment algorithm).
(“Information regarding ambient light conditions may include the color and brightness of any ambient light sources, as well as the angle and distance from the ambient light source to the display device. For example, soft orange-white 2700K light from a 60 watt incandescent light bulb shielded by a lamp shade at a distance from the display device combines with light driven by the display differently than bright white sunlight from a large window directly to one side of the display device... optical sensors 104 may include a light field camera, which provides information indicative of light intensity and direction of light rays. This additional information regarding the direction of light rays may enable per-pixel adjustments to compensate for unintended light, specular ions, and/or mirror-like ions.”; Greenebaum, [0032])
The light source can be, for example, a soft orange-white 2700 K light from a 60 watt incandescent light bulb shielded by a lamp shade. Optical sensors provide information indicating light intensity and direction of light rays, which enables per-pixel adjustments to compensate for unintended light (use at least the pixels that correspond to the light source to execute the color adjustment algorithm and the brightness adjustment algorithm). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of segment pixels that correspond to a light source from the image sensor data; and use at least the pixels that correspond to the light source to execute the color adjustment algorithm and the brightness adjustment algorithm, in order to continuously and dynamically adjust display 340 to compensate for changing unintended light, and modulate the adjustments, e.g., at a rate commensurate with a viewer's ability to perceive the adjustments, such that the adaptation appears seamlessly to the viewer, as taught by Greenebaum ([0029]).
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iyer as applied to claims 1 and 10 above, and further in view of Marcu et al. U.S. Patent No. 10,672,363.
Re: claim 6, Iyer is silent regarding use the image sensor data to detect a low light condition or a mixed light condition, however, Marcu teaches
6. (Previously Presented) The system of claim 1, further comprising instructions executable by the second processor to use the image sensor data to detect a low light condition or a mixed light condition. (“Image processing circuit 145 may aid in the capture of still and video images from image sensor 110 and include at least one video codec.”; Marcu, col. 5, lines 43-45, Fig. 1)
The image processing circuit (second processor) aids the image sensor in the capture of still and video images.
(“For example, perceptual model 212 may take into consideration information indicative of ambient light conditions obtained from one or more optical sensors 216 (e.g., ambient light sensors, image sensors, and the like).”; Marcu, col. 8, lines 31-35)
(34)
The perceptual model considers ambient light condition information obtained from, for example, an image sensor.
(“For example, when perceptual model 212 determines based on ambient light levels and viewing conditions that a user is adapted to a low light (e.g., dim or night-time) viewing environment, perceptual model 212 may modify transfer function 222 to set the white point (e.g., a white luminance value of the display panel) of display 250 to be lower than the target or native white point (e.g., the maximum luminance value of the display panel 250.”; Marcu, col. 10, lines 18-26)
The perceptual model determines, using the ambient light levels obtained from the image sensor, that the user is in a low light viewing environment (use the image sensor data to detect a low light condition or a mixed light condition). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of use the image sensor data to detect a low light condition or a mixed light condition, in order to perform improved color rendering in extended dynamic range mode, as taught by Marcu (col. 6, lines 34-37).
Re: claim 15, Iyer is silent regarding using the image sensor data to detect a low light condition or a mixed light condition, however, Marcu teaches
15. (Previously Presented) The method of claim 10, further comprising using the image sensor data to detect a low light condition or a mixed light condition. (“Image processing circuit 145 may aid in the capture of still and video images from image sensor 110 and include at least one video codec.”; Marcu, col. 5, lines 43-45, Fig. 1)
The image processing circuit aids the image sensor in the capture of still and video images.
(“For example, perceptual model 212 may take into consideration information indicative of ambient light conditions obtained from one or more optical sensors 216 (e.g., ambient light sensors, image sensors, and the like).”; Marcu, col. 8, lines 31-35)
(34)
The perceptual model considers ambient light condition information obtained from, for example, an image sensor.
(“For example, when perceptual model 212 determines based on ambient light levels and viewing conditions that a user is adapted to a low light (e.g., dim or night-time) viewing environment, perceptual model 212 may modify transfer function 222 to set the white point (e.g., a white luminance value of the display panel) of display 250 to be lower than the target or native white point (e.g., the maximum luminance value of the display panel 250.”; Marcu, col. 10, lines 18-26)
The perceptual model determines, using the ambient light levels obtained from the image sensor, that the user is in a low light viewing environment (using the image sensor data to detect a low light condition or a mixed light condition). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date, to modify the system of Iyer by adding the feature of using the image sensor data to detect a low light condition or a mixed light condition, in order to perform improved color rendering in extended dynamic range mode, as taught by Marcu (col. 6, lines 34-37).
Response to Arguments
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“The claims are directed to systems and methods that determine whether an image sensor is in use by an application at a first processor. If the image sensor is not in use by the application at the first processor, the system blocks image sensor data from the first processor and routes the image data to a second processor to execute color and brightness adjustment algorithms. Advantageously, in this manner the first processor is prevented from receiving potentially private information from the image sensor data when the image sensor data is not already in use at the first processor, while still enabling the computing device to accurately determine ambient lighting conditions and display content accordingly using a combination of the image sensor data and ambient light data. By contrast, Iyer discloses methods for controlling a privacy indicator for a camera and microphone. Iyer discloses determining that data transmitted from a camera is only metadata and does not include image data. Based on this determination, Iyer turns off a camera indicator. For example, at paragraph [0034] Iyer discloses that "[t]he software camera and microphone indicator will not be displayed when the camera or microphone is used only to gather environmental metadata." However, Applicant can find no disclosure in Iyer of blocking image sensor data from being received by a processor. As noted, Iyer merely determines a type of data transmitted from a camera and based on that determination turns a camera indicator on or off. Applicant can find no disclosure in Iyer of affirmatively blocking camera data from a processor to prevent the processor from receiving potentially private information from the image sensor data when the image sensor data is not already in use at the first processor. Accordingly and for at least the reasons provided above, Applicant submits that Iyer does not disclose the configurations of claims 1 or 10. Therefore, Applicant respectfully requests reconsideration and withdrawal of the rejection of independent claims 1 and 10. Claims 2, 8, 11, and 17 depend directly or indirectly from claims 1 or 10. Accordingly, Applicant also respectfully requests that the rejection of claims 2, 8, 11, and 17 be withdrawn for at least the reason of dependence from an allowable base claim.”
Examiner disagrees. Iyer teaches that the software suppresses (block the image sensor data from the first processor) the camera/indicator when metadata is transmitted (instead of multimedia data) to the application processor, which is considered to be the first processor (determine that the image sensor is not in use by an application executed at the first processor). The camera is not in use to provide images/video/multimedia data to the application processor and is therefore considered to be blocked. (Iyer, [0056]) .
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Roberts, Dahlkamp and Park fail to cure the deficiencies of Iyer discussed above. As such, Applicant respectfully requests withdrawal of the rejection of claim 19 under 35 U.S.C. 103.”
Examiner disagrees. Claim 19 has been rejected. Please see the rejection for claim 19.
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Greenebaum fails to cure the deficiencies of Iyer, Roberts, Dahlkamp, and Park discussed above. As such, Applicant respectfully requests withdrawal of the rejection of claim 20 under 35 U.S.C. 103.”
Examiner disagrees. Claim 19 as well as claim 20 have been rejected. Please see the corresponding rejections.
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Roberts fails to cure the deficiencies of Iyer as discussed above. As such, Applicant respectfully requests withdrawal of the rejection of claims 3 and 12 under 35 U.S.C. 103.”
Examiner disagrees. Claim 1 and claims 3 and 12 have been rejected. Please see the corresponding rejections.
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Dahlkamp fails to cure the deficiencies of Iyer as discussed above. Additionally, in the interest of advancing prosecution, claims 9 and 18 are amended to recite that, on condition that the image sensor is not in use by the application, performance of the image sensor is downgraded to reduce a resolution of the image sensor data below a threshold resolution that enables facial recognition. None of the cited references discloses this limitation. Accordingly and for this additional reason, Applicant respectfully requests withdrawal of the rejection of claims 9 and 18 under 35 U.S.C. 103.”
Examiner disagrees. This amended limitation is taught by Iyer, Dahlkamp and Fleming. For example, Fleming teaches to reduce power (downgrade performance), the image sensor captures image data at a low resolution (below a threshold) that is sufficient to enable the low power operations controller to perform facial recognition. (Fleming, [0038]).
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Greenebaum fails to cure the deficiencies of Iyer discussed above. As such, Applicant respectfully requests withdrawal of the rejection of claims 4, 5, 7, 13, 14, and 16 under 35 U.S.C. 103.”
Examiner disagrees. Claim 1 and claims 4, 5, 7, 13, 14 and 16 have been rejected. Please see the corresponding rejections.
Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. Applicant argues:
“Applicant respectfully traverses this rejection as Marcu fails to cure the deficiencies of Iyer discussed above. As such, Applicant respectfully requests withdrawal of the rejection of claims 6 and 15 under 35 U.S.C. 103”
Examiner disagrees. Claim 1 and claims 6 and 15 have been rejected. Please see the corresponding rejections.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONNA J RICKS whose telephone number is (571)270-7532. The examiner can normally be reached on M-F 7:30am-5pm EST (alternate Fridays off).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached on 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Donna J. Ricks/Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights