Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This communication is in response to the Application filed on 09/27/2023.
Claims 1-30 are pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-4, 6-9, 12-15, 17-19, 22-25, 27 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (U.S Patent 10,021,313) (hereafter, "Chen") in view of Prentice (U.S Publication No. 2012/0257071).
Regarding claim 1, Chen teaches an apparatus comprising (Column 4, line 3, smartphone 102-1): a processing system that includes one or more memories and one or more processors coupled to the one or more memories, the processing system configured to (Column 4, line 8-10, Computing device 102 includes processor(s) 202 and computer-readable media 204, which includes memory media 206 and storage media 208): … generate a composite image in accordance with the second quantity of frames (Column 12, line 60-61, 63-65, 67 & Column 13, line 1-4, 6-9 and 13-15, step 604 captures multiple frames with the image sensor at a frame rate ... a frame rate used to capture the multiple frames decreases as an amount of detectable light decreases ... the frame rate used to capture a scene increases as an amount of detectable light increases ... Frames of a scene are captured at the determined frame rate for a threshold amount of time ... the computing device 102 captures the multiple image frames 116 of scene 114 using the image sensor 106 at the determined frame rate for a threshold amount of time ... Step 606 aligns a subset of the captured multiple frames and generates a multi-frame image by combining pixel values from the aligned subset of frames into a single image).
Chen does not teach determine a first quantity of frames for a multi-frame image capture operation; initiate capture of one or more frames of the multi-frame image capture operation using a camera; after capturing the one or more frames and prior to completing the multi-frame image capture operation, determine a second quantity of frames for the multi-frame image capture operation; and.
However, Prentice teaches determine a first quantity of frames for a multi-frame image capture operation ([0085] When the digital camera 10 is set to operate in the burst capture mode, an initialize frame rate step 200 is used to initialize the frame rate 205 (R) to some initial frame rate, having an associated image capture time interval ΔT=1/R); initiate capture of one or more frames of the multi-frame image capture operation using a camera ([0086] An initiate image capture sequence step 210 is used to initiate the capture of a sequence (i.e., a “burst”) of digital images … the initiate image capture sequence step 210 is triggered by the activation of an image capture control such as a shutter button when the digital camera 10 is set to operate in the burst capture mode); after capturing the one or more frames and prior to completing the multi-frame image capture operation ([0100] while digital images 225 are being captured at the initial frame rate 205 ... The modify frame rate step 240 is then used to determine a new reduced frame rate 205 ... additional digital images 225 can continue to be captured and stored in the buffer memory 18 according to the new frame rate 205; [0102] The terminate image capture sequence step 260 terminates the capture of any additional digital images; FIG. 4 shows the element 240 [MODIFY FRAME RATE] happens after the element 215 [CAPTURE DIGITAL IMAGE] and before the element 260 [TERMINATE IMAGE CAPTURE SEQUENCE]), determine a second quantity of frames for the multi-frame image capture operation; and ([0090] A modify frame rate step 240 is used to modify the frame rate 205 ... the frame rate 205 is adjusted so that the new frame rate is reduced by a factor of 2× relative to the previous frame rate (R′=R/2); [0094] The modify frame rate step 240 (FIG. 4) then updates the frame rate 205 (FIG. 4) by reducing it by a factor of two. The new image capture time interval will therefore be 2ΔT).
It would have been obvious before the effective filing date of the claimed invention to one having ordinary skill in the art to modify the method and device of Chen to incorporate the step/system of initializing the capture of a sequence of images by setting the frame rate (quantity of frames) for a burst capture mode, and modifying the frame rate after capturing images and before terminating the capture of images taught by Prentice.
The suggestion/motivation for doing so would have been to improve the stability of composite image generation by reducing unexpected outcomes ([0039] the burst image capture mode can be used to generate a composite image highlighting a moving object. This invention provides a method for managing buffer memory in a manner such that a user does not need to manually set image capture settings specifying a total time duration for the burst of image captures, thus reducing the number of unacceptable results; [0044] It will be obvious that there are many variations of this embodiment that are possible and are selected to reduce the cost, add features or improve the performance of the camera). Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predicted results. Therefore, it would have been obvious to combine Chen with Prentice to obtain the invention as specified in claim 1.
Regarding claim 2, the combination of Chen and Prentice teaches all the limitations of claim 1 above. Chen teaches wherein the processing system is further configured to perform a plurality of image blending operations associated with the composite image (Column 3, line 26-29, multi-frame image module 108 represents functionality that combines the multiple frames 116 to generate a multi-frame image 118; Column 3, line 37-41, multi-frame image module 108 combines pixel values describing color and brightness values for individual pixels in the multiple frames 116 and averages combined pixel values to generate pixel values for the multi-frame image 118).
Regarding claim 3, the combination of Chen and Prentice teaches all the limitations of claim 2 above. Chen teaches wherein the processing system is further configured to adjust a quantity of the plurality of image blending operations from the first quantity to the second quantity (Column 10, line 57-63, consider a multi-frame image generated using five individual image frames in comparison to a multi-frame image generated using two individual image frames. As described above, the multi-frame image generated from five image frames would result in noise less noise and increased sharpness in comparison to the multi-frame image generated using only two image frames; The number of input frames directly corresponds to the number of blending operations performed, as each pixel from every input frame must be processed and combined to produce the final image).
Regarding claim 4, the combination of Chen and Prentice teaches all the limitations of claim 2 above. Chen teaches wherein the processing system is further configured to determine a plurality of candidate quantity values of the multi-frame image capture operation (Column 11, line 11-14, adjustment parameter table 502 is illustrated as including row 504, which includes information specifying different numbers of possible frames “m” that were captured and buffered to generate the multi-frame image 118) respectively associated with at least a subset of the plurality of image blending operations (Column 13, line 13-15 & 32-35, Step 606 aligns a subset of the captured multiple frames and generates a multi-frame image by combining pixel values from the aligned subset of frames into a single image ... Step 608 determines a set of image adjustment parameters for the multi-frame image based on a number of individual frames in the subset of frames that were used to generate the multi-frame image).
Regarding claim 6, the combination of Chen and Prentice teaches all the limitations of claim 1 above. Chen teaches wherein the processing system is further configured to determine the first quantity of frames in accordance with a first ambient lighting condition associated with the camera and (Column 12, line 63-65, a frame rate used to capture the multiple frames decreases as an amount of detectable light decreases; Column 8, line 47-50, an appropriate frame rate to use when capturing individual frames of scene 114 depends on an amount of light emitted by scene 114 that can be detected by image sensor 106) to determine the second quantity of frames in accordance with one or more second ambient lighting conditions associated with the camera (Column 12, line 67 & Column 13, line 1, the frame rate used to capture a scene increases as an amount of detectable light increases; Column 8, line 50-53, the image capture module 104 is configured to automatically adjust a frame rate at which the image sensor 106 captures the multiple image frames 116 based on an amount of detectable light).
Regarding claim 7, the combination of Chen and Prentice teaches all the limitations of claim 6 above. Chen teaches wherein the processing system is further configured to determine the first ambient lighting condition (Column 12, line 63-65, a frame rate used to capture the multiple frames decreases as an amount of detectable light decreases; Column 8, line 47-50, an appropriate frame rate to use when capturing individual frames of scene 114 depends on an amount of light emitted by scene 114 that can be detected by image sensor 106) in accordance with one or more of a first autoexposure (AE) parameter, a first auto white balance (AWB) parameter, a first autofocus (AF) parameter, a first AE, AWB, and AF (3A) parameter, or a first sensitivity parameter (Column 5, line 60-64, longer exposure times are often implemented in low-light scenarios in order to gather additional light photons emitted from scene 114, which enables the sensors 402 to register luminance values that otherwise go unregistered at shorter exposure times).
Regarding claim 8, the combination of Chen and Prentice teaches all the limitations of claim 7 above. Chen teaches determine the one or more second ambient lighting conditions (Column 12, line 67 & Column 13, line 1, the frame rate used to capture a scene increases as an amount of detectable light increases; Column 8, line 50-53, the image capture module 104 is configured to automatically adjust a frame rate at which the image sensor 106 captures the multiple image frames 116 based on an amount of detectable light) in accordance with one or more of a second AE parameter different than the first AE parameter, a second AWB parameter different than the first AWB parameter, a second AF parameter different than the first AF parameter, a second 3A parameter different than the first 3A parameter, or a second sensitivity parameter different than the first sensitivity parameter (Column 3, line 18-22, different ones of the multiple image frames 116 are generated using different exposure times, thereby producing multiple image frames 116 having varying brightness levels among different frames of the scene 114).
Chen does not teach wherein the processing system is further configured to, after initiating the multi-frame image capture operation and prior to completing the multi-frame image capture operation.
However, Prentice teaches wherein the processing system is further configured to, after initiating the multi-frame image capture operation and prior to completing the multi-frame image capture operation ([0100] while digital images 225 are being captured at the initial frame rate 205 ... The modify frame rate step 240 is then used to determine a new reduced frame rate 205 ... additional digital images 225 can continue to be captured and stored in the buffer memory 18 according to the new frame rate 205; [0102] The terminate image capture sequence step 260 terminates the capture of any additional digital images; FIG. 4 shows the element 240 [MODIFY FRAME RATE] happens after the element 215 [CAPTURE DIGITAL IMAGE] and before the element 260 [TERMINATE IMAGE CAPTURE SEQUENCE]).
It would have been obvious before the effective filing date of the claimed invention to one having ordinary skill in the art to modify the method and device of Chen to incorporate the step/system of modifying the frame rate after capturing images and before terminating the capture of images taught by Prentice.
Motivation for this combination has been stated in claim 1.
Regarding claim 9, the combination of Chen and Prentice teaches all the limitations of claim 1 above. Chen teaches wherein the one or more memories include a circular buffer (Column 13, line 9-10, Frame stacking module is configured to buffer each of the multiple image frames 116; Column 8, line 28-31, computing device 102 from FIG. 1 can limit the frame stacking module 108 to a threshold amount of time for capturing and buffering individual image frames; Column 4, line 20-22, multi-frame image module 108 … are illustrated here as residing on computer-readable media 204).
With respect to claim 12, arguments analogous to those presented for claim 1, are applicable.
With respect to claim 13, arguments analogous to those presented for claim 2, are applicable.
With respect to claim 14, arguments analogous to those presented for claim 3, are applicable.
With respect to claim 15, arguments analogous to those presented for claim 4, are applicable.
With respect to claim 17, arguments analogous to those presented for claim 6, are applicable.
With respect to claim 18, arguments analogous to those presented for claim 7, are applicable.
With respect to claim 19, arguments analogous to those presented for claim 8, are applicable.
With respect to claim 22, arguments analogous to those presented for claim 1, are applicable.
With respect to claim 23, arguments analogous to those presented for claim 2, are applicable.
With respect to claim 24, arguments analogous to those presented for claim 3, are applicable.
With respect to claim 25, arguments analogous to those presented for claim 4, are applicable.
With respect to claim 27, arguments analogous to those presented for claim 6, are applicable.
With respect to claim 28, arguments analogous to those presented for claim 7, are applicable.
Claim 10, 11, 20, 21, 29 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (U.S Patent 10,021,313) (hereafter, "Chen") in view of Prentice (U.S Publication No. 2012/0257071) and further in view of Gandhi et al. (U.S. Publication No. 2021/0127050) (hereafter, "Gandhi").
Regarding claim 10, the combination of Chen and Prentice teaches all the limitations of claim 9 above. Chen teaches wherein the processing system is further configured to retrieve at least one previously stored image frame from the circular buffer (Column 9, line 38-40, a subset of the multiple frames 116 can include any number of image frames that is less than or equal to a number of image frames in the multiple frames 116; Column 13, line 9-10, Frame stacking module is configured to buffer each of the multiple image frames 116; Column 4, line 20-22, multi-frame image module 108 … illustrated here as residing on computer-readable media 204) … wherein the composite image is generated further based on the at least one previously stored image frame (Column 9, line 36-38 & 45-48, the multi-frame image 118 is often generated from the multiple image frames 116 using only a subset of the multiple image frames ... although m different image frames 116 were captured and buffered, the multi-frame image 118 is generated by combining a subset of the multiple image frames 116 that includes image frame 116-1, 116-2, and 116-m).
Chen does not teach wherein the at least one previously stored image frame is associated with a zero shutter lag (ZSL) mode of operation, and.
However, Gandhi teaches wherein the at least one previously stored image frame is associated with a zero shutter lag (ZSL) mode of operation, and ([0005] The processor is configured to set the image capture device to a zero-shutter-lag (ZSL) mode in response to a capture of a first image; [0070] In the ZSL mode, image sensors of the camera 402 may be configured to receive images (such as preview images); [0071] The received preview images can be saved to a circular buffer 406).
It would have been obvious before the effective filing date of the claimed invention to one having ordinary skill in the art to modify the method and device of Chen to incorporate the step/system of storing received images associated with a zero shutter lag (ZSL) mode taught by Gandhi.
The suggestion/motivation for doing so would have been to improve the accuracy and efficiency of generating a combined image and to reduce power consumption ([0004] Disclosed herein are implementations of reducing power consumption for enhanced zero shutter-lag; [0048] the image capture device 200 may maintain information indicating the location and orientation of the lenses 204, 206 and the image sensors 240, 242 such that the fields-of-view 210, 212, stitch points 270, 272, or both may be accurately determined, which may improve the accuracy, efficiency, or both of generating a combined image). Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predicted results. Therefore, it would have been obvious to combine Chen with Gandhi to obtain the invention as specified in claim 10.
Regarding claim 11, the combination of Chen, Prentice and Gandhi teaches all the limitations of claim 9 above. Gandhi teaches wherein the composite image is generated ([0051] Images or frames captured by an image capture device, such as the image capture device 100 shown in FIGS. 1A-D or the image capture device 200 shown in FIGS. 2A-C, may be combined, merged, or stitched together to produce a combined image, such as a spherical or panoramic image) further based on a live image frame prior to or without the live image frame being stored to the circular buffer ([0095] the non-ZSL mode can be a preview mode; [0096] a video mode (whereby the image capture device captures video) may be a non-ZSL mode; [0104] The processing pipelines of the examples 600 and 650 can be included in or performed (e.g., executed, etc.) by an image capture device; [0108] In the non-ZSL mode (e.g., when the image capture device is configured to perform the operations of the example 650)).
With respect to claim 20, arguments analogous to those presented for claim 9 and 10, are applicable.
With respect to claim 21, arguments analogous to those presented for claim 11, are applicable.
With respect to claim 29, arguments analogous to those presented for claim 9 and 10, are applicable.
With respect to claim 30, arguments analogous to those presented for claim 11, are applicable.
Allowable Subject Matter
Claim 5, 16 and 26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL C. CHANG whose telephone number is (571)270-1277. The examiner can normally be reached Monday-Thursday and Alternate Fridays 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan S. Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL C CHANG/Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669