DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
The information disclosure statement (lDS) submitted on July 3, 2026 is in compliance with the provisions of 37 CFR 1.97 and has been considered by the Examiner.
Terminal Disclaimer
The terminal disclaimer filed on July 3, 2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. Patent No. 12,200,359 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Response to Arguments
Applicant's arguments filed July 3, 2026 have been fully considered but they are not persuasive.
Applicant argues, with respect to claim 1, that “even if Zhou and Lee are combined, the combination does not teach or suggest the limitation of "synchronizing the first and second streams of frames at a frame rate of the image sensor that operates at a full operation rate”. And to support this argument, Applicant states that”
“Lee does not remedy this deficiency. Lee is concerned with synthesizing images obtained from plural photographing units. Lee describes two alternatives: first, determining that plural images are synchronized and then synthesizing them, or using delay processing; and second, when delay processing is difficult, transmitting photographing timing control signals to make photographing timing signals identical. Neither alternative teaches the claimed synchronization scheme.
In the first alternative, Lee at most waits for frames that can be treated as synchronized or delay-processes frames for synthesis. That is not the same as synchronizing operation of two image sensors such that one operates at a full operation rate, another operates at a fractional partial operation rate, and the two streams are synchronized at the frame rate of the full-rate image sensor. In the second alternative, Lee makes the photographing timing signals identical. Making timing signals identical also does not teach synchronizing two streams at the full-rate sensor's frame rate while maintaining another sensor at a fractional partial operation rate.
This distinction is important because the present specification describes a specific synchronization architecture in which the synchronization and operation control module operates at the same frame rate as the high-frame-rate sensor, receives valid frames from the high-rate sensor and lower-rate valid frames from the partial-rate sensor, and may use dummy invalid frames when there is no corresponding lower-rate frame. The module then compares timestamps of valid frame pairs and modifies timing to minimize the time difference between the valid frames. Thus, the claimed synchronization is not merely image synthesis, post-capture delay alignment, or making photographing timing signals identical. It is a control scheme that permits continued partial operation of one image sensor while synchronizing the resulting streams at the full-rate sensor's frame rate.”
However, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., a specific synchronization architecture in which the synchronization and operation control module operates at the same frame rate as the high-frame-rate sensor, receives valid frames from the high-rate sensor and lower-rate valid frames from the partial-rate sensor, and may use dummy invalid frames when there is no corresponding lower-rate frame, and/or comparing timestamps of valid frame pairs and modifies timing to minimize the time difference between the valid frames) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (US Pub. 2017/0359494), in view of Lee et al. (US Patent 9,813,615).
In regard to claim 1, note Zhou discloses a method comprising providing a dual-aperture camera that includes a first camera with a first image sensor operation rate, the first camera operative to output a first stream of frames (paragraphs 0014, 0022, 0024, and figure 1: 110, 114, 140; camera 110 includes image sensor 114 which is coupled to processing system 140, which is operative to output a first stream of frames), and a second camera comprising a second image sensor with a second image sensor operation rate, the second camera operative to output a second stream of frames (paragraphs 0014, 0022, 0024, and figure 1: 120, 124, 140; camera 120 includes image sensor 124 which is coupled to processing system 140, which is operative to output a second stream of frames), synchronizing in time operation of the first camera and of the first image sensor with operation of the second camera and of the second image sensor such that one image sensor operates at a full operation rate and another image sensor operates at a partial operation rate that is a fraction of the full operational rate (paragraph 0014, 0019-0023, 0032, 0036, and figure 1: 130; the capture control system controls which camera is set as the primary, i.e., fully operational frame rate, and which camera is set as the secondary, i.e., partially operational frame rate, wherein either camera can be selected as the primary, and the other is selected as the secondary that operates at a fractional rate).
Therefore, it can be seen that the primary reference fails to explicitly disclose synchronizing the first and second streams of frames at a frame rate of the image sensor that operates at a full operation rate.
In analogous art, Lee discloses a dual aperture imaging system which performs synchronization in time operation of the first camera in a fully operational mode and the second camera in a partially operational mode, and synchronizes the first and second streams of frames at a frame rate of the image sensor that operates at a full operation rate (column 14, lines 6-21, column 15, lines 9-22, column 16, lines 24-32, figure 7: “Camera 1” & “Camera 3”, and figure 9: 140). Lee teaches that the synchronizing the first and second streams of frames at a frame rate of the image sensor that operates at a full operation rate is preferred in order to ensure that the images are captured with identical timing, and thereby prevent the generation of unclear synthesized images (column 11, lines 39-55, column 14, lines 6-21). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the primary reference to include synchronizing the first and second streams of frames at a frame rate of the image sensor that operates at a full operation rate, in order to ensure that the images are captured with identical timing, and thereby prevent the generation of unclear synthesized images, as suggested by Lee.
In regard to claim 2, note Lee discloses that the frame rate of the image sensor that operates at a full operation rate is in a range of 30 frames per second to 60 frames per second (column 13, lines 41-46, and figure 7: “Camera 1”).
In regard to claim 3, note Lee discloses that a frame rate of the image sensor that operates at a partial operation rate is in a range of 5 frames per second to 10 frames per second (column 13, lines 41-46, and figure 7: “Camera 3”).
In regard to claim 4, note Lee discloses outputting a synchronized pair of frames having a frame from the first stream of frames and a frame from the second stream of frames (column 13, lines 41-46, and figure 7: “Camera 1”, “Camera 3”, 620; the synchronized pair of frames are output at 620).
In regard to claim 5, note Zhou discloses that the first image sensor is associated with a first field of view (FOV) and wherein the second image sensor is associated with a second FOV that is narrower than the first FOV (paragraphs 0014, 0019-0021; camera 110 has a smaller FOV than camera 120, therefore, when camera 120 is set as the first, fully operational camera, and camera 110 is set as the second, partially operational camera, the second camera, i.e., camera 110 has a narrower FOV).
In regard to claim 6, note Zhou discloses that the first image sensor is associated with a first field of view (FOV) and wherein the second image sensor is associated with a second FOV that is wider than the first FOV (paragraphs 0014, 0019-0021; camera 110 has a smaller FOV than camera 120, therefore, when camera 110 is set as the first, fully operational camera, and camera 120 is set as the second, partially operational camera, the second camera, i.e., camera 120 has a wider FOV).
In regard to claim 12, note Zhou discloses that the method is performed by a host device (paragraphs 0003, 0013).
In regard to claim 14, note Zhou discloses that the host device is a cellular telephone (paragraphs 0003, 0013).
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (US Pub. 2017/0359494), in view of Lee et al. (US Patent 9,813,615), and further in view of Corephotonics (WO 2015/015383 – hereinafter referred to as Corephotonics ‘383)
In regard to claim 7, note the primary reference of Zhou in view of Lee discloses the system of claim 1, as discussed above. Therefore, it can be seen that the primary reference fails to explicitly disclose that the first image sensor is a color image sensor and wherein the second image sensor is a monochromatic image sensor.
In analogous art, Corephotonics ‘383 discloses the use of a dual aperture imaging system, which includes a first image sensor that is a color image sensor and a second image sensor that is a monochromatic image sensor (page 7, lines 11-18, page 8, lines 23-29, page 10, lines 3-9, and figure 3: “Sensor 1”, “Sensor 2”). Corephotonics ‘383 teaches that a first image sensor that is a color image sensor and a second image sensor that is a monochromatic image sensor is preferred in order to generate a fused image having an increase in SNR (page 7, lines 11-18, page 8, lines 23-29, page 10, lines 3-9). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the primary reference such that a first image sensor is a color image sensor and a second image sensor is a monochromatic image sensor in order to generate a fused image having an increase in SNR, as suggested by Corephotonics ‘383. Additionally, based on the combination of the primary reference and Corephotonics ‘383, since the primary reference of Zhou can set either camera as the first, fully operational camera, and the other camera as the secondary, partially operational camera, the first camera can be set as the image sensor that is a color.
In regard to claim 8, note the primary reference of Zhou in view of Lee discloses the system of claim 1, as discussed above. Therefore, it can be seen that the primary reference fails to explicitly disclose that first image sensor is a monochromatic image sensor and wherein the second image sensor is a color image sensor.
In analogous art, Corephotonics ‘383 discloses the use of a dual aperture imaging system, which includes a first image sensor that is a color image sensor and a second image sensor that is a monochromatic image sensor (page 7, lines 11-18, page 8, lines 23-29, page 10, lines 3-9, and figure 3: “Sensor 1”, “Sensor 2”). Corephotonics ‘383 teaches that a first image sensor that is a color image sensor and a second image sensor that is a monochromatic image sensor is preferred in order to generate a fused image having an increase in SNR (page 7, lines 11-18, page 8, lines 23-29, page 10, lines 3-9). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the primary reference such that a first image sensor is a color image sensor and a second image sensor is a monochromatic image sensor in order to generate a fused image having an increase in SNR, as suggested by Corephotonics ‘383. Additionally, based on the combination of the primary reference and Corephotonics ‘383, since the primary reference of Zhou can set either camera as the first, fully operational camera, and the other camera as the secondary, partially operational camera, the first camera can be set as the image sensor that is a monochromatic.
Claims 9-11, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (US Pub. 2017/0359494), in view of Lee et al. (US Patent 9,813,615), and further in view of Corephotonics (WO 2014/199338 – hereinafter referred to as Corephotonics ‘338).
In regard to claim 9, note the primary reference of Zhou in view of Lee discloses the system of claim 1, as discussed above. Therefore, it can be seen that the primary reference fails to explicitly disclose that the synchronizing in time operation of the first image sensor with operation of the second image sensor includes configuring a respective vertical blanking time of the first image sensor and of the second image sensor.
In analogous art, Corephotonics ‘338 discloses a dual aperture imaging system, which includes synchronizing in time operation of the first image sensor with operation of the second image sensor, which includes configuring a respective vertical blanking time of the first image sensor and of the second image sensor (page 10, lines 4-19). Corephotonics ‘338 teaches that synchronizing in time operation of the first image sensor with operation of the second image sensor, which includes configuring a respective vertical blanking time of the first image sensor and of the second image sensor is preferred in order to ensure that the same points of the object in each view oar obtained simultaneously (page 10, lines 4-19). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the primary reference such that synchronizing in time operation of the first image sensor with operation of the second image sensor, includes configuring a respective vertical blanking time of the first image sensor and of the second image sensor, in order to ensure that the same points of the object in each view oar obtained simultaneously, as suggested by Corephotonics ‘338.
In regard to claim 10, note Corephotonics ‘338 discloses that configuring the respective vertical blanking time includes increasing or decreasing the vertical blanking time of the first image sensor and the second image sensor (page 10, lines 4-19; in order to remove any delay between the vertical blanking of each image sensor, with respect to one another, the blanking time is considered to be increased or decreased so that they are identical to one another).
In regard to claim 11, note the primary reference of Zhou in view of Lee discloses the system of claim 1, as discussed above. Therefore, it can be seen that the primary reference fails to explicitly disclose using a smooth transition library for providing an instruction used for synchronizing in time the operation of the first and second cameras and of the first and second image sensors
In analogous art, Corephotonics ‘338 discloses the use of a dual aperture imaging system, which includes the use of a smooth transition library for providing an instruction used for synchronizing in time the operation of the first and second cameras and of the first and second image sensors (page 13, line 30 – page 14, line 8). Corephotonics ‘338 teaches that the use of smooth transition library for providing an instruction used for synchronizing in time the operation of the first and second cameras and of the first and second image sensors is preferred in order to minimize the jump effect that occurs when switching between cameras (page 13, line 30 – page 14, line 8). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the primary reference to include the use of a smooth transition library for providing an instruction used for synchronizing in time the operation of the first and second cameras and of the first and second image sensors, in order to minimize the jump effect that occurs when switching between cameras, as suggested by Corephotonics ‘338.
In regard to claim 13, note Zhou discloses that the method is performed by a host device (paragraphs 0003, 0013).
In regard to claim 15, note Zhou discloses that the host device is a cellular telephone (paragraphs 0003, 0013).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISS S YODER III whose telephone number is (571)272-7323. The examiner can normally be reached M-F 9:00-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lin Ye can be reached at (571) 272-7372. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.Y./Examiner, Art Unit 2638
/LIN YE/Supervisory Patent Examiner, Art Unit 2638
Prosecution Insights