DETAILED ACTION
Response to Arguments
Applicant’s arguments with respect to claims 1-9, 11-14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al.(US 20220359597 A1; hereinafter Lee) in view of Galor Gluskin et al. (US 20170230649 A1;hereinafter Galor Gluskin).
As of Claim 1: Lee teaches FIGS. 2&3 an electronic device (¶0044 and note that an image sensor 200 included in a camera module (e.g., a camera module 110 of FIG. 1)) and comprising: an image sensor (¶0044 and note that a 4PD image sensor in FIG. 2) comprising a plurality of unit pixels (¶0045 and note that one 210 of a plurality of pixels may include a microlens 205, a color filter 206, a first PD (or a first sub-pixel) (PD1) 211, a second PD (or a second sub-pixel) (PD2) 212, a third PD (or a third sub-pixel) (PD3) 213, and a fourth PD (or a fourth sub-pixel) (PD4) 214.); and at least one processor electrically connected with the image sensor, wherein a first unit pixel of the plurality of unit pixels comprises: a micro lens (¶0049 and exhibit microlens 205) ; and a first photodiode (PD), a second PD disposed in a first direction of the first PD (¶0053 and note that the two pixels (e.g., the first PD 211 and the second PD 212) separated horizontally ), a third PD disposed in a second direction of the first PD, and a fourth PD disposed in the second direction of the second PD, the first PD, the second PD, the third PD, and the fourth PD being disposed under the micro lens (¶¶0049-0053 and note that one 210 of a plurality of pixels may include a microlens 205, a color filter 206, a first PD (or a first sub-pixel) (PD1) 211, a second PD (or a second sub-pixel) (PD2) 212, a third PD (or a third sub-pixel) (PD3) 213, and a fourth PD (or a fourth sub-pixel) (PD4) 214.), wherein the at least one processor is configured to: acquire a first image frame from the image sensor (¶0057 and note that the image sensor 410 may generate image data for taking a picture or a moving image or image data); in response to the operation mode of the image sensor being determined to a first mode, control the image sensor to output first auto focus (AF) data corresponding to a quantity of light inputted to at least one PD of the first PD or the third PD, and second AF data corresponding to a quantity of light inputted to at least one PD of the second PD or the fourth PD (¶¶0068-0070 and note that the PAF circuit 451 may perform PAF calculation using AF data extracted from pixels (e.g., the first PD and the second PD or the third PD and the fourth PD) separated horizontally or may perform PAF calculation using AF data extracted from pixels (e.g., the first PD and the third PD or the second PD and the fourth PD) separated vertically. Hereinafter, a description will be given of a 2PD PAF circuit), and perform an AF function based on a first phase difference of the first direction which is acquired based on the first AF data and the second AF data; and in response to the operation mode of the image sensor being determined to a second mode (¶¶0097,0112), control the image sensor to output third AF data corresponding to a quantity of light inputted to at least one PD of the first PD or the second PD, and fourth AF data corresponding to a quantity of light inputted to at least one PD of the third PD or the fourth PD, and perform an AF function based on a second phase difference of the second direction which is acquired based on the third AF data and the fourth AF data (¶¶0066-0069 and note that the AF processing unit 450 may perform AF (PAF) using a phase difference based on the received AF data. The AF processing unit 450 may include at least one or more PAF circuits 451).
Galor Gluskin is a similar or analogous system to the claimed invention as evidenced Galor Gluskin teaches capturing an image of a scene, determining that the image of the scene is out of focus, and estimating, via a first AF process of the hybrid AF system, an initial lens position at which the image is in focus in response to determining that the image is out of focus; moving the lens to the initial lens position. The method may also include determining, via a second AF process of the hybrid AF system that would have prompted a predictable variation of Lee by applying Galor Gluskin’s known principal of determine an operation mode of the image sensor to a first mode in case a first contrast value is greater than or equal to a threshold value or to a second mode in case the first contrast value is less than the threshold value (¶¶0043-0044, 0058 and note that the images sensor evaluates confidence metrics associated with PDAF and switches to either PDAF of contrast detection AF. Also, the focus quality), in response to the first contrast value being greater than or equal to the threshold value and the operation mode of the image sensor being determined to a first the first mode; and in response to the first contrast value is less than the threshold value and the operation mode of the image sensor being determined to the second mode (¶¶0043-0044, 0058 and note that the images sensor evaluates confidence metrics associated with PDAF and switches to either Contrast AF detection).
In view of the motivations such as adaptability of the hybrid AF process to changing environmental conditions thereby further improving image quality by improving quality of images and one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Lee.
Therefore, the claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 2-5 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al.(US 20220359597 A1; hereinafter Lee) in view of Galor Gluskin et al. (US 20170230649 A1;hereinafter Galor Gluskin), and further in view of Chino et al. (US 20210067704 A1; hereinafter Chino).
As of Claim 2: Chino is a similar or analogous system to the claimed invention as evidenced Chino teaches determining the tilt/focus driving/driving amount based on the distance information, such as a phase difference, or the method of acquiring the tilt angle, the focal length information, and the object distance information of the image sensor 100 that would have prompted a predictable variation of Lee by applying Chino’s known principal of the first contrast value corresponds to contrast information of a first direction associated with the first image frame (¶¶0022,0024,0034).
In view of the motivations such as focus thereby further improving image quality by improving quality of images and one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Lee.
Therefore, the claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
As of Claim 3: Lee in view of Galor Gluskin in view of Chino further teaches the at least one processor is configured to: identify the first contrast value corresponding to contrast information of the first direction associated with the first image frame (Chino ¶¶0022,0024,0034 and note that for each target object area, and acquires a contrast related evaluation value of a specific frequency); acquire a second contrast value corresponding to contrast information of the second direction associated with the first image frame from the image sensor; and based on the first contrast value and the second contrast value, determine the operation mode of the image sensor (Chino ¶¶0022,0024,0034 and note that FIG. 7 is mainly executed by the in-focus evaluation value calculator 112 that evaluates contrast as well ).
As of Claim 4: Lee in view of Galor Gluskin in view of Chino further teaches the at least one processor is configured to: identify the first contrast value corresponding to contrast information of the first direction associated with the first image frame (Chino ¶¶0022,0024,0034 and note that FIG. 7 is mainly executed by the in-focus evaluation value calculator 112 that evaluates contrast as well ); and determine the operation mode of the image sensor, based on whether the first contrast value is less than the threshold value sensor (Chino ¶¶0022,0024,0034 and FIG. 7).
As of Claim 5: Lee in view of Galor Gluskin in view of Chino further teaches the at least one processor is configured to: while controlling the image sensor to operate in the second mode (Lee ¶¶0097,0112), acquire a second contrast value corresponding to contrast information of the second direction associated with the first image frame; and determine the operation mode of the image sensor, based on the first contrast value and the second contrast value (Chino ¶¶0022,0024,0034).
Claims 6-9 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al.(US 20220359597 A1; hereinafter Lee) in view of Galor Gluskin et al. (US 20170230649 A1;hereinafter Galor Gluskin), and further in view of LI et al.(US 20190082130 A1).
As of Claim 6: LI is a similar or analogous system to the claimed invention as evidenced LI teaches providing phase detection auto-focus (PDAF) information based on edges having more than one orientation (e.g., vertical and horizontal edges), to improve PDAF performance (especially in low light conditions), to reduce or eliminate the need for signal correction, or to increase the resolution of an image sensor that would have prompted a predictable variation of Lee by applying LI’s known principal of the first unit pixel comprises: one floating diffusion (FD) (¶0067 and note that the set of charge transfer transistors 706 to simultaneously transfer charge from multiple photodetectors 702 (e.g., a pair of photodetectors) to the sense region 708 or floating diffusion node) connected with the first PD, the second PD, the third PD, and the fourth PD; and a first transfer gate (TG) configured to connect the first PD and the FD (¶0067 and note that the gates of first and second charge transfer transistors 706a (TX_A) and 706b (TX_B) (i.e., the charge transfer transistors of the first row) may be simultaneously driven to transfer charges accumulated by the first and second photodetectors 702a, 702b to the sense region 708), a second TG configured to connect the second PD and the FD (¶0067 and note that), a third TG configured to connect the third PD and the FD, and a fourth TG configured to connect the fourth PD and the FD (¶0067 and note that), and wherein the image sensor is configured to: when the operation mode of the image sensor is the first mode, switch at least one TG of the first TG or the third TG into an ON state and acquire the first AF data corresponding to at least one PD of the first PD or the third PD (¶0067 and note that), and switch at least one TG of the second TG or the fourth TG to an ON state, and acquire the second AF data corresponding to at least one PD of the second PD or the fourth PD, and when the operation mode of the image sensor is the second mode, switch at least one TG of the first TG or the second TG into an ON state and acquire the third AF data corresponding to at least one PD of the first PD or the second PD, and switch at least one TG of the third TG or the fourth TG into an ON state and acquire the fourth AF data corresponding to at least one PD of the third PD or the fourth PD (¶0067 and note that).
In view of the motivations such as improved phase detection auto-focus (PDAF) performance for a device having a camera or other image capture device thereby further improving image quality by improving focus and one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Lee.
Therefore, the claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
As of Claim 7: Lee in view of Galor Gluskin in view of LI further teaches the image sensor is configured to: acquire first readout data, second readout data, third readout data, and fourth readout data by reading out from the first PD, the second PD, the third PD, and the fourth PD, respectively; when the operation mode of the image sensor is the first mode (Lee ¶¶0169-0176), acquire the first AF data resulting from addition of the first readout data and the third readout data, and the second AF data resulting from addition of the second readout data and the fourth readout data; and when the operation mode of the image sensor is the second mode, acquire the third AF data resulting from addition of the first readout data and the second readout data, and the fourth AF data resulting from addition of the third readout data and the fourth readout data (Lee ¶¶0068 and note that the PAF circuit 451 may perform PAF calculation using AF data extracted from pixels (e.g., the first PD and the second PD or the third PD and the fourth PD) separated horizontally or may perform PAF calculation using AF data extracted from pixels (e.g., the first PD and the third PD or the second PD and the fourth PD) separated vertically. Hereinafter, a description will be given of a 2PD PAF circuit).
As of Claim 8: Lee in view of Galor Gluskin in view of LI further teaches a lens assembly which is electrically connected with the at least one processor and comprises at least one lens aligned along an optical axis (LI ¶0041 and note that an image capture device (e.g., a camera 200), including an image sensor 202, a lens 204, and an auto-focus mechanism 206), wherein the at least one processor is configured to: determine the operation mode of the image sensor to the first mode, based on the information related to at least part of the first image frame; acquire, from the image sensor, the first AF data (Lee ¶0138 and note that the PAF circuit 1235 performs determined calculation irrespective of information about directionality, it may perform PAF calculation depending on directionality of an edge to obtain a PAF result with high reliability. The PAF circuit 1235 may extract a statistical value for each direction and may perform AF determination), the second AF data, and a second image frame related to the first AF data and the second AF data; acquire the first phase difference of the first direction, based on the first AF data and the second AF data; control the lens assembly to move on the optical axis based on the first phase difference; and acquire, from the image sensor (LI ¶0044 and note that the auto-focus mechanism 206 may include (or the functions of the auto-focus mechanism 206 may be provided by) a processor. The auto-focus mechanism 206 may receive signals from the image sensor 202 and, in response to the signals, adjust a focus setting of the camera 200. In some embodiments, the signals may include PDAF information. The PDAF information may include both horizontal phase detection signals and vertical phase detection signals. In response to the PDAF information (e.g., in response to an out-of-focus condition identified from the PDAF information), the auto-focus mechanism 206 may adjust a focus setting of the camera 200 by, for example, adjusting a relationship between the image sensor 202 (or plurality of pixels) and the lens 204 (e.g., by adjusting a physical position of the lens 204 or the image sensor 202), a third image frame corresponding to light passing through the moved lens assembly.
As of Claim 9: Lee in view of Galor Gluskin in view of LI further teaches a lens assembly which is electrically connected with the at least one processor and comprises at least one lens aligned along an optical axis, wherein the at least one processor is configured to: determine the operation mode of the image sensor to the second mode based on the first image frame (LI ¶0087-0090 and note that analyzing horizontal phase detection signals output from a first set of the multiple pixels while capturing the one or more images. The operation(s) at 1104 may be performed, for example, by the auto-focus mechanism described with reference to FIG. 2, the image processor described with reference to FIG. 4, or the processor or image processor described with reference to FIG. 12); acquire, from the image sensor, the third AF data, the fourth AF data, a second image frame related to the third AF data and the fourth AF data, and correlation data between the third AF data and the fourth AF data; acquire the second phase difference of the second direction, based on the third AF data, the fourth AF data, and the correlation data; control the lens assembly to move on the optical axis, based on the second phase difference; and acquire, from the image sensor, a third image frame corresponding to light passing through the moved lens assembly (LI ¶¶0053-0057).
Claims 11-15: Claims 11-15 are method claims for Claims 1,3,4,8,9 and are addressed above.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
As of Claim 10 : the prior art of record fails to teach or fairly suggest the limitations of claim 10, combination with claim 1, that includes, “further comprising an illuminance sensor configured to measure ambient illuminance of the electronic device, wherein the at least one processor is configured to: determine whether the ambient illuminance of the electronic device is greater than or equal to an illuminance threshold value by using the illuminance sensor; in response to the ambient illuminance of the electronic device being greater than or equal to the illuminance threshold value, control the image sensor operating in the first mode to output the first AF data corresponding to a quantity of light inputted to any one PD of the first PD or the third PD, and the second AF data corresponding to a quantity of light inputted to any one PD of the second PD or the fourth PD; and in response to the ambient illuminance of the electronic device being less than the illuminance threshold value, control the image sensor operating in the first mode to output the first AF data corresponding to a quantity of light inputted to the first PD and the third PD, and the second AF data corresponding to a quantity of light inputted to the second PD and the fourth PD.”
Claim 16 is allowed, because the prior/related art fails to explicitly teach or suggest “a first PD, a second PD disposed in a first direction of the first PD, a third PD disposed in a second direction of the first PD, and a fourth PD disposed in the second direction of the second PD; determining an operation mode of the image sensor, based on information related to at least part of the first image frame; in response to the operation mode of the image sensor being determined to a first mode, controlling the image sensor to output first AF data corresponding to a quantity of light inputted to at least one PD of the first PD or the third PD, and second AF data corresponding to a quantity of light inputted to at least one PD of the second PD or the fourth PD, and performing an AF function based on a first phase difference of the first direction which is acquired based on the first AF data and the second AF data; and in response to the operation mode of the image sensor being determined to a second mode, controlling the image sensor to output third AF data corresponding to a quantity of light inputted to at least one PD of the first PD or the second PD, and fourth AF data corresponding to a quantity of light inputted to at least one PD of the third PD or the fourth PD, and performing an AF function based on a second phase difference of the second direction which is acquired based on the third AF data and the fourth AF data.
Regarding claims 17-20: claims 17-20 are allowed based on its dependence to the allowed independent claim 16.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contacts
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEKONNEN D DAGNEW whose telephone number is (571)270-5092. The examiner can normally be reached on 8:00AM-5:00PM M-Th.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lin Ye can be reached on 571-272-7372. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEKONNEN D DAGNEW/Primary Examiner, Art Unit 2638
Prosecution Insights