DETAILED ACTION
Claims 1-16 are pending in the application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 12, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Shen (WO 2022/048460 A1) in view of Geiss et al (US 2015/0163400 A1). *Should be noted Shen (US 2024/0040261 A1) was used for reference/English translation regarding WO 2022/048460 A1 throughout*.
As per claim 1, Shen discloses a method of controlling an electronic device including a camera (fig. 1, flowchart for adjusting camera gain), the method comprising:
obtaining an image frame, which is captured at a current frame rate by using an image sensor included in the camera (fig. 1, flowchart for adjusting camera gain, S102);
determining whether the current frame rate is lower than a reference frame rate (Shen teaches when an adjustment period overruns the current frame brightness adjustment period is an abnormal period (i.e. higher or lower), fig. 5, para 0012 and 0067);
based on determining that the current frame rate is lower than the reference frame rate, determining whether a brightness value of the image frame is lower than a reference brightness value (figs. 5 and 8, flowcharts for gain adjustment(s), an universal adjustment mode is executed to adjust the camera gain when an abnormal period overruns or exceeds a second preset value, para 0009, 0015, and 0072);
Shen fails to teach based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera is in a first state in which the camera is blocked by an obstacle and based on determining that the camera is in the first state, changing an exposure setting of the camera to a first exposure setting. However, Geiss discloses an digital camera device configured to detect occlusion with a camera being detected based on a brightness of a scene detected by the camera, an exposure setting may be determined (i.e. adjusted) (Geiss, fig. 1A, digital camera device 100, cameras 104, 112A, 112B, para 0019, 0022, 0039, 0082).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Shen in view of Geiss, as a whole, by incorporating the ability to detect occlusion and adjust/determine a exposure setting as disclosed by Geiss into the imaging device as taught by Shen, because doing so would provide a more efficient way of correcting brightness within an image, thus enhancing the quality of the image(s) being captured.
As per claim 2, the combined teachings of Shen in view of Geiss, as a whole, further discloses the method of claim 1, further comprising, based on determining that the brightness value of the image frame is higher than or equal to the reference brightness value:
setting the current frame rate to a second frame rate;
obtaining an image frame at the second frame rate; and
recalculating the exposure setting of the camera based on the image frame obtained at the second frame rate (Shen, fig. 5, discloses after an abnormal period is overrun and the universal adjustment mode are executed when in an abnormal period adjustment process, the adjustment period is marked as a stable period, when the next frame is acquired in fig. 3, the method enters in a stable period adjustment process of fig. 6, see para 0092-0098, also see exposure determining/adjustment in claim 1 above).
As per claim 3, the combined teachings of Shen in view of Geiss, as a whole, further discloses the method of claim 1, further comprising, based on determining that the camera is not in the first state:
setting the current frame rate to a second frame rate;
obtaining an image frame at the second frame rate; and
recalculating the exposure setting of the camera based on the image frame obtained at the second frame rate (Shen, para 0015 and 0086, also see rejected claim 2 above).
As per claim 4, the combined teachings of Shen in view of Geiss, as a whole, further discloses the method of claim 1, wherein the determining of whether the brightness value of the image frame is lower than the reference brightness value comprises:
obtaining a first image frame from the image sensor based on determining that the camera operates at a frame rate lower than the reference frame rate; and
determining whether a brightness value of the obtained first image frame is lower than the reference brightness value (Shen, para 0015 and 0086, also see rejected claim 1 above).
As per claim 5, the combined teachings of Shen in view of Geiss, as a whole, further discloses the method of claim 1, wherein the brightness value of the image frame corresponds to an average of brightness values of all pixels in the image frame (Shen, para 0014, grayscale long-term mean).
As per claim 12, the combined teachings of Shen in view of Geiss, as a whole, further discloses the method of claim 1, further comprising increasing the current frame rate of the camera to a preset first frame rate based on determining that the camera is in the first state (Shen, para 0069).
As per claim 15, Shen discloses an electronic device (camera, para 0039) comprising:
a camera (camera, para 0039);
obtain an image frame, which is captured at a current frame rate by using the image sensor;
determine whether the current frame rate is lower than a reference frame rate;
based on determining that the current frame rate is lower than the reference frame rate, determine whether a brightness value of the image frame is lower than a reference brightness value (claim limitations have been discussed and rejected, see claim 1 above);
Shen fails to teach an image sensor, a memory configured to store at least one instruction, and at least one processor configured to execute the at least one instruction to: based on determining that the brightness value of the image frame is lower than the reference brightness value, determine whether the camera is in a first state in which the camera is blocked by an obstacle; and based on determining that the camera is in the first state, change an exposure setting of the camera to a first exposure setting. However, Geiss discloses a digital camera device 100 comprising an image sensor 224, data storage 208, a processor 206, configured to detect occlusion with a camera being detected based on a brightness of a scene detected by the camera, an exposure setting may be determined (i.e. adjusted) (Geiss, figs. 1A and 2, digital camera device 100, cameras 104, 112A, 112B, image sensor 224, data storage 208, processor 206, para 0039, 0053, and 0082).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Shen in view of Geiss, as a whole, by incorporating the digital camera and the ability to detect occlusion and adjust/determine a exposure setting as disclosed by Geiss into the imaging device as taught by Shen, because doing so would provide a more efficient way of correcting brightness within an image, thus enhancing the quality of the image(s) being captured.
As per claim 16, the combined teachings of Shen in view of Geiss, as a whole, further discloses a non-transitory computer-readable recording medium having recorded thereon a program, which, when executed by at least one processor (Shen, claim 14), causes the at least one processor to perform:
obtaining an image frame, which is captured at a current frame rate by using an image sensor included in a camera;
determining whether the current frame rate is lower than a reference frame rate;
based on determining that the current frame rate is lower than the reference frame rate, determining whether a brightness value of the image frame is lower than a reference brightness value;
based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera is in a first state in which the camera is blocked by an obstacle; and
based on determining that the camera is in the first state, changing an exposure setting of the camera to a first exposure setting (claim limitations have been discussed and rejected, see claims 1 and 15 above).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Shen (WO 2022/048460 A1) and Geiss et al (US 2015/0163400 A1), in further view of Decrop et al (US 2022/0303442 A1).
As per claim 9, the method of claim 1, wherein the determining whether the camera is in the first state comprises determining whether the camera is in the first state based on a detection value of a proximity sensor that is positioned adjacent to a lens of the camera.
The combined teachings of Shen in view of Geiss, as a whole, fails to teach the limitations as recited above in claim 9. However, Decrop discloses a primary device 202 (i.e. mobile phone) comprising a camera sensor and a obstacle mitigation component 206 which may be a proximity sensor placed near the camera sensor to detect a physical object or obstacle (i.e. finger) (Decrop, fig. 2, primary device 202, obstacle mitigation component 206, sensor(s) 208, para 0017 and 0023).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Shen and Geiss, in further view of Decrop, as a whole, by incorporating the proximity sensor as taught by Decrop, into the imaging device as taught by Shen and Geiss, because doing so would provide a more efficient way of detect occlusion of the camera, thus enhancing the imaging device as a whole.
Allowable Subject Matter
Claims 6-8, 10, 11, 13, and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, further comprising: each time an image frame is obtained from the image sensor, storing an exposure setting of the camera used in capturing the image frame and a brightness value of the image frame in a buffer of the camera, wherein the changing the exposure setting of the camera to the first exposure setting comprises: determining, based on determining that the camera is in the first state, whether the brightness value of the image frame stored in the buffer is a valid value; changing the exposure setting of the camera to the first exposure setting based on determining that the brightness value of the image frame stored in the buffer is valid; and increasing the current frame rate of the camera to a second frame rate based on determining that the brightness value of the image frame stored in the buffer is invalid.
Regarding claim 7, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, further comprising: each time an image frame is obtained from the image sensor, storing an exposure setting of the camera used in capturing the image frame and a brightness value of the image frame in a buffer of the camera, wherein the changing the exposure setting of the camera to the first exposure setting comprises: determining, as the first exposure setting, an exposure setting for an image frame corresponding to a highest brightness value among exposure settings for a plurality of image frames stored in the buffer.
Regarding claim 8, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, further comprising: each time an image frame is obtained from the image sensor, storing an exposure setting of the camera used in capturing the image frame and a brightness value of the image frame in a buffer of the camera, wherein the changing the exposure setting of the camera to the first exposure setting comprises: calculating an average of brightness values of a plurality of frames stored in the buffer; and determining, as the first exposure setting, an exposure setting for an image frame having a brightness value closest to the average of brightness values.
As per claim 10, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, wherein the determining whether the camera is in the first state comprises: obtaining a depth map from a time of flight (ToF) camera that is positioned adjacent to a lens of the camera; and determining, based on determining that a minimum depth value in the depth map is less than a reference depth value, that the camera is in the first state.
As per claim 11, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, wherein the determining whether the camera is in the first state comprises: obtaining, based on a detection value of a Hall sensor included in the electronic device, state information of a protective case coupled to the electronic device, the state information indicating whether the protective case is in a folded state in which a front side of the electronic device is covered by the protective case or an open state in which the front side of the electronic device is not covered by the protective case; and determining that the camera is in the first state based on the state information indicating that the protective case is in the folded state.
As per claim 13, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, wherein the determining whether the camera is in the first state comprises: obtaining a detection value of an acceleration sensor included in the electronic device, based on determining that the brightness value of the image frame is lower than the reference brightness value; determining whether the electronic device is placed in a fixed position while a first side of the electronic device on which the camera is disposed is in contact with the obstacle; and determining that the camera is in the first state based on determining that the electronic device is placed in the fixed position while the first side of the electronic device on which the camera is disposed is in contact with the obstacle.
As per claim 14, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the method of claim 1, wherein the electronic device is a foldable device, wherein the camera is disposed on an interior surface of the foldable device, the interior surface being positioned inside the foldable device while the foldable device is in a folded state, and wherein the determining whether the camera is in the first state comprises: based on a folding angle of the foldable device being less than a reference angle value, determining that the camera is in the first state.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H MOREHEAD III whose telephone number is (571)270-3845. The examiner can normally be reached M - F 0930-1800 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H MOREHEAD III/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights