DETAILED ACTION
I. Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
II. Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
III. Claim Rejections - 35 USC § 103
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
A. Claims 1-4,8-10,15, and 22-25 are rejected under 35 U.S.C. 103 as being unpatentable over the WIPO publication of Geiss et al. (WO 2021/034311 A1) in view of Harris et al. (US 2015/0350533 A1)
As to claim 24, Geiss et al. teaches a computing device (Fig. 5, computing device “500”), comprising:
one or more processors (Fig. 5, one or more processors “503”); and
data storage (Fig. 5, data storage “504”), wherein the data storage has stored thereon computer-executable instructions (Fig. 5, computer-readable instructions “504”) that, when executed by the one or more processors, cause the computing device to carry out functions ([0080]) comprising:
receiving, by the computing device, an object of interest in an image ([0054], lines 1-3);
providing, by a graphical user interface of the computing device (Figs. 2A-2C, interface “200”), a user-adjustable control to adjust a desired local brightness exposure level for the object of interest (Fig. 2A, highlight control feature “204”);
receiving, by the user-adjustable control, a user indication of the desired local brightness exposure level for the object of interest (Fig. 1, step “106”); and
responsive to the user indication, adjusting the desired local brightness exposure level for the object of interest (Fig. 1, step “108”) by:
applying a first local tonemapping to the object of interest ([0061), and
applying a second local tonemapping to a portion of the image outside the object of interest ([0061]; {Tone-mapping associated with a local area/control point will necessarily be different than that associated with an area outside of the control point. That is, the control point will have a set of gains corresponding to a user’s shadow or highlight adjustment, and areas outside of that control point will not correspond to that adjustment.}).
Claim 1 differs from Geiss et al. in that it requires that the user-adjustable control is provided responsive to the receiving of the object of interest. In Geiss et al., it is unclear whether the shadow and highlight slider controls are provided after the user specifies a control point for localized adjustment. However, in the same field of endeavor as the instant application, Harris et al. discloses a camera system (Fig. 1, device “100” with lens “303” and image sensor “306” of Fig. 3) that captures and displays a live-view image ([0020], lines 1-5). A user can then tap on the display to indicate a portion of the image that serves as a basis for exposure/brightness adjustment ([0060], lines 1-5). That is, after the user taps the image portion of the display, a slider bar is displayed next to the image portion that the user operates to specify brightness adjustment (Figs. 2A and 2B, slider “102”; [0060], lines 5-7).
In light of the teaching of Harris et al., the examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to display the shadow and highlight slider controls of Geiss et al. in response to the user’s indication of a control point in the live-view image. One of ordinary skill in the art would recognize that this would effectively communicate to the user which portions of the image are being adjusted. Conversely, if the slider controls are displayed without a connection to a specific control point, the user might unintentionally adjust the brightness of image portions with which he or she is already satisfied.
Claim 1 is a method claim reciting steps substantially similar to the computing device functions of claim 24. Therefore, it is rejected as detailed above.
As to claim 2, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, wherein the object of interest comprises a portrait (see Geiss et al., Figs. 2A-2C, image “201” is a portrait of a man).
As to claim 3, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, wherein the receiving of the object of interest comprises receiving a user selection of the object of interest (see Geiss et al., [0054], lines 1-3).
As to claim 4, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 3, further comprising:
providing a live-view preview of the image prior to a capture of the image (see Geiss et al., Fig. 1, step “102” occurring before step “116”), and
wherein the receiving of the user selection of the object of interest comprises receiving the user selection in the live-view preview (see Geiss et al., [0054], lines 1 and 2, “…specify control points in a live-view preview image.”).
As to claim 8, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, further comprising:
capturing the image subsequent to the adjusting of the desired local brightness exposure level of the object of interest (see Geiss et al., Fig. 1, step “116”).
As to claim 9, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, further comprising:
providing, by the graphical user interface, a second user-adjustable control to adjust a desired global brightness exposure level for the image;
receiving, by the second user-adjustable control, a second user indication of the desired global brightness exposure level; and
responsive to the second user indication, adjusting the desired global brightness exposure level for the image (see Geiss et al., [0038]).
As to claim 10, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, further comprising:
providing, by the graphical user interface, a third user-adjustable control to adjust a desired shadow exposure level for the image;
receiving, by the third user-adjustable control, a third user indication of the desired shadow exposure level; and
responsive to the third user indication, adjusting the desired shadow exposure level for the image (see Geiss et al., Fig. 2A, shadow control feature “206”).
As to claim 15, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, comprising one or more user-adjustable controls to a shadow exposure level (see Geiss et al., [0034], lines 7-9), a local brightness (see Geiss et al., [0034], lines 4 and 5), or a global brightness (see Geiss et al., [0038]), wherein the one or more user-adjustable controls comprise respective slider controls (see Geiss et al., Figs. 2A-2C, slider “202”).
As to claim 22, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, wherein the computing device is a mobile device (see Geiss et al., [0024], lines 2-4).
As to claim 23, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1, wherein the image is a frame of a plurality of frames of a video ([0060], line 1, “…live-view preview image may be a video feed….”).
Geiss et al. discloses an article of manufacture (Fig. 5, computing device “500”) comprising a computer-readable medium (Fig. 5, data storage “504”) storing processor-executable instructions (Fig. 5, computer-readable instructions “504”) for accomplishing the method steps of claim 1 and computing device functions of claim 24 ([0080]). Therefore, in light of this disclosure, the examiner submits that the limitations of claim 25 are satisfied.
B. Claims 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Geiss et al. (WO 2021/034311 A1) in view of Harris et al. (US 2015/0350533 A1) and further in view of Meshkin et al. (US 2023/0146181 A1)
As to claim 5, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1. The claim differs from Geiss et al., as modified by Harris et al., in that it requires that the receiving of the object of interest comprises detecting the object of interest in the image. However, in the same field of endeavor as the instant application, Meshkin et al. discloses a mobile platform that provides a graphical user interface (Fig. 4A), allowing a user to locally adjust the brightness of a face ([0061]). In another embodiment (i.e., one that presumably does not entail user input), a neural network may semantically segment local features in an image, like a face, that may be enhanced ([0051]).
In light of the teaching of Meshkin et al., the examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to automatically detect a salient region in Geiss’s live-view image that serves as localized brightness adjustment area in the manner disclosed by Meshkin et al. An artisan of ordinary skill in the art would recognize that this would automate the user’s selection of a control point whose brightness he or she may likely want to refine, thereby leading to an interface with increased versatility and user-friendliness.
As to claim 6, Geiss et al., as modified by Harris et al. and Meshkin et al., teaches the computer-implemented method of claim 5, wherein the detecting of the object of interest comprises applying a machine learning model to detect the object of interest (see Meshkin et al., [0051], lines 1-3).
As to claim 7, Geiss et al., as modified by Harris et al. and Meshkin et al., teaches the computer-implemented method of claim 6, wherein the machine learning model comprises one or more of an object detection model, a face detection model (see Meshkin et al., [0051], lines 3-8), or a segmentation model.
C. Claims 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Geiss et al. (WO 2021/034311 A1) in view of Harris et al. (US 2015/0350533 A1) in view of Meshkin et al. (US 2023/0146181 A1) and further in view of Imai (US 2012/0050565 A1)
As to claim 16, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1. The claim differs from Geiss et al., as modified by Harris et al., in that it further requires the steps of providing, by the graphical user interface, one or more user-selectable portions of the image, that the one or more user-selectable portions are determined by applying machine learning based image segmentation to generate a segmentation mask, and that the receiving of the object of interest comprises receiving a user selection of a user-selectable portion of the one or more user-selectable portions.
However, in the same field of endeavor as the instant application, Meshkin et al. discloses a mobile platform that provides a graphical user interface (Fig. 4A), allowing a user to locally adjust the brightness of a face ([0061]). In another embodiment (i.e., one that presumably does not entail user input), a neural network may semantically segment local features in an image that may be candidates for enhancement ([0075]). Meshkin et al., however, does not specifically disclose that a user can select one of the network-segmented features for the local face brightness adjustment described in para. [0061]. Further in the same field of endeavor, however, Imai discloses a mobile device providing a graphical user interface (Figs. 4A-4C; [0029]). A plurality of regions of a captured image are segmented and displayed on the interface (Fig. 4B; [0081]). Upon selection of a region (Fig. 5, “S504”; [0096]), the interface allows the user manually adjust brightness of the region using a displayed control object, like a slider ([0099], lines 1-8).
In light of the teaching of Meshkin et al. and Imai et al., the examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to provide Geiss’s interface with selectable regions that may be used as local brightness control points, where the regions are semantically segmented using a neural network as disclosed by Meshkin et al., because an artisan of ordinary skill in the art would recognize that this would highlight specific regions, like a face, for the user whose brightness he or she may likely want to refine, thereby leading to an interface with increased versatility and user-friendliness.
As to claim 17, Geiss et al., as modified by Harris et al., Meshkin et al., and Imai, teaches the computer-implemented method of claim 16, wherein the machine learning based image segmentation comprises one or more of edge detection segmentation, region-based segmentation, semantic segmentation (see Meshkin et al., [0075]), or instance segmentation.
C. Claims 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Geiss et al. (WO 2021/034311 A1) in view of Harris et al. (US 2015/0350533 A1) and further in view of Matsushima et al. (US 2021/0165562 A1)
As to claim 11, Geiss et al., as modified by Harris et al., teaches the computer-implemented method of claim 1. The claim differs from Geiss et al., as modified by Harris et al., in that it further requires the steps of providing, by the graphical user interface, a fourth user-adjustable control to adjust a desired light direction for the image, receiving, by the fourth user-adjustable control, a fourth user indication of the desired light direction, and responsive to the fourth user indication, adjusting the desired light direction for the image.
However, in the same field of endeavor as the instant application, Matsushima et al. discloses a camera (Figs. 1A and 1B, digital camera “100”) that performs a relighting operation on a displayed image (Fig. 3A). A user can select an object in the image to be relit, like a face (Fig. 3A, “S304”), and a virtual light source is displayed (Figs. 6G and 6H, virtual light source “615”). The user can change a position of the virtual light source that results in a change in its lighting direction ([0114]). In light of the teaching of Matsushima et al., the examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to allow a user to adjust localized brightness regions of Geiss’s live-view image by adjusting the position of a virtual light source as disclosed by Matsushima et al. because this would allow for a more realistic brightness adjustment that would blend more naturally with surrounding areas.
As to claim 12, Geiss et al., as modified by Harris et al. and Matsushima et al., teaches the computer-implemented method of claim 11, wherein the fourth user-adjustable control comprises a virtual object configured to be positioned by a user at a desired location of the graphical user interface (see Matsushima et al., Figs. 6G and 6H), wherein the virtual object is representative of a virtual light source providing the desired light direction (see Matsushima et al., [0114]).
IV. Allowable Subject Matter
Claims 13,14, and 18-21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: As to claims 13 and 18, their reasons for allowability can be found in the Office action dated December 28, 2023 (corresponding to claims 10 and 15, respectively) for parent application, 17/704,275. Claims 14 and 19-21 are allowed because they depend on either claim 13 or claim 18.
V. Additional Pertinent Prior Art
Johnson et al. (US # 10,740,959 B2) discloses another example of a technique for providing virtual lighting adjustment for localized regions of an image. See parent application, 17/704,275, for the citation of other pertinent prior art.
VI. Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J DANIELS whose telephone number is (571)272-7362. The examiner can normally be reached M-F 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 571-272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J DANIELS/Primary Examiner, Art Unit 2637
7/23/2026
Prosecution Insights