DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The response received on 9/16/2026 has been placed in the file and was considered by the examiner. An action on the merit follows.
Response to Amendment
The amendments filed on September 16th, 2026 have been fully considered. Response to these amendments is provided below.
Summary of Amendment/ Arguments and Examiner’s Response:
The applicant has amended the claims and has argued that the prior art rejections are overcome.
It is noted that the 101 rejection is overcome, as amended and argued on pages 8-10 of the remarks, because the applicant has amended claim language that provides a practical application, i.e. image enhancement.
On pages 10-12 of the remarks, the applicant repeats the claim, previous rejection and cites Lin. The applicant states Lin does not disclose the newly claimed “selecting” step.
The examiner disagrees, as Lin describes such a step when evaluating selected images 156-b1- 156-b3, and as described in page 5, paragraph 56. Further explanation follows in the rejection below.
On page 13 of the remarks, the applicant states that Lin does not teach the predicting of a distractor score for at least one candidate object…based on pairwise relations.
As explained below, Lin et al and Fried et al combined teach the claimed limitation.
ON pages 13-15, the applicant argues Lin does not teach “selecting’’ objects, and “providing… score” then recites the previous rejection for claim 20. The applicant states nothing in Fried teaches the “applying a neural network to predict a distractor score…based on pairwise relations…”
The examiner disagrees. Fried et al discloses the distractor score of a region (fig. 7, item 706) is based on pairwise relations between regions corresponding to the candidate object/ region being processed, i.e. people, and region coordinates of another bounding region corresponding to another identified object, i.e. main person (page 3, paragraph 32) in the input image, because the input image draws the same features of the training images (page 4, paragraph 44), the “relation” being pairwise relations between region coordinates/ location of the regions, as clarified by claim 15.
On pages 15-16, the applicant argues that the dependent claims are allowable because the independent claim is allowable, and that new claims are added.
As argued above and rejected below, the independent claims are not allowable. New claims are addressed below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7-19 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent Application Publication No. 20220129670 (Lin et al) al in view of U.S. Patent Application Publication No. 20170032551 (Fried et al).
Regarding claim 7, Lin et al discloses a computer-implemented method (fig. 2, 3), comprising: receiving, by a computing device (fig. 1, item 110), an input image (fig. 3, item 114); identifying one or more objects in the input image (fig. 3, item 153); generating a bounding region for each of the one or more objects, wherein the bounding region is associated with region coordinates indicative of a location and a size of the bounding region in the input image, i.e. the bounding boxes shown in fig. 3, item 153 that indicate the location and size; selecting, from the one or more objects, at least one candidate object, i.e. by selecting the rest of the objects as described in page 5, paragraph 56, associated with a detection score higher than threshold detection score, the detection score that determines the are not the main subject 156-a, and are of 156-b, determined to be so by being above a certain threshold (page 4, paragraph 42), applying a neural network (fig. 3, item 158, 159, fig. 5) to predict a distractor score (fig. 3, item 159, fig. 5) for the at least one candidate object of the one or more objects (fig. 3, regions processed as shown in item 159, page 5, paragraph 56), based on region coordinates of bounding regions corresponding to the candidate object, because the region coordinates define the bounding regions (fig. 3, item 158), wherein the distractor score for an object of the one or more objects is indicative of a perceived visual distraction caused by a presence of the object in the input image (page 4, paragraph 42, fig. 3, output of 159, fig. 4, item 132b), the neural network having been trained on training data comprising a plurality of images, i.e. training data images (page 4, paragraph 45), one or more identified objects in each of the plurality of images, i.e. regions marked or annotated (page 4, paragraph 45), and a detection score associated with each of the one or more identified objects, wherein the detection score is indicative of a degree to which a portion of an image corresponds to an object in an image, i.e. any score that is calculated in generating the detection masks of page 4, paragraph 46; providing the predicted distractor score for the at least one candidate object in the input image, the score output in fig. 5 and shown in fig. 3, output of 159; and modifying the input image to remove/ delete a distractor object (page 5, paragraph 57) based on the predicted distractor object exceeding a distractor threshold score (page 5, paragraph 59).
Lin et al does not disclose expressly that the distractor score is based on pairwise relations between regions corresponding to the candidate object and region coordinates of another bounding region corresponding to another identified object in the input image.
Fried et al discloses the distractor score of a region (fig. 7, item 706) is based on pairwise relations between regions corresponding to the candidate object/ region being processed, i.e. people, and region coordinates of another bounding region corresponding to another identified object, i.e. main person (page 3, paragraph 32) in the input image, because the input image draws the same features of the training images (page 4, paragraph 44), the “relation” being pairwise relations between region coordinates/ location of the regions, as clarified by claim 15.
Lin et al and Fried et al are combinable because they are from the same field of endeavor, i.e. categorizing distractor areas
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use relations to determine distractor areas.
The suggestion/motivation for doing so would have been to provide a more robust system by considering context.
Therefore, it would have been obvious to combine the method of Fried et al with the pairwise relations of Fried et al to obtain the invention as specified in claim 7.
Claim 22 is rejected for the same reasons as claim 7. Thus, the arguments analogous to that presented above for claim 7 are equally applicable to claim 22. Claim 22 distinguishes from claim 7 only in that claim 22 is a computing device comprising one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out operations of the method of claim 7. Lin et al teaches further this feature, i.e. a computing device (fig. 6) comprising one or more processors (fig. 6, item 614); and data storage (fig. 6, item 612), wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out operations (page 6, paragraphs 66-67) of the method of claim 7 (fig. 2, 3).
Regarding claim 8, Lin et al discloses the identifying of the one or more objects comprises identifying the one or more objects by an object detection neural network, an object detection network (page 4, paragraph 43).
Regarding claim 9, Lin et al discloses the providing of the predicted distractor score comprises providing the input image with the one or more objects within respective bounding regions (fig. 4, item 132b).
Regarding claim 10, Lin et al discloses identifying, based on the predicted distractor score, an object, of the one or more objects, of high significance in the input image, i.e. main subjects (fig. 4, item 132a, fig. 5, output).
Regarding claim 11, Lin et al discloses all of the claimed elements as set forth above and incorporated herein by reference. Lin et al further discloses the identifying of the object of high significance is based on a determination that a predicted object score for the object is above a threshold score (page 5, paragraph 59). Fried et al discloses using a predicted distractor score and finding it is below a threshold for non-distractor areas by classifying areas that are above a threshold as distractor areas (page 4, paragraph 38).
Regarding claim 12, Fried et al discloses modifying an image editing parameter to prevent removal of the object of high significance from the input image i.e. modifying the image parameters that remove the distractors and leave the non-distractors as shown in fig. 5, where the chair remains in item 502.
Regarding claim 13, Lin et al discloses identifying, based on the predicted distractor score, an object, of the one or more objects, of low significance in the input image, i.e. distractors (fig. 4, item 132a, fig. 5, output).
Regarding claim 14, Lin et al further discloses the identifying of the object of low- significance/ distractors is based on a determination that a predicted object score for the object is below a threshold score (page 6, paragraph 59). Fried et al discloses using a predicted distractor score and finding it is above a threshold for distractor areas (page 4, paragraph 38).
Regarding claim 15, Fried et al discloses providing, via a graphical user interface (Fig. 1), a recommendation to remove the object of low significance from the input image by highlighting the potential distractors (page 4, paragraph 38); receiving a user indication to remove the object of low significance from the input image with a selection by the user (page 4, paragraph 38); and in response to the user indication, removing the object of low significance from the input image (page 4, paragraph 38-39).
Regarding claim 16, Fried et al discloses the removing of the object of low significance comprises generating a segmentation mask of the object of low significance, i.e. the mask on the segments and pixels to transform them only in the location of distractors as described by page 4, paragraph 39 or the distractor map (page 4, paragraph 40).
Regarding claim 17, Fried et al discloses the removing of the object of low significance comprises inpainting a region of the input image corresponding to the object of low significance as removed from the input image, i.e. hole filling (page 4, paragraph 39).
Regarding claim 18, Fried et al discloses removing of the object of low significance comprises compositing the input image to maintain consistency with a background region of the input image by blurring (page 4, paragraph 39) and as seen in fig. 5 by the window region being removed.
Regarding claim 19, Lin et al discloses the predicted distractor score for a given object of the one or more objects is based on a region coordinate of a bounding region corresponding to the given object by considering the global information with the local information as described by page 3, paragraph 41-42.
Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lin et al in view Fried et al, as applied to claim 7 above and further in view of U.S. Patent Application Publication No. 20210110174 (Ohgushi et al).
Regarding claim 21, Lin et al (as modified by Fried et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Lin et al further discloses applying of the neural network to predict the distractor score (fig. 5) is based on training (page 4, paragraph 45) that models a context of an object in the input image because a user marks the image and thus considers the objects context (page 4, paragraph 45).
Lin et al (as modified by Fried et al) does not disclose expressly the training uses conditional random field.
Ohgushi et al discloses the training uses conditional random field (page 2, paragraph 28).
Lin et al (as modified by Fried et al) and Ohgushi et al are combinable because they are from the same field of endeavor, i.e. training attention areas.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use a CRF to train.
The suggestion/motivation for doing so would have been to provide a more robust method by using a known learning technique.
Therefore, it would have been obvious to combine the method of Lin et al (as modified by Fried et al) with training with CRFs of Ohgushi et al to obtain the invention as specified in claim 21.
Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lin et al in view of Fried et al, as applied to claim 7 above, and further in view of U.S. Patent Application Publication NO. 20220138913 (Huang et al).
Regarding claim 23, Lin et al (as modified by Fried et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Lin et al discloses providing the data to remove the distractor object / providing predicted distractor score and modifying the input image to remove the distractor object is performed on an image (fig. 7, item 706, 708).
Lin et al (as modified by Fried et al) does not disclose expressly the input image is a live camera preview feed, and wherein providing the data to remove the distractor object and removing the distractor object are performed on the live camera preview feed prior to capturing a final image.
Huang et al discloses the input image is a live camera preview feed (Page 5, paragraph 52), and wherein providing the data to remove the distractor object and removing the distractor object are performed on the live camera preview feed prior to capturing a final image (page 5, paragraph 52).
Lin et al (as modified by Fried et al) & Huang et al are combinable because they are from the same field of endeavor, i.e. modifying images removing undesired objects.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use a live feed.
The suggestion/motivation for doing so would have been to provide a more robust method by utilizing the removal in video settings.
Therefore, it would have been obvious to combine the method of Lin et al (as modified by Fried et al) with the live modification of Huang et al to obtain the invention as specified in claim 23.
Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lin et al in view of Fried et al, as applied to claim 7 above, and further in view of U.S. Patent Application Publication NO. 20090016449 (Cheung et al).
Regarding claim 24, Lin et al (as modified by Fried et al) discloses all of the claimed elements as set forth above and incorporated herein by reference. Lin et al further discloses the identifying of the object of high significance is based on a determination that a predicted object score for the object is above a threshold score (page 5, paragraph 59). Fried et al discloses using a predicted distractor score and finding it is below a second threshold for non-distractor areas by classifying areas that are above a threshold as distractor areas (page 4, paragraph 38).
Lin et al (as modified by Fried et al) does not disclose expressly providing, via a graphical user interface, a recommendation to lock the object of high significance from being edited; and modifying an image editing parameter of the computing device to lock the object of high significance to prevent removal of the object of high significance from the input image during image editing operations.
Cheung et al discloses providing, via a graphical user interface, a display (page 2, paragraph 21), a recommendation to lock the object of high significance from being edited, by editing the area outside of the object of high significance (fig. 1c, item 130); and modifying an image editing parameter of the computing device to lock the object of high significance to prevent removal of the object of high significance from the input image during image editing operations (fig. 1B, area 155 is not removed after editing in fig. 1C).
Lin et al (as modified by Fried et al) and Cheung et al are combinable because they are from the same field of endeavor, i.e. image editing.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to lock areas of importance for non removal.
The suggestion/motivation for doing so would have been to provide a more robust method by ensuring areas of significance.
Therefore, it would have been obvious to combine the method of Lin et al (as modified by Fried et al) with the locking of the significant areas to obtain the invention as specified in claim 24.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kathleen Yuan Dulaney whose telephone number is (571)272-2902. The examiner can normally be reached M-F: 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at 5712703717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN Y DULANEY/Primary Examiner, Art Unit 2666 10/7/2026
