DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/24/2026 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sollami et al., U.S. Patent Number 11,250,572 B2, in view of Banerjee et al., U.S. Patent Number 12,154,231 B2.
Regarding claim 1, Sollami discloses a method comprising: accessing, by a user system, a first image that depicts a real-world object (col. 3, lines 1-2, receive a first image that include at least a portion of a person); retrieving, by the user system, a second image that depicts an augmented reality (AR) fashion item (col. 3, lines 25-26, an image of a second fashion item (i.e., a second image that includes a second fashion item); providing as concurrent inputs to a machine learning model both the first image that depicts the real-world object and the second image that depicts the AR fashion item (col. 4, lines 36-40, the first image may be masked to occlude pixels of the first fashion item to be replaced; key points and masking may be convolved using the convolution encoder; col. 4, lines 42-45, portion of the GAN may be the second image that includes the second fashion item, which may be convolved by the convolution encoder; FIG. 3A): analyzing the first image and the second image by the machine learning model to estimate an occlusion map for overlaying the AR fashion item on the real-world object depicted in the first image (col. 4, lines 1-14, server may mask the first image to occlude pixels of the first fashion item that is to be replaced with the second fashion item; determine which portions of the body of the person are not to be covered with the second fashion item; masking of the image; occlusion of the portions of the person; figure 2A, col. 4, lines 7-9, masking of the image 200 so that the shirt 222a, pants 222b, and shoes 222c may be replace with the fashion item); generating a modified image by overlaying the AR fashion item depicted in the second image on the real-world object depicted in the first image (col. 5, lines 6-10, using the masked first image, the second segmentation image, and the second image that includes the second fashion item, the server may generate a third image that includes the person with the second fashion item; generate the third image such that the second fashion item disposed on the person appears real; that is, the fashion item may have the desired positioning, texture, color, orientation, and the like on the person).
However, it is noted that Sollami discloses a generated third image that includes the modified image having an after overlaying the AR fashion item on the real-world object and col. 2, lines 21-25, the pixels of the occluded portions of the person may be located; the pixels of an identified hand may be retained in the input segmentation map (i.e., the hand is not masked); col. 4, lines 1-3, may mask the first item to occlude pixels of the first fashion item that is to be replaced with the second fashion item. However Sollami fails to disclose applying the estimated occlusion map to selectively determine which pixels of the AR fashion item are occluded and which pixels of the real-world object remain visible in the modified image.
Banerjee discloses after overlaying the AR object on the real-world object, applying the estimated occlusion map to selectively determine which pixels of the AR object are occluded and which pixels of the real-world object remain visible in the modified image (col. 4, lines 35-47, after an image has been rendered; the re-projected real and virtual depth may be used to perform occlusion rendering when modifying the previously rendered image; col. 19, lines 36-41, may compare the re-project depth of the virtual object to the re-projected depths of the real-world objects to ensure that the virtual object and/or the real-world objects are correctly occluded with respect to one another; determine which second pixels overlay which first pixels , and for these overlapping pixels, perform a depth comparison to determine how to accurately occlude the virtual object and/or the real-world objects).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the generated third image, i.e. the rendered image having the overlaid fashion item as disclosed by Sollami, using the reprojection and occlusion rendering for modifying the previously rendered image, of the real and virtual object display as disclosed by Banerjee, to ensure that virtual content and/or the real-world scene are correctly occluded with respect to one another, when real-world objects change.
Regarding claim 2, Sollami discloses wherein the second image comprises a three-dimensional (3D) rendering of the AR fashion item overlaid on a portion of the real-world object depicted in the first image (col. 5, lines 18-20, generate a photo-realistic image of the person wearing the transferred fashion item based on the segmentation GAN).
Regarding claim 3, Sollami discloses wherein the AR fashion item comprises at least one of an AR shoe, AR glasses, an AR watch, an AR hat, or AR jewelry (col. 3, lines 2-4, fashion item (e.g., shirt, pants, dress, belt, shoes, jewelry, hat, scarf, garment, and/or fashion accessory)).
Regarding claim 4, Sollami discloses wherein the occlusion map indicates occlusions and visibility associated with the first image (col. 4, lines 16-19, image of the person may be occluded as shown in image 250, where portions of the person 252b and 252c are covered, portion 252d remains visible).
Regarding claim 5, Sollami discloses further comprising: selecting a region of the first image over which to overlay the AR fashion item (col. 54, lines 25-26, first image may be masked to occlude pixels of the first fashion item with the second fashion item); identifying a first pixel of the first image within the region of the first image (col. 29-30, predict pixel-wise semantic segmentation); and replacing, based on the occlusion map, the first pixel with a corresponding pixel of the AR fashion item to occlude the first pixel with the corresponding pixel of the AR fashion item (col. 4, lines 1-3, may mask the first time to occlude pixels of the first fashion item that is to be replaced with the second fashion item; col. 4, lines 16-19, image of the person may be occluded as shown in image 250, where portions of the person 252b and 252c are covered, portion 252d remains visible; col. 4, lines 36-38, first image may be masked to occlude pixels of the first fashion item with the second fashion item).
Regarding claim 6, Sollami discloses further comprising: identifying a second pixel of the first image within the region of the first image (col. 2, lines 23-24, pixels of an identified hand); and preventing replacing, based on the occlusion map, the second pixel with a corresponding pixel of the AR fashion item to prevent occluding the second pixel with the corresponding pixel of the AR fashion item (col. 2, lines 24-25, the hand may be retained; col. 6, lines 53-63, when the parts of the body of the person in the first image overlap or self-occlude, the pixels of the overlapped or self-occluded parts in the first semantic segmentation image may be retained; col. 6, lines 59-62, pixels associated with these portions of the body may be identified; the identified pixels may be retained in the input segmentation map).
Regarding claim 7, Sollami discloses wherein the AR fashion item is selected in response to input that identifies the AR fashion item from a list of AR fashion items (col. 2, lines 2-6, use two images: (1) a picture of the target fashion item, and (2) an image including the customer; generated output image includes the customer wearing the fashion item; col. 3, lines 2-4, person wearing a first fashion item (e.g., shirt, pants, dress, belt, shoes, jewelry, hat, scarf, garment, and/or fashion accessory; col.4 , lines 8-9, FIG. 2A, shows masking of the image so that the shirt 222a, pants 222b and shoes 222c may be replaced with a fashion item).
Regarding claim 8, Sollami discloses wherein the first image comprises a frame of a real-time video captured by a camera of the user system (col. 3, lines 5-8, first image may be received by the server from a camera, a computer and/or mobile device (e.g., computer 550 shown in FIG.4) that may capture; col. 3, lines 30-32, a plurality of images may be provided where the person is in a different pose, or the images are taken from different points of view).
Regarding claim 9, Sollami discloses further comprising: applying one or more machine learning models to the real-time video to generate tracking information of the real-world object depicted in the real-time video; continuously updating the real-time video; and modifying placement of the AR fashion item, adjusted based on the estimated occlusion map, on the depiction of the real-world object (col. 3, lines 7-8, may capture and/or store the first image, which Examiner interprets as real time; col. 3, lines 28-37, in some implementation, more than one image of person may be provided to the server; a plurality of images may be provided where the person is in a different pose, or the images are taken from different points of view; the semantic segmentation of the first image may retain the details of the one or more garments and/or fashion items in the image (e.g., color, cloth, texture and the like)
Regarding claim 10, Sollami discloses wherein the machine learning model comprises an image-to-image artificial neural network (col. 3, lines 23-26, GAN may use an image that includes a person (i.e., the first image including a person wearing a first fashion item) and an image of a fashion item (i.e., a second image that includes a second fashion item)).
Regarding claim 11, Sollami discloses further comprising training the machine learning model by performing training operations comprising: accessing training data comprising a first set of training images that depict one or more real-world objects, a second set of training images that depict training AR objects overlaid on the one or more real-world objects, and corresponding ground-truth occlusion maps (col. 4, line 61 – col. 5, line 5, form image 318 of the person wearing the transferred fashion item; m ay compare the image 318 with the first image and/or second image to determine whether the image is comparatively realistic in order to train the GAN); applying the machine learning model to a first training image of the first set of training images and a second training image of the second set of training images to estimate a training occlusion map (col. 18-30, segmentation GAN of FIG. 3A; occludes pixels of the first fashion item to be replaced with the second fashion item; key points and masking may be convolved by the convolution encoder so as to concatenate the key points and masking); computing a deviation between the training occlusion map and an individual ground-truth occlusion map of the ground-truth occlusion maps corresponding to the first and second training images (col. 7, lines 2-4, may determine a loss between the second semantic segmentation image and the first semantic segmentation image, and determine adversarial loss); and updating one or more parameters of the machine learning model based on the computed deviation (col. 7, lines 5-14, losses between the second semantic segmentation image and the first semantic segmentation image may be determined from the loss (i.e., the least absolute deviation); the determined loss may be used to train the server and/or one or more of the above-described GANs).
Regarding claim 12, Sollami discloses wherein the first and second training images are synthetically generated, further comprising: synthetically generating a depiction of the one or more real-world objects to provide the first training image (col. 3, lines 54-55, the first semantic segmentation which may form an image that appears to be non-realistic with the person and the second fashion item); and synthetically generating a depiction of the training AR objects of the one or more real- world objects to provide the second training image; col. 7, lines 27-37, the server may generate a fourth image of the person that includes the second fashion item by using the first semantic segmentation image, the second masked image, and the second image that includes the second fashion item; determine an error gradient by combining perceptual loss, feature matching loss, and adversarial loss; server may be trained by the determined error gradient).
Regarding claim 13, Sollami discloses further comprising: automatically generating the individual ground-truth occlusion map for the synthetically generated first and second training images (col. 7, lines 21-26, masking at the server, the first image to occlude pixels of the first fashion item to be replaced to form a second masked image; occluding may be performed by the server deleting minimal sub-images whose pixels are to be changed during the transference of the image of the second fashion onto the image of the first fashion item on the person).
Regarding claim 14, Sollami discloses further comprising: accessing the first training image depicting the one or more real-world objects; automatically overlaying the training AR objects on the one or more real-world objects depicted in the first training image to generate the second training image (col. 5, lines 2-5, the discriminator may compare the image with the first image and/or second image to determine whether the image is comparatively realistic in order to train the GAN); and receiving input that specifies portions of the first training image to occlude by a first set of portions of the training AR objects and portions of the first training image to prevent from being occluded by a second set of portions of the training AR objects (col. 5, lines 27-32, key points and masking may be convolved by using the convolution encoder so as to concatenate the key points and the masking; the output may be combined).
Regarding claim 15, Sollami discloses further comprising: generating the individual ground-truth occlusion map for the first and second training images based on the input that specifies the portions of the first training image to occlude by a first set of portions of the training AR objects and the portions of the first training image to prevent from being occluded by a second set of portions of the training AR objects (col. 2, lines 23-24, pixels of an identified hand; col. 2, lines 24-25, the hand may be retained; col. 6, lines 53-63, when the parts of the body of the person in the first image overlap or self-occlude, the pixels of the overlapped or self-occluded parts in the first semantic segmentation image may be retained; col. 6, lines 59-62, pixels associated with these portions of the body may be identified; the identified pixels may be retained in the input segmentation map; col. 5, lines 2-5, the discriminator may compare the image with the first image and/or second image to determine whether the image is comparatively realistic in order to train the GAN).
Regarding claims 16- 18, they are rejected based upon similar rational as above claims 1, 2, and 11, respectively. Sollami further discloses a system comprising: at least one processor configured to perform operations (col 10, lines 61-67).
Regarding claim 19, Sollami discloses estimating the occlusion map, rendering a presentation of the AR fashion item on the real-world object based on a static predefined occlusion map associated with the AR fashion item (col. 2, lines 21-25, the pixels of the occluded portions of the person may be located; the pixels of an identified hand may be retained in the input segmentation map (i.e., the hand is not masked); col. 4, lines 1-3, may mask the first item to occlude pixels of the first fashion item that is to be replaced with the second fashion item).
However, it is noted that Sollami fails to disclose prior to estimating the occlusion map, rendering a presentation of the AR object on the real-world object based on a static predefined occlusion map associated with the AR object; and in response to estimating the occlusion map, applying the estimated occlusion map to subsequent frames of the real-time video to correct visual inconsistencies caused by the static predefined occlusion map
Banerjee discloses prior to estimating the occlusion map, rendering a presentation of the AR object on the real-world object based on a static predefined occlusion map associated with the AR object (col. 16, lines 48-51, semantic segmentation component is a real-time semantic segmenter used to segment static and dynamic object (e.g., referred to herein as real-world objects) in a real-world image to generate a segmentation map); and in response to estimating the occlusion map, applying the estimated occlusion map to subsequent frames of the real-time video to correct visual inconsistencies caused by the static predefined occlusion map (col. 12, lines 58-61, gaps and/or improper overlay between real and virtual objects may result in an image displayed to the user due to inaccurate depth estimates and contours of real-world objects when performing occlusion; col. 13, lines 59-54, late stage occlusion rendering techniques described herein may be sued to re-project depth information for real-world objects and virtual objects in a rendered image due to changed content; see also figures 3A, 3B, 4, and 5A).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to include in the occlusion map as disclosed by Sollami, prior to the estimate rendering a presentation based on a static occlusion as disclosed by Banerjee and applying the estimated to subsequent frames to correct visual inconsistencies caused by the stat as disclosed by Banerjee, to account for estimates of performing occlusion of virtual object with real-world objects that change.
Regarding claim 20, it is rejected based upon similar rational as above claim 1. Sollami further discloses a non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a user system, cause the user system to perform operations (col. 10, lines 41-45).
Response to Arguments
Applicant’s arguments, see page 8, filed 03/23/2026, with respect to the rejection(s) of claim(s) 1-20 under 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of 103, Sollami in view of Banerjee.
Applicant argues Sollami fails to disclose “after overlaying the AR object on the real-world object, applying the estimated occlusion map to selectively determine which pixels of the AR object are occluded and which pixels of the real-world object remain visible in the modified image”.
Examiner responds newly cited prior art Banerjee discloses col. 4, lines 35-47, after an image has been rendered; the re-projected real and virtual depth may be used to perform occlusion rendering when modifying the previously rendered image; col. 19, lines 36-41, may compare the re-project depth of the virtual object to the re-projected depths of the real-world objects to ensure that the virtual object and/or the real-world objects are correctly occluded with respect to one another; determine which second pixels overlay which first pixels, and for these overlapping pixels, perform a depth comparison to determine how to accurately occlude the virtual object and/or the real-world objects.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Motilewa Good-Johnson whose telephone number is (571)272-7658. The examiner can normally be reached Monday - Friday 6am-2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOTILEWA . GOOD JOHNSON
Primary Examiner
Art Unit 2616
/MOTILEWA GOOD-JOHNSON/Primary Examiner, Art Unit 2619
Prosecution Insights