DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Oath/Declaration
2. The receipt of Oath/Declaration is acknowledged.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 09/01/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
4. The drawing(s) filed on 05/02/2023 are accepted by the Examiner.
Specification
5. The disclosure is objected to because of the following informalities:
6. The title of the invention has a typographical error. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested:
ARTIFICIAL INTELLIGENCE POWERED STYLING AGENT.
Appropriate correction is required.
Status of Claims
7. Claims 1-20 are pending in this application.
Claim Rejections - 35 USC § 102
8. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
10. Claims 1, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dahl et al. (US 2020/0372560), hereinafter ‘Dahl’.
Regarding Claim 1:
Dahl discloses a system (Dahl: Fig. 1 ‘product recommendation system 100’ ¶[0033-0034]) comprising:
one or more processors (Dahl: Fig. 9 ‘processor(s) 914’); and
one or more non-transitory computer-readable media storing computing instructions (Dahl: Fig. 9 e.g., ‘memory 912’ storing ‘instructions 924), that when executed on the one or more processors, cause the one or more processors to perform:
receiving stock images comprising an anchor garment (Dahl: Fig. 2 “product recommendation system 204 can receive a product image set 202” (wherein ‘product image set’ reasonable corresponds to “stock images”) ¶[0048]);
automatically identifying the anchor garment and complementary garments within the stock images (Dahl: ‘analyzes images using object detection to identify the primary product and secondary products (wherein ‘primary product’ and ‘secondary products’ reasonably corresponds to “anchor garment” and “complementary garments”, respectively); (Fig.7, step 704 ¶[0018]));
selecting an image of the stock images in which a mask area of a first complementary garment of the complementary garments as a ratio of an area of the anchor garment is largest over other complementary garments of the complementary garments (Dahl isolates detected product regions via object detection and generates per-object feature vectors; object detection inherently identifies region size (pixel area/bounding box), enabling dominance-based selection (¶[0018]). Dahl further discloses object detection models such as YOLO, RCNN and Multibox (¶[0051]) that expressly use ratios and various scales between objects which further support selecting of regions based on size);
performing an image search, using the image, in an item catalog (Dahl: Fig. 3 ‘product image catalog 302’) for similar garments to the first complementary garment (Dahl generates descriptive feature vectors and queries a search index to identify similar products (¶¶[0019-0020]); Fig. 8, steps 804-806); and
displaying, on a user interface (Dahl: Fig. 1 ‘user devices 102a – 102n’ such as a PC or mobile device), an avatar wearing the anchor garment and at least one of the similar garments (Dahl displays coordinated garments worn on a model with associated style panels (Figs. 4-6; ¶¶[0020-0021]). “For example, if the primary product is a t-shirt, the product image set may contain at least one isolated product image of the t-shirt as well as multiple images of models wearing the t-shirt along with other secondary products such as jeans or pants.” ¶[0018]).
Regarding Claim 11: (drawn to a method)
The proposed rejection of system claim 1, over Dahl et al. is similarly cited to reject the steps of the method of claim 11 because these steps occur in the operation of the system as discussed above. Thus, the arguments similar to that presented above for claim 1 are equally applicable to claim 11.
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
14. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
15. Claims 2-4, and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Dahl et al. (US 2020/0372560) in view of Ahmadi et al. (US 2021/0272295), hereinafter ‘Dahl’ and ‘Ahmadi’.
Regarding Claim 2:
Dahl further disclose the system of claim 1, wherein automatically identifying the anchor garment and the complementary garments comprises:
using a (Dahl ¶[0044]) to identify the anchor garment and the complementary garments within the stock images. (Dahl: Fig. 7 flowchart Step 704 ‘analyzing each image from the plurality of images using object detection to identify the primary product and each secondary product from the plurality of secondary products’ (¶0070]).
Dahl discloses wherein the object detection can use “any suitable object detection method” and names some models, including Mask RCNN ¶[0051]. Mask RCNN is a well-known segmentation model, however Dahl does not explicitly recite using as segmentation model.
Ahmadi discloses a segmentation model for object detection and identification (Ahmadi discloses neural-network based instance segmentation producing masks (¶¶[0093-0095]); Fig. 2, Fig. 5).
Dahl in view of Ahmadi are combinable because they are from the same field of endeavor of image processing; e.g., both disclose segmenting and identifying regions of interest in an image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose a segmentation model.
The suggestion/motivation for doing so is to improve the performance of object detection by using “instance segmentation” to further extract features from each region/mask as disclosed by Ahmadi. These objects can then be classified by the features that are obtained.
Therefore, it would have been obvious to combine Dahl with Ahmadi to obtain the invention as specified in claim 2.
Regarding Claim 3:
The proposed combination of Dahl in view of Ahmadi further discloses the system of claim 2, wherein automatically identifying the anchor garment and the complementary garments comprises:
identifying the anchor garment based on which garment is most commonly found in the stock images (Dahl: “A “primary product” of a product image set comprises the product that is the focus of the product image set, and as such, typically appears in each image of the product image set.” ¶[0018]).
Regarding Claim 4:
The proposed combination of Dahl in view of Ahmadi further discloses the system of claim 1, wherein selecting the image comprises: filtering out the stock images in which the complementary garments are partially cropped out (Ahmadi teaches rejecting low-quality/partial object instances, i.e., low quality masks (Fig. 3 flowchart step 320)).
Dahl in view of Ahmadi are combinable because they are from the same field of endeavor of image processing; e.g., both identify and segment regions of interest in an image using masking and feature vectors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein selecting the image comprises: filtering out the stock images in which the complementary garments are partially cropped out. The suggestion/motivation for doing so is to “reject objects whose confidence score is below a predetermined threshold, object-instances whose mask confidence score is below a predetermined threshold are removed; and object instances whose mask area is below a predetermined threshold are removed.” as disclosed by Ahmadi. Therefore, it would have been obvious to combine Dahl with Ahmadi to obtain the invention as specified in claim 4.
Regarding Claims 12-14: (drawn to a method)
The proposed rejection of system claims 2-4, over Dahl in view of Ahmadi is similarly cited to reject the method of claims 12-14 because these steps occur in the operation of the system as discussed above. Thus, the arguments similar to that presented above for claims 2-4 are equally applicable to claims 12-14.
16. Claims 5-10 and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Dahl in view of Ahmadi as applied to claims 1 and 11 above, and further in view of Adeyoola et al. (US 2016/0180419), hereinafter ‘Adeyoola’.
Regarding Claim 5:
Dahl in view of Ahmadi further disclose the system of claim 1, wherein performing the image search further comprises:
pre-training a visual search model (Dahl requires a trained feature extractor/visual embedding model in order to index a large catalog of garment images, to generate stable descriptive feature vectors (¶¶[0007; 0019; 0096]); performing (Dahl performs vector-space search over its indexed feature vectors which corresponds to k-nearest-neighbor retrieval in an embedding space. (Fig. 8, step 806)), with (Dahl discloses “the categories of the products are determined using a trained classifier and indexed with the corresponding product in the search index” (¶¶[0003; 0019]). Dahl uses the garment metadata (e.g., shirt, pants, dress) when determining which vectors are valid neighbors (complements vs. mismatches) which reasonably corresponds to metadata-conditioned negative mining).
Dahl does not expressly disclose performing deep clustering and hard negative mining.
Ahmadi discloses performing deep clustering (Ahmadi: ‘clustering the potential object-instances based on the feature vectors and splitting clusters’; ‘k-means, elbow method (Fig. 3 step 330; ¶¶[0100-0102]) and hard data mining; (Dahl discloses rejecting low-quality/non-matching instances (¶¶[0022-0027], [0101]).
Dahl in view of Ahmadi are combinable because they are from the same field of endeavor of image processing; e.g., both segment and identify regions of interest in an image using masking and feature vectors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose performing deep clustering.
The suggestion/motivation for doing so is to “split apart clusters that are not sufficiently coherent”. Ahmadi further discloses that by rejecting object-instances whose mask area falls below a predetermined threshold are removed from the cluster, thereby improving the segmenting of objects in an image. Therefore, it would have been obvious to combine Dahl with Ahmadi to obtain the invention as specified.
Dahl in view of Ahmadi do not disclose performing active learning.
Adeyoola discloses performing active learning (Adeyoola: ‘here is also provided a method of generating a virtual body model, in which a user takes, or has taken for them, an image of their body which is then processed by a computer system to generate and display a virtual body model using, at least, the body image and the user is presented with controls that enable the shape and/or measurements of the virtual body model to be altered to more accurately match the user's real or perceived shape and/or measurements, in which the user is presented with an on-screen field or control that enables the user to provide feedback about the accuracy of the virtual body model.’ [0256])
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose performing active learning. The suggestion/motivation for doing so is to enable the user to interact with the virtual body models as disclosed by Adeyoola in the Background of Invention. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 5.
Regarding Claim 6:
The proposed combination of Dahl, Ahmadi & Adeyoola further disclose the system of claim 5, wherein pre-training the visual search model further comprises:
augmenting batch images for training the visual search model with positive examples or negative examples (Ahmadi: Fig. 7 flowchart for generating a training dataset. Step 730 ‘generate synthetic images’ (¶¶[0038-0044; 0047; 0136]). Also note that Ahmadi further discloses using objects from OpenImages datasets which also has positive and negative examples and using a random trajectory for each object, which further support using positive and negative examples to train the model).
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein pre-training the visual search model further comprises: augmenting batch images for training the visual search model with positive examples or negative examples.
The suggestion/motivation for doing so is to develop a better and more reliable model helping prevent misclassifying. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 6.
Regarding Claim 7:
The proposed combination of Dahl, Ahmadi & Adeyoola further disclose the system of claim 6, wherein augmenting the batch images comprises:
generating new images to be the positive examples, based on the stock images that comprise the first complementary garment, by at least one of:
changing hues of the first complementary garment (Dahl: Fig. 6 ‘user interface 600’ containing selectable ‘color panel 606’):
changing an angle of or skewing the first complementary garment (Ahmadi: ‘modifying appearance, orientation, scale’ (¶¶[0040, 0044, 0136]);
changing a size of the first complementary garment (Adeyoola: e.g., Fig. 19 ‘changing bra, hip and waist size’ on user interface);
adding holes in the stock images of the first complementary garment (Ahmadi: ‘preserving a region of the garment fabric in occluded regions’ ¶[0753-0754]); or
changing an avatar model wearing the first complementary garment using a virtual try on (VTO) model (Adeyoola teaches rendering garments on different virtual body models ¶[0036-0040] Figs. 6-7).
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose generating new images to be the positive examples, based on the stock images that comprise the first complementary garment, by at least one of:
changing hues of the first complementary garment:
changing an angle of or skewing the first complementary garment;
changing a size of the first complementary garment;
adding holes in the stock images of the first complementary garment; or
changing an avatar model wearing the first complementary garment using a virtual try on (VTO) model.
The suggestion/motivation for doing is so the user can see what they look like with different clothes on so that they can better see if they like it. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 7.
Regarding Claim 8:
The proposed combination of Dahl, Ahmadi & Adeyoola further discloses the system of claim 6, wherein augmenting the batch images further comprises:
automatically selecting the negative examples from images of other garments in the item catalog (Ahmadi teaches the rejection of non-matching instances as negatives’ ¶[0023-0027, 101]).
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein augmenting the batch images further comprises: automatically selecting the negative examples from images of other garments in the item catalog.
The suggestion/motivation for doing so is to remove items of clothing that a user would not desire or fit and not present these items to them. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 8.
Regarding Claim 9:
The proposed combination of Dahl, Ahmadi & Adeyoola further discloses the system of claim 6, wherein augmenting the batch images further comprises:
generating new images to be the negative examples, based on the stock images that comprise the first complementary garment, by changing a color of the first complementary garment (Ahmadi teaches the rejection of non-matching instances as negatives ¶[0023-0027, 0101]), and synthetic modification of object appearance’ ¶¶[0044])
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein augmenting the batch images further comprises: generating new images to be the negative examples, based on the stock images that comprise the first complementary garment, by changing a color of the first complementary garment.
The suggestion/motivation for doing so is to remove items of clothing that a user would not desire or fit and not present these items to them. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 9.
Regarding Claim 10:
The proposed combination of Dahl, Ahmadi & Adeyoola further discloses the system of claim 5, wherein performing the active learning comprises:
submitting style proposals to individuals for feedback, wherein the style proposals each comprise the anchor garment and at least one of the similar garments as a group (Adeyoola teaches using voter/preference feedback from the customer (¶¶[0026; 0030] and Figs. 8-10);
receiving feedback from the individuals; and using the style proposals that are rejected as negative examples in a feedback loop (Ahmadi teaches training with positive and negative samples as described above (¶¶[0019-0122]).
Dahl, Ahmadi & Adeyoola are combinable because they are from the same field of endeavor of image processing; e.g., all disclose segmenting objects from images, such as foreground and background objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein wherein performing the active learning comprises: submitting style proposals to individuals for feedback, wherein the style proposals each comprise the anchor garment and at least one of the similar garments as a group; receiving feedback from the individuals; and using the style proposals that are rejected as negative examples in a feedback loop.
The suggestion/motivation for doing so is to remove items of clothing that a user would not desire or fit by presenting items to them and let them decide how they feel. Therefore, it would have been obvious to combine Dahl, Ahmadi & Adeyoola to obtain the invention as specified in claim 10.
Regarding Claims 15-20: (drawn to a method)
The proposed rejection of system claims 5-10, over Dahl, Ahmadi and Adeyoola is similarly cited to reject the method of claims 15-20 because these steps occur in the operation of the system as discussed above. Thus, the arguments similar to that presented above for claims 5-10 are equally applicable to claims 15-20.
Conclusion
17. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Siddique et al. (US 10,872,322) relate to online methods of collaboration in community environments. The methods and systems are related to an online apparel modeling system that allows users to have three-dimensional models of their physical profile created. Users may purchase various goods and/or services and collaborate with other users in the online environment.
18. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL R MCLEAN whose telephone number is (571)270-1679. The examiner can normally be reached Monday-Thursday, 6AM - 4PM, PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NEIL R MCLEAN/Primary Examiner, Art Unit 2681