DETAILED ACTION
Claim(s) 1,4,5,14 and 15,16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1):
Claim(s) 2,3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Timoshenko (Identifying Customer Needs from User-Generated Content):
Claim(s) 6,7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Amthor et al. (DE 10 2021 125 576 A1) with machine translation:
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Kim et al. (Improving Cross-Modal Retrieval with Set of Diverse Embeddings):
Claim(s) 9,10,11,12,13 and 18,19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Piety et al. (US 5,637,781):
Response to Arguments
I. Information Disclosure Statement of February 6, 2024
Applicant’s arguments, see remarks, page 8, filed 01/16/2026, with respect to the IDS of 6 FEB 2024 have been fully considered and are persuasive. The IDS of 6 FEB 2024 is considered by the examiner.
II. Statement of the Interview
Agree: the annotations in the Interview Summary of 9 JAN 2026 are by the examiner.
III. Rejections under 35 USC 101
Applicant’s arguments, see remarks, pages 9,10, filed 1/16/2026, with respect to 35 USC 101 have been fully considered and are persuasive1. The 35 USC 101 rejection of claims 1-20 has been withdrawn.
IV. Rejections under 35 USC 102
Applicant’s arguments, see remarks, pages 10,11, filed 1/16/2026, with respect to 35 USC 102 have been fully considered and are persuasive. The 35 USC 102 rejection of 1-20 in the Office action of 10/16/2025, starting page 11 has been withdrawn.
V. Rejections under 35 USC 103
Applicant’s arguments, see remarks, pages 11,12, filed 1/16/2026, with respect to the rejection(s) of claim(s) 1,4-5,9-16, and 18-20 under 35 USC 103 (in the Office action of 10/16/2025, starting page 26) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of 35 USC 103:
Claim(s) 1,4,5,14 and 15,16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1), wherein Burnap was previously applied (in the Office action of 10/16/2025, starting page 26) and Bala teaches a Graphical User Interface (GUI), fig. 5:180:
PNG
media_image1.png
752
967
media_image1.png
Greyscale
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1,4,5,14 and 15,16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1):
PNG
media_image2.png
727
393
media_image2.png
Greyscale
Re 1. (Currently Amended), Burnap teaches A computer-implemented method, comprising:
receiving, by a computing system, an (“Ratings”, pg. 20, 5th para, 1st S) input [[from]] via a (“SUV” (a vehicle), pg. 20, 1st S) user (fig. 1: “CONSUMER EVALUATION”) interface relating to a (car-design) concept;
automatically obtaining (via an automatic, fed device via Fig. 1: “Augmenting Aesthetic Design with Machine Learning”2), by the computing system, a first (“training”, pg. 15, last para, penult S) set of images from an unlabeled (fig. 2: “UNLABLED IMAGES”) dataset (resulting in “a smaller set”, pg. 3, 2nd para, 2nd S: fig. 3: “DESIGN CONCEPT TESTING” at said smaller training set) of images based on the input;
obtaining, by the computing system, a first rating (of “7000”, pg. 5, 2nd param 2nd S) via the (“SUV” (a vehicle), pg. 20, 1st S) user interface for each image from the first set (resulting in “a smaller set”, pg. 3, 2nd para, 2nd S) of images;
training, by the computing system, a machine-learned (or “trained”, pg. 4, penult para, 1st S: fig. 2: trained: “GENERATIVE MODEL (MACHINE)”: “PREDICTIVE MODEL (MACHINE)”) model (i.e., a “training” “encoder”, pg. 13, 4th para, 4th S: fig. 2: “ENCODER MODEL”: “trained”-“machine learning models”, pg. 4, penult para, 1st S) relating to the (car-design) concept based on the first set (resulting in “a smaller set”, pg. 3, 2nd para, 2nd S) of images rated (fig. 3: “Aesthetic Rating”, pg. 17) [[by]] via the (“SUV” (a vehicle), pg. 20, 1st S) user interface;
automatically obtaining, by the computing system, a second (“validation”, pg. 15, last para, 2nd to last S) set of images from the unlabeled dataset of images based on the machine-learned (or “trained”, pg. 4, penult para, 1st S: fig. 2: trained: “GENERATIVE MODEL (MACHINE)”: “PREDICTIVE MODEL (MACHINE)”) th para, 4th S: fig. 2: “ENCODER MODEL”: “trained”-“machine learning models”, pg. 4, penult para, 1st S) trained based on the first (training) set of images;
obtaining, by the computing system, a second rating (of 7000) via the (“SUV” (a vehicle), pg. 20, 1st S) user interface for each image from the second (validation) set (represented in fig. 2 as validation “IMAGES”) of images; and
retraining (via “retrain the blocks”, pg. 17, last para, 2nd S: fig. 3: blocks), by the computing system, the machine-learned (or “trained”, pg. 4, 1st para, 1st S: fig. 2: trained: “GENERATIVE MODEL (MACHINE)”: “PREDICTIVE MODEL (MACHINE)”) th para, 4th S: fig. 2: “ENCODER MODEL”: “trained”-“machine learning models”, pg. 4, 1st para, 2nd S) relating to the concept based on the first (“training”, pg. 15, last para, penult S) set of images rated [[by]] via the (“SUV” (a vehicle), pg. 20, 1st S) user interface and the second set of images rated [[by]] via the (“SUV” (a vehicle), pg. 20, 1st S) user interface to obtain an updated (“competing”, pg. 18, last para, 1st S) machine-learned (or “trained”, pg. 4, penult para, 1st S: fig. 2: trained: “GENERATIVE MODEL (MACHINE)”: “PREDICTIVE MODEL (MACHINE)”) th para, 4th S: fig. 2: “ENCODER MODEL”: “trained”-“machine learning models”, pg. 4, penult para, 1st S via:
PNG
media_image3.png
1023
1133
media_image3.png
Greyscale
.
Burnap does not teach the difference3 of claim 1 of:
a (user)4 interface…5
the (user) interface…
the (user) interface…
the (user) interface…
the (user) interface…
the (user) interface.
Bala teaches the difference of claim 1 in the context of a person spending a long time to recognize an image (similar to the speed-problem faced by applicants):
[0032] According to various features described herein with regard to FIGS. 2-4, when uploading a given image for personalization, locations or regions within the image are suggested that can be effectively personalized by the design tool, thus reducing the time and iterations spent in the design process (i.e., minimizing or eliminating manual identification of suitable images and/or image regions by a user). In another example, if the user wishes to select a single image for personalization from a large collection, the "suitability for personalization" (SFP) metric can be pre-calculated for all images in that collection, stored, and fed to or retrieved by a file browsing application, which presents to the user the images sorted or ranked by SFP. The user can then quickly select from the top candidates. In a third scenario, the user may upload an image for a general image processing/editing task (i.e. not necessarily for personalization). The personalization analysis and SFP metric are calculated in the background. Only when the metric exceeds a predetermined threshold does the processor make a suggestion that this image is a good candidate for personalization, and offers an option to initiate the personalization process. All of these scenarios minimize wasted time and effort, and offer a productive design experience.
(“The method further comprises presenting the candidate regions to the user via”) a (user)6 interface (“GUI” [0005] last S)…7
the (user) interface (“that can be employed to permit designers to rate a set of images” [0012]) …
(“At 106, the candidate locations are presented to the user via” [0031] 5th S) the (user) interface (fig. 1:106: “PRESENT CANDIDATE LOCATIONS TO USER VIA GUI”)…
(“FIG. 5 illustrates a screenshot of” [0036] 2nd S) the (user) interface…
(“The designers can also provide rationale for their ratings and choices via” [0036] penult S) the (user) interface…
(“a number of images (e.g., 16, 20, etc.) are sequentially displayed on the left side of” [0036] last S) the (user) interface (fig. 5).
Since Burnap teaches a similar concept problem (i.e., a conceptual free-hand design-concept drawing lacking in perceptivity/understanding) to applicant’s, via Brunap page 4:
Aesthetic designers create hundreds to thousands of freehand sketches8 that are converted to 2D images (Box 2). For example, Dyson and General Motors generate several hundred sketches per new device or vehicle, while IKEA generates fewer sketches given its product line variety and turnover (Bouchard, Aoussat, and Duchamp 2006; Toffoletto 2013). The human design team next screens potential designs to a smaller set of testable design concepts in a process known as “downselecting” (Box 3). Consumers evaluate the remaining designs in theme clinics resulting in more screening. Successful designs are advanced downstream for further development, including engineering, manufacturing, and marketing communications (advertising, social media, websites, etc.). The process is highly iterative and asynchronous across both generation and testing; our augmentation applies to all iterations.
one of skill in the art of design can make Burnap’s be as Bala’s seeing in the change “a productive design experience”, Bala [0032] last S:
PNG
media_image4.png
1514
1012
media_image4.png
Greyscale
Re claim 4. (Original), Burnap of the combination of Burnap,Bala teaches The computer-implemented method of claim 1, further comprising providing a rating tool (or a rating-to-page or tool rate pages via “web…pages”910,Burnap, pg. 20, 2nd & 3rd paragraphs: page 31, A4. Example Rating Page from Aesthetic Rating Survey used in Theme Clinic) to obtain the first rating and the second rating.
Re claim 5. (Currently Amended), Burnap of the combination of Burnap,Bala teaches The computer-implemented method of claim 4, wherein the (page) rating tool (or rating-to-page or tool rate pages via “web…pages”1112,Burnap, pg. 20, 2nd & 3rd paragraphs: page 31, A4. Example Rating Page from Aesthetic Rating Survey used in Theme Clinic) comprises (via the above combination, illustrated above) [[a]] the user (file) interface (of the combination illustrated above) which displays (page 31, A4. Example Rating Page from Aesthetic Rating Survey used in Theme Clinic) each image from among the first set of images and the second set of images, and
the user (file) interface (of the combination illustrated above) includes (GUI) user interface elements which are selectable to indicate whether an image is a positive (Bala, fig. 5: “Very Good”) image (via Burnap, page 24: fig. 8: Generated Designs – High Appeal) corresponding to the (car-design) concept or a negative (appeal: Bala, fig. 5: “Very Poor”) image that does not correspond to the (car-design) concept.
Re claim 14. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computer-implemented method of claim 1, wherein training the machine-learned model relating to the (design) concept based on the first (training) set of images rated [[by]] via the user interface comprises:
implementing one or more (three) pretrained models (in Burnap’s fig. 3: “until we reach the target image resolution of 256 x 256”, pg. 17, 5.2 Progressive Training, 1st para, 2nd S) to train a neural network (“augmented through adversarial training”, pg. 8, 2nd S) using image embeddings (“in the embedding space”, pg. 14, 4th para, 2nd S) provided by the one or more (three) pretrained models (via Burnap’s page 17, fig. 3:
PNG
media_image5.png
892
1116
media_image5.png
Greyscale
Claim 15 is rejected like claim 1:
15. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala teaches A computing system, comprising:
one or more processors (comprised by machine learning13); and
one or more non-transitory computer-readable media (comprised by machine learning) that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
receiving an input (via said arrows of fig. 1) [[from]] via a user interface relating to a (car-design) concept;
automatically obtaining (via said automated, fed device) a first (training) set of images from an unlabeled dataset of images based on the input;
obtaining a first (aesthetic) rating via the (web-page) user interface for each image from the first (training) set of images;
training a machine-learned via the user interface;
automatically obtaining a second (validation) set of images from the unlabeled dataset of images based on the machine-learned
obtaining a second rating (via fig. 3: “Predictive Model”: “Aesthetic Rating”: “2.3”) via the user interface for each image from the second (validation) set of images; and
retraining the machine-learned nd S) relating to the concept based on the first set of images rated [[by]] via the user interface and the second set of images rated [[by]] via the user interface to obtain an updated machine-learned
Re claim 16. (Original), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computing system of claim 15, wherein the input comprises a plurality of text phrases (i.e., “product attributes”, pg. 2, penult para, 2nd S), the plurality of text phrases including at least one positive textual description (to create an appealing car image) relating to the concept and at least one negative textual description (to create a not appealing car image) relating to the (car-design) concept.
Claim 20 is rejected like claims 1 and 15:
Re 20. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala teaches One or more non-transitory computer-readable media (comprised by machine learning) that collectively store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
receiving an (arrow) input [[from]] via a user interface relating to a (car-design) concept;
automatically obtaining a first set of images (via said automated, fed device) from an unlabeled dataset of images based on the input;
obtaining a first rating (of 7,000) via the user interface for each image from the first (training) set of images;
training a machine-learned via the user interface;
automatically obtaining a second (validation) set of images from the unlabeled dataset of images based on the machine-learned
obtaining a second rating (of said 7,000) via the user interface for each image from the second (validation) set of images; and
retraining (i.e., updating) the machine-learned via [[by]] the user interface and the second (validation) set of images rated (on said scale of 1 to 5) via [[by]] the (web-)user interface to obtain an machine-learned .
Claim(s) 2,3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Timoshenko (Identifying Customer Needs from User-Generated Content):
PNG
media_image6.png
727
575
media_image6.png
Greyscale
Re 2. (Original), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computer-implemented method of claim 1, wherein the input comprises a plurality of text phrases.
Burnap of the combination (illustrated above) of Burnap,Bala does not teach “a plurality of text phrases”. Timoshenko teaches “a plurality of phrases” or “a small set of sentences”, pg. 7, 4. Methodology, 1st bullet.
Since Burnap of the combination (illustrated above) of Burnap,Bala cites to Timoshenko, via Burnap, page 5, last para:
PNG
media_image7.png
414
1115
media_image7.png
Greyscale
, one of skill in the art of attributes14 can make Burnap’s of the combination (illustrated above) of Burnap,Bala be as Timoshenko’s predictably recognizing the change being a “satisfied” “customer”, Timoshenko, page 3, 2nd para:
PNG
media_image8.png
958
1118
media_image8.png
Greyscale
PNG
media_image9.png
1465
1132
media_image9.png
Greyscale
Re claim 3., Burnap of the combination (illustrated above) of Burnap,Bala, Timoshenko teaches The computer-implemented method of claim 2, wherein the plurality of text phrases (or attributes15 or sentences) includes at least one positive textual description (to “create credible product aesthetic image”, Burnap, pg. 23, 7.2 Generative Capability From embedding to images, 1st S: page 24, fig. 5: “Generated Designs – High Appeal) relating to the (design) concept and at least one negative textual description (or attribute not to “create credible product aesthetic image”: page 24: fig. 5: “Generated Designs – Low Appeal”) relating to the concept.
Claim(s) 6,7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Amthor et al. (DE 10 2021 125 576 A1) with machine translation:
PNG
media_image10.png
727
575
media_image10.png
Greyscale
Re claim 6. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computer-implemented method of claim 1, wherein the machine-learned (encoder) model is a binary classifier model.
Burnap of the combination (illustrated above) of Burnap,Bala does not teach “a binary classifier model”. Amthor teaches “binary classifiers”, abstract. Thus one of skill in the art of classification can make Burnap’s of the combination (illustrated above) of Burnap,Bala be as Amthor’s predictably recognizing the change “improved by implementations according to the invention”, Amthor.
Re claim 7 (Original), Burnap of the combination (illustrated above) of Burnap,Bala,Amthor teaches claim 7 of The computer-implemented method of claim 6, wherein the first rating (of 7,000) is a binary (encoder) rating indicating whether an image from the first (training) set of images is a positive image (to “create credible product aesthetic image”, Burnap, pg. 23, 7.2 Generative Capability From embedding to images, 1st S: page 24, fig. 5: “Generated Designs – High Appeal) corresponding to the (design) concept or a negative image (not to “create credible product aesthetic image”: page 24: fig. 5: “Generated Designs – Low Appeal”) that does not correspond to the (car-design) concept.
Claim 17 is rejected like claims 6 and 7:
Re 17. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Amthor teaches The computing system of claim 15, wherein the machine-learned .
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Kim et al. (Improving Cross-Modal Retrieval with Set of Diverse Embeddings):
PNG
media_image11.png
727
575
media_image11.png
Greyscale
Re 8. (Original), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computer-implemented method of claim 1, wherein obtaining the first set of images from the unlabeled dataset of images based on the input comprises:
co-embedding (pg. 7, fig. 2: “IMAGE EMBEDDING”) the unlabeled dataset of images and the input into a same (“embedding”, pg. 14, 2nd para, last S) space, and
performing a nearest-neighbor (“image…not…too far from the prior”, pg. 9, penult para, 5th S) search to retrieve the first (training) set of images from among the unlabeled (training) dataset of images to obtain (via said automated, fed device) images which are nearest to each text (attribute) embedding.
Burnap of the combination (illustrated above) of Burnap,Bala does not teach “performing a nearest-neighbor search to retrieve…nearest to”.
Kim teaches “performing a nearest-neighbor search to retrieve…nearest to” :
Page 5, 3.4 Training and Inference:
PNG
media_image12.png
199
967
media_image12.png
Greyscale
PNG
media_image13.png
474
1021
media_image13.png
Greyscale
Since Burnap of the combination (illustrated above) of Burnap,Bala teaches text embedding for finding what a customer wants, page 5, last para:
PNG
media_image14.png
421
1116
media_image14.png
Greyscale
, one of skill in the art of embedding can make Burnap’s of the combination (illustrated above) of Burnap,Bala be as Kim’s:
PNG
media_image15.png
894
1126
media_image15.png
Greyscale
predictably recognizing the change accurately searching car products for a customer16, Kim, page 8:
PNG
media_image16.png
853
972
media_image16.png
Greyscale
Claim(s) 9,10,11,12,13 and 18,19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Bala et al. (US 2013/0182946 A1) as applied in claims 1,4,5,14 and 15,16 and 20 above further in view of Piety et al. (US 5,637,781):
PNG
media_image17.png
727
571
media_image17.png
Greyscale
Re claim 9. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala teaches The computer-implemented method of claim 1, further comprising:
automatically obtaining (via said automated, fed device), by the computing system, a third set (via “more screening”, pg. 3, 2nd para, 3rd S: fig. 3: “CONSUMER EVALUATION (HUMAN)” resulting in an even smaller consumer test-set, comprised by a 3rd function17 in page 16, Table 1) of images from the unlabeled (fig. 2: “UNLABLED IMAGES”) dataset (represented in fig. 3 as “GENERATIVE MODEL (MACHINE)”: trained) of images based on the (“competing”, pg. 18, last para, 1st S) updated machine-learned (encoder) st function in page 16, Table 1) of images and the second (“validation”, pg. 15, last para, 2nd to last S) set (comprised by a 2nd function in page 16, Table 1) of images;
obtaining, by the computing system, a third rating (via said 7,000 consumer ratings) [[via]] by a plurality of (“eliminated” pg. 22, 1st S) users via a further (“SUV” (a vehicle), pg. 20, 1st S) user interface nd para, 3rd S: fig. 3: “CONSUMER EVALUATION (HUMAN)” resulting in an even smaller consumer test-set, comprised by a 3rd function18 in page 16, Table 1) of images; and
retraining, by the computing system, the (“often”19, pg. 18, last para, 2nd S) updated machine-learned (encoder) nd S) relating to the (car-design) concept based on the first (“training”, pg. 15, last para, 2nd to last S) set (comprised by a 1st function in page 16, Table 1) of images rated (fig. 2: “images w/ ratings”) [[by]] via the (“SUV” (a vehicle), pg. 20, 1st S) user interface (via the combination (illustrated above) of Burnap,Bala), the second (“validation”, pg. 15, last para, 2nd to last S) set (comprised by a 2nd function in page 16, Table 1) of images rated (fig. 2: “images w/ ratings”) [[by]] via the (remaining- “SUV” (a vehicle), pg. 20, 1st S) user interface (via the combination (illustrated above) of Burnap,Bala), and the third set (via “more screening”, pg. 3, 2nd para, 3rd S: fig. 3: “CONSUMER EVALUATION (HUMAN)” resulting in an even smaller consumer “testing set”, pg. 15, last para, 2nd to last S, comprised by a 3rd function20 in page 16, Table 1 comprised by a 3rd function21 in page 16, Table 1) of images rated (fig. 2: “images w/ ratings”) [[by]] via the further user interface, to obtain a further updated (often) machine-learned
Burnap of the combination (illustrated above) of Burnap,Bala does not teach the difference22 of claim 9 of:
via a further (user) interface…
the further (user) interface.
Piety teaches the difference of claim 9:
via a further (user) interface (via “by23 additional24 user interfaces”, c. 13,ll. 57-61: fig. 4:294,296: mouse & keyboard) …
the further (user) interface (via “by additional user interfaces”, c. 13,ll. 57-61: fig. 4:294,296: mouse & keyboard):
PNG
media_image18.png
773
1138
media_image18.png
Greyscale
Since Burnap of the combination (illustrated above) of Burnap,Bala teaches a user, one of skill in the art of users can make Burnap’s of the combination (illustrated above) of Burnap,Bala be as Pierty’s seeing in the change “ergonomic25 features”, Piety, c. 13, ll. 57-61, designed to be comfortable, safe, and efficient to use, especially in or as a work environment:
PNG
media_image19.png
2287
1138
media_image19.png
Greyscale
Re 10. (Original), Burnap of the combination (illustrated above) of Burnap,Bala, Piety teaches The computer-implemented method of claim 9, wherein a number (or “all” “7,308 rated images”, pg. 20, 4th full para, 1st S) of26 the third set (via “more screening”, pg. 3, 2nd para, 3rd S: fig. 3: “CONSUMER EVALUATION (HUMAN)” resulting in an even smaller consumer test-set, comprised by a 3rd function27 in page 16, Table 1) of images is greater (understood given all 7,308 rated images:100%) than a number (or “50%”, pg. 20, 4th para, last S of said all 7,308 rated images) of the first (training) set of images and greater (understood given all 7,308 rated images:100%) than a number (or “25%”, pg. 20, 4th para, last S of said all 7,308 rated images) of the second (validation) set of images.
Re 11. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Piety teaches The computer-implemented method of claim 10, wherein retraining (via “retrain the blocks”, pg. 17, last para, 2nd S: fig. 3: blocks), by the computing system, the updated28 (“often”29, pg. 18, last para, 2nd S, via fig. 2: “NEW IMAGES”) machine-learned st full para, 2nd S, resulting in “aggregated”30 “Ratings”, Burnap, pg. 20, 4th full para, 1st S) obtained via the user (GUI) interface higher (“to be rated high”, Burnap, page 24, 1st full para, 1st S: fig. 5: “Generated Designs – Low Appeal” via “most unappealing to most appealing”, Burnap, pg. 20, 1st full para, 2nd S) than (“low rated”, Burnap, page 24, 1st full para, 1st S: fig. 5: “Generated Designs – Low Appeal”) ratings obtained via the further (mouse/keyboard) user interface
Re 12. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Piety teaches The computer-implemented method of claim 9, further comprising providing a first rating tool (via “machine learning tools”, Burnap, pg. 2, 3rd para, 1st S) to obtain the first rating and the second rating [[from]] via the (remaining/ non-eliminated) user interface (via the combination (illustrated above) of Burnap,Bala) and providing a second rating tool (or “controllable tool”, Burnap, pg. 2, 4th para, 3rd S) to obtain the third rating (via “rated ten sequential pages”, pg. 20, 2nd full para, 1st S) [[from]] via the further (high quality user) user interface (via the combination (illustrated above) of Burnap,Bala,Piety).
Re 13. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Piety teaches The computer-implemented method of claim 12, wherein
the first (machine learning) rating tool comprises the user (“web…page”31, pg. 20, 2nd and 3rd Ss:GUI) interface (via the combination (illustrated above) of Burnap,Bala) which is configured to display each image from among the first set of images and the second set of images, the
the second (control) rating tool comprises the further user interface (or a 2nd mouse, keyboard web page via the combination (illustrated above) of Burnap,Bala,Piety) which is configured to display each image from among the third set of images, the the further (GUI, mouse, keyboard) user interface (via the combination (illustrated above) of Burnap,Bala,Piety) including information providing an explanation (Burnap: pg. 31: “RATE FROM 1 (VERY UNAPPEALING) TO 5 (VERY APPEALING)”) relating to the (car-design) concept for the plurality of (“eliminated”, Burnap: pg. 22, 1st S) users and user interface elements which are selectable to indicate whether an image from among the third set (via “more screening”, Burnap: pg. 3, 2nd para, 3rd S: fig. 3: “CONSUMER EVALUATION (HUMAN)” resulting in an even smaller consumer test-set, comprised by a 3rd function32 in page 16, Table 1) of images is a positive (appealing) image corresponding to the concept or a negative (appealing) image that does not correspond to the concept.
Claim 18 is rejected like claim 9:
Re 18. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Piety teaches The computing system of claim 15, further comprising:
automatically obtaining, by the computing system, a third set of images from the unlabeled dataset of images based on the updated machine-learned
obtaining, by the computing system, a third rating [[via]] by a plurality of users via a further user interface for each image from the third set of images; and
retraining, by the computing system, the updated classifier model relating to the concept based on the first set of images rated [[by]] via the user interface, the second set of images rated [[by]] via the user interface, and the third set of images rated by the plurality of users via a further user interface , to obtain a further updated machine-learned model.
Claim 19 is rejected like claims 10 and 11:
Re 19. (Currently Amended), Burnap of the combination (illustrated above) of Burnap,Bala,Piety teaches The computing system of claim 18, wherein
a number (said 7,000) of the third set of images is greater than a number (or “50%”, pg. 20, 5th param 2nd S, of said 7,000) of the first set of images and greater than a number (or “25%” of said 7000) of the second set of images, and
retraining (often) the updated (via inputting new data) machine-learned 33 (via a scale) a rating (resulting in “aggregate”34 “Ratings”, pg. 20, 5th para, 1st S) obtained via the (visual-GUI) user interface higher (or very appealing) than (lower-appealing) ratings obtained via 35further (ergonomic mouse, keyboard, visual-GUI: “ ‘clicking through’ ”, Burmap, pg. 24, 1st S) user interface.
Conclusion
The prior art “nearest to the subject matter defined in the claims” (MPEP 707.05) made of record and not relied upon is considered pertinent to applicant's disclosure.
The following table lists several references that are relevant to the subject matter claimed and disclosed in this Application. The references are not relied on by the Examiner, but are provided to assist the Applicant in responding to this Office action.
Citation
Relevance
IDS cited Amershi (Power to the People: The Role of Humans in Interactive Machine Learning)
Amershi teaches, page 112, Novel Interfaces for Interactive Machine Learning, 1st para, last S:
“Therefore, we believe there is an opportunity to explore new36, richer interfaces that can leverage human knowledge and capabilities more efficiently and effectively.”
as the closest to the claimed “further user interface” of claim 9.
IDS cited Schuhmann ( LAION-5B: An open large-scale dataset for training next generation image-text models)
Schuhmann teaches, Abstract:
“Additionally we provide several nearest neighbor indices, an improved37 web-interface for dataset exploration and subset generation, and detection scores for watermark, NSFW, and toxic content detection.”
as the closest to the claimed “further user interface” of claim 9.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENNIS ROSARIO whose telephone number is (571)272-7397. The examiner can normally be reached Monday-Friday, 9AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENNIS ROSARIO/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
1 The claims reflect applicant’s disclosed machine-learning improvement at [0105] as one of skill in the art would recognize (MPEP 2106.04(d)(1) Evaluating Improvements in the Functioning of a Computer, or an Improvement to Any Other Technology or Technical Field in Step 2A Prong Two [R-10.2019], 2nd para, 3rd S: “ In short, first the specification [0105] should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an (machine-learning) improvement.”), wherein machine learning is defined: a branch of artificial intelligence in which a computer (Claim 1: “user interface”) generates (via “training” & “retraining”) rules (model) underlying or based on raw (“unlabeled”) data that has been fed into it (Dictionary.com)
2 machine learning: a branch of artificial intelligence in which a computer generates rules underlying or based on raw data that has been fed into it, where artificial intelligence is defined: Computers, Digital Technology. the capacity of a computer, robot, programmed device, or software application to perform operations and tasks analogous to learning and decision making in humans, such as speech recognition or question answering. AI, A.I., wherein robot is defined: any machine or mechanical device that operates automatically with humanlike skill. (Dictionary.com)
3 THE CLAIMED INVENTION AS A WHOLE: Regarding the claimed “interface”:
The problem faced by applicants is in view of applicant’s disclosure:
[0003] Existing works describe automating the process of large-scale annotation by having users provide a single positive example and asking the crowd to determine whether other images are similar to it. For subjective concepts, particularly those with multiple visual modes, a single image may be insufficient to convey the meaning of the concept to the crowd.
[0005] Personalization is an existing topic in building classification, detection, and image synthesis, however these methods are often devoid of real user interactions and test their resultant models on standard vision datasets.
[0006] Few-shot properties present in vision-language models (e.g., found in CLIP and ALIGN), illustrate it is possible to bootstrap classifiers with language descriptions. Besides functioning as a baseline, good representations have shown to similarly bootstrap active learning. However, few-shot learning has limited capabilities, especially for subjective concepts where a single language description or a single prototype is unlikely to capture the variance in the concept. Therefore, iterative approaches like active learning provide an appropriate formalism to maximize information about the concept while minimizing labels. Active learning methods derive their name by “actively” asking users to annotate data which the model currently finds most uncertain, or believes is most representative of the unlabeled set, or both. Unfortunately, most active learning methods are computationally intensive and take too long for real-world and real-time applications, reducing their utility. Methods to speed up active learning limit the search for informative data points, use low-performing proxy models for data selection, or use heuristics.
The solution to the above (concept-user interaction-speed) problem includes the claimed “user interface”: figs.3A,3B,6 (an indication of non-obviousness?) is:
[0051] Aspects of the disclosure are directed to user-centric approaches for developing real-world classifiers for subjective concepts. According to examples of the disclosure, a computing system may obtain an input from a user (e.g., a domain expert) to define a concept (e.g., a subjective concept) for training a classifier model. In some implementations, the classifier model can be developed by the computing system providing an interactive interface (e.g., a graphical user interface) such that users who are not machine learning experts can easily provide information to define a subjective decision boundary (e.g., by the user indicating positive and negative examples of the concept). According to examples of the disclosure, the computing system may be configured to train the classifier model without the user sifting through and annotating thousands of training instances that are typical for most image classification datasets. For example, ImageNet annotated over 160 million images to arrive at their final 14 million dataset version.
I don’t see in claim 1 applicant’s solution’s [0051]’s “subjective decision boundary (e.g., by the user indicating positive and negative examples of the concept). ” This absence of the solution in claim 1 leans to being obvious.
4 (italics) represent claim limitations already taught
5 ellipses (…) represent claim limitations already taught
6 (italics) represent claim limitations already taught
7 ellipses (…) represent claim limitations already taught
8 sketches: a rough design, plan, or draft, as of a book, wherein rough is defined: crude, unwrought, nonprocessed, or unprepared, wherein crude is defined: lacking in intellectual subtlety, perceptivity, etc.; rudimentary; undeveloped, wherein perceptivity is defined: having or showing keenness of insight, understanding, or intuition. (Dictionary.com)
9 web: Digital Technology., Sometimes Web World Wide Web (preceded by the, except when used before a noun), where World Wide Web is defined: Usually the World Wide Web (except when used before a noun) a system of extensively interlinked hypertext documents: a branch of the internet. WWW, wherein documents is defined: Digital Technology., a computer data file, especially one with formatted text, where file is defined: Computers., a collection of related data or program records stored on some input/output or auxiliary storage medium, wherein record is defined: Computers., a group of related fields, or a single field, treated as a unit and comprising part of a file or data set, for purposes of input, processing, output, or storage by a computer, wherein field is defined: An interface element in a graphical user interface that accepts the input of text. (Dictionary.com)
10 page: a screenful of information from a website, teletext service, etc, displayed on a television monitor or visual display unit, wherein monitor is defined: A device that accepts video signals from a computer and displays information on a screen. Monitors generally employ cathode-ray tubes or flat-panel displays to project the image, wherein device is defined: a machine or tool used for a specific task (Dictionary.com)
11 web: Digital Technology., Sometimes Web World Wide Web (preceded by the, except when used before a noun), where World Wide Web is defined: Usually the World Wide Web (except when used before a noun) a system of extensively interlinked hypertext documents: a branch of the internet. WWW, wherein documents is defined: Digital Technology., a computer data file, especially one with formatted text, where file is defined: Computers., a collection of related data or program records stored on some input/output or auxiliary storage medium, wherein record is defined: Computers., a group of related fields, or a single field, treated as a unit and comprising part of a file or data set, for purposes of input, processing, output, or storage by a computer, wherein field is defined: An interface element in a graphical user interface that accepts the input of text. (Dictionary.com)
12 page: a screenful of information from a website, teletext service, etc, displayed on a television monitor or visual display unit, wherein monitor is defined: A device that accepts video signals from a computer and displays information on a screen. Monitors generally employ cathode-ray tubes or flat-panel displays to project the image, wherein device is defined: a machine or tool used for a specific task (Dictionary.com)
13 machine learning: a branch of artificial intelligence in which a computer generates rules underlying or based on raw data that has been fed into it
14 attribute: Grammar., a word or phrase that is syntactically subordinate to another and serves to limit, identify, particularize, describe, or supplement the meaning of the form with which it is in construction. In the red house, red is an attribute of house. (Dictionary.com)
15 attribute: Grammar., a word or phrase that is syntactically subordinate to another and serves to limit, identify, particularize, describe, or supplement the meaning of the form with which it is in construction. In the red house, red is an attribute of house. (Dictionary.com)
16MPEP 2143 I. F. Known Work in One Field of Endeavor May Prompt Variations of It for Use in Either the Same Field or a Different One Based on Design Incentives or Other Market Forces if the Variations Are Predictable to One of Ordinary Skill in the Art
To reject a claim based on this rationale, Office personnel must resolve the Graham factual inquiries (as shown above). Then, Office personnel must articulate the following:
(1) a finding that the scope and content of the prior art [Burnap et al. (Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach) in view of Kim et al. (Improving Cross-Modal Retrieval with Set of Diverse Embeddings)], whether in the same field of endeavor as that of the applicant’s invention or a different field of endeavor, included a similar or analogous device ((embedding) method, or product);
(2) a finding that there were design incentives or market forces (in Burnap) which would have prompted adaptation of the known device ([embedding] method, or product);
(3) a finding that the differences (as shown above) between the claimed invention (claim 8) and the prior art were encompassed in known variations or in a principle known in the prior art (of Kim’s fig. 2);
(4) a finding that one of ordinary skill in the art, in view of the identified (car-) design incentives or other market forces, could have implemented the claimed variation (Kin’s fig. 2) of the prior art, and the claimed variation would have been predictable (via accurate searching of consumer products) to one of ordinary skill in the art; and
(5) whatever additional findings based on the Graham factual inquiries may be necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
The rationale to support a conclusion that the claimed invention would have been obvious is that design incentives or other market forces could have prompted one of ordinary skill in the art to vary the prior art in a predictable manner to result in the claimed invention. If any of these findings cannot be made, then this rationale cannot be used to support a conclusion that the claim would have been obvious to one of ordinary skill in the art.
17 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
18 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
19 often: many times (Dictionary.com)
20 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
21 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
22 THE CLAIMED INVENTION AS A WHOLE: Regarding the claimed--a further… interface—(applicant’s figure 6):
The multi-faceted (concept-user interaction-speed) problem faced by applicants is discussed in the rejection of claim 1.
The solution to this (concept-user interaction-speed) problem is in applicant’s disclosure:
[0125] FIG. 6 depicts an example graphical user interface screen for crowd raters (e.g., the plurality of users) to label images, according to example embodiments of the disclosure. Different from the graphical user interface of FIG. 3B, in FIG. 6 the graphical user interface 6100 may include additional information to provide the plurality of users with information relating to the concept as understood and defined by the user, to better align the interpretation of the concept of the user and the interpretations of the concept of the plurality of users. For example, the concept may be identified to the plurality of users. In FIG. 6, the plurality of users are given a description 6200 that identifies the concept as an “astronaut” and asks the plurality of users to indicate whether an image 6400 (from the third set of images) is indicative of the concept (e.g., by selecting one of the options 6500). The computing system may also be configured to provide a concept description 6300. In FIG. 6 the concept description 6300 describes the user’s interpretation of the concept for reference by the plurality of users in rating the image 6400. The computing system may also be configured to provide positive examples 6600 and negative examples 6700 relating to the concept. In FIG. 6 the graphical user interface 6100 includes images of positive examples 6600 with corresponding explanations of why the image corresponded to the concept and images of negative examples 6700 with corresponding explanations of why the image did not correspond to the concept.
I don’t see in claim 9 applicant’s solution’s [0125]’s “ Different from the graphical user interface of FIG. 3B, in FIG. 6 the graphical user interface 6100 may include additional information…For example, the concept may be identified to the plurality of users”.
This absence of applicant’s recognition/identification/interpretation solution in claim 9 is an indication of obviousness.
23 by: via; through (Dictionary.com: BRITISH)
24 additional: added; more; supplementary, wherein more is defined: additional or further (Dictionary.com)
25 ergonomic: designed to be comfortable, safe, and efficient to use, especially in or as a work environment. (Dictionary.com)
26 of: (used to indicate inclusion in a number [7,308:100%], class, or whole). (Dictionary.com)
27 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
28 update: Computers. to incorporate new or more accurate information in (a database, program, procedure, etc.). (Dictionary.com)
29 often: many times (Dictionary.com)
30 aggregate: collect into one sum, mass, or body, wherein mass is defined: Physics., the quantity of matter as determined from its weight or from Newton's second law of motion. M, wherein matter is defined: importance or significance. (Dictionary.com)
31 Web page: a single, usually hypertext document on the World Wide Web that can incorporate text, graphics, sounds, etc., wherein document is defined: Digital Technology., a computer data file, especially one with formatted text, where file is defined: Computers., a collection of related data or program records stored on some input/output or auxiliary storage medium, wherein record is defined: Computers., a group of related fields, or a single field, treated as a unit and comprising part of a file or data set, for purposes of input, processing, output, or storage by a computer, wherein field is defined: An interface element in a graphical user interface that accepts the input of text. (Dictionary.com)
32 function: A relationship between two sets that matches each member of the first set with a unique member of the second set. (Dictionary.com)
33 weighting: Statistics., to give a statistical weight to, wherein weight is defined: importance, moment, consequence, or effective influence. (Dictionary.com)
34 aggregate: collect into one sum, mass, or body, wherein mass is defined: Physics., the quantity of matter as determined from its weight or from Newton's second law of motion. M, wherein matter is defined: importance or significance. (Dictionary.com)
35 quality: high grade, wherein high is defined: of great consequence, wherein consequence is defined: importance or significance. (Dictionary.com)
36 new: coming or occurring afresh; further; additional. (Dictioanry.com)
37 improved: to bring into a more desirable or excellent condition, wherein more is defined: additional or further. (Dictionary.com)
Prosecution Insights