Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Applicant’s submission filed on 05/26/2026 has been entered. The status of claims is as follows:
Claims 1-20 remain pending in the application.
Claims 1, 4, 7-8, 11, 14-15 and 18 are amended.
Response to Arguments
In reference to the Claim Rejections under 35 U.S.C 101:
Applicant asserts in Remarks pg. 1-5 that the claims are not directed to a mental process because the recited machine-learning operations, including generating and encoding intent embeddings from visual design context into a shared embedding space, cannot practically be performed in the human mind or with pencil and paper. Even if the claims were found to recite an abstract idea, Applicant contends that the additional ML-based embedding generation integrates any alleged exception into a practical application by improving image-search technology, reducing the search space and computational resources, and improving retrieval accuracy through a less noisy dataset. Applicant further relies on the USPTO 2019 PEG, the October 2019 Update, and Ex parte Desjardins to argue that the claims recite a technological improvement to computer functionality rather than merely implementing an abstract idea on a generic computer.
Applicant’s arguments are not persuasive. As amended, claim 1 remains directed to generating an intent embedding from a design context, comparing embeddings in a shared embedding space, and identifying recommended background images based on similarity, which recites collecting, analyzing, and using information to produce a recommendation. Although the claim recites an embedding generator including a machine learning model, the claim does not recite any improvement to the operation of the machine learning model, the embedding generation technique, or the functioning of the computer itself. Rather, the machine learning model is used as a tool to perform the abstract process of recommending background images. Accordingly, the additional elements merely implement the judicial exception on generic computer technology and do not integrate the exception into a practical application or demonstrate an improvement to computer functionality.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under U.S.C 101 for containing an abstract idea without significantly more.
Regarding claim 1:
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
determining one or more candidate background embeddings based on a similarity between the intent embedding and a plurality of candidate background embeddings in embedding space - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
identifying one or more recommended background images based on one or more background classes corresponding to the one or more candidate background embeddings - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
obtaining a design context, the design context including one
or more visual elements on a digital canvas; – This limitation is directed to insignificant extra-solution activity (see MPEP 2106.05(g)).
generating, [by an embedding generator], the embedding generator including a machine learning model trained to generate an intent embedding from the design context based on the one or more visual elements; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
by an embedding generator – This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
wherein the embedding generator encodes the intent embedding and a plurality of background embeddings corresponding to background classes into a shared embedding space; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
obtaining a design context, the design context including one
or more visual elements on a digital canvas; – This limitation is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.).
generating, [by an embedding generator], the embedding generator including a machine learning model trained to generate an intent embedding from the design context based on the one or more visual elements; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
by an embedding generator – This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
wherein the embedding generator encodes the intent embedding and a plurality of background embeddings corresponding to background classes into a shared embedding space; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 2,
Claim 2 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
determining an intent from the design context This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 3,
Claim 3 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 2 which includes an abstract idea (see rejection for claim 2). The additional limitations:
generating the intent embedding from the intent. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 4,
Claim 4 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein determining one or more candidate background embeddings based on a similarity between the intent embedding and a plurality of candidate background embeddings in embedding space, further comprises: This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
calculating a distance metric between the intent embedding and the plurality of candidate background embeddings in the shared embedding space; and This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
selecting the one or more candidate background embeddings based on the distance metric. This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 5,
Claim 5 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein identifying one or more recommended background images based on one or more background classes corresponding to the one or more candidate background embeddings, further comprises: This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
searching an image library using the one or more background classes. This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 6,
Claim 6 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the embedding generator is a transformer network and – This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
wherein the embedding generator is trained using a triplet loss on a training dataset comprising a plurality of sets of background class and query pairs. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 7,
Claim 7 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
presenting, [via a user interface], the one or more recommended background images by adding it to the design context. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
via a user interface This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
Independent claims 8 and 15 are analogous claims to independent claim 1, therefore the same rejection and rationale apply to them.
In addition, claims 8 and 15 recite additional elements analyzed under Step 2A- Prong Two and Step 2B:
Claim 8:
A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
Claim 15:
a memory component; and This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
a processing device coupled to the memory component, the processing device to perform operations comprising: This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
Dependent Claims 9 and 16, as described above, are analogous claims to claim 2, therefore the same rejection and rationale apply to them.
Dependent Claims 10 and 17, as described above, are analogous claims to claim 3, therefore the same rejection and rationale apply to them.
Dependent Claims 11 and 18, as described above, are analogous claims to claim 4, therefore the same rejection and rationale apply to them.
Dependent Claims 12 and 19, as described above, are analogous claims to claim 5, therefore the same rejection and rationale apply to them.
Dependent Claims 13 and 20, as described above, are analogous claims to claim 6, therefore the same rejection and rationale apply to them.
Dependent Claim 14, as described above, is analogous claims to claim 7, therefore the same rejection and rationale apply to them.
Viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amount to significantly more than the abstract idea itself. Therefore, the claim(s) are rejected under 35 U.S.C 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5, 7-10, 12 and 14-17, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Mei et al. (US 9,411,830 B2) (hereafter referred to as “Mei”) in view of Radford et al. (“Learning Transferable Visual Models From Natural Language Supervision”)
Regarding Claim 1, Mei teaches a method comprising: obtaining a design context, , the design context including one or more visual elements on a digital canvas; (Mei, Col. 4, Lines 10-14: “mobile device 112 receives a natural sentence input via a microphone and voice processor to initiate a voice query, as shown at 118. For example, a mobile device 112 receives a sentence like "find an image with a lake, the sky, and a tree," as illustrated at 118.”, Mei, Col. 6, Lines 12-17: “the interactive multi-modal image search tool records the location of the lake as being lower in the frame of the canvas area 210 than the tree and the sky. Meanwhile, the tree is recorded as being positioned to the right in the frame of canvas area 210 and below the sky, while the sky is at the top of the canvas area 210.”))
determining one or more candidate background embeddings based on a similarity between the intent embedding and a plurality of candidate background embeddings in embedding space; and (Mei, Col. 11, Lines 44-50: “For example, image search module 538, which can be executed by processor 504, can identify image search results based on vector matching of one or more image patches that make up the composite visual query. Image search module 538 can make results of the image search available to be displayed on the screen of mobile device 112.”, Col. 13, Lines 31-35: “In at least one implementation, a clustering-based approach based on visual features and a similarity metric is used to identify candidate images for a given entity by exploiting a known image database and results from image search engines.”) [Examiner’s note: the “embeddings” here is being interpreted as the vector matching of one or more image patches]
identifying one or more recommended background images based on one or more background classes corresponding to the one or more candidate background embeddings. (Mei, Col. 16, Lines 21-38: “the interactive multi-modal image search tool selects the centers of a predetermined number of images from the top clusters ( e.g., the top 10) as candidate images for this entity. For example, potential candidate images showing different subjects may have tags that match an entity. While the potential candidate images may be collected by searching for a certain tag, the interactive multi-modal image search tool can cluster these potential candidate images into groups according to their appearance to identify representative images of the different subjects presented in the images. The interactive multi-modal image search tool can rank the groups, for example, according to the number of images in the respective groups, such that the group with the largest number of images is ranked first. In addition, in some instances, the interactive multi-modal image search tool retains a predetermined number, e.g., the top ten or the top five, groups deemed most representative. In some instances the number of groups retained is user configurable.”)
Mei fails to teach:
generating, by an embedding generator, the embedding generator including a machine learning model trained to generate an intent embedding from the design context based on the one or more visual elements
wherein the embedding generator encodes the intent embedding and a plurality of background embeddings corresponding to background classes into a shared embedding space
However, Radford explicitly discloses:
generating, by an embedding generator, the embedding generator including a machine learning model trained to generate an intent embedding from the design context based on the one or more visual elements,(Radford, Pg. 4, Col. 1, Section 2.3, ¶[4]: “Given a batch of N (image, text) pairs, CLIP is trained to predict which of the N × N possible (image, text) pairings across a batch actually occurred. To do this, CLIP learns a multi-modal embedding space by jointly training an image encoder and text encoder to maximize the cosine similarity of the image and text embeddings of the N real pairs in the batch while minimizing the cosine similarity of the embeddings of the N 2 − N incorrect pairings.”) [Examiner’s note: CLIP teaches a trained visual encoder that converts visual content into a semantic representation]
wherein the embedding generator encodes the intent embedding and a plurality of background embeddings corresponding to background classes into a shared embedding space;(Radford, Pg. 2, Figure 1:
PNG
media_image1.png
415
1119
media_image1.png
Greyscale
“Summary of our approach. While standard image models jointly train an image feature extractor and a linear classifier to predict some label, CLIP jointly trains an image encoder and a text encoder to predict the correct pairings of a batch of (image, text) training examples. At test time the learned text encoder synthesizes a zero-shot linear classifier by embedding the names or descriptions of the target dataset’s classes.”) [Examiner’s note: CLIP teaches jointly trained representations that associate visual concepts and images in a common semantic representation space]
The combination of Mei and Radford are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Mei and Radford before them, to modify the teachings of Mei to include the teachings of Radford to retrieve background images based on semantic similarity to the user’s design context, thereby improving the relevance of recommended backgrounds, reducing dependence on manually assigned keywords or categories, and increasing retrieval accuracy for visually similar content.
Regarding Claim 2, the combination of Mei and Radford explicitly discloses all the limitation from Claim 1 (as shown in the rejection above).
Mei in view of Radford further discloses:
determining an intent from the design context. (Mei, Col. 4, Lines 12-23: “a mobile device 112 receives a sentence like "find an image with a lake, the sky, and a tree," as illustrated at 118. The system employs a speech recognition (SR) engine 120 to transfer the speech received at 118 to a piece of text. The system then employs entity extraction engine 122 to extract entities, which are nouns, from the text. As a result, the tool recognizes "lake," "sky," and "tree" as three entities from lexicon 124. An image clustering engine 126 identifies candidate images from an image database 128 that correspond to each of the three entities and that can be used as respective image patches to represent the recognized entities.”)
Regarding Claim 3, the combination of Mei and Radford explicitly discloses all the limitation from Claim 2 (as shown in the rejection above).
Mei in view of Radford further discloses:
generating the intent embedding from the intent.(Radford, Pg. 4, Col. 1, Section 2.3, ¶[4]: “Given a batch of N (image, text) pairs, CLIP is trained to predict which of the N × N possible (image, text) pairings across a batch actually occurred. To do this, CLIP learns a multi-modal embedding space by jointly training an image encoder and text encoder to maximize the cosine similarity of the image and text embeddings of the N real pairs in the batch while minimizing the cosine similarity of the embeddings of the N 2 − N incorrect pairings.”) [Examiner’s note: CLIP teaches a trained visual encoder that converts visual content into a semantic representation]
Regarding Claim 5, the combination of Mei and Radford explicitly discloses all the limitation from Claim 1 (as shown in the rejection above).
Mei in view of Radford further discloses:
wherein identifying one or more recommended background images based on one or more background classes corresponding to the one or more candidate background embeddings, further comprises: searching an image library using the one or more background classes. (Mei, Col. 4, Lines 20-23: “An image clustering engine 126 identifies candidate images from an image database 128 that correspond to each of the three entities and that can be used as respective image patches to represent the recognized entities.”, Col. 4, Lines 35-38: “The interactive multi-modal image search tool exploits the composite visual query to search for relevant images from image database 128 or in some instances from other sources such as the Internet.”) [Examiner’s note: “an image library” is being interpreted as the “image database”]
Regarding Claim 7, the combination of Mei and Radford explicitly discloses all the limitation from Claim 1 (as shown in the rejection above).
Mei in view of Radford further discloses:
presenting, via a user interface, the one or more recommended background images by adding it to the design context. (Mei, Col. 5, Lines 59- 67: “Meanwhile, candidate images for the entities can be presented on the screen of mobile device 112 as shown at 208. In the example shown, candidate images for one entity, "tree," are presented in a single horizontal ribbon format, from which a particular image is being selected by dragging onto a canvas area 210 of the screen of mobile device 112. Meanwhile, particular candidate images for the entities "lake" and "sky" have already been selected via dragging onto a canvas area 210 of the screen of mobile device 112.”)
Referring to Independent claim 8 and 15, they are rejected on the same basis as independent claim 1 since they are analogous claims.
In addition, Claim 8 recites additional limitation:
A non-transitory computer-readable medium storing executable instructions, which
when executed by a processing device, cause the processing device to perform operations comprising: (Mei, Col. 9, Lines 10-15: “An operating system (OS) 512, a browser application 514, a global positioning system (GPS) module 516, a compass module 518, an interactive multi-modal image search tool 520, and any number of other applications 522 are stored in memory 510 as computer-readable instructions, and are executed, at least in part, on processor 504.”)
Claim 15 recites additional limitation:
a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising: (Mei, Col. 9, Lines 10-15: “An operating system (OS) 512, a browser application 514, a global positioning system (GPS) module 516, a compass module 518, an interactive multi-modal image search tool 520, and any number of other applications 522 are stored in memory 510 as computer-readable instructions, and are executed, at least in part, on processor 504.”)
Referring to dependent claim 9 and 16, they are rejected on the same basis as dependent claim 2 since they are analogous claims.
Referring to dependent claim 10 and 17, they are rejected on the same basis as dependent claim 3 since they are analogous claims.
Referring to dependent claim 12 and 19, they are rejected on the same basis as dependent claim 5 since they are analogous claims.
Referring to dependent claim 14, they are rejected on the same basis as dependent claim 7 since they are analogous claims
Claim(s) 4, 11, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mei et al. (US 9,411,830 B2) (hereafter referred to as “Mei”) in view of Radford et al. (“Learning Transferable Visual Models From Natural Language Supervision”), and further in view of Zhang et al. (US 11,138,285 B2) (hereafter referred to as “Zhang”).
Regarding Claim 4, the combination of Mei and Radford explicitly discloses all the limitation from Claim 1 (as shown in the rejection above).
Mei in view of Radford further teaches:
selecting the one or more candidate background embeddings based on the distance metric. (Mei, Col. 3, Lines 38-46: “The mobile interactive multi-modal image search tool described herein provides a context-aware approach to image search that takes into consideration the spatial relationship among separate images, which are treated as image patches, e.g., small sub-images that represent visual words. The mobile interactive multi-modal image search tool presents an interface for a new search mode that enables users to formulate a composite query image by selecting particular candidate images”, Col. 14, Lines 52-57: “in one implementation, the interactive multi-modal image search tool weights visual words based on relative distance of their respective image patches from the center of the image, with image patches that are closer to the center being more heavily weighted than those that are farther from the center.”, Col. 16, Lines 21-24: “At block 814, the interactive multi-modal image search tool selects the centers of a predetermined number of images from the top clusters ( e.g., the top 10) as candidate images for this entity”)
Mei in view of Radford fails to disclose:
wherein determining one or more candidate background embeddings based on a similarity between the intent embedding and a plurality of candidate background embeddings in the share embedding space, further comprises: calculating a distance metric between the intent embedding and the plurality of candidate background embeddings in the embedding space; and
However, Zhang explicitly discloses:
wherein determining one or more candidate background embeddings based on a similarity between the intent embedding and a plurality of candidate background embeddings in the shared embedding space, further comprises: calculating a distance metric between the intent embedding and the plurality of candidate background embeddings in the embedding space; and (Zhang, Col. 6, Lines 35-43: “the neighbor search component 118 finds any neighbor query intent vectors (NEIVs) within a specified distance of a given intent vector (which serves as a search key), such as the intent vector associated with the input query Iq1. The neighbor search component 118 can determine the distance between the two intent vectors in intent vector space using cosine similarity or some other distance metric. The cosine similarity between any two vectors (A,B) is defined as (A·B)/(IIAII IIBII).”)
The combination of Mei, Radford and Zhang are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Mei, Radford and Zhang before them, to modify the teachings of Mei and Radford to include the teachings of Zhang to improve semantic normalization which help placing the input queries near each other in vector space so the system behaves consistently even when phrasing changes.
Referring to dependent claim 11 and 18, they are rejected on the same basis as dependent claim 4 since they are analogous claims.
Claim(s) 6, 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mei et al. (US 9,411,830 B2) (hereafter referred to as “Mei”) in view of Radford et al. (“Learning Transferable Visual Models From Natural Language Supervision”), and further in view of ZhangNPL et al. (“Composed query image retrieval based on triangle area triple loss function and combining CNN with transformer”) (hereafter referred to as “ZhangNPL”)
Regarding Claim 6, the combination of Mei and Radford discloses all the limitation of Claim 1 (as shown in the rejection above).
Mei in view of Radford fails to disclose:
wherein the embedding generator is a transformer network and wherein the embedding generator is trained using a triplet loss on a training dataset comprising a plurality of sets of background class and query pairs.
However, ZhangNPL explicitly discloses:
wherein the embedding generator is a transformer network and wherein the embedding generator is trained using a triplet loss on a training dataset comprising a plurality of sets of background class and query pairs. (ZhangNPL, Pg. 2, ¶[1]: “We combine CNN with Transformer to capture local and edge feature information of reference images, which can reduce the loss of information. Specifically, the local feature information of reference images is extracted by CNN. Meanwhile, the edge feature information of reference images is focused through Transformer.”, Pg. 10, ¶[3]: “As shown in Tables 7 and 8, “Ours(Ed)” refers to training our network model by Triplet Loss Function, Euclidean distance as sample distance measurement. “Ours(Cd)” refers to training our network model by Triplet Loss Function, Cosine distance as sample distance measurement. “Ours” refers to training our network model by Triangle Area Triplet Loss Function.”,)
The combination of Mei, Radford and ZhangNPL are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Mei, Radford and ZhangNPL before them, to modify the teachings of Mei and Radford to include the teachings of ZhangNPL to reduce the loss of information.
Referring to dependent claims 13 and 20, they are rejected on the same basis as dependent claim 6 since they are analogous claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY TRAN whose telephone number is (571)270-0693. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMY TRAN/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights