DETAILED ACTION
This correspondence is responsive to the application filed on April 23, 2024. Claims 1-20 are pending in the case with claims 1, 8 and 14 in independent form.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Summary of Detailed Action
I. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
II. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
III. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
IV. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
V. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
VI. Claims 5, 12 and 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
VII. Claims 6, 13 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
VIII. Claims 7 and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
I. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 8 and 15 recite the limitation to identify a first “source” of object label representation and a second “source” of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query. It is not clear what a “source” of an object label means or what it is referring to or not referring to. For example, does a source of an object label representation mean a source machine, system, model, company, enterprise, software, labeler, or person that was involved in providing, producing, storing or creating the object label representation? Or, is a source of an object label representation something else entirely? Thus, the boundaries of the claim are indefinite.
Applicant may cancel claims 1, 8 and 15 or amend claims 1, 8 and 15 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-7, 9-14 and 16-20 depend directly or indirectly from claims 1, 8 and 15 and are rejected for the same reasons discussed above with respect to claims 1, 8, and 15.
II. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 8 and 15 recite the limitation to identify a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query. It is not clear if “a first source of object label representation and a second source of object label representation” is referring to two different sources of the same object label representation or if it is referring to sources of two different label representations for the same object, or if it is referring to sources of two different label representations of two different objects, or if it is referring to sources of the same label and object but different representations, or if it means all of the above or none of the above. Thus, the boundaries of the claim are indefinite.
Applicant may cancel claims 1, 8 and 15 or amend claims 1, 8 and 15 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-7, 9-14 and 16-20 depend directly or indirectly from claims 1, 8 and 15 and are rejected for the same reasons discussed above with respect to claims 1, 8, and 15.
III. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 8 and 15 recite the limitation to identify a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query. It is not clear how a source of object label representation includes an estimated label prototype vector associated with an input text-based object query. It is not clear what an estimated label prototype vector is or is not, much less how it is associated with an input text-based object query. Thus, the boundaries of the claim are indefinite.
Applicant may cancel claims 1, 8 and 15 or amend claims 1, 8 and 15 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-7, 9-14 and 16-20 depend directly or indirectly from claims 1, 8 and 15 and are rejected for the same reasons discussed above with respect to claims 1, 8, and 15.
IV. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 8 and 15 recite the limitation to determine a first contextualized embedding for the first source and a second contextualized embedding for the second source. It is not clear what information is or is not being embedded in the first and second contextualized embeddings for the first and second sources. It is further unclear what first and second sources are much less what the context of the first and second sources is referring to or includes. Thus, the boundaries of the claim are indefinite.
Applicant may cancel claims 1, 8 and 15 or amend claims 1, 8 and 15 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-7, 9-14 and 16-20 depend directly or indirectly from claims 1, 8 and 15 and are rejected for the same reasons discussed above with respect to claims 1, 8, and 15.
V. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 8 and 15 recite the limitation to combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query. It is not clear how to generate a candidate object representation associated with the input text-based object query by combining the first source contextual embedding and second source contextual embedding. It is not clear what the source contextual embeddings include or do not include, much less how they are combined in order to generate a candidate object representation that is associated with the input text-based object query. Thus, the boundaries of the claim are indefinite.
Applicant may cancel claims 1, 8 and 15 or amend claims 1, 8 and 15 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 2-7, 9-14 and 16-20 depend directly or indirectly from claims 1, 8 and 15 and are rejected for the same reasons discussed above with respect to claims 1, 8, and 15.
VI. Claims 5, 12 and 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 5, 12 and 19 depend from independent claims 1, 8 and 15 respectively. Claims 5, 12 and 19 and recite the limitation to initiate an action based on “the type of recommendation” associated with the candidate object representation. There is insufficient antecedent basis for this limitation in the claim.
Applicant may cancel claims 5, 12 and 19 or amend claims 5, 12 and 19 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
VII. Claims 6, 13 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 6, 13 and 20 depend from independent claims 1, 8 and 15 respectively, and recite the limitation to identify the estimated label prototype vector using at least one of (1) a document associated with an object-identifying label or (2) a normalization operator. It is not clear what relationship there is, if any, between the identifying label and the object label representations recited in parent claims 1, 8 and 15. Applicant may cancel claims 6, 13 and 20 or amend claims 6, 13 and 20 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
VIII. Claims 7 and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 7 and 14 depend from independent claims 1 and 9 respectively, and recite the limitation to identify a loss function to match a similarity of document embedding and label embedding in a semantic space. It is not clear how to identify a loss function to match a similarity of document embedding and label embedding because it is not clear what document, or document embedding, or label embedding is being referred to or not being referred to and there is no document or document embedding or label embedding that has been identified or defined in either dependent claims 7 and 14 or in their respective parent claims 1 and 8. Thus, the claim boundaries are indefinite.
Applicant may cancel claims 7 and 14 or amend claims 7 and 14 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) subject matter at a general, high-level to identify a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query; determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). Additionally, the recitations to determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query are which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). This judicial exception is not integrated into a practical application and the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 1-20 recite one of the four statutory categories of patent able subject matter and belong to the statutory class(es) of a process (method claims 8-14), a machine (system/apparatus claims 1-7), and an article of manufacture (non-transitory computer readable media claims 15-20).
Claim 1 recites an apparatus, thus a machine and one of the four statutory categories of patentable subject matter. However, claim 1 further recites to identify a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query; determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). Additionally, the recitations to determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query are also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
An apparatus, comprising: interface circuitry; machine-readable instructions; and at least one processor circuit to be programmed by the machine-readable instructions to: (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Thus, the claim is ineligible.
Claim 2, dependent on claim 1, recites only additional abstract ideas for wherein the first source of object label representation is a text-based embedding vector and the second source of object label representation is an auxiliary parameter vector, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
Claim 3, dependent on claim 2, recites abstract ideas to determine the text-based embedding vector, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein one or more of the at least one processor circuit is to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
by performing a pooling operation to average contextualized vectors associated with the text-based object query (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 4, dependent on claim 2, recites additional abstract ideas to combine the first contextualized embedding and the second contextualized embedding, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein one or more of the at least one processor circuit is to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
using a self-attention module (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 5, dependent on claim 1, does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein one or more of the at least one processor circuit is to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
transmit the candidate object representation for processing to initiate an action based on the type of recommendation associated with the candidate object representation (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.)
Claim 6, dependent on claim 1, recites additional abstract ideas to identify the estimated label prototype vector using at least one of (1) a document associated with an object-identifying label or (2) a normalization operator, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2) (III). Additionally, the recitation of (2) a normalization operator is also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein one or more of the at least one processor circuit is to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Claim 7, dependent on claim 1, recites additional abstract ideas to identify a loss function to match a similarity of document embedding and label embedding in a semantic space, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2) (III). Additionally, this recitation is also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein one or more of the at least one processor circuit is to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Claim 8 recites a method, thus a process and one of the four statutory categories of patentable subject matter. However, claim 8 further recites for identifying a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query; determining, a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combining, the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2) (III). Additionally, the recitations for determining, a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combining, the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query are also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
by at least one processor circuit programmed by at least one instruction, (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
by one or more of the at least one processor circuit, (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Thus, the claim is ineligible.
Claim 15 recites at least one non-transitory machine-readable medium, thus an article of manufacture and one of the four statutory categories of patentable subject matter. However, claim 8 further recites to identify a first source of object label representation and a second source of object label representation, the first source or the second source including an estimated label prototype vector associated with an input text-based object query; determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). Additionally, the recitations to determine a first contextualized embedding for the first source and a second contextualized embedding for the second source; and combine the first contextualized embedding and the second contextualized embedding to generate a candidate object representation associated with the input text-based object query are also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
At least one non-transitory machine-readable medium comprising machine-readable instructions to cause at least one processor circuit to at least: (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Dependent claims 9-14 and 16-20 are comparably rejected as set forth above with respect to dependent claims 2-7.
Allowable Subject Matter
Claims 1-20 would be allowable if the rejections of claims 1-20 as being directed to an abstract idea without significantly more are overcome AND if the rejections of claim 1-20 as being indefinite are overcome.
The closest known prior art of record is Kunal Dahiya et al. Deep Encoders with Auxiliary Parameters for Extreme Classification, August 2023, Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’23), August 6–10, 2023, Long Beach, CA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3580305.3599301), which is cited on Applicant’s Information Disclosure Statement filed on April 23, 2024. Dahiya discloses and proposes a lightweight alternative DEXA that augments encoder training with auxiliary parameters. Dahiya, Abstract, p. 358. Examiner Notes that according to claim 2, the auxiliary parameter vector encompasses the second source including an estimated label prototype vector. Dahiya teaches identifying a first source and second source of object label representation including an estimated label prototype vector associated with an input text-based object query in that Dahiya illustrates encoder object label representations with first source and second auxiliary parameters source (“ak”) and estimated label prototype vector in Figure 1 (c): “label l”, “label m”, “ak”, and “nl “, “nm”. Dahiya, Figure 1, (c) page 359; Section 4, page 361. 4 DEXA: Deep Encoders with Auxiliary Parameters, “Model Architecture…encoder .. to embed data points and labels. Additionally, during training ... also uses K auxiliary vectors A…to train the encoder. Dahiya, Section 4, page 361. Encoder Training – Module I: …Next an auxiliary vector ak .., is assigned to each cluster Ck.(estimated label prototype vector). Dahiya, Section 4, page 361, 362; Figure 1, (c) page 359. SponsoredSearch-40M dataset, is the primary metric for the application to match user queries to advertiser keywords (associated with an input text-based object query). This is because the retrieval algorithm generally retrieves a large number of keywords which are progressively pruned in downstream relevance, click-prediction, and bidding stages. Dahiya, Section 6, page 365.
Dahiya illustrates In Figure 1(c), the result of the encoding of “label l” or “label m”, is contextualized embeddings produced by clustering. Dahiya, Figure 1, page 359, Section 4 DEXA: Deep Encoders with Auxiliary Parameters, page 361. “Encoder Training – Module I: First, DEXA creates K << L disjoint label clusters …These embeddings are then subject to … clustering to create K balanced label clusters (first contextualized embeddings). Next, an auxiliary vector ak is assigned to cluster Ck . Dahiya, Section 4 DEXA: Deep Encoders with Auxiliary Parameters, page 361, Figure 1(c), page 359.
Dahiya discloses that The task of annotating a data point with labels most relevant to it from a large universe of labels is referred to as Extreme Classification (XC). State-of-the-art XC methods have applications in ranking, recommendation, and tagging and mostly employ a combination architecture comprised of a deep encoder and a high-capacity classifier (combine first contextualized embedding and second embedding). These two components are often trained in a modular fashion to conserve compute. This paper shows that in XC settings where data paucity and semantic gap issues abound, this can lead to sub optimal encoder training which negatively affects the performance of the overall architecture. The paper then proposes a lightweight alternative DEXA that augments encoder training with auxiliary parameters. Dahiya, Abstract, Section 4. Classifier Training – Module II, page 361, Figure 1 (c), Module I and Module II. Classifier Training– Module II: Once encoder training is complete, the encoder is frozen and augmented embedding of each label is computed … The augmented label embeddings are preserved …. Next, the label classifiers are initialized to the augmented label embeddings (classifier combines the encoder first contextualized embedding and the second embedding), … and the classifiers are learnt. Dahiya, Section 4. Classifier Training – Module II, page 361, Figure 1 (c), Module I and Module II. Dahiya and the known prior art of record does not disclose or suggest the second contextualized embedding or combining the second contextualized embedding.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US-20200356851-A1, US-10949907-B1, US-20220414137-A1, US-20230034495-A1, US-20240111794-A1, US-20250238474-A1, US-20250284721-A1, US-20250291867-A1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BARBARA LEVEL whose telephone number is (303)297-4748. The examiner can normally be reached Monday through Friday 8:00 AM - 5:00 PM MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BARBARA M LEVEL/ Examiner, Art Unit 2142
Prosecution Insights