DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the original application filed on 4/8/2024.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 12, and 19 recite the limitation “one or more training operations for a machine learning classification model configured for the performance of the predictive task” (emphasis added). There is insufficient antecedent basis for the claimed “the performance”. For examination purposes, the limitation will be interpreted to mean “one or more training operations for a machine learning classification model configured for [[the]] performance of the predictive task” (emphasis added). Dependent claims 2-11, 13-18, and 20 depend on indefinite claims 1, 12, and 19, respectively, and are also rejected under 35 USC § 112(b) by virtue of this dependency. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Claim 1
Step 1: The claim recites a method; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: The claim recites, inter alia:
identifying, …, a plurality of entity predictive identifiers from one or more historical data objects for an entity: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifiers from data objected, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally identify identifiers or tags from data.
generating, …, a textual entity document for the entity by concatenating a set of contextual text descriptions from the task-specific data store that correspond to a set of task relevant identifiers from the plurality of entity predictive identifiers: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a textual document by concatenating or combining textual descriptions, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally make a text document by combining text descriptions.
generating, …, a contextual entity embedding for the entity based on the textual entity document: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a an embedding for an entity based on a document, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally generate an embedding based on a text document.
Step 2A Prong 2: The claim does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “by one or more processors”, “by the one or more processors and using a task-specific data store corresponding to a predictive task”, “by the one or more processors and using a machine learning encoder model”, and “initiating, by the one or more processors and using the contextual entity embedding, one or more training operations for a machine learning classification model configured for the performance of the predictive task”.
The additional elements of “by one or more processors”, “by the one or more processors and using a task-specific data store corresponding to a predictive task”, and “by the one or more processors and using a machine learning encoder model” amount to generic computer components and models used as a tool to perform an existing process. The additional element of “initiating, by the one or more processors and using the contextual entity embedding, one or more training operations for a machine learning classification model configured for the performance of the predictive task” amounts to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the generic classification model is broadly trained. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Thus, even when viewed individually and as an ordered combination, these additional elements do not integrate the abstract idea into a practical application, and the claim is thus directed to the abstract idea.
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “by one or more processors”, “by the one or more processors and using a task-specific data store corresponding to a predictive task”, and “by the one or more processors and using a machine learning encoder model” amount to generic computer components and models used as a tool to perform an existing process. The additional element of “initiating, by the one or more processors and using the contextual entity embedding, one or more training operations for a machine learning classification model configured for the performance of the predictive task” amounts to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the generic classification model is broadly trained. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 2
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the machine learning encoder model comprises a neural network architecture and the contextual entity embedding comprises a fixed length numerical vector representing a semantic relationship between the set of task relevant identifiers to the predictive task” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 3
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
identifying the set of task relevant identifiers from the plurality of entity predictive identifiers based on a comparison between the plurality of entity predictive identifiers and a plurality of task relevant identifiers from the task-specific data store: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifiers based on a comparison, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating the textual entity document by concatenating the set of contextual text descriptions: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating text based on combining descriptions, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “receiving, from the task-specific data store, the set of contextual text descriptions corresponding to the set of task relevant identifiers” is insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 4
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
identifying, using a normalized co-occurrence matrix, one or more identifier clusters from the set of task relevant identifiers, wherein each identifier cluster comprises a subset of related task relevant identifiers from the set of task relevant identifiers: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifier clusters using a matrix, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
modifying the textual entity document by grouping a subset of contextual text descriptions from the set of contextual text descriptions that correspond to the subset of related task relevant identifiers: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of modifying text by grouping descriptions, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 5
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the normalized co-occurrence matrix is generated based on a plurality of identifier frequencies across a plurality of historical data objects for a plurality of training entities” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 6
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the plurality of identifier frequencies identifies a co-occurrence count reflective of a number of historical data objects that comprise a first predictive identifier and a second predictive identifier” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 7
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the normalized co-occurrence matrix comprises a normalized co-occurrence measure for the first predictive identifier and the second predictive identifier that is based on the co-occurrence count and a number of the plurality of historical data objects” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 8
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
identifying a plurality of task relevant identifiers for the predictive task: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifiers for a task, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating, using a machine learning contextualization model, a contextual text description for each of the plurality of task relevant identifiers: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a description for identifiers, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “using a machine learning contextualization model” amounts to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how the generic contextualization model is broadly used to generate text descriptions. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)). The additional element of “storing the plurality of task relevant identifiers and the contextual text description for each of the plurality of task relevant identifiers” is insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Storing and retrieving information in memory”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 9
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
identifying a predefined textual description for the task relevant identifier: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying a description for an identifier, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating a generative text prompt based on the predefined textual description: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a text prompt based on a description, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “inputting the generative text prompt to the machine learning contextualization model to receive the particular contextual text description for the task relevant identifier” amounts to reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is not clear how inputting of the prompt into a generic contextualization model results in a text description. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)). The additional element of “inputting the generative text prompt to the machine learning contextualization model” is also insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 10
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
generating a positive class identifier document by: identifying a first plurality of training entities associated with a positive class of the binary classification task: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a class identifier documents by identifying training entities associated with a task, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating the positive class identifier document by concatenating a plurality of positive class identifiers from each of the first plurality of historical data objects: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a class identifier document bay combining identifiers from data objects, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating a negative class identifier document by: identifying a second plurality of training entities associated with a negative class of the binary classification task: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a class identifier document by identifying training entities associated with a task, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating the negative class identifier document by concatenating a plurality of negative class identifiers from each of the second plurality of historical data objects: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating a class identifier document bay combining identifiers from data objects, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
identifying the plurality of task relevant identifiers based on a comparison between the positive class identifier document and the negative class identifier document: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifiers based on a comparison, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional elements of “receiving a first plurality of historical data objects corresponding to the first plurality of training entities … receiving a second plurality of historical data objects corresponding to the second plurality of training entities” are insignificant extra-solution activities required for any uses of the mental processes (see MPEP § 2106.05(g)), and are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). The additional element of “wherein the predictive task is a binary classification task” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 11
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
generating a plurality of positive frequency scores for the plurality of positive class identifiers based on the positive class identifier document;: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating frequency scores based on a document, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating a plurality of negative frequency scores for the plurality of negative class identifiers based on the negative class identifier document: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating frequency scores based on a document, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
generating a plurality of feature importance measures based on a comparison between the plurality of positive frequency scores and the plurality of negative frequency scores: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of generating importance measures based on a comparison of scores, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
identifying the plurality of task relevant identifiers based on the plurality of feature importance measures: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying identifiers based on importance measures, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claims 12-18
Claims 12-18 recite a computing system (step 1: a machine) using a processor and program to perform the steps of claims 1-7, respectively, which by MPEP 2106.05(f) (“apply it”) cannot integrate an abstract idea into a practical application or provide significantly more than the abstract idea by itself, and are thus rejected for the same reasons set forth in the rejection of claims 1-7, respectively.
Claims 19-20
Claims 19-20 recite one or more non-transitory computer-readable storage media (step 1: a manufacture) using a processor and instructions to perform the steps of claims 1 and 8, respectively, which by MPEP 2106.05(f) (“apply it”) cannot integrate an abstract idea into a practical application or provide significantly more than the abstract idea by itself, and are thus rejected for the same reasons set forth in the rejection of claims 1 and 8, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8, 12-14, 19, and 20 are rejected under 35 USC § 103 as being obvious over Hegselmann et al. (Hegselmann et al., “TabLLM: Few-shot Classification of Tabular Data with Large Language Models”, Mar. 17, 2023, arXiv:2210.10723v2, pp. 1-33, hereinafter “Hegselmann”) in view of Hore et al. (US 20230177270 A1, hereinafter “Hore”).
Regarding claim 1, Hegselmann discloses [a] computer-implemented method comprising: (Abstract; “We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem”, which discloses a method that is inherently performed using a computer; and §4; the experiments section is inherently implemented using a computer)
identifying, by one or more processors, a plurality of entity predictive identifiers from one or more historical data objects for an entity; (§3.1; “Problem Formalization. Suppose we have a tabular dataset with 𝑛 rows and 𝑑 columns or features. We can formalize this as 𝐷 x𝑖 𝑦𝑖 𝑛 𝑖 1, where each x𝑖 is a 𝑑 dimensional feature vector”, which discloses, under a broadest reasonable interpretation of the claim language, a tabular dataset “D” in which each row x𝑖 is a d-dimensional feature vector derived from the historical relational-database record described in §1 of Hegselmann as the source of tabular data. The column/feature identifiers f1…fd and their values, read for a given row, are the “entity predictive identifiers” that are identified “from one or more historical data objects for an entity”; and Supplemental section Page 13, §1.1 and 1.2; the sections disclose the databases or datasets from which the “one or more historical data objects for an entity” are sourced)
generating, by the one or more processors [[and using a task-specific data store corresponding to a predictive task]], a textual entity document for the entity by concatenating a set of contextual text descriptions [[from the task-specific data store]] that correspond to a set of task relevant identifiers from the plurality of entity predictive identifiers; (§3.1; “To use an LLM for tabular data, the table must be transformed into a natural text representation. Typically, when prompting an LLM, there is a template used to both serialize the inputs into one natural-language string, and to provide the prompt itself (e.g., the string “Does this person make more than 50,000 dollars? Yes or no?”), which is usually located after the serialized input”, which discloses generat6ing a textual document from the entity’s tabular identifiers; and §3.2; the section further discloses the “Text Template” serialization, which is a textual “enumeration of all features” that results in a textual document by joining a natural-language phrase for every feature or identifier through concatenation; and Supplement §1.2.2; “Hence, we had to determine which concepts of a patient should be included during the serialization. We evaluated four different concept selection strategies in the zero-shot setting for the List Template serialization. Choosing the least frequent, most frequent, oldest, or most recent concepts per patient. We tested these for all concepts (conditions and procedures), only conditions, or only procedures”, which discloses four different concept selection strategies that are used to choose which of the entity’s full identifier set are serialized, with “most frequent conditions per patient” identified as the best performing selection or a “task-relevant” subset selected from the set of task-relevant identifiers)
generating, by the one or more processors and using a machine learning encoder model, a [[contextual entity]] embedding for the entity based on the textual entity document; and (§3.2; “We employ the T0 encoder-decoder model with 11 billion parameters as the LLM for TabLLM”, which discloses using a ML encoder model/T0 encoder/decoder model that processes the serialized textual document input to generate the model’s internal/output representation used for classification, the representation being the ML encoder-generated embedding)
initiating, by the one or more processors [[and using the contextual entity embedding]], one or more training operations for a machine learning classification model configured for the performance of the predictive task (§4.2; “For fine-tuning, we adopted the default hyperparameters of the T-Few method without any additional parameter tun ing (Liu et al., 2022). The authors used a setup of 𝑘 32 shots and 1,000 training steps for most of their experiments, which corresponds to 31.25 epochs. Hence, we fixed 30 training epochs for all few-shot experiments on the public tabular datasets”, which discloses fine-tuning or training the LLM using the model’s representation of the serialized or prompted input; and Supplement §8; the templates discloses that the trained model is configured for specific tasks).
Hegselmann fails to explicitly disclose but Hore discloses using a task-specific data store corresponding to a predictive task … from the task-specific data store (Figure 1, Element 103; and [0010]; “a database or storage device 103”; and [0014]; “a database or storage device 103 stores datasets or records 105 and 107 including data values associated with entities”; and Figure 6, Elements 605-607)
a contextual entity embedding … and using the contextual entity embedding ([0019]; “compute contextual information and/or semantic meaning utilizing a word embedding technique”; and [0031]; and [0034]; and [0038]; “The vector obtained is then connected to the fully connected layer which calculates the probability that the input sentence belongs to a category and then through back propagation through time the Bi-LSTM cell weights can be updated, and likewise any random embeddings/vector that originally assigned to each word”).
Hegselmann and Hore are analogous art because both are concerned with machine learning and LLMs. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in LLMs and machine learning to combine the task-specific data store and contextual embeddings of Hore with the method of Hegselmann to yield to the predictable result of generating, by the one or more processors and using a task-specific data store corresponding to a predictive task, a textual entity document for the entity by concatenating a set of contextual text descriptions from the task-specific data store that correspond to a set of task relevant identifiers from the plurality of entity predictive identifiers; generating, by the one or more processors and using a machine learning encoder model, a contextual entity embedding for the entity based on the textual entity document. The motivation for doing so would be to provide for improved and accurate machine learning predictions for the classification of categories and subcategories of an entity or entities (Hore; [0009]).
Regarding claim 12, it is a system claim corresponding to the steps of claim 1 and is rejected for the same reasons as claim 1.
Regarding claim 19, it is a non-transitory computer-readable storage media claim corresponding to the steps of claim 1 and is rejected for the same reasons as claim 1.
Regarding claims 2 and 13, the rejection of claims 1 and 12 are incorporated and Hegselmann further discloses wherein the machine learning encoder model comprises a neural network architecture (§3.2).
Hegselmann fails to explicitly disclose but Hore discloses the contextual entity embedding comprises a fixed length numerical vector representing a semantic relationship between the set of task relevant identifiers to the predictive task ([0018-0020]; and [0038-0039]).
The motivation to combine Hegselmann and Hore is the same as discussed above with respect to claim 1.
Regarding claims 3 and 14, the rejection of claims 1 and 12 are incorporated and Hegselmann fails to explicitly disclose but Hore discloses identifying the set of task relevant identifiers from the plurality of entity predictive identifiers based on a comparison between the plurality of entity predictive identifiers and a plurality of task relevant identifiers from the task-specific data store; receiving, from the task-specific data store, the set of contextual text descriptions corresponding to the set of task relevant identifiers; and generating the textual entity document by concatenating the set of contextual text descriptions ([0036]; “ In some implementations, an input vector can be produced from input 601, using already the computed word embeddings available from category/sub-category against whom it is being matched, and the cosine similarity technique can be applied between the input vector and vectors derived from the category prototypes represented by word embeddings and/or the vectors derived from the subcategory prototypes represented by word embeddings”; and Figure 1, 103; and [0032]).
The motivation to combine Hegselmann and Hore is the same as discussed above with respect to claim 1.
Regarding claims 8 and 20, the rejection of claims 1 and 12 are incorporated and Hegselmann further discloses identifying a plurality of task relevant identifiers for the predictive task; generating, using a machine learning contextualization model, a contextual text description for each of the plurality of task relevant identifiers (§3.2; and Figure 1).
Hegselmann fails to explicitly disclose but Hore discloses storing the plurality of task relevant identifiers and the contextual text description for each of the plurality of task relevant identifiers (Figure 1, 103; and [0014]).
The motivation to combine Hegselmann and Hore is the same as discussed above with respect to claim 1.
Claims 4-7 and 15-18 are rejected under 35 USC § 103 as being obvious over Hegselmann in view of Hore and further in view of Shazeer et al. (US 20170228414 A1, hereinafter “Shazeer”).
Regarding claims 4 and 15, the rejection of claims 1 and 12 are incorporated and Hegselmann fails to explicitly disclose but Shazeer discloses identifying, using a normalized co-occurrence matrix, one or more identifier clusters from the set of task relevant identifiers, wherein each identifier cluster comprises a subset of related task relevant identifiers from the set of task relevant identifiers; and modifying the textual entity document by grouping a subset of contextual text descriptions from the set of contextual text descriptions that correspond to the subset of related task relevant identifiers ([0005]; “obtaining a set of sub matrices of a feature co-occurrence matrix, wherein each row of the feature co-occurrence matrix corresponds to a feature from a first feature vocabulary and each column of the feature co-occurrence matrix corresponds to a feature from a second feature vocabulary; selecting a sub matrix from the set of sub matrices, wherein the sub matrix is associated with a particular row block of the feature co-occurrence matrix and a particular column block of the feature co-occurrence matrix; assigning a respective d-dimensional initial row embedding vector to each row from the particular row block and a respective d-dimensional initial column embedding vector to each column from the particular column block, wherein d represents a predetermined target dimensionality; and determining a final row embedding vector for each row from the particular row block and a final column embedding vector for each column from the particular column block by iteratively adjusting the initial row embedding vectors from the particular row block and the initial column embedding vectors from the particular column block using the feature co-occurrence matrix”, where clusters are interpreted as feature embeddings; and Figure 1, Elements 102 and 112).
Hegselmann, Hore, and Shazeer are analogous art because all are concerned with LLMs and word embeddings. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in LLMs and machine learning to combine the co-occurrence matrix and clustering of Shazeer with the method of Hegselmann and Hore to yield to the predictable result of identifying, using a normalized co-occurrence matrix, one or more identifier clusters from the set of task relevant identifiers, wherein each identifier cluster comprises a subset of related task relevant identifiers from the set of task relevant identifiers; and modifying the textual entity document by grouping a subset of contextual text descriptions from the set of contextual text descriptions that correspond to the subset of related task relevant identifiers. The motivation for doing so would be to compress the distributional structure of raw language co-occurrence statistics to generate compact representations that retain properties of the original space (Shazeer; [0003]).
Regarding claims 5 and 16, the rejection of claims 1, 4, 12, and 15 are incorporated and Hegselmann fails to explicitly disclose but Shazeer discloses wherein the normalized co-occurrence matrix is generated based on a plurality of identifier frequencies across a plurality of historical data objects for a plurality of training entities ([0014-0015]; and Figure 3; and [0066-0068]).
The motivation to combine Hegselmann, Hore, and Shazeer is the same as discussed above with respect to claim 4.
Regarding claims 6 and 17, the rejection of claims 1, 4, 5, 12, 15, and 16 are incorporated and Hegselmann fails to explicitly disclose but Shazeer discloses wherein the plurality of identifier frequencies identifies a co-occurrence count reflective of a number of historical data objects that comprise a first predictive identifier and a second predictive identifier ([0014-0015]; and Figure 3; and [0066-0068]).
The motivation to combine Hegselmann, Hore, and Shazeer is the same as discussed above with respect to claim 4.
Regarding claims 7 and 18, the rejection of claims 1, 4-6, 12, and 15-17 are incorporated and Hegselmann fails to explicitly disclose but Shazeer discloses wherein the normalized co-occurrence matrix comprises a normalized co-occurrence measure for the first predictive identifier and the second predictive identifier that is based on the co-occurrence count and a number of the plurality of historical data objects ([0014-0015]; and Figure 3; and [0066-0068]; and [0005]; and Figure 1).
The motivation to combine Hegselmann, Hore, and Shazeer is the same as discussed above with respect to claim 4.
Claim 9 is rejected under 35 USC § 103 as being obvious over Hegselmann in view of Hore and further in view of Saxena et al. (US 20240330579 A1, hereinafter “Saxena”).
Regarding claim 9, the rejection of claims 1 and 8 are incorporated and Hegselmann fails to explicitly disclose but Saxena discloses identifying a predefined textual description for the task relevant identifier; generating a generative text prompt based on the predefined textual description; and inputting the generative text prompt to the machine learning contextualization model to receive the particular contextual text description for the task relevant identifier (Figure 4, Elements 410-414; and [0047-0049]).
Hegselmann, Hore, and Saxena are analogous art because all are concerned with LLMs. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in LLMs and machine learning to combine the text prompts of Saxena with the method of Hegselmann and Hore to yield to the predictable result of identifying a predefined textual description for the task relevant identifier; generating a generative text prompt based on the predefined textual description; and inputting the generative text prompt to the machine learning contextualization model to receive the particular contextual text description for the task relevant identifier. The motivation for doing so would be to generate a prompt to a large language model to generate text for the section of the webpage (Saxena; Abstract).
Claims 10-11 are rejected under 35 USC § 103 as being obvious over Hegselmann in view of Hore and further in view of Miller et al. (US 20210224534 A1, hereinafter “Miller”).
Regarding claim 10, the rejection of claims 1 and 8 are incorporated and Hegselmann fails to explicitly disclose but Miller discloses generating a positive class identifier document by: identifying a first plurality of training entities associated with a positive class of the binary classification task, receiving a first plurality of historical data objects corresponding to the first plurality of training entities, and generating the positive class identifier document by concatenating a plurality of positive class identifiers from each of the first plurality of historical data objects; generating a negative class identifier document by: identifying a second plurality of training entities associated with a negative class of the binary classification task, receiving a second plurality of historical data objects corresponding to the second plurality of training entities, and generating the negative class identifier document by concatenating a plurality of negative class identifiers from each of the second plurality of historical data objects; and identifying the plurality of task relevant identifiers based on a comparison between the positive class identifier document and the negative class identifier document (Figure 4, Elements 440 and 460; and [0085-0086; and 0019; and 0024]).
Hegselmann, Hore, and Miller are analogous art because all are concerned with LLMs. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in LLMs and machine learning to combine the positive and negative class identifiers of Miller with the method of Hegselmann and Hore to yield to the predictable result of generating a positive class identifier document by: identifying a first plurality of training entities associated with a positive class of the binary classification task, receiving a first plurality of historical data objects corresponding to the first plurality of training entities, and generating the positive class identifier document by concatenating a plurality of positive class identifiers from each of the first plurality of historical data objects; generating a negative class identifier document by: identifying a second plurality of training entities associated with a negative class of the binary classification task, receiving a second plurality of historical data objects corresponding to the second plurality of training entities, and generating the negative class identifier document by concatenating a plurality of negative class identifiers from each of the second plurality of historical data objects; and identifying the plurality of task relevant identifiers based on a comparison between the positive class identifier document and the negative class identifier document. The motivation for doing so would be to provide a discriminative binary text classification model that is scalable, computationally efficient, and self-tuning (Miller; [0024]).
Regarding claim 11, the rejection of claims 1, 8, and 10 are incorporated and Hegselmann fails to explicitly disclose but Miller discloses wherein identifying the plurality of task relevant identifiers based on the comparison between the positive class identifier document and the negative class identifier document comprises: generating a plurality of positive frequency scores for the plurality of positive class identifiers based on the positive class identifier document; generating a plurality of negative frequency scores for the plurality of negative class identifiers based on the negative class identifier document; generating a plurality of feature importance measures based on a comparison between the plurality of positive frequency scores and the plurality of negative frequency scores; and identifying the plurality of task relevant identifiers based on the plurality of feature importance measures ([0103]; “in block 620, for each feature, classification engine 326 uses Bayes Rule to compute a positive and a negative “accuracy” score, as a measure of the feature's positive/negative predictive power”; and Figure 4, Elements 440 and 460; and [0085-0086; and 0019; and 0024]).
The motivation to combine Hegselmann, Hore, and Miller is the same as discussed above with respect to claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brent Hoover whose telephone number is (303)297-4403. The examiner can normally be reached Monday - Friday 9-5 MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT JOHNSTON HOOVER/ Primary Examiner, Art Unit 2127
Prosecution Insights