DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/4/26 has been entered.
Notice to Applicant
The following is a Non-Final Office action. In response to Examiner’s Final Rejection of 2/4/26, Applicant, on 5/4/26 filed an RCE for the claims of 4/3/26 to be entered. Claims 1-20 are pending in this application and have been rejected below.
Response to Amendment
Applicant’s amendments are acknowledged.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without reciting significantly more.
Step One - First, pursuant to step 1 in MPEP 2106.03, the claim 1 is directed to a method which is a statutory category.
Step 2A, Prong One - MPEP 2106.04 - The claim 1 recites–
“A … method, comprising:
determining, …, a contextualized score based on contextualized metadata describing a user and at least one additional individual… (Applicant’s [0043] as filed states “contextualized data (including metadata) refers to all information about the context where the user and/or other individuals are present, with a focus on elements that are measurable. For example, in embodiments contextual metadata may be job, role, and/or profile data including job role skill set (JRSS) coding associated with a job role, project profile data, social data, feedback data, and performance data. In embodiments, project profile data may include internal and external projects, project type… daily/weekly/monthly schedules, and/or other data that describes a user’s day-to-day efforts and tasks. In embodiments, social data may include a level of interest in a meeting, number of times speaking and/or commenting during a meeting... In embodiments, feedback data may include feedback from peers, managers, and stakeholders, including 360-degree feedback … In embodiments, performance data may include career goals, … Furthermore, as used herein, context may be a workplace, sport, activity, career, an individual’s role within a job or team, projects the individual is working on, or any other characterizations of the contextual data examples provided herein.”;
model parameters… updated using a loss function based on the historical parameters (Applicant’s [0061] as published states that mean square error is an example of a loss function- computing error based on historical information is a mathematical relationship);
determining, …, a personalized score by comparing personal parameters describing the user against historical parameters (Applicant’s [0045] as filed states “user performance module 210 may be further configured to determine, using a second LLM built using a neural network framework such as a second RNN, a personalized score by comparing personal parameters (i.e., dimensions) describing the user against historical parameters”. Applicant’s [0046] as filed states “personal parameters may include one or more of interaction data, communication data, skills data, habit data, and/or tracked data/status. Interaction data may include coaching, mentoring, learning, volunteering, participation in community activities, innovating, inventing, patenting, and the like, with each interaction data being quantified (e.g., how often, how many, feedback score, diversity, etc.).” Applicant’s [0047] as filed states “Historical parameters refer to existing data and benchmarks used to measure and assess performance of individuals within the same context as the personal parameters. For example, if a personal parameter measures a user’s ability to communicate in a large group setting, the historical parameters provide a benchmark for other individuals and their ability to communicate in large group settings. In another example, if the personal parameter describes a user’s number of vacation days taken per year, the corresponding historical parameter provides a benchmark for other similar individuals and their number of vacation days taken per year.”);
determining, …, an individual benchmark based on a normalized aggregation of the contextualized score and the personalized score (Applicant’s [0048] as filed states “, root cause analysis module 215 is configured to determine an individual benchmark based on an aggregation of the contextualized score and the personalized score. That is, root cause analysis module 215 may be configured to break down the contextualized score and the personalized score into each dimension and may optionally offer an in-depth report of the user’s performance… In embodiments, the other individuals with whom the scores are compared may be individuals in the same context of the user and/or individuals that the user aspires to be and/or be more like. In embodiments, the aggregation of the contextualized score and the personalized score are normalized to better understand the insights provided by the aggregated scores; “normalized” = mathematical relationship)
determining, …, an industry benchmark based on historical industry benchmarks; and
generating, …, an objective roadmap for the user based on the individual benchmark and the industry benchmark, the objective roadmap comprising first actions for improvement that are generated by measuring a first distance between a first tracked status, the individual benchmark, and the industry benchmark, wherein the first tracked status is … received from a user … based on user interactions … (Applicant’s [0046] as filed states “Tracked data/status may include any data (e.g., status data) that the user enters as related to trackable tasks (e.g., goals, steps, roadmap stages, etc.) and/or a level of completion of such tasks.” Applicant’s [0050] as filed states “user may perform/complete to achieve a desired improvement. In embodiments, the roadmap may include a plurality of actions to be performed or completed in a sequential order to achieve the desired improvement. For example, a roadmap may comprise sequential steps that build on one another, such as perform task 1, then complete action 2, and finish action 3).”
dynamically updating, …, the objective roadmap in response to updates to the first tracked status.
As drafted, this is, under its broadest reasonable interpretation, within the Abstract idea grouping of “mathematical relationships” and “certain methods of organizing human activity” (managing personal behavior or relationships between people – including… teaching, and following rules or instructions) because we have contextualized scoring a user for a context (e.g. job, projects, career, person’s role with a job or team, etc), computing a loss function (mean square error in [0061] is example) relative to historical information, then determining a personalized score by comparing a user’s personal parameters describing a user (e.g. interactions, communications, skills, etc) against historical parameters (e.g. assess performance of other individuals and their abilities… to communicate; how similar users spend their time or take vacation, etc), determining an individual benchmark based on aggregating contextual score and personalized score (e.g. user’s performance), determining an industry benchmark, and then generating an objective roadmap for user to improve (e.g. learn something) based on distance between a first status (e.g. completion in claim 3 and example in [0046]), the individual benchmark (e.g. performance level) and industry benchmark, where tracked status based on user interactions and then updating the objective roadmap in response to updates to tracked status. Accordingly, claim 1 is directed to an abstract idea as it is directed to performing a number of mathematical calculations (e.g. contextualized score, loss function, personalized score, individual benchmark, industry benchmark, distance between status and benchmarks), and scoring user performance relative to industry and others to recommend the user follow a roadmap of content/actions for a user to learn/improve.
Step 2A, Prong Two - MPEP 2106.04 - This judicial exception is not integrated into a practical application. Claim 1 recites Additional elements that are:
“A computer-implemented method, comprising:
determining, by a processor set using a first large language model, a contextualized score based on contextualized metadata describing a user and at least one additional individual, wherein the first large language model is trained using a test-train repository where the contextualized metadata is embedded into vectors within the first large language model and compared against the at least one additional individual;
training, by the processor set, a second large language model using historical parameters as training data, wherein model parameters of the second large language model are updated using a loss function based on the historical parameters;
determining, by the processor set using the second large language model, a personalized score by comparing personal parameters describing the user against the historical parameters wherein the personal parameters are weighted based on correlation to user performance associated with a task;
determining, by the processor set, an individual benchmark based on a normalized aggregation of the contextualized score and the personalized score;
determining, by the processor set, an industry benchmark based on historical industry benchmarks; and
generating, by the processor set, an objective roadmap for the user based on the individual benchmark and the industry benchmark, the objective roadmap comprising first actions for improvement that are generated by measuring a first distance between a first status, the individual benchmark, and the industry benchmark, wherein the first tracked status is automatically received from a user device based on user interactions with one or more applications;
dynamically updating, by the processor set, the objective roadmap in response to updates to the first tracked status.”
(Additional elements of computer, processor set, large language models, training, where status of a user was received from a user “device” are considered “apply it [abstract idea] on a computer” (See MPEP 2106.05f); combination of “first large language model,” that is trained using a test-train repository, for contextualized score and “second large language model” for personalized score and processor set using the models are also considered “field of use” (MPEP 2106.05h).
Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim also fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, and/or an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See 84 Fed. Reg. 55. The claim is directed to an abstract idea.
Step 2B in MPEP 2106.05 - The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of a computer, processor, large language model that is trained using test-train repository, are treated as MPEP 2106.05(f) (Mere Instructions to Apply an Exception – “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235) and “field of use” (MPEP 2106.05h). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Regarding claim 10, it recites “a computer program product comprising one or more computer readable storage media…” Applicant’s [0023] as filed states “A computer readable storage medium… is not to be construed as storage in the form of transitory signal per se…” Accordingly, independent claim 10 is directed to an article of manufacture at step 1, which is a statutory category. Claim 10 recites similar limitations as claim 1 and is rejected for the same reasons at step 2a, prong one, 2a, prong 2, and step 2b.
Independent claim 17 is directed to an apparatus at step 1, which is a statutory category. Claim 17 recites similar limitations as claim 1 and claim 10 and is rejected for the same reasons at step 2a, prong one; step 2a, prong 2 and step 2b.
Claims 2, 11, 18 have an additional element stating that the metadata is received “from an external device or storage medium.” This is considered at step 2a, prong two and step 2B to be MPEP 2106.05(f) (Mere Instructions to Apply an Exception – “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235) and “field of use” (MPEP 2106.05h). At step 2B, this is also considered a conventional computer function – See MPEP 2106.05d(II) i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321.
Claims 3, 12, 19 narrow the abstract idea by stating that there is “completion” as the status for different actions a user undertakes for improvement/learning.
Claims 4, 11, 18 narrow the abstract idea by having a recommended roadmap with second actions for a user to undertake for improvement by repeating steps in claim 1 – a second distance [math] analyzed between second status, second individual benchmark, and industry benchmark.
Claims 5, 14 narrow the abstract idea by generally stating that the action for improvement are generated using a machine learning model. To extent this is “using a computer”, this is considered MPEP 2106.05f (apply it [abstract idea] on a computer) and MPEP 2106.05h (field of use).
Claims 6, 15 narrows the abstract idea by training a model based on the second status, second individual benchmark, industry benchmark. To extent the training is “by a computer”, this is considered MPEP 2106.05f (apply it [abstract idea] on a computer) and MPEP 2106.05h (field of use).
Claim 7 narrows the abstract idea by reciting that metadata describes user and additional individual are in the “same context.”
Claims 8, 16 narrow the abstract idea by giving the description of various personal parameters – past interactions, communications, habits, and actions of a user, that are for trackable tasks of the user.
Claim 9 recites additional element of specific neural networks, and this claim is considered “by a computer”, that is considered MPEP 2106.05f (apply it [abstract idea] on a computer) and MPEP 2106.05h (field of use).
Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
For more information on 101 rejections, see MPEP 2106.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Amatriain (US 2025/0005288) and Sabet (US 2016/0260044) and Zheng, “Generative Learning Plan Recommendation for Employees: A Performance-aware Reinforcement Learning Approach,” 2023, Proceedings of the 17th ACM Conference on Recommender Systems, pages 442-454.
Concerning claim 1, Amatriain discloses:
A computer-implemented method (Amatriain – see par 267 - The term “machine-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any of the methodologies of the present disclosure), comprising:
determining, by a processor set (Amatriain - see par 41 - In the example of FIG. 1A, an example computing system 100 is shown, which includes an example directive generative thread-based user assistance system 102 and an example thread-based user assistance interface 118. See par 256 - The machine can operate in the capacity of a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment) using a first large language model, a contextualized score (Amatriain – see par 41 - The directive generative thread-based user assistance system 102 of FIG. 1A includes a thread classification prompt generator 104, a first large language model 108, a thread label generator 109, a plan execution prompt generator 112, and a second large language model 116; see par 46 - To create and operate various portions of directive generative thread-based user assistance system 102 and/or thread-based user assistance interface 118, components of the directive generative thread-based user assistance system 102 and/or thread-based user assistance interface 118 can access one or more contextual resources to, for example, obtain parameter values that can be used to constrain the operations of one or more large language models. Examples of contextual resources shown in FIG. 1A include entity graph 103, knowledge graph 105 and data sources 107; see par 59 - Data sources 107 can be used to supply retrieved data 130 to thread classification prompt generator 104 and/or plan execution prompt generator 112 in a similar manner. Examples of retrieved data 130 include online dialog history 113, web content 115 (e.g., web pages, such as user profile pages, company pages, articles, and posts.) based on contextualized metadata describing a user … (Applicant’s [0043] as filed states “contextualized data (including metadata) refers to all information about the context where the user and/or other individuals are present, with a focus on elements that are measurable
Amatriain discloses the limitations based on broadest reasonable interpretation in light of the specification – see par 63 - Online dialog history 113 includes historical threads and thread portions associated with an online dialog involving a particular user. That is, each user will have a separate online dialog history 113; see par 65 - Data retrieved from web content 115 can be used to constrain the operation of one or more large language models. Examples of web content 115 that can be extracted and used by thread classification prompt generator 104 and/or plan execution prompt generator 112 to constrain the operations of a large language model include user experience, interests, areas of expertise, educational history, job titles, skills, job history, etc., as well as similar information related to other types of entities, e.g., new articles related to a company associated with a job posting, etc. For example, a negative news article about a company, e.g., an article that discusses recent layoffs, can be used to exclude that company from a plan execution prompt. Similarly, if a company's web page mentions the company's involvement in an emerging technology that matches the user's interests, that company can be included in a plan execution prompt. See par 108 - Functional component 192 maps the received thread classification (alone or in combination with data extracted from stored thread(s) or retrieved thread context) to a plan identifier and then retrieves the plan template that matches the plan identifier. In some implementations, functional component uses portions of stored thread(s) and/or retrieved thread context 186 to select a plan).
Amatriain discloses looking at skills associated with a job entity, to determine overlap between the user’s skills and skills associated with a job entity (See par 56) and also discusses including news about a “company” (See par 65).
Sabet discloses:
determining, by a processor set using a first large language model, a contextualized “score” based on contextualized metadata describing a user “and at least one additional individual” (Applicant’s [0043] as filed states “contextualized data (including metadata) refers to all information about the context where the user and/or other individuals are present, with a focus on elements that are measurable. For example, in embodiments contextual metadata may be job, role, and/or profile data including job role skill set (JRSS) coding associated with a job role, project profile data, social data, feedback data, and performance data. In embodiments, project profile data may include internal and external projects, project type… daily/weekly/monthly schedules, and/or other data that describes a user’s day-to-day efforts and tasks. In embodiments, social data may include a level of interest in a meeting, number of times speaking and/or commenting during a meeting... In embodiments, feedback data may include feedback from peers, managers, and stakeholders, including 360-degree feedback … In embodiments, performance data may include career goals, … Furthermore, as used herein, context may be a workplace, sport, activity, career, an individual’s role within a job or team, projects the individual is working on, or any other characterizations of the contextual data examples provided herein.” Sabet discloses the limitations based on broadest reasonable interpretation in light of the specification – see par 83, FIG. 2 - Classification module 204 may also generating descriptive text for each performance metrics data. In this system the descriptive text is generated using either pre-defined associations of text to performance attributes or using heuristic engines. The performance metrics data generated from the processing of responses includes a comparison of data generated for the person against data collected for a cohort of people, and wherein the performance metrics data presented is displayed in comparison to aggregate performance metrics data for the cohort of people. see par 158-161 - Benchmark the performance attributes of an individual against the information provided in the Cohort High Performance Profile to determine whether the individual has the requisite performance attributes to perform at a high level.; benchmarks score used for deciding where allocate resources on training and professional development to have the greatest impact by focusing on the attributes of performance correlated with the high achievement in the desired job duties; see par 162-167 – PAM system to consider the variety of possible attributes, and to determine which ones are key to achievement in particular role (cohort) is a unique and emergent property of the PAM systems benchmark data; Performance analytics metrics System (PAM) 210).
Amatriain discloses:
wherein the first large language model is trained using a… repository (Amatriain –see par 23 - the score associated by the model with a given task description-output pair represents a probabilistic or statistical likelihood of there being a relationship between the output and the corresponding task description in the task description-output pair. The score for a given task description-output pair is dependent upon the way the generative model has been trained and the data used to perform the model training. see par 73 - In some implementations, reinforcement learning is used to further improve the output of one or more models of directive generative thread-based user assistance system. In reinforcement learning, ground-truth examples of desired model output are paired with respective inputs, and these input-example output pairs are used to train or fine tune one or more models of directive generative thread-based user assistance system. see par 87, 89-91, FIG. 1B – computing system 140 includes thread history 142).
Amatriain discloses training and fine tuning the training (see par 23, 73) and then using vectors with large language models (See par 87, 89-91). Sabet discloses having performance data for different cohorts of people (See par 83, 92, 98).
Zheng disclose:
wherein the first large language model is trained using a “test-train” repository (Zheng -see page 449, col. 2, equation 14 – train performance predictor with cross-entropy loss as: [equation 14], where 𝑃Ψ is our performance predictor; page 449, col. 2, section 4.3.2 - given an employee 𝑒 and his/her historical learning plan L, we first use the generator to get the extended record ˆL by greedy search. And the goal of our rationality discriminator 𝐷Φ is to distinguish L and ˆL based on the employee’s information as follows: [equation 15]; see page 450, col. 1, 1st paragraph - Note that the models 𝑃Ψ and 𝐷Φ used for evaluation are all pre-trained and the parameters are fixed in the testing phase).
Amatriain, Sabet, and Zheng disclose:
and wherein the contextualized metadata is embedded into vectors within the first large language model and compared against the at least one additional individual (Amatriain –see par 23 - the score associated by the model with a given task description-output pair represents a probabilistic or statistical likelihood of there being a relationship between the output and the corresponding task description in the task description-output pair; see par 73 - In reinforcement learning, ground-truth examples of desired model output are paired with respective inputs, and these input-example output pairs are used to train or fine tune one or more models of directive generative thread-based user assistance system. see par 87, 89-91, FIG. 1B – computing system 140 includes thread history 142; Match or matching as used herein may refer to an exact match or an approximate match, e.g., a match based on a computation of similarity between two pieces of data. Other approaches that can be used to determine similarity between or among pieces of data include … neural network-based vectorization techniques such as WORD2VEC. In some implementations, generative language models, such as large language models, are used to determine similarity of pieces of data; see par 196 - Examples of concepts include topics, industries, and skills. The knowledge graph 534 can be used to generate and export content and entity-level embeddings that can be used to discover or infer new interrelationships between entities and/or concepts, which then can be used to identify related entities. As with other portions of entity graph 532, knowledge graph 534 can be used to compute various types of relationship weights, affinity scores, similarity measurements, and/or statistical correlations between or among entities and/or concepts;
see also Zheng –see page 445, col. 1, last paragraph – inspired by above methods using large language models, we design learning plan generation with a generator and two reward models; see page 445, col. 2, Table 2 – Mathematical notations – f = feature vector of an employee; history record of employee; see page 446, section 3.2, 2nd paragraph – for each employee, we can formulate his/her historical record with f as “profile feature vector” and p as “annual work performance”; See page 446, Section 4.1.1 – Employee Encoder Given an employee 𝑒 with the corresponding profile feature f which is a multi-hot vector, we use a simple yet effective MLP layer as the employee encoder to get the employee embedding as MLP (f), where e is the employee embedding corresponding to employee 𝑒.; see FIG. 2, overview of GLAD Framework is Generative Learning Plan Recommendation for Employees).
Amatriain and Sabet and Zheng disclose:
training, by the processor set, a second large language model using historical parameters as training data, wherein model parameters of the second large language model are updated… based on the historical parameters (Amatriain see par 41 - The directive generative thread-based user assistance system 102 of FIG. 1A includes a thread classification prompt generator 104, a first large language model 108, a thread label generator 109, a plan execution prompt generator 112, and a second large language model 116; see par 48-49 – entity graph 103 represents users, content items (e.g. user profiles)…skills; portions of entity graph 103 can be updated from time to time based on changes and updates to stored data, e.g. in response to updates to entity data and/or activity data. see par 79 - In some implementations, second large language model 116 and first large language model 108 have the same model architecture but are trained differently. For example, in some implementations, first large language model 108 is trained on a large dataset of digital content while second large language model 116 is pre-trained on the same large dataset but then tuned for a specific task type, such as job searching, resume generation, interview preparation, etc; see par 80 - second large language model 116 machine-generates and outputs machine-generated thread portion 134. Examples of machine-generated thread portion 134 include natural language text and/or multi-model content, such as conversational questions, job recommendations including links to relevant job postings, personalized task lists that are customized based on thread context data)
Zhang discloses using a loss function:
training, by the processor set, a second large language model using historical parameters as training data, wherein model parameters of the second large language model are updated “using a loss function” based on the historical parameters are (See Zhang page 446, FIG. 2 – overview of GLAD framework includes “RL Loss” in bottom left between Reward and feeing back into “Learning Plan Generator”; see page 448, Section 4.3.1 – Training for Performance predictor – to ensure correctness of knowledge learned, we utilize data of employees with “excellence” performance and “to be improved” in training process for performance predictor, and train performance predictor with cross-entropy loss [see equation 14]; page 449, col. 1 –equation 18 – loss function for RL (reinforcement learning) training);
Amatriain and Sabet and Zheng disclose:
determining, by the processor set using the second large language model (Amatriain – see par 41 - The directive generative thread-based user assistance system 102 of FIG. 1A includes a thread classification prompt generator 104, a first large language model 108, a thread label generator 109, a plan execution prompt generator 112, and a second large language model 116), a personalized score by comparing personal parameters describing the user against the historical parameters Applicant’s [0045] as filed states “user performance module 210 may be further configured to determine, using a second LLM built using a neural network framework such as a second RNN, a personalized score by comparing personal parameters (i.e., dimensions) describing the user against historical parameters”. Applicant’s [0046] as filed states “personal parameters may include one or more of interaction data, communication data, skills data, habit data, and/or tracked data/status. Interaction data may include coaching, mentoring, learning, volunteering, participation in community activities, innovating, inventing, patenting, and the like, with each interaction data being quantified (e.g., how often, how many, feedback score, diversity, etc.).” Applicant’s [0047] as filed states “Historical parameters refer to existing data and benchmarks used to measure and assess performance of individuals within the same context as the personal parameters. For example, if a personal parameter measures a user’s ability to communicate in a large group setting, the historical parameters provide a benchmark for other individuals and their ability to communicate in large group settings. In another example, if the personal parameter describes a user’s number of vacation days taken per year, the corresponding historical parameter provides a benchmark for other similar individuals and their number of vacation days taken per year..)
Amatriain –see par 31 - automatically generate suggestions of new skills for the user to develop to advance their career; see par 80 - second large language model 116 machine-generates and outputs machine-generated thread portion 134. Examples of machine-generated thread portion 134 include natural language text and/or multi-model content, such as conversational questions, job recommendations including links to relevant job postings, personalized task lists that are customized based on thread context data, personalized job assessments that are customized based on thread context data, push notifications, pull notifications, etc; see par 154 - The system-generated thread portion 317 of user interface 315 also includes a user-personalized job assessment 319. The user-personalized job assessment 318 is machine-generated and output by a large language model based on a plan execution prompt supplied to the large language model that contains an instruction to compare the user's experience to the job descriptions associated with the job titles mentioned in the question 311.
Sabet – see par 91 - Data collected through the Input Modules can be used to build out each person's profile and classify a person into system templates for Cohorts; A person profile may relate to a number of different cohorts—a one to many relationships. see par 158-161 - deciding where to allocate resources on training and professional development to have the greatest impact by focusing on the attributes of performance correlated with the high achievement in the desired job duties; par 246 - Collection of data that is available for performance development resources for a cohort and processing recommended actions or goals of all people in a cohort to allocate available performance development resources based upon the processed recommended actions or goals is done using PAM. The resources are allocated in a manner to achieve the greatest impact on performance of a person or of the cohort based on performance data collected for persons identified having high performance indicators) wherein the personal parameters are weighted based on correlation to user performance associated with a task (Amatriain – see par 193, FIG. 5 - For instance, as described in more detail with reference to FIG. 6, entity graph 532 and/or knowledge graph 534 can be used to compute various types of relationship weights, affinity scores, similarity measurements, and/or statistics between, among, or relating to entities.
see also Zheng – see page 446, col. 2, section 4 - the performance predictor and the rationality discriminator are regarded as two reward models, and the weighted sum of the outputs from these two models are leveraged by the reinforcement learning method to train the generator following the GAN paradigm; section 4.1 - the performance predictor and the discriminator are the two reward models in our RL-based framework. In this paper, the input and output formats of these two models are completely identical. Specifically, the input is the profile features and the learning record of a specific employee, and the output is a real number between 0 and 1. Therefore, we use the same architecture to build these two models, but the parameters and functionalities of them are completely different. see page 448, col. 2, section 4.3. 1 - The first step of the entire training process of our model is to train the performance predictor. As we mentioned before, the performance predictor aims to capture the relationship between course learning and work performance; We set the label 𝑙 as 1 for "excellence" and 0 for "to be improved", and we train our performance predictor with cross-entropy loss as: [see equation 14]; After the training process is completed, the parameters in 𝑃Ψ will be fixed, and 𝑃Ψ will be used as a reward model to teach the generator how to help employees improve their work performance;
Sabet see par 157 - If the number of people in the entire cohort having a high score in that attribute is only 55%, but the number of people in the high performing cohort having a high score for that attribute is 90%, then through displaying this, the system may reveal to a person of the system that this particular attribute is a critical performance attribute for ABC Company sales people. see par 252 - using system templates to generate timely, accurate, relevant and actionable performance metrics that workforce participants can use to guide them towards better career outcomes);
determining, by the processor set, an individual benchmark based on a normalized aggregation of the contextualized score and the personalized score (Amatriain discloses the limitations based on broadest reasonable interpretation in light of the specification – See par 141 - The technologies described herein evaluate entities related to the user (e.g., people in the user's network, skills that the user has and/or does not have, comparison of the user's resume to the job requirements, etc.) and generate a prompt that causes the large language model 206 to create a personalized strategic task list to assist the user with the overall job seeking process as opposed to the process of pursuing a specific identified job opportunity. see par 169 - the plan execution prompt instructs the large language model to generate and output recommended next steps based on the current state of the user's job search (determined, e.g., based on the thread history), the match between the user's skills and the requirements of the job, and potentially other information obtained from one or more contextual resources; see par 238, FIG. 7 – operation 704 uses 1st large language model for thread as in FIG. 1A, 104; see par 245, FIG. 7 – steps 710-714 from second large language model in response to “1st thread” for task recommendations;
Sabet and Zheng disclose:
determining, by the processor set, an individual benchmark based on a normalized aggregation of the contextualized score and the personalized score (Applicant’s [0048] as filed states “In accordance with aspects of the invention, root cause analysis module 215 is configured to determine an individual benchmark based on an aggregation of the contextualized score and the personalized score. That is, root cause analysis module 215 may be configured to break down the contextualized score and the personalized score into each dimension and may optionally offer an in-depth report of the user’s performance””
Sabet - See par 130 - FIG. 12 shows an example of how different activity types may be generated by the PAM. From the results of the Classification Modules, the Analysis Module 206 may generate an activity map showing the person how he spends his time among different kinds of activity types. From the results of the Classification Modules, the Analysis Module 206 may generate various benchmark dashboards showing where the person's profile stands as compared with others within selected cohorts. As an example, the person can choose to see a benchmark dashboard of how his salary compares with other Finance Directors or how his title compares with others in medium sized technology companies with 10 years of experience; see par 145 - In addition, because the system has activity map results aggregated over many persons of the same cohort type, a Display Module 210 can also display a comparison dashboard showing a particular person's activity map as compared with an aggregated map of a particular cohort type. This kind of display allows a person not only to see how he has been allocating his time, but also how his time allocation compares against the average for a certain type of person profile; see par 157 - After constructing a Cohort High Performance Profile, the system may determine which performance attributes are more commonly present in high performers within the cohort as compared with the cohort overall—this defines a set of key performance attributes for this cohort. For example, it may be that 90% of the cohort “salespeople at ABC Company” who are determined to be high performers are consistently rated in the system with a high score the particular attribute of “listening”. See par 158- Benchmark the performance attributes of an individual against the information provided in the Cohort High Performance Profile to determine whether the individual has the requisite performance attributes to perform at a high level; see par 212 - Natural Language Processing, where the system uses available natural language processing techniques and algorithms to analyze the goal and suggest most likely mappings.)
see also Zheng see page 6, FIG. 2 – RG-Transformer in lower right uses “Layer norm; col. 1, 1st paragraph – use residual connection and layer normalization mechanisms to fuse contextual and original representation… to get a course embedding by a Self-Fusion layer with gating mechanism ;
determining, by the processor set, an industry benchmark based on historical industry benchmarks (Amatriain – see par 53 - thread classification prompt generator 104 may determine, based on a search of entity graph 103, an industry associated with a particular type of job (e.g., tech, healthcare, sales, etc.) or a geographic region associated with a job posting, and then select a thread classification prompt template based on that industry or geographic region. see par 57 - For instance, plan execution prompt generator 112 may determine, based on a search of entity graph 103, an industry associated with a particular type of job (e.g., tech, healthcare, sales, etc.) or a geographic region associated with a job posting, and then select a plan execution prompt template based on that industry or geographic region (e.g., to draft a resume appropriate for a particular industry or geographic region). See par 58 - For instance, if the plan execution prompt generator 112 selects a plan execution prompt template containing instructions to generate a resume for the software industry, the plan execution prompt generator 112 can use entity graph 103, 105 to extract relevant skills from the user's profile and include those skills in the plan execution prompt. see also Sabet – see par 40 - for example, a cohort might be people in a sales role selling software. A cohort might be a group of people within a company, across companies, across industries, or another combination of people. see par 124 - Performance management 844 sub function calculates the performance of the person, group or company based on metrics. Dependency grouping sub function 846 provides the function to group various persons and functionalities. For example, a sales team within a particular director can be grouped and metrics tracked. Heuristic engine sub function 848 uses hysteresis 836 and real-time analysis 834 to provide intelligence. It correlates with past data, similar persons across market, similar groups across company, similar companies across industry and similar persons across the company. see par 232 - FIG. 18 allows the person to compare their performance against the industry average for various activities and displays it in a nutshell as graphical display); and
generating, by the processor set, an objective roadmap for the user based on the individual benchmark and the industry benchmark, the objective roadmap comprising first actions for improvement that are generated (Applicant’s [0046] as filed states “Tracked data/status may include any data (e.g., status data) that the user enters as related to trackable tasks (e.g., goals, steps, roadmap stages, etc.) and/or a level of completion of such tasks.” Applicant’s [0050] as filed states “user may perform/complete to achieve a desired improvement. In embodiments, the roadmap may include a plurality of actions to be performed or completed in a sequential order to achieve the desired improvement. For example, a roadmap may comprise sequential steps that build on one another, such as perform task 1, then complete action 2, and finish action 3
Amatriain – see par 31 - For instance, a user assistance system configured with the disclosed technologies can automatically generate job recommendations based on the user's goals, skills, experience, and preferences, automatically generate comparative insights between multiple jobs based on the user's preferences, automatically generate suggestions of new skills for the user to develop to advance their career. See par 141 - large language model 206 to create a personalized strategic task list to assist the user with the overall job seeking process as opposed to the process of pursuing a specific identified job opportunity. In other embodiments, examples of the contextual task list 220 include lists of tasks to help the user accomplish another type of goal or objective, such as … managing a project, or organizing a to-do list; see par 180 - the job-specific task lists that are machine-generated using the disclosed technologies are specific to each user-job pair. For example, if the same user applies to two different jobs, that user's job-specific task lists will be different for each job in that the tasks included in the task list may be different and/or the order in which the tasks are ranked may be different. See par 224 - In the example of FIG. 6, entity graph 600 includes entity nodes, which represent entities, such as content item nodes (e.g., Post U21, Article 1), user nodes (e.g., User 1, User 2, User 3, User 4), and job nodes (e.g., Job 1, Job 2). Entity graph 600 also includes attribute nodes, which represent attributes (e.g., job title data, article title data, skill data, topic data) of entities. Examples of attribute nodes include title nodes (e.g., Title U1, Title A1), company nodes (e.g., Company 1), topic nodes (Topic 1, Topic 2), and skill nodes (e.g., Skill A1, Skill U11, Skill U31, Skill U41) by measuring a first distance between a first tracked status, the individual benchmark, and the industry benchmark, wherein the first tracked status is automatically received from a user device based on user interactions with one or more applications (Amatriain – see FIG. 1A – showing user devices 118; se par 172 – examples from user devices of desktop, laptop, tablet, phones; see par 158 - For example, a call is made to a social network service to determine how many connections the user has that work at the company that posted the job. User interface 330 also includes an assess user interface control mechanism 334 and a set of user-selectable options 312. The set of user-selectable options is updated dynamically as the thread classification changes. see par 170 - The task list 382 includes tasks 383 and 386. Each task 383 has a task description (e.g., task description 385), and a check box 384. The task list 382 is generated, using the disclosed technologies, for example by applying a large language model to a plan execution prompt that instructs the large language model to generate a user-personalized task list for applying to the job posting 381. For example, the plan execution prompt instructs the large language model to generate, output, and rank or prioritize recommended next steps based on the current state of the user's job search (determined, e.g., based on the thread history), the match between the user's skills and the requirements of the job, and potentially other information obtained from one or more contextual resources. For example, the system has obtained information from a social network service that indicates that the user has a connection that works at the company who posted the job 381
Sabet – see par 161 - Deciding where to allocate resources on training and professional development to have the greatest impact by focusing on the attributes of performance correlated with the high achievement in the desired job duties; see par 203-206 - Example: A manager of product marketing with 5 years of job experience wants to become a VP of Products at a Fortune 500 company. PAM shows that not only the typical performance profile of a VP of Products at a Fortune 500 company, encompassing the competencies and skills of such a PAM, but also evaluate the cohort of VP of Products at Fortune 500 companies who in the past were a manager of Product Marketing, and then show the path between those two roles: [0204] How long it took? [0205] What the skill profile was like at the Manager level, and how it changed on the way to the VP of Products title? [0206] How many intermediate promotions or titles existed); see par 210-212 - Management Module 212 may map a goal to a set of analysis results. As an example, a goal of “75 percentile of Cohort X” may be mapped to a calculated performance dashboard for Cohort X. In performing this mapping function, the system may use one or a combination of techniques, including: Natural Language Processing, where the system uses available natural language processing techniques and algorithms to analyze the goal and suggest most likely mappings); and
dynamically updating, by the processor set, the objective roadmap in response to updates to the first tracked status (Amatriain -see par 174 - The figures FIG. 4A and FIG. 4B, See par 180 - the disclosed technologies are capable of determining how well the user's background, skills, experiences, and preferences match each particular job; See par 195 - Portions of entity graph 532, 534 can be automatically re-generated or updated from time to time based on changes and updates to the stored data, e.g., updates to entity data and/or activity data. See par 227 - combinations of nodes and edges are used to compute various scores, and those scores are used by various components of the directive generative thread-based user assistance system to, for example…select execution plans, generate. Any one or more of the paths p1, p2, p3, p4 and/or other paths through the graph 600 (FIG. 6) can be used to compute scores that represent affinities, relationships, or statistical correlations between different nodes. a user-skill affinity score computed between User 3 and Skill U31 might be higher than the user-skill affinity score computed between User 3 and Skill U11. As another example, a job-skill affinity score computed between Job 1 and Skill U31 might be higher than a job-skill affinity score computed between Job 1 and Skill U41. See par 228 - Sub-graph E includes skills (e.g., skills that may be associated with users and/or jobs) and links involving the skills. See par 229 – sub-graphs facilitate the efficient determination of relevant thread context data that can be used for thread classification and/or plan execution).
Amatriain and Sabet and Zheng are analogous art as they are directed to analyzing natural language, aspects of a user, to help give user’s advice for improvement, e.g. jobs/careers (see Amatriain Abstract, par 22, 24, 80; Sabet Abstract, par 203, 212; Zheng See Abstract). 1) Amatriain discloses looking at skills associated with a job entity, to determine overlap between the user’s skills and skills associated with a job entity (See par 56) and also discusses including news about a “company” (See par 65). Sabet improves upon Amatriain by having performance metrics collected for cohorts of people (see par 83), generating benchmark dashboards showing where the person’s profile stands compared with others for selected cohorts (See par 130), and focusing on attributes of performance correlated with high achievement as “training” development with greatest impact (See par 161) and showing paths of skills and job titles for a goal (See par 203-206). One of ordinary skill in the art would be motivated to further include having metrics and data for many other individuals and comparing across cohorts to efficiently improve upon the recommended next steps for a user in a job search in Amatriain. 2) Amatriain discloses training and fine tuning the training (see par 23, 73) and then using vectors with large language models (See par 87, 89-91). Sabet discloses having performance data for different cohorts of people (See par 83, 92, 98). Zheng improves upon Amatriain and Sabet by testing and training the model (See page 450) and using normalization in layers for embeddings. One of ordinary skill in the art would be motivated to further include having testing and training for an LLM to efficiently improve upon the recommended next steps for a user in a job search using large language models in Amatriain and the performance data for different people in Sabet.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model assisting users with plans and job goals, customized job lists, and affinities between skills and jobs in Amatriain (Abstract, par 65, 80, FIG. 3A, FIG. 6) to further calculate performance metrics for cohorts of people, compare person’s profile to others (par 83, 130), and show paths of skills and job titles for a goal (See par 203-206) as disclosed in Sabet, and to further use testing and training for reinforcement learning for learning plans for users as disclosed in Zheng, since the claimed invention is merely a combination of old elements, and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable and there is a reasonable expectation of success.
Concerning independent claim 10, Amatriain and Sabet and Zheng disclose:
A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable (Amatriain – see par 41 - In the example of FIG. 1A, an example computing system 100 is shown, which includes an example directive generative thread-based user assistance system 102 and an example thread-based user assistance interface 118. See par 255 - In FIG. 8, an example machine of a computer system 800 is shown, within which a set of instructions for causing the machine to perform any of the methodologies discussed herein can be executed. In some embodiments, the computer system 800 can correspond to a component of a networked computer system (e.g., as a component of the computing system 100 of FIG. 1A or the computer system 500 of FIG. 5) that includes, is coupled to, or utilizes a machine to execute an operating system to perform operations corresponding to one or more components of the directive generative thread-based user assistance system 102 of FIG. 1A; see par 267 - The term “machine-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any of the methodologies of the present disclosure) to:
The remaining limitations are similar to claim 1 above. It would be obvious to combine Amatriain and Sabet and Zheng for the same reasons as claim 1.
Concerning independent claim 17, Amatriain and Sabet and Zheng disclose:
A system comprising:
a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to: (Amatriain – see par 41 - In the example of FIG. 1A, an example computing system 100 is shown, which includes an example directive generative thread-based user assistance system 102 and an example thread-based user assistance interface 118. See par 255 - In FIG. 8, an example machine of a computer system 800 is shown, within which a set of instructions for causing the machine to perform any of the methodologies discussed herein can be executed. In some embodiments, the computer system 800 can correspond to a component of a networked computer system (e.g., as a component of the computing system 100 of FIG. 1A or the computer system 500 of FIG. 5) that includes, is coupled to, or utilizes a machine to execute an operating system to perform operations corresponding to one or more components of the directive generative thread-based user assistance system 102 of FIG. 1A; see par 267 - The term “machine-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any of the methodologies of the present disclosure).
The remaining limitations are similar to claim 1 above. It would be obvious to combine Amatriain and Sabet and Zheng for the same reasons as claim 1.
Concerning claims 2, 11, and 18, Amatriain and Sabet disclose:
The computer-implemented method of claim 1, further comprising receiving the contextualized metadata describing the user and the at least one additional individual from an external device or storage medium (Amatriain – see par 66 - In the example of FIG. 1A, thread context data are received via one or more user devices or systems, such as portable user devices like smartphones, wearable devices, tablet computers, or laptops, one or more web servers, and/or one or more database servers; see also Sabet par 81, Fig. 1 - Particularly, the PAM is supported by server 104, computing devices 106, 104, 108, and 112 (some, such as device 112, which may be mobile devices), a network 102, a database 110. A network 102 may be a local area network (LAN), wide area network (WAN), metropolitan area network (MAN), extranet, intranet, internet, peer-to-peer network or the like or a combination thereof. ).
Concerning claims 3, 12, and 19, Amatriain and Sabet discloses:
The computer-implemented method of claim 1, further comprising:
determining a completion of at least one of the first actions for improvement as the user performs the first actions (Amatriain – see par 170, FIG. 3V - The task list 382 includes tasks 383 and 386. Each task 383 has a task description (e.g., task description 385), and a check box 384 – “ask for referral”, then in 386 “update your profile – highlight your expertise with updates to your headline, summary, and skills”;
see also Sabet – see par 35 – progress against goals; see par 108 - . Individual CSAT allows the solution to gauge company resource pulse, and leads to targeted training goals. See par 201-206 – identify career trajectory; current skills and job titles when looking at a goal of a new job title/role; show path and skill profile and how it changed on the way to the VP of Products; See par 246 - Collection of data that is available for performance development resources for a cohort and processing recommended actions or goals of all people in a cohort to allocate available performance development resources based upon the processed recommended actions or goals is done using PAM. The resources are allocated in a manner to achieve the greatest impact on performance of a person or of the cohort based on performance data collected for persons identified having high performance indicators).
Amatriain disclose having a current state of a user’s job search and current user skills (See par 170) and having a checkmark to show completion for updating a profile with updates to skills (See FIG. 3V, 386). Sabet discloses having “progress against goals”, targeted training goals (See par 35), and by aggregating data, identifying a typical career trajectory and skills and timelines to obtaining career objectives (See par 44) and showing a path and skills profile for how it changes on the way to the VP title (See par 201-206).
Zheng discloses:
determining a second tracked status based on the completion of at least one of the first actions (Zheng page 447, FIG. 2 – “next” recommended course; see page 448, col. 1, last paragraph – use an MLP layer to get course generation probability (equation 12), to prevent generator from generating repeated courses)
It would be obvious to combine Amatriain and Sabet and Zheng for the same reasons as claim 1. Amatriain disclose having a current state of a user’s job search and current user skills (See par 170) and having a checkmark to show completion for updating a profile with updates to skills (See FIG. 3V, 386). Sabet discloses having “progress against goals”, targeted training goals (See par 35), and by aggregating data, identifying a typical career trajectory and skills and timelines to obtaining career objectives (See par 44) and showing a path and skills profile for how it changes on the way to the VP title (See par 201-206). Zheng improves upon Amatriain and Sabet by generating a “next” recommended course and preventing “repeating” any course.
Concerning claims 4, 13, and 20, Amatriain, Sabet, and Zheng disclose:
The computer-implemented method of claim 3, wherein the roadmap further comprises second actions for improvement that are generated by measuring a second distance between the second status, a second individual benchmark, and the industry benchmark (Amatriain – see par 49, 195 - Portions of entity graph 103 can be automatically re-generated or updated from time to time based on changes and updates to the stored data, e.g., in response to updates to entity data and/or activity data. see par 60 - Examples of recommendation systems include machine learning models that have been trained based on historical data to score user-entity pairs, rank the user-entity pairs based on the scores, and select one or more of the top ranking user-entity pairs to formulate and output a user recommendation; see par 222 - For example, the entity graph 600 is updated in response to updates of user profiles, … the creation and distribution of new content items, such as messages, posts, articles, comments, and shares. As another example, the entity graph 600 is updated as new computations are computed, for example, as new relationships between nodes are created based on statistical correlations or machine learning model output).
Concerning claims 5, 14, Amatriain, Sabet, and Zheng disclose:
The computer-implemented method of claim 3, wherein at least one of the first actions for improvement and the second actions for improvement are generated using a machine learning model (Amatriain – see par 60 - Examples of recommendation systems include machine learning models that have been trained based on historical data to score user-entity pairs, rank the user-entity pairs based on the scores, and select one or more of the top ranking user-entity pairs to formulate and output a user recommendation. Examples of data obtained from recommendation systems include user connection recommendations and job recommendations (e.g., people you may know, jobs you may be interested in); see par 79 - Second large language model 116 includes one or more neural network-based machine learning models, such as any of the types of models described above with reference to first large language model 108. see par 222 - . For example, the entity graph 600 is updated in response to updates of user profiles, the creation or deletion of user connections with other users, and the creation and distribution of new content items, such as messages, posts, articles, comments, and shares. As another example, the entity graph 600 is updated as new computations are computed, for example, as new relationships between nodes are created based on statistical correlations or machine learning model output;
Zheng – see page 447, FIG. 2 – uses Reinforcement Learning; see page 452, col. 2, section 6 - we proposed an reinforcement learning based
generative recommendation framework, namely GLAD, which can
generate personalized learning plans for employees by imitative
learning from the behaviors of different employees).
It would be obvious to combine Amatriain and Sabet and Zheng for the same reasons as claim 3 and 4.
Concerning claims 6 and 15, Amatriain, Sabet, and Zheng disclose:
The computer-implemented method of claim 5, further comprising training the
machine learning model using the second status, the second individual benchmark, and the industry benchmark as inputs (Amatriain – see par 23 - The score associated by the model with a given task description-output pair represents a probabilistic or statistical likelihood of there being a relationship between the output and the corresponding task description in the task description-output pair. The score for a given task description-output pair is dependent upon the way the generative model has been trained and the data used to perform the model training. The generative model can sort the task description-output pairs by score and output only the pair or pairs with the top scores. See par 24 - A large language model (LLM) is a type of generative language model that is trained in an unsupervised way on massive amounts of unlabeled data, such as publicly available texts extracted from the Internet, using deep learning techniques. See par 205 - when a user creates a thread portion via directive generative thread-based user assistance system 580, or reacts to a system-generated thread portion received from directive generative thread-based user assistance system 580, event logging service 570 stores the corresponding event data in a log. Event logging service 570 generates a data stream that includes a record of real-time event data for each user interface event that has occurred. Event data logged by event logging service 570 can be pre-processed and anonymized as needed so that it can be used, for example, to generate relationship weights, affinity scores, similarity measurements, and/or to formulate training data for artificial intelligence models. see par 208 - LLM data store 560 stores data that can be used to configure, train or tune one or more large language models of the directive generative thread-based user assistance system 580).
It would be obvious to combine Amatriain and Sabet and Zheng for the same reasons as claim 3 and 4.
Concerning claim 7, Amatriain and Sabet disclose:
The computer-implemented method of claim 1, wherein the contextualized metadata describing the user and the at least one additional individual comprise a same context (Amatriain – see par 193 - , as described in more detail with reference to FIG. 6, entity graph 532 and/or knowledge graph 534 can be used to compute various types of relationship weights, affinity scores, similarity measurements, and/or statistics between, among, or relating to entities. See par 205 - Event data logged by event logging service 570 can be pre-processed and anonymized as needed so that it can be used, for example, to generate relationship weights, affinity scores, similarity measurements, and/or to formulate training data for artificial intelligence models. see par 196 - discover or infer new interrelationships between entities and/or concepts, which then can be used to identify related entities. As with other portions of entity graph 532, knowledge graph 534 can be used to compute various types of relationship weights, affinity scores, similarity measurements, and/or statistical correlations between or among entities and/or concepts. See par 220 - Nodes can be weighted based on, for example, similarity with other nodes, edge counts, or other types of computations, and edges can be weighted based on, for example, affinities, relationships, activities, similarities, or commonalities between the nodes connected by the edges, such as common attribute values (e.g., two users have the same job title or employer, or two users are n-degree connections in a user connection network, where n is a positive integer);
see also Zhen page 444, col. 1, 3rd paragraph - Therefore, the model needs to take both the job context of employees and the mutual correlation between courses into consideration, and generate appropriate personalized learning plans; see page 446, col. 2, section 4.1.2, last paragraph – capture contextual representation of course title T).
Concerning claims 8 and 16, Examiner notes that the “names” of data as the various categories here (interactions, communications, habits, and actions) have no functional relationship with the other operations (how do these change the score if at all?), and a result, are not entitled to patentable weight (See MPEP 2111.05), similar to conveying a message to a human reader independent of the computer system. Nonetheless, art is applied for compact prosecution purposes; Amatriain and Sabet disclose:
The computer-implemented method of claim 1, wherein the personal parameters describing the user comprise past interactions, communications, habits, and data of the user entered related to trackable tasks of the user (Examiner notes this claim covers a listing of alternative descriptions of what the parameters represents; the last of which is “data of the user entered related to trackable tasks of the user”
[0046] as filed states “In embodiments, personal parameters may include one or more of interaction data, communication data, skills data, habit data, and/or tracked data/status. [1] Interaction data may include coaching, mentoring, learning, volunteering, participation in community activities, innovating, inventing, patenting, and the like, with each interaction data being quantified (e.g., how often, how many, feedback score, diversity, etc.). [2] Communication data may include mail or email correspondences, social media interactions, classes taken or taught, training taken or taught, webinars attended or given, speaking engagements, and the like, with each communication data being quantified (e.g., how often, how many, feedback, etc.). [3] Habit data may include vacation (e.g., duration and times), working time (e.g., regional and/or global), time zone, business travel, health habits such as being a part of professional well-being communities, breaks between meetings (e.g., going for a walk, consuming a refreshment, or other activities to unwind and/or reset). [4] Tracked data/status may include any data (e.g., status data) that the user enters as related to trackable tasks (e.g., goals, steps, roadmap stages, etc.) and/or a level of completion of such tasks.”
Amatriain discloses the limitations based on broadest reasonable interpretation in light of the specification –
[1] interactions - see par 59 - Examples of retrieved data 130 include online dialog history 113; see par 63 - Online dialog history 113 includes historical threads and thread portions associated with an online dialog involving a particular user. For instance, in some implementations, a text file is created to store the online dialog history 113 and is updated each time a new thread or thread portion is added to the online dialog, such that the text file contains the entire dialog history involving the user. see par 64 - Data retrieved from online dialog history can be used to constrain the operation of one or more large language models;
(2) communications – Amatriain - See par 204 - Examples of network activity data include thread creations, thread edits, thread views, page loads, clicks on messages or graphical user interface control elements, the creation, editing, sending, and viewing of messages, and social action data such as likes, shares, comments, and social reactions (e.g., “insightful,” “curious,” etc.).
(3) habits – Sabet – see par 92 - , from calendar data, the Classification Module 204 may calculate how much time the person spends in meetings versus working alone. system may assign a likelihood that each specific calendar activity falls into a more general activity type. Activity types might include, without limitation: travel time, 1-on-1 meetings, group meetings, presentations, training, social event, customer meeting, support call, individual working session or conference. From this analysis, the system can generate an activity map for the person showing the person how he spends his time among these different kinds of activity types.
(4) action of the user Amatriain – see par 141 - The contextual task list 220 can be generated based on a target entity or based on a more generalized intent, objective, or goal of the user. For example, in the jobs context, the contextual task list 220 can be configured as tool to help the user plan their career, job search; see par 204 - Examples of network activity data include thread creations, thread edits, thread views, page loads, clicks on messages or graphical user interface control elements, the creation, editing, sending, and viewing of messages, and social action data such as likes, shares, comments, and social reactions (e.g., “insightful,” “curious,” etc.). see par 205 - Event data logged by event logging service 570 can be pre-processed and anonymized as needed so that it can be used, for example, to generate relationship weights, affinity scores, similarity measurements, and/or to formulate training data for artificial intelligence models; see also Sabet par 35 – progress against goals).
It would be obvious to combine Amatriain and Sabet for the same reasons as claim 1.
Concerning claims 9, Amatriain and Sabet disclose:
The computer-implemented method of claim 1, wherein the first large language model comprises a first recurrent neural network and the second large language model comprises a second recurrent neural network (Amatriain – see par 71 - In some examples, the neural network-based machine learning model architecture includes or is based on one or more generative transformer models, one or more generative pre-trained transformer (GPT) models, one or more bidirectional encoder representations from transformers (BERT) models, one or more large language models (LLMs), one or more XLNet models, and/or one or more other natural language processing (NL) models. In some examples, the neural network-based machine learning model architecture includes or is based on one or more predictive text neural models that can receive text input and generate one or more outputs based on processing the text with one or more neural network models. Examples of predictive neural models include, but are not limited to, Generative Pre-Trained Transformers (GPT), BERT, and/or Recurrent Neural Networks (RNNs).
Response to Arguments
Applicant’s arguments of 12/15/25 have been considered but are not persuasive and/or moot over the revised rejections.
With regards to 101, Applicant argues that claim 1 is eligible because of a technical solution by “continuously tracks user status and dynamically updates the roadmap based on the tracked status.” Remarks, page 10. In response, Examiner respectfully disagrees. Applicant’s arguments are difficult to follow because most of them are not connected to the claims. It appears this argument is referring to new limitation, 2nd from the end. However, as stated in the revised rejection, this limitation is viewed as just gathering more data on the user status; the additional element of using a computer to do so, with tracked data from any software/application, is viewed as “apply it [abstract idea] on a computer” (MPEP 2106.05f) and “field of use” (MPEP 2106.05h).
Applicant then quotes Specification [0015] – which has many things not in the claims (e.g. billions of weights; backpropagation). It is unclear how this is helpful to the claims as they don’t require these aspects. Applicant then quotes Specification [0061] in reference to new limitation of “loss function”. This is addressed in the new rejection – scoring the parameters based on the loss/error relative to historical parameters. Examiner notes that claim does not include the activation functions to help LLM learn complex relationships.
In light of the arguments, the claims here are still viewed as ineligible even in view of Enfish and Desjardins. See MPEP 2106.04(d)(1) “The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.” Here, just having the model already trained using test-train repository is not sufficient detail to be eligible. There is no further explanation in the specification, other than this is the general kind of training for the model [0052] – “In embodiments a machine learning model may be employed to generate the first actions for improvement, the individual tracked status, individual benchmark, and the industry benchmark may be used as inputs to help train the machine learning model to generate the first actions”; [0059] “the first LLM/RNN uses a test-train repository where the contextualized metadata is embedded into vectors within the LLM/RNN and is compared with/against other individuals within the same contexts… In such embodiments, a contextualized score that is less than 30 indicates that the user is not connected to their professional environment, a contextualized score that is more than 70 indicates that the user is on a career path promotion, and a contextualized score between 30-70 indicates that the user is connected to their environment and is stable in their career progression.”
The 103 arguments are moot in light of the new rejections.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IVAN R GOLDBERG whose telephone number is (571)270-7949. The examiner can normally be reached 830AM - 430PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Coupe can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IVAN R GOLDBERG/Primary Examiner, Art Unit 3619
Prosecution Insights