Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Non-Final action is in reply to the RCE filed 12/23/2025.
Claims 1, 8, 11, 18, 25 have been amended.
Claims 6, 10, 16, and 20-24 were previously canceled.
Claims 1-5, 7-9, 11-15, 17-19 and 25 are pending.
Request for Continued Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/23/2025 has been entered.
Response to Arguments/Amendments
Applicant’s amendment to claims 1 and 25 overcome the prior 35USC 112(b) rejection, it has been withdrawn.
With respect to the 35USC 103 rejection, applicant argues the amended claim limitations, however the limitations were not previously presented nor applied against the prior art. Examiner has modified the rejection to further explain how the claims are being interpreted and addressed each of applicant’s claim limitations as noted below in this Non-Final action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained through the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, 7-9, 11-15, 17-19 and 25 are rejected under 35 USC 103(a) as being unpatentable over Jungmeisteris et al., US Patent Application Publication No US 2022/0374956 A1, Thibodeaux et al., US Patent Application No US US2019/0370719 A1 in further view of Cella (WO 2024/186954 A2).
With respect to Claims 1 and 11,
Jungmeisteris discloses,
extracting subject matter items, from one or more webpages of one or more internet platforms (Abstract: “A text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. A user's freeform text input is analyzed using machine learning algorithms to derive the meaning of the input text as well as to determine the user sentiment expressed therein”; ¶18: “The intercept survey is a freeform text survey present during the user's interaction with a computer system (e.g., on a website or application), presented before the initialization of a help ticket and without the need for default or generic questions in a post-event survey”; ¶20: “the sentiment analysis system is capable of performing semantic parsing to capture the meaning of the input text. The sentiment analysis system includes a pre-trained natural language processing (NLP) model capable of generating vector representations of the input text. In an exemplary embodiment, the model is capable of generating a series of vectors (each corresponding to a word) so as to be representative of a sentence, or another measurement of text”; ¶41: “memory 210 may also, in one embodiment, include communication logic 224, including one or more algorithms or models for obtaining information from or communicating information with web server 140, database 250 (or other external or third-party databases), and/or via network 130 (FIG. 1)”; Fig 6, #602 “Pretrain language model, NLP topic classification model, and sentiment analysis model on exemplary chat data”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like”)
associating analyzed collected data of the subject matter items with the one or more consultant workflows (¶17: “The questions asked in the survey may be guided by the user’s activity with the service, e.g., on an e-commerce website, the workflow followed by the user and a determination of when and how the user engaged customer support in the workflow process. In some embodiments, the questions and/or responses presented to the user via the survey or chatbot are associated with a “tone” of response, the tone of response being dictated by the derived sentiment score”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like. ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like. Therefore, each of the entries in workflow data 232 can be associated with a particular user or user device, for example, a user ID (if logged in), a device (by device ID, IP address, MAC address, or the like), session ID, other information sufficient to identify a user (such as a unique code or password), or any other appropriate identifying mechanism. In an exemplary embodiment, data regarding a user's activity is pulled from one or more of web server(s) 140 and external databases 250 to populate workflow data 232 in real-time, in response to any action by the user or any of one or more scheduled events”; ¶58: “Each of these navigational, click, search, or other interactions may be stored as workflow data 232 in association with a session ID (and/or in some cases, a user ID)”; ¶93: “system 110 may also, in step 604, function to recognize circumstances that trigger the presentation of the chatbot (or other UI) from which sentiment analysis and/or workflow analysis can be conducted. In one embodiment, this trigger may be based on a semantic analysis of text entered into a field on a website, e.g., into a messenger app or widget, search bar or other query field, etc. (typically accessed from workflow data 232). ¶98: “user sentiment, circumstantial evidence, and the availability of self-serve options may be factors that are variously weighted by one or more machine learning algorithms (e.g., linear regression models). Such algorithms may be trained on a training set of various scenarios with selected or procedurally generated combinations of circumstantial and workflow scenarios. In another embodiment, machine learning analyses may be applied to determine sentiment, but the determination of workflow may be subsequently applied to a rules-based process, e.g., where escalation or other alternate remediation is always applied in certain defined circumstances”; Fig 4B, ¶99: “based on the determinations of step 650, calculated user sentiment, and/or circumstantial data, the workflow logic 220 may present to the user different workflows”)
combining the one or more consultant workflows with the one or more client preferences (¶19: “information about the path or workflow the user took during their support journey (their activity) can be used to dynamically select a set of one or more questions to use in the user interaction”;¶48: “With reference to FIG. 2B, user data 231 may include information associated with a user, e.g., user ID, user account information (e.g., name, contact information (e.g., email address, mailing address, telephone number), date of birth, payment card information), a type of user (e.g., guest/registered, business/vacation travelers, etc.), length of membership, booking history, demographic data about the user, such as age, gender, location, language, and device information, GPS, IP address, or location data, operating system, browser, or other device data, user preferences or interests, connected third party accounts (e.g., social networks) if applicable, and the like”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like.)
tracking and updating the one or more selections of the one or more clients to update the one or more client preferences (¶17: A numeric sentiment score is derived and updated in real-time based on implicit signals of customer satisfaction obtained from a user's freeform text entry (obtained, for example, from a survey, a chatbot, an interaction with a customer service agent or an interaction with another user) and/or the users selection interaction with an intercept survey or a post-activity survey in the same channel as the users activity, as well as other relevant information. The questions asked in the survey may be guided by the user’s activity with the service, e.g., on an e-commerce website, the workflow followed by the user and a determination of when and how the user engaged customer support in the workflow process”; ¶58: “The user may also have clicked on one or more of the displayed results 330. Each of these navigational, click, search, or other interactions may be stored as workflow data 232 in association with a session ID (and/or in some cases, a user ID). FIGS. 4A-4D describe various progressive interfaces that may be displayed to the user and updated in real-time, for example through chatbot interface 340 or similar type of interface”)
memory; a processor operatively coupled with the memory, wherein the processor is configured to execute program code (¶39: “The customer support system 110 may include a memory 210. As used herein, memory 210 may refer to any suitable storage medium, either volatile and non-volatile (e.g., RAM, ROM, EPROM, EEPROM, SRAM, flash memory, disks or optical storage, magnetic storage, or any other tangible or non-transitory medium) that stores information that is accessible by a processor. Memory 210 may store instructions and data used in the systems and methods described herein. Customer support system 110 may also include a RAM or other volatile or other memory accessible by a CPU/processor 245”)
Jungmeisteris discloses all of the above limitations, Jungmeisteris does not distinctly describe the following limitations, but Thibodeaux as shown discloses,
semantically processing and analyzing text data or image data of the one or more webpages using natural language processing to determine one or more sentiments of the one or more expert third-party evaluators for the subject matter items based on analysis from the one or more third-party evaluators considering one or more client preferences and analysis of anonymous aggregate client data (Abstract: “a tiered set of questions to a subject matter expert and, based on how the subject matter expert answers the questions, and identifies a best known path for answering the questions. At a second time, the system provides the tiered set of questions to a candidate, captures how the candidate answers the questions, compares that information to the best-known path, and generates, based on that comparison, a live score as the candidate answers the questions”; ¶16: “the ability to take into account all available data, including behavioral characteristics, to score competency; (5) access to a library of curated resources to answer questions; (6) machine learning/artificial intelligence which analyzes all aspects of the candidates interaction with the adaptive competency assessment model (not just their answers to the questions); and (7) a relative scoring schema to weight job role objectives appropriately for the situation”; ¶18: “Upon organizing the questions into tiers for a given job, the SMEs for that job can be provided the tiered question set for that job. As the SMEs complete the questions, the system can record various factors about how the SMEs answer the questions. For example, rather than just collecting the correct answers for each question, the system can record information about how the SMEs answer the questions”; ¶25: “report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; ¶26: “The process may require a sample of individuals across different, predetermined competency levels. For example, for a three-level system three groups of equal size may be required: one generally underqualified for the job role, a group generally qualified for the job role, and a group over qualified. These groups can provide sufficient data to enable the machine learning/artificial intelligence algorithm to modify the ACA model for a specific deployment”; ¶32; ¶36: “each question in the tiered set of questions can have a list of associated answers, each answer in the list of answers having a weighted core competency associated with at least one job in a plurality of job roles”)
Applicant’s disclosure generally teaches on page 9, lines 28 and 29: “the sentiments determined by the aforementioned tools can include strongly negative, negative, neutral, positive or strongly positive”, and page 14, lines 3-9: “A client's 304 subject matter expertise can be determined, tracked, and compared to the sentiment from expert analysis and anonymous aggregate client preferences and selections over time. In the process of automatically tracking a client's preferences and selections, the system 302 also automatically computes the percentage match between a client's sentiment and the overall sentiment from analysis by experts in the SMA 312. The closer the client's preferences and selections match expert sentiment, the more their future choices will match expert choices”.
Jungmeisteris teaches a text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. Jungmeisteris teaches a sentiment analysis system including a pre-trained natural language processing (NLP) model capable of performing semantic parsing to capture and understand the meaning of input text, a statement, request, or question input by a user. Jungmeisteris also teaches training one or more machine learning models on a set of character string data simulating potential input text typed by a user whereby features from training data are extracted to develop one or more trained models capable of sentiment analysis and topic classification. Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for the assessment, prediction and identification of competency levels of a candidate based on how SMEs answer questions, the best-known path created by SMEs and other factors which identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role. Jungmeisteris and Thibodeaux are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the adaptive competency assessment (ACA) system as taught by Thibodeaux with the sentiment analysis techniques of Jungmeisteris to achieve the claimed invention and there would have been a reasonable expectation of success in doing so." DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006). Further, one of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that applying the known techniques for developing an adaptive competency assessment (ACA) model to evaluate the knowledge/skills of a candidate (compared to SMEs) as taught by Thibodeaux to include all available data including behavioral characteristics, and various factors regarding how the SME answered questions (sentiment analysis of one or more expert third-party evaluators) for scoring candidate competency (compared to best known path created by SMEs) would have been predictable to one of ordinary skill in the art, resulting in a more accurate and improved competency assessment and prediction for a candidate by analyzing all aspects of candidate interactions with the ACA model, not just their answers to the questions (¶16-¶19, ¶22-¶26).
automatically processing and creating one or more best-fitting subject matter items options for one or more selections by one or more clients after the determination of the one or more sentiments of the one or more expert third-party evaluators is made (¶16: “the ability to take into account all available data, including behavioral characteristics, to score competency; ¶18: “Upon organizing the questions into tiers for a given job, the SMEs for that job can be provided the tiered question set for that job. As the SMEs complete the questions, the system can record various factors about how the SMEs answer the questions. For example, rather than just collecting the correct answers for each question, the system can record information about how the SMEs answer the questions”; ¶22: “ACA models configured as disclosed herein can identify if a candidate has reached a benchmark level of competency in one or more categories. For example, each level of competency identifiable by the ACA model can have a representative set of questions covering the most important objectives for a particular job role at that particular level… the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs”)
Applicant’s disclosure generally teaches, Fig 10, page 15, lines 14-27: “associating analyzed collected data of the subject matter items with one or more consultant workflows at 406, combining the one or more consultant workflows with the one or more client preferences at 408, automatically creating one or more best-fitting subject matter items in an option list for one or more selections by one or more clients”. Giving the broadest reasonable interpretation of the claim limitation in light of the disclosure, Examiner interprets the ACA model for providing competency assessment levels of a candidate based on how SMEs answer the questions, and the best-known path created by the SMEs as taught by Thibodeaux as teaching applicant’s best fitting subject matter items options.
comparing the one or more selections of the one or more best-fitting subject matter items options of the one or more clients to the one or more sentiments of the expert third- party evaluators for the subject matter items to determine a level of client expertise after the automatic processing and creating of the one or more best-fitting subject matter items options (¶22: “there can be a predefined set of questions for a candidate seeking a “Level 1” competency, and distinct sets of predefined set of questions for candidates seeking “Level 2” or “Level 3” competencies. Each succeeding level may reference the same objectives, but each level contains questions which are demonstrably more challenging than the previous level. To advance to a higher, or subsequent, level, the candidate may need to demonstrate a predefined competency. In some scenarios, to demonstrate their competency, the candidate may need to obtain a threshold score (e.g., 80% or 100%) to move to the next level. In other scenarios, the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “at the end of each level, and at the end of each level they fail to complete, the candidate can be presented with a score report that includes information about their score including: 1) how they did on each objective, 2) the quality of their responses, 3) the speed of their responses, 4) areas where they should seek additional training. This report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; Fig 1, ¶28; ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc. The system organizes the questions by level (112) and creates an algorithm (114) for the questions. At this point, a preliminary ACA model is defined, and a test group is created (116) to test the preliminary ACA model. The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”).
wherein the level of client expertise is automatically calculated based on a percentage comparing how close the one or more client selections of the one or more best-fitting subject matter items options are to the one or more sentiments of the one or more expert third-party evaluators for the subject matter items (¶16: “(4) the ability to take into account all available data, including behavioral characteristics, to score competency”; ¶22: “to demonstrate their competency, the candidate may need to obtain a threshold score (e.g., 80% or 100%) to move to the next level. In other scenarios, the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “at the end of each level, and at the end of each level they fail to complete, the candidate can be presented with a score report that includes information about their score including: 1) how they did on each objective, 2) the quality of their responses, 3) the speed of their responses, 4) areas where they should seek additional training. This report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc. The system organizes the questions by level (112) and creates an algorithm (114) for the questions. At this point, a preliminary ACA model is defined, and a test group is created (116) to test the preliminary ACA model. The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”).
wherein the greater the percentage is calculated that the one or more client selections of the one or more best-fitting subject matter items options match the one or more sentiments of the one or more expert third-party evaluators for the subject matter items the greater the level of client expertise (¶22: “there can be a predefined set of questions for a candidate seeking a “Level 1” competency, and distinct sets of predefined set of questions for candidates seeking “Level 2” or “Level 3” competencies. Each succeeding level may reference the same objectives, but each level contains questions which are demonstrably more challenging than the previous level. To advance to a higher, or subsequent, level, the candidate may need to demonstrate a predefined competency. In some scenarios, to demonstrate their competency, the candidate may need to obtain a threshold score (e.g., 80% or 100%) to move to the next level. In other scenarios, the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “at the end of each level, and at the end of each level they fail to complete, the candidate can be presented with a score report that includes information about their score including: 1) how they did on each objective, 2) the quality of their responses, 3) the speed of their responses, 4) areas where they should seek additional training. This report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; Fig 1, ¶28; ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc. The system organizes the questions by level (112) and creates an algorithm (114) for the questions. At this point, a preliminary ACA model is defined, and a test group is created (116) to test the preliminary ACA model. The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”; ¶36: “each question in the tiered set of questions can have a list of associated answers, each answer in the list of answers having a weighted core competency associated with at least one job in a plurality of job roles”)
based on the comparing, matching the best-fitting subject matter items options to one or more particular steps in the plurality of steps of the one or more consultant workflows (¶22: “ACA models configured as disclosed herein can identify if a candidate has reached a benchmark level of competency in one or more categories. For example, each level of competency identifiable by the ACA model can have a representative set of questions covering the most important objectives for a particular job role at that particular level. That is, there can be a predefined set of questions for a candidate seeking a “Level 1” competency, and distinct sets of predefined set of questions for candidates seeking “Level 2” or “Level 3” competencies. Each succeeding level may reference the same objectives, but each level contains questions which are demonstrably more challenging than the previous level. To advance to a higher, or subsequent, level, the candidate may need to demonstrate a predefined competency… if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶26: “Each ACA model, for each job role, can be calibrated before it is deployed. The process may require a sample of individuals across different, predetermined competency levels. For example, for a three-level system three groups of equal size may be required: one generally underqualified for the job role, a group generally qualified for the job role, and a group over qualified. These groups can provide sufficient data to enable the machine learning/artificial intelligence algorithm to modify the ACA model for a specific deployment. Each group will attempt to reach their maximum competency level”; ¶28-32; Fig 1, Fig 2, Fig 4, ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc… The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”; ¶32: “provide, via the processor, the tiered set of questions to a candidate (412); capture, via the processor as the candidate answers at least a portion of the tiered set of questions, second scoring factors of the candidate in answering the at least a portion of the tiered set of questions (414); compare, as the candidate answers the at least a portion of the tiered set of questions, the second scoring factors to the best known path, resulting in a comparison (416); and generate, via the processor, based on the comparison, as the candidate answers the at least a portion of the tiered set of questions, a live score as the candidate answers the at least a portion of the tiered set of questions (418)”)
Applicant’s disclosure generally teaches on page 4 line 28- page 5, line 5: “Consultants construct workflows to ensure that the options presented to clients are ones that can be offered by the consultant and all required work for a client is completed in the necessary order. A workflow consists of a number of workflow steps, each containing a list of subject matter keywords which are a subset of the keywords used by the system operator to locate SMIs for the purpose of matching to client preferences. For example, for a financial consultant, workflow steps could include gathering information on investments, qualifying a client for a set of funds, determining investment types, and qualifying particular potential investments. Each consultancy has its own workflow, even those using the same subject matter. The workflow steps help define the context needed for matching items to client preferences”. Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for identifying competency levels of a candidate based on how subject matter experts (SMEs) answer questions, best known path created by SMEs and other factors to identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role. Giving the broadest reasonable interpretation of applicant’s claim limitation in light of the specification, Examiner interprets the ACA model for assessing and identifying candidate competency levels based on how SMEs answer questions, and the best-known path created by SMEs as taught by Thibodeaux as teaching applicant’s consultant workflow.
Jungmeisteris teaches a text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. User input is analyzed via machine learning algorithms during session activity to infer workflow and derive the meaning of user input. Jungmeisteris further teaches a sentiment analysis system including a pre-trained natural language processing (NLP) model capable of performing semantic parsing to capture and understand the meaning of input text, a statement, request, or question input by a user. Jungmeisteris discloses training one or more machine learning models on a set of character string data simulating potential input text typed by a user whereby features from training data are extracted to develop one or more trained models capable of sentiment analysis and topic classification.
Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for identifying competency levels of a candidate based on how subject matter experts (SMEs) answer questions, best known path created by SMEs and other factors to identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role.
Jungmeisteris and Thibodeaux are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. One of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that applying the known techniques for developing an adaptive competency assessment (ACA) model to evaluate the knowledge/skills of a candidate (compared to SMEs) as taught by Thibodeaux to include all available data including behavioral characteristics, and various factors regarding how the SME answered questions (sentiment analysis of one or more expert third-party evaluators) for scoring candidate competency (compared to best known path created by SMEs) would have been predictable to one of ordinary skill in the art, resulting in a more accurate and improved competency assessment and prediction for a candidate by analyzing all aspects of candidate interactions with the ACA model, not just their answers to the questions (¶16-¶19, ¶22-¶26). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the method/system for inferring workflow and determining user sentiment of Jungmeisteris with the adaptive competency assessment (ACA) system of Thibodeaux to achieve the claimed invention and there would have been a reasonable expectation of success in doing so." DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006).
Jungmeisteris and Thibodeaux disclose all of the above limitations, the combination of Jungmeisteris and Thibodeaux does not distinctly describe the following limitations but Cella however as shown discloses,
generating one or more consultant workflows using generative artificial intelligence (AI) and based on consultant-selected drag-and-drop inputs, wherein the one or more consultant workflows comprise a plurality of steps to be performed by particular consultants in particular subject matters (¶234: “a generative artificial intelligence engine (GAIE) may be combined with a machine learning system; ¶254: “The GAIE may enhance the intelligence system with a supervisory generative Al capability that decides how and when to apply various Al tools and modules. For a workflow development application area, a pretrained GAIE may identify, refine, and/or create various transaction (e.g., data, or financial) workflows that may be modularized, re-used, and further refined based on data. For an expert training application area, a GAIE may interact with experts, approvers, etc. to build domain- specific capabilities that may be used to enhance workflows, governance, fraud detection, and the like.”; ¶255: “a pre-trained GAIE may generate a summary of customer profiles based on contextual analysis of information sourced, for example, from social media. Such a pre-trained GAIE may facilitate iterating between conversation and user behavior tracking/observation to determine how conversational parameters influence user behavior (group/cohort level). Also, it may facilitate iterating between conversation and user behavior tracking/observation to determine how conversational parameters influence user behavior, such as at an individual level”; ¶265: “user interfaces that adapt to the user and context, hybrid content generation, collaboration units of humans and generative Al, purpose- specific data integration, a selected, set of data sources, curation of data as models as input to generative AL and the like”; ¶320: “platform 800 for the application of generative Al may include a set of subject matter-specific pretrained examples and prompts 804. This set of examples and prompts 804 may be configured by analyzing (e.g., by a human expert and/or computer-based expert and/or digital twin) information that characterizes various aspects of the domain to generate example prompts and preferred and/or correct responses. Pretraining may also include training the next-token prediction Al engine 802 by sampling some text (e.g., prompt/response sets) from the set of subject matter-specific pretrained examples and prompts 804 and training it to predict a next word, object, and/or term. Pretraining may also include sampling some images, contracts, architectures, and the like to predict a next token. These prompt-response sub-sets may facilitate pre-training the prediction Al engine 802 for predicting a next token (e.g., word, object, image element, and the like) tor various aspects”; ¶324: “A high level of accuracy and integration with operational systems may enable such a tool to go beyond just generating new content to be more productive; through integration with workflows, it may facilitate automating workflow actions… a pre-training optimizing engine 806 may provide a wider range of prompts and responses based on user preferences (e.g., speaking styles) to enrich the platform’s ability to provide user-centric responses. In example embodiments, user-centric responses may include fine tuning the platform 800 for different roles in an organization”; ¶329: “The platform 800 may further include an expert, review and approval portal 818 through which an expert (e.g., human / digital twin, and the like) can review, edit, and approve content generated. Examples include review and adaptation by a subject matter specific data story expert; a data, scientist, and the like. The expert review and. approval portal 818 may operate cooperatively with, for example, the pre-training optimizing engine 806 that may receive and analyze expert feedback (e.g., edits to the content and the like) for opportunities to further optimize the platform 800”; ¶379: “the deployment of the EAL 1000 may be configurable. For example, the enterprise 900 or some associated developer can function as a type of architect for the EAL 1000 that best serves the particular enterprise 900. Additionally, or alternatively, the deployed location of the EAL 1000 may influence its configuration. For instance, the EAL 1000 may be embedded within an enterprise (e.g., non-dynamically) where it can be specifically configured using various module libraries, interface tools, etc”; ¶381: “The configurations can be done by selecting pre-defined configurations/plugins, by building customized modules, and/or by connecting to third party services that provide certain functionalities”; ¶389: “The GUI 1114 may present an interface for configuring workflows in the workflow definition system 1142, for configuring the capabilities, such as by selecting subsystems, of the EAL 1000, for defining data pool templates in the data pool system 1136, etc. The GUI 1114 may also provide access to the reporting system 1180 by regulators, auditors, government entities, etc”; ¶450: “the EAL may include a workflow system 1140. In some embodiments, the workflow system 1140 provides tools and capabilities for defining, selecting, deploying, and/or managing workflows that are executed on behalf of respective enterprises… To create, manage, and implement workflow processes, the workflow system 1140 may include a workflow definition system 1142, a workflow library system 1144, a workflow optimization system 1146, and a workflow management system 1148”; ¶452: “the workflow definition system may include a set of tools that allow an enterprise to configure, define, and deploy workflows. In some embodiments, the workflow definition system provides GUIs that assist a user (e.g., an enterprise user) in selecting existing default workflows and/or defining custom workflows. In the case of selecting default workflows, the workflow definition system may allow authorized users to select from a menu of available workflows that can be used to perform respective tasks”; ¶453: “The workflow definition system receives workflow configurations from a user and generates executable workflows based thereon. In some of these embodiments, the workflow definition system includes a workflow builder that provides an interface where users can build workflows based on pre-defined or configured business rules and processes, transaction models, or the like. In some embodiments, the workflow builder may include a GUI that allows users to configure new workflows”)
Cella teaches a workflow management using generative artificial intelligence to facilitate workflow orchestration for a process that uses a conversational, generative Al agent and another Al-supported process in an orchestrated sequence. In example embodiments, a GAIE may generate, perform, maintain, and/or supervise one or more workflows in a robotic process automation (RPA) environment. For example, a GAIE may be trained to monitor expressions and/or actions of an individual during interaction with other individuals, and may generate similar expressions and/or perform similar actions during similar interactions between the GAIE and other individuals. Cella also teaches that a GAIE may interact with experts, approvers, etc. to build domain- specific capabilities that may be used to enhance workflows, governance, fraud detection, and the like for an expert training application area. Jungmeisteris, Thibodeaux and Cella are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the method/system for inferring workflow and determining user sentiment of Jungmeisteris, and the adaptive competency assessment (ACA) system of Thibodeaux with the method/system for workflow management using generative artificial intelligence as taught by Cella since it allows for receiving workflow configurations from a user and generating executable workflows based on pre-defined or configured business rules and processes (¶379, ¶389, ¶450-¶453).
With respect to Claims 2 and 12,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more webpages include expert analyses, third- party reviews, social media posts, or newsfeed webpages (¶17: “The methods and systems described herein may be used to collect customer feedback and leverage sentiment analysis as a proxy to determine customer satisfaction with a service, a customer support interaction or other customer interaction. For example, a customer’s messaging or dialog with another user (e.g., a host or customer service agent) or interactions with a service can be monitored and analyzed to determine the customer's current sentiment”; ¶20: “a user accessing a website may initiate communication with a customer support application or widget, by either clicking on a support feature or responding to a presented customer support inquiry. In particular, the user may input a character string (e.g., as freeform text) with the intent of receiving a responsive message or instruction. The input character string is transmitted (in some embodiments, the string being tokenized, and in others, not being tokenized) from a user device to a web server, and then to a sentiment analysis system. In some embodiments, the sentiment analysis system is capable of performing semantic parsing to capture the meaning of the input text”; ¶30: “the customer support system and methods described herein may target users over various channels, such as a website or application's help center (or help/feedback widget), chat function or bot (also referred to herein as a chatbot), and/or messaging features”)
With respect to Claims 3 and 13,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the natural language processing includes processing by a Bidirectional Encoder Representations from Transformers (BERT) model (¶66: “The specific method of generating the vector from the textual input may be performed through any of a variety of known methods. In one exemplary embodiment, Google's BERT (Bidirectional Encoder Representations from Transformers) model or a model based on BERT, such as XLM-RoBERTa, may be used as an applied natural language processing (NLP) model. However, in other embodiments, any other appropriate pre-trained model (e.g., Generative Pretrained Transformer (GPT), Text-to-Text Transfer Transformer (TS), ULMFiT, etc.) may be used”; ¶74: “a transformer-based deep learning technique is used for sentiment analysis and other large scale NLP processing tasks. The transformer may be trained on the dataset described above with regard to step 502. Exemplary transformer models may include XLM-RoBERTa, or other models based on BERT”)
With respect to Claims 4 and 14,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more sentiments are determined from keywords or key phrases using the natural language processing (¶74: “a transformer-based deep learning technique is used for sentiment analysis and other large scale NLP processing tasks. The transformer may be trained on the dataset described above with regard to step 502. Exemplary transformer models may include XLM-RoBERTa, or other models based on BERT. In some embodiments, sentiment analysis task is modeled as a classification problem, whereby a classifier is fed a text input and returns a category, e.g., positive, negative, or neutral. This may involve feature extraction from freeform text e.g., to generate vectors for words or sentences. Exemplary classification algorithms are NLP classification models such as Na″fve Bayes, Support Vector Machines (SVM), and/or linear regression techniques. The output of the models is a probability distribution across different sentiment categories, then a sentiment score is generated based on the distribution. In other embodiments, this score may then be compared to a range of possible scores to classify the sentiment as positive, negative, or neutral…”)
With respect to Claims 5 and 15,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more sentiments include strongly negative data, negative data, neutral data, positive data, or strongly positive data (¶74: “The output of the models is a probability distribution across different sentiment categories, then a sentiment score is generated based on the distribution. In other embodiments, this score may then be compared to a range of possible scores to classify the sentiment as positive, negative, or neutral. In some embodiments, other classification schemes may be used, e.g., highly/slightly positive/negative. In an exemplary embodiment, the sentiment score is a value from 0-1, with 0 being the most negative, and 1 being the most positive”)
With respect to Claims 7 and 17,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more associated providers interact with a system operator to confirm identified goods or services (¶53: “where system 110 is a customer support system for use with a website facilitating an online reservation platform, such themes or classifications may be, for instance, “user”, “user account”, “payment”, “booking”, “cancellation”, “confirmation”, and so on”)
With respect to Claims 8 and 18,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more consultant workflows contain one or more workflow steps or one or more conditional statements, and the one or more workflow steps contain descriptions, keywords, plans, or advisor notices (¶17: “a customer’s messaging or dialog with another user (e.g., a host or customer service agent) or interactions with a service can be monitored and analyzed to determine the customer's current sentiment. If the customers sentiment is determined to be falling or getting worse, the system can intervene to attempt to resolve the situation for the customer…The questions asked in the survey may be guided by the users activity with the service, e.g., on an e-commerce website, the workflow followed by the user and a determination of when and how the user engaged customer support in the workflow process. In some embodiments, the questions and/or responses presented to the user via the survey or chatbot are associated with a “tone” of response, the tone of response being dictated by the derived sentiment score”; ¶19: “the computer system may dynamically generate or select text, in real-time, to present to the user in the intercept survey or a chatbot based on information about the user and the workflow of the user in the particular session. For example, information about the path or workflow the user took during their support journey (their activity) can be used to dynamically select a set of one or more questions to use in the user interaction”)
With respect to Claims 9 and 19,
Jungmeisteris, Thibodeaux and Cella disclose all of the above limitations, Jungmeisteris further discloses,
wherein the one or more associated providers include one or more associated asset managers, the one or more consultant workflows include one or more financial advisor workflows, and the subject matter items include financial assets (¶26: “the computer system may be an online reservation system that displays to potential customers properties, such as houses, condominiums, rooms, apartments, lots, and other real estate, offered to the potential customer (or guest) by an owner/manager of the property for reservation (sometimes referred to as a “booking” or “rental”) for a specified time period (e.g., a day, week, month, or another period of interest). The owner of a property may contract with the merchant managing the online reservation system to use the system to display a property “listing” for the reservable property. When a customer books a property, the merchant's online reservation system may allow for intake, from the customer, of a booking fee, an initial setup fee, a recurring fee, no fee, and/or any other appropriate value. In some instances, the merchant may also handle and/or facilitate one or more financial transactions relating to the purchase or booking of that property, and may receive a fee in relation thereto. The systems and methods described herein are of course not limited to systems relating to property listings; rather, they may be provided for any website, application, or e-commerce system that may potentially provide text-based customer support features”) Examiner interprets at least the online reservation system displaying//providing potential customers properties (houses, real estate) by an owner/manager of the property, and merchant managing the reservation system, to facilitate intake, handling of financial transactions relating to the purchase or booking of that property listing as taught by Jungmeisteris as teaching applicant’s “providers include one or more associated asset managers, the one or more consultant workflows include one or more financial advisor workflows, and the subject matter items include financial assets”)
With respect to Claim 25,
Jungmeisteris discloses,
extracting subject matter items, from one or more webpages of one or more internet platforms (Abstract: “A text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. A user's freeform text input is analyzed using machine learning algorithms to derive the meaning of the input text as well as to determine the user sentiment expressed therein”; ¶18: “The intercept survey is a freeform text survey present during the user's interaction with a computer system (e.g., on a website or application), presented before the initialization of a help ticket and without the need for default or generic questions in a post-event survey”; ¶20: “the sentiment analysis system is capable of performing semantic parsing to capture the meaning of the input text. The sentiment analysis system includes a pre-trained natural language processing (NLP) model capable of generating vector representations of the input text. In an exemplary embodiment, the model is capable of generating a series of vectors (each corresponding to a word) so as to be representative of a sentence, or another measurement of text”; ¶41: “memory 210 may also, in one embodiment, include communication logic 224, including one or more algorithms or models for obtaining information from or communicating information with web server 140, database 250 (or other external or third-party databases), and/or via network 130 (FIG. 1)”; Fig 6, #602 “Pretrain language model, NLP topic classification model, and sentiment analysis model on exemplary chat data”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like”)
associating analyzed collected data of the subject matter items with the one or more consultant workflows (¶17: “The questions asked in the survey may be guided by the user’s activity with the service, e.g., on an e-commerce website, the workflow followed by the user and a determination of when and how the user engaged customer support in the workflow process. In some embodiments, the questions and/or responses presented to the user via the survey or chatbot are associated with a “tone” of response, the tone of response being dictated by the derived sentiment score”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like. ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like. Therefore, each of the entries in workflow data 232 can be associated with a particular user or user device, for example, a user ID (if logged in), a device (by device ID, IP address, MAC address, or the like), session ID, other information sufficient to identify a user (such as a unique code or password), or any other appropriate identifying mechanism. In an exemplary embodiment, data regarding a user's activity is pulled from one or more of web server(s) 140 and external databases 250 to populate workflow data 232 in real-time, in response to any action by the user or any of one or more scheduled events”; ¶58: “Each of these navigational, click, search, or other interactions may be stored as workflow data 232 in association with a session ID (and/or in some cases, a user ID)”; ¶93: “system 110 may also, in step 604, function to recognize circumstances that trigger the presentation of the chatbot (or other UI) from which sentiment analysis and/or workflow analysis can be conducted. In one embodiment, this trigger may be based on a semantic analysis of text entered into a field on a website, e.g., into a messenger app or widget, search bar or other query field, etc. (typically accessed from workflow data 232). ¶98: “user sentiment, circumstantial evidence, and the availability of self-serve options may be factors that are variously weighted by one or more machine learning algorithms (e.g., linear regression models). Such algorithms may be trained on a training set of various scenarios with selected or procedurally generated combinations of circumstantial and workflow scenarios. In another embodiment, machine learning analyses may be applied to determine sentiment, but the determination of workflow may be subsequently applied to a rules-based process, e.g., where escalation or other alternate remediation is always applied in certain defined circumstances”; Fig 4B, ¶99: “based on the determinations of step 650, calculated user sentiment, and/or circumstantial data, the workflow logic 220 may present to the user different workflows”)
combining the one or more consultant workflows with the one or more client preferences (¶19: “information about the path or workflow the user took during their support journey (their activity) can be used to dynamically select a set of one or more questions to use in the user interaction”;¶48: “With reference to FIG. 2B, user data 231 may include information associated with a user, e.g., user ID, user account information (e.g., name, contact information (e.g., email address, mailing address, telephone number), date of birth, payment card information), a type of user (e.g., guest/registered, business/vacation travelers, etc.), length of membership, booking history, demographic data about the user, such as age, gender, location, language, and device information, GPS, IP address, or location data, operating system, browser, or other device data, user preferences or interests, connected third party accounts (e.g., social networks) if applicable, and the like”; ¶49: “Workflow data 232 includes a variety of information collected as users interact with a website or app, make transactions, and the like.)
tracking and updating the one or more selections of the one or more clients to update the one or more client preferences (¶17: A numeric sentiment score is derived and updated in real-time based on implicit signals of customer satisfaction obtained from a user's freeform text entry (obtained, for example, from a survey, a chatbot, an interaction with a customer service agent or an interaction with another user) and/or the users selection interaction with an intercept survey or a post-activity survey in the same channel as the users activity, as well as other relevant information. The questions asked in the survey may be guided by the user’s activity with the service, e.g., on an e-commerce website, the workflow followed by the user and a determination of when and how the user engaged customer support in the workflow process”; ¶58: “The user may also have clicked on one or more of the displayed results 330. Each of these navigational, click, search, or other interactions may be stored as workflow data 232 in association with a session ID (and/or in some cases, a user ID). FIGS. 4A-4D describe various progressive interfaces that may be displayed to the user and updated in real-time, for example through chatbot interface 340 or similar type of interface”)
Jungmeisteris discloses all of the above limitations, Jungmeisteris does not distinctly describe the following limitations, but Thibodeaux as shown discloses,
semantically processing and analyzing text data or image data of the one or more webpages using natural language processing to determine one or more sentiments of the one or more expert third-party evaluators for the subject matter items based on analysis from the one or more third-party evaluators considering one or more client preferences and analysis of anonymous aggregate client data (Abstract: “a tiered set of questions to a subject matter expert and, based on how the subject matter expert answers the questions, and identifies a best known path for answering the questions. At a second time, the system provides the tiered set of questions to a candidate, captures how the candidate answers the questions, compares that information to the best-known path, and generates, based on that comparison, a live score as the candidate answers the questions”; ¶16: “the ability to take into account all available data, including behavioral characteristics, to score competency; (5) access to a library of curated resources to answer questions; (6) machine learning/artificial intelligence which analyzes all aspects of the candidates interaction with the adaptive competency assessment model (not just their answers to the questions); and (7) a relative scoring schema to weight job role objectives appropriately for the situation”; ¶18: “Upon organizing the questions into tiers for a given job, the SMEs for that job can be provided the tiered question set for that job. As the SMEs complete the questions, the system can record various factors about how the SMEs answer the questions. For example, rather than just collecting the correct answers for each question, the system can record information about how the SMEs answer the questions”; ¶25: “report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; ¶26: “The process may require a sample of individuals across different, predetermined competency levels. For example, for a three-level system three groups of equal size may be required: one generally underqualified for the job role, a group generally qualified for the job role, and a group over qualified. These groups can provide sufficient data to enable the machine learning/artificial intelligence algorithm to modify the ACA model for a specific deployment”; ¶32; ¶36: “each question in the tiered set of questions can have a list of associated answers, each answer in the list of answers having a weighted core competency associated with at least one job in a plurality of job roles”)
Applicant’s disclosure generally teaches on page 9, lines 28 and 29: “the sentiments determined by the aforementioned tools can include strongly negative, negative, neutral, positive or strongly positive”, and page 14, lines 3-9: “A client's 304 subject matter expertise can be determined, tracked, and compared to the sentiment from expert analysis and anonymous aggregate client preferences and selections over time. In the process of automatically tracking a client's preferences and selections, the system 302 also automatically computes the percentage match between a client's sentiment and the overall sentiment from analysis by experts in the SMA 312. The closer the client's preferences and selections match expert sentiment, the more their future choices will match expert choices”.
Jungmeisteris teaches a text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. Jungmeisteris teaches a sentiment analysis system including a pre-trained natural language processing (NLP) model capable of performing semantic parsing to capture and understand the meaning of input text, a statement, request, or question input by a user. Jungmeisteris also teaches training one or more machine learning models on a set of character string data simulating potential input text typed by a user whereby features from training data are extracted to develop one or more trained models capable of sentiment analysis and topic classification. Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for the assessment, prediction and identification of competency levels of a candidate based on how SMEs answer questions, the best-known path created by SMEs and other factors which identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role. Jungmeisteris and Thibodeaux are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the adaptive competency assessment (ACA) system as taught by Thibodeaux with the sentiment analysis techniques of Jungmeisteris to achieve the claimed invention and there would have been a reasonable expectation of success in doing so." DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006). Further, one of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that applying the known techniques for developing an adaptive competency assessment (ACA) model to evaluate the knowledge/skills of a candidate (compared to SMEs) as taught by Thibodeaux to include all available data including behavioral characteristics, and various factors regarding how the SME answered questions (sentiment analysis of one or more expert third-party evaluators) for scoring candidate competency (compared to best known path created by SMEs) would have been predictable to one of ordinary skill in the art, resulting in a more accurate and improved competency assessment and prediction for a candidate by analyzing all aspects of candidate interactions with the ACA model, not just their answers to the questions (¶16-¶19, ¶22-¶26).
automatically processing and creating one or more best-fitting subject matter items options for one or more selections by one or more clients after the determination of the one or more sentiments of the one or more expert third-party evaluators is made (¶16: “the ability to take into account all available data, including behavioral characteristics, to score competency; ¶18: “Upon organizing the questions into tiers for a given job, the SMEs for that job can be provided the tiered question set for that job. As the SMEs complete the questions, the system can record various factors about how the SMEs answer the questions. For example, rather than just collecting the correct answers for each question, the system can record information about how the SMEs answer the questions”; ¶22: “ACA models configured as disclosed herein can identify if a candidate has reached a benchmark level of competency in one or more categories. For example, each level of competency identifiable by the ACA model can have a representative set of questions covering the most important objectives for a particular job role at that particular level… the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs”)
Applicant’s disclosure generally teaches, Fig 10, page 15, lines 14-27: “associating analyzed collected data of the subject matter items with one or more consultant workflows at 406, combining the one or more consultant workflows with the one or more client preferences at 408, automatically creating one or more best-fitting subject matter items in an option list for one or more selections by one or more clients”. Giving the broadest reasonable interpretation of the claim limitation in light of the disclosure, Examiner interprets the ACA model for providing competency assessment levels of a candidate based on how SMEs answer the questions, and the best-known path created by the SMEs as taught by Thibodeaux as teaching applicant’s best fitting subject matter items options.
automatically calculating a level of client expertise based on a percentage comparing how close the one or more client selections of the one or more best-fitting subject matter items options are to the one or more sentiments of the one or more expert third-party evaluators for the subject matter items (¶16: “(4) the ability to take into account all available data, including behavioral characteristics, to score competency”; ¶22: “to demonstrate their competency, the candidate may need to obtain a threshold score (e.g., 80% or 100%) to move to the next level. In other scenarios, the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “at the end of each level, and at the end of each level they fail to complete, the candidate can be presented with a score report that includes information about their score including: 1) how they did on each objective, 2) the quality of their responses, 3) the speed of their responses, 4) areas where they should seek additional training. This report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc. The system organizes the questions by level (112) and creates an algorithm (114) for the questions. At this point, a preliminary ACA model is defined, and a test group is created (116) to test the preliminary ACA model. The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”).
wherein the greater the percentage is calculated that the one or more client selections of the one or more best-fitting subject matter items options match the one or more sentiments of the one or more expert third-party evaluators for the subject matter items the greater the level of client expertise (¶22: “there can be a predefined set of questions for a candidate seeking a “Level 1” competency, and distinct sets of predefined set of questions for candidates seeking “Level 2” or “Level 3” competencies. Each succeeding level may reference the same objectives, but each level contains questions which are demonstrably more challenging than the previous level. To advance to a higher, or subsequent, level, the candidate may need to demonstrate a predefined competency. In some scenarios, to demonstrate their competency, the candidate may need to obtain a threshold score (e.g., 80% or 100%) to move to the next level. In other scenarios, the score may be a factor, but how the candidate answers the question may also factor into the determination of competency. For example, if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶25: “at the end of each level, and at the end of each level they fail to complete, the candidate can be presented with a score report that includes information about their score including: 1) how they did on each objective, 2) the quality of their responses, 3) the speed of their responses, 4) areas where they should seek additional training. This report can be customized to the candidate based on how their responses to questions, and the process of their responses, compared to the best-known path created by the SMEs. In other words, this generated report can identify subject matter which the candidate needs to study further as well as test taking behavior which the candidate needs to further improve”; Fig 1, ¶28; ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc. The system organizes the questions by level (112) and creates an algorithm (114) for the questions. At this point, a preliminary ACA model is defined, and a test group is created (116) to test the preliminary ACA model. The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”; ¶36: “each question in the tiered set of questions can have a list of associated answers, each answer in the list of answers having a weighted core competency associated with at least one job in a plurality of job roles”)
based on the comparing, matching the best-fitting subject matter items options to one or more particular steps in the plurality of steps of the one or more consultant workflows (¶22: “ACA models configured as disclosed herein can identify if a candidate has reached a benchmark level of competency in one or more categories. For example, each level of competency identifiable by the ACA model can have a representative set of questions covering the most important objectives for a particular job role at that particular level. That is, there can be a predefined set of questions for a candidate seeking a “Level 1” competency, and distinct sets of predefined set of questions for candidates seeking “Level 2” or “Level 3” competencies. Each succeeding level may reference the same objectives, but each level contains questions which are demonstrably more challenging than the previous level. To advance to a higher, or subsequent, level, the candidate may need to demonstrate a predefined competency… if the user obtained 100% but took twice as long as the SMEs, the candidate may not be considered to be as competent as necessary for a given job or role”; ¶26: “Each ACA model, for each job role, can be calibrated before it is deployed. The process may require a sample of individuals across different, predetermined competency levels. For example, for a three-level system three groups of equal size may be required: one generally underqualified for the job role, a group generally qualified for the job role, and a group over qualified. These groups can provide sufficient data to enable the machine learning/artificial intelligence algorithm to modify the ACA model for a specific deployment. Each group will attempt to reach their maximum competency level”; ¶28-32; Fig 1, Fig 2, Fig 4, ¶29: “For each question, the system defines the known best answer (110). In addition, the system may assign values to other answers (i.e., rank the answers), or assign values to time spent answer the question, resources used, etc… The test group is pre-assessed (118) to identify characteristics of the test group, then the ACA model is presented to the test group (120). Based on these results, the system fine tunes/modifies the ACA algorithm (122) to provide improved accuracy in identifying competencies, then deploys the ACA model (124)”; ¶32: “provide, via the processor, the tiered set of questions to a candidate (412); capture, via the processor as the candidate answers at least a portion of the tiered set of questions, second scoring factors of the candidate in answering the at least a portion of the tiered set of questions (414); compare, as the candidate answers the at least a portion of the tiered set of questions, the second scoring factors to the best known path, resulting in a comparison (416); and generate, via the processor, based on the comparison, as the candidate answers the at least a portion of the tiered set of questions, a live score as the candidate answers the at least a portion of the tiered set of questions (418)”)
Applicant’s disclosure generally teaches on page 4 line 28- page 5, line 5: “Consultants construct workflows to ensure that the options presented to clients are ones that can be offered by the consultant and all required work for a client is completed in the necessary order. A workflow consists of a number of workflow steps, each containing a list of subject matter keywords which are a subset of the keywords used by the system operator to locate SMIs for the purpose of matching to client preferences. For example, for a financial consultant, workflow steps could include gathering information on investments, qualifying a client for a set of funds, determining investment types, and qualifying particular potential investments. Each consultancy has its own workflow, even those using the same subject matter. The workflow steps help define the context needed for matching items to client preferences”. Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for identifying competency levels of a candidate based on how subject matter experts (SMEs) answer questions, best known path created by SMEs and other factors to identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role. Giving the broadest reasonable interpretation of applicant’s claim limitation in light of the specification, Examiner interprets the ACA model for assessing and identifying candidate competency levels based on how SMEs answer questions, and the best-known path created by SMEs as taught by Thibodeaux as teaching applicant’s consultant workflow.
Jungmeisteris teaches a text-based real-time communication interface, such as a chatbot, is presented to a user for the exchange of customer support information. User input is analyzed via machine learning algorithms during session activity to infer workflow and derive the meaning of user input. Jungmeisteris further teaches a sentiment analysis system including a pre-trained natural language processing (NLP) model capable of performing semantic parsing to capture and understand the meaning of input text, a statement, request, or question input by a user. Jungmeisteris discloses training one or more machine learning models on a set of character string data simulating potential input text typed by a user whereby features from training data are extracted to develop one or more trained models capable of sentiment analysis and topic classification.
Thibodeaux teaches an Adaptive Competency Assessment (ACA) system and ACA model for identifying competency levels of a candidate based on how subject matter experts (SMEs) answer questions, best known path created by SMEs and other factors to identify if a candidate has reached a benchmark level of competency in one or more categories and/or performing a job role.
Jungmeisteris and Thibodeaux are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. One of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that applying the known techniques for developing an adaptive competency assessment (ACA) model to evaluate the knowledge/skills of a candidate (compared to SMEs) as taught by Thibodeaux to include all available data including behavioral characteristics, and various factors regarding how the SME answered questions (sentiment analysis of one or more expert third-party evaluators) for scoring candidate competency (compared to best known path created by SMEs) would have been predictable to one of ordinary skill in the art, resulting in a more accurate and improved competency assessment and prediction for a candidate by analyzing all aspects of candidate interactions with the ACA model, not just their answers to the questions (¶16-¶19, ¶22-¶26). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the method/system for inferring workflow and determining user sentiment of Jungmeisteris with the adaptive competency assessment (ACA) system of Thibodeaux to achieve the claimed invention and there would have been a reasonable expectation of success in doing so." DyStar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick Co., 464 F.3d 1356, 1360, 80 USPQ2d 1641, 1645 (Fed. Cir. 2006).
Jungmeisteris and Thibodeaux disclose all of the above limitations, the combination of Jungmeisteris and Thibodeaux does not distinctly describe the following limitations but Cella however as shown discloses,
generating one or more consultant workflows using generative artificial intelligence (AI) and based on consultant-selected drag-and-drop inputs, wherein the one or more consultant workflows comprise a plurality of steps to be performed by particular consultants in particular subject matters (¶234: “a generative artificial intelligence engine (GAIE) may be combined with a machine learning system; ¶254: “The GAIE may enhance the intelligence system with a supervisory generative Al capability that decides how and when to apply various Al tools and modules. For a workflow development application area, a pretrained GAIE may identify, refine, and/or create various transaction (e.g., data, or financial) workflows that may be modularized, re-used, and further refined based on data. For an expert training application area, a GAIE may interact with experts, approvers, etc. to build domain- specific capabilities that may be used to enhance workflows, governance, fraud detection, and the like.”; ¶255: “a pre-trained GAIE may generate a summary of customer profiles based on contextual analysis of information sourced, for example, from social media. Such a pre-trained GAIE may facilitate iterating between conversation and user behavior tracking/observation to determine how conversational parameters influence user behavior (group/cohort level). Also, it may facilitate iterating between conversation and user behavior tracking/observation to determine how conversational parameters influence user behavior, such as at an individual level”; ¶265: “user interfaces that adapt to the user and context, hybrid content generation, collaboration units of humans and generative Al, purpose- specific data integration, a selected, set of data sources, curation of data as models as input to generative AL and the like”; ¶320: “platform 800 for the application of generative Al may include a set of subject matter-specific pretrained examples and prompts 804. This set of examples and prompts 804 may be configured by analyzing (e.g., by a human expert and/or computer-based expert and/or digital twin) information that characterizes various aspects of the domain to generate example prompts and preferred and/or correct responses. Pretraining may also include training the next-token prediction Al engine 802 by sampling some text (e.g., prompt/response sets) from the set of subject matter-specific pretrained examples and prompts 804 and training it to predict a next word, object, and/or term. Pretraining may also include sampling some images, contracts, architectures, and the like to predict a next token. These prompt-response sub-sets may facilitate pre-training the prediction Al engine 802 for predicting a next token (e.g., word, object, image element, and the like) tor various aspects”; ¶324: “A high level of accuracy and integration with operational systems may enable such a tool to go beyond just generating new content to be more productive; through integration with workflows, it may facilitate automating workflow actions… a pre-training optimizing engine 806 may provide a wider range of prompts and responses based on user preferences (e.g., speaking styles) to enrich the platform’s ability to provide user-centric responses. In example embodiments, user-centric responses may include fine tuning the platform 800 for different roles in an organization”; ¶329: “The platform 800 may further include an expert, review and approval portal 818 through which an expert (e.g., human / digital twin, and the like) can review, edit, and approve content generated. Examples include review and adaptation by a subject matter specific data story expert; a data, scientist, and the like. The expert review and. approval portal 818 may operate cooperatively with, for example, the pre-training optimizing engine 806 that may receive and analyze expert feedback (e.g., edits to the content and the like) for opportunities to further optimize the platform 800”; ¶379: “the deployment of the EAL 1000 may be configurable. For example, the enterprise 900 or some associated developer can function as a type of architect for the EAL 1000 that best serves the particular enterprise 900. Additionally, or alternatively, the deployed location of the EAL 1000 may influence its configuration. For instance, the EAL 1000 may be embedded within an enterprise (e.g., non-dynamically) where it can be specifically configured using various module libraries, interface tools, etc”; ¶381: “The configurations can be done by selecting pre-defined configurations/plugins, by building customized modules, and/or by connecting to third party services that provide certain functionalities”; ¶389: “The GUI 1114 may present an interface for configuring workflows in the workflow definition system 1142, for configuring the capabilities, such as by selecting subsystems, of the EAL 1000, for defining data pool templates in the data pool system 1136, etc. The GUI 1114 may also provide access to the reporting system 1180 by regulators, auditors, government entities, etc”; ¶450: “the EAL may include a workflow system 1140. In some embodiments, the workflow system 1140 provides tools and capabilities for defining, selecting, deploying, and/or managing workflows that are executed on behalf of respective enterprises… To create, manage, and implement workflow processes, the workflow system 1140 may include a workflow definition system 1142, a workflow library system 1144, a workflow optimization system 1146, and a workflow management system 1148”; ¶452: “the workflow definition system may include a set of tools that allow an enterprise to configure, define, and deploy workflows. In some embodiments, the workflow definition system provides GUIs that assist a user (e.g., an enterprise user) in selecting existing default workflows and/or defining custom workflows. In the case of selecting default workflows, the workflow definition system may allow authorized users to select from a menu of available workflows that can be used to perform respective tasks”; ¶453: “The workflow definition system receives workflow configurations from a user and generates executable workflows based thereon. In some of these embodiments, the workflow definition system includes a workflow builder that provides an interface where users can build workflows based on pre-defined or configured business rules and processes, transaction models, or the like. In some embodiments, the workflow builder may include a GUI that allows users to configure new workflows”)
Cella teaches a workflow management using generative artificial intelligence to facilitate workflow orchestration for a process that uses a conversational, generative Al agent and another Al-supported process in an orchestrated sequence. In example embodiments, a GAIE may generate, perform, maintain, and/or supervise one or more workflows in a robotic process automation (RPA) environment. For example, a GAIE may be trained to monitor expressions and/or actions of an individual during interaction with other individuals, and may generate similar expressions and/or perform similar actions during similar interactions between the GAIE and other individuals. Cella also teaches that a GAIE may interact with experts, approvers, etc. to build domain- specific capabilities that may be used to enhance workflows, governance, fraud detection, and the like for an expert training application area. Jungmeisteris, Thibodeaux and Cella are related to the same field of endeavor since they are directed to the semantic analysis of user data/information in a computing environment. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of applicant’s invention to combine the method/system for inferring workflow and determining user sentiment of Jungmeisteris, and the adaptive competency assessment (ACA) system of Thibodeaux with the method/system for workflow management using generative artificial intelligence as taught by Cella since it allows for receiving workflow configurations from a user and generating executable workflows based on pre-defined or configured business rules and processes (¶379, ¶389, ¶450-¶453).
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Korada et al., US Patent Application Publication No US 2022/0092635 A1, “Adaptive Real Time Modeling and Scoring”, relating to systems and methods for adaptive real-time modeling and scoring; the methods may be directed to customer acquisition and customer relationship management to generate web leads or new customer insights across multiple channels (email, display, call center, and so forth).
Williams et al., US Patent Application Publication No US 20240281410 A1, “Multi-Service Business Platform System having Custom Workflow Action Systems and Methods”, relating to methods and systems of an application for developing a strategy for development of online presence content, the application accessing the content cluster data store and having a set of tools for exploring and selecting suggested topics for online presence content generation.
Kraus et al., US Patent Application Publication No US 20240303437 A1, “Systems and Methods for Implementing an Advanced Content Intelligence Platform”, relating to systems, methods, schemes, techniques and processes implemented on an advanced content intelligence platform in a manner that creates predictably high performing content to meet a the objectives of users and enterprises by combining human intelligence with deep machine learning to determine a full body of content relevant to the objectives of a user or enterprise regarding a content output involving the steps of applying content intelligence to perform a deep dive into the information available on a specific topic, and to determine which of the available content may be particularly adapted to achieving the objectives of the user or enterprise in delivering optimized output content.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIMBERLY L EVANS whose telephone number is (571)270-3929. The examiner can normally be reached M-F 730a-5p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lynda Jasmin can be reached at (571)272-6782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KIMBERLY L. EVANS
Examiner
Art Unit 3629
/KIMBERLY L EVANS/Examiner, Art Unit 3629
/NATHAN C UBER/Supervisory Patent Examiner, Art Unit 3626
Prosecution Insights