DETAILED ACTION
This communication is a Final Office Action rejection on the merits. Claims 1-20 are currently pending and have been addressed below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement (IDS)
The information disclosure statement(s) filed on 03/17/2025 and 04/09/2026 comply with the provisions 37 CFR 1.97, 1.98, and MPEP 609 and is considered by the Examiner.
Claim Rejections - 35 USC § 101
Claims 12 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the computer readable-medium as claimed and described does not explicitly exclude transitory media. The specification, in Paragraph 0043, describes a computer readable-medium as “a computer-readable storage medium, as that term is used in the disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media.” Although the specification excludes transitory signals, the claims do not exclude transitory signals. Examiner recommends to change “computer readable-medium” to “non-transitory computer readable-medium.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without reciting significantly more.
Independent Claim 1
Step One - First, pursuant to step 1 in the January 2019 Revised Patent Subject Matter Eligibility Guidance (“2019 PEG”) on 84 Fed. Reg. 53, the claim 1 is directed to a method which is a statutory category.
Step 2A, Prong One - Claim 1 recites: A method, comprising: receiving assessment data associated with a first entity, wherein the assessment data comprises at least a set of questions and a first set of responses for the set of questions, and the set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity; detecting a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules, wherein the set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, an accuracy of the assessment data, and compliance of the assessment data; applying a model on the assessment data and the first set of anomalies; generating anomaly data associated with the first set of anomalies on the assessment data and the first set of anomalies, wherein the generating of the anomaly data comprises: evaluating free-format text associated with the assessment data based on a set of instructions; transforming unstructured assessment data including the free-format text into a structured format including key extracted information; and generating the anomaly data associated with the first set of anomalies based on the structured format, and the anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data; generating a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data, wherein the set of recommendations is generated to resolve the first set of anomalies; outputting the anomaly data and the set of recommendations on a first electronic device associated with the first entity; receiving one or more inputs based on the outputting of the set of recommendations on the first electronic device associated with the first entity to update the assessment data; and updating the assessment data based on the one or more inputs, wherein the updated assessment data comprises at least one updated response associated with the first set of responses. These claim elements are considered to be abstract ideas because they are directed to “certain methods of organizing human activity” which include “managing personal behavior.” In this case, evaluating risk parameters associated with an entity based on one or more rules is merely following rules or instructions (see MPEP 2106.04(a)(2)). Also, the limitations recite “collecting information, analyzing it, and displaying certain results of the collection and analysis," where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind (see MPEP 2106.04(a)). Examiner notes that the limitations of: receiving assessment data …; transforming unstructured data into structured data …; and generating the anomaly data … based on the structured format are merely describing evaluation of assessment data to identify anomalies, which is considered a mental process. Although the claim further requires “applying a first artificial intelligence model” and “generating recommendations based on the anomaly data,” the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process (see MPEP 2106.04(a), a claim that requires a computer may still recite a mental process). If a claim limitation, under its broadest reasonable interpretation, covers evaluations, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 - The judicial exception is not integrated into a practical application. Claim 1 includes additional elements: a computer; and a first artificial intelligence (Al) model.
The computer-implemented method includes receiving assessment data associated with a first entity. The assessment data includes at least a set of questions and a first set of responses for the set of questions. The set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity. The computer-implemented method further includes detecting a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules. The set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, an accuracy of the assessment data, and compliance of the assessment data. Further, the computer-implemented method includes generating anomaly data associated with the first set of anomalies based on the assessment data and the first set of anomalies. The anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data. The computer-implemented method further includes generating a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data. The set of recommendations is generated to resolve the first set of anomalies. The computer-implemented method further includes outputting the anomaly data and the set of recommendations (Paragraph 0003). The first AI uses a text processing unit configured to analyze and transform raw and unstructured assessment data into a structured format that can be further processed and utilized within the system (Paragraph 0090). These elements of “computer” and “first AI model” are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer element (MPEP 2106.05f). In this case, the first AI model includes inputs (e.g., responses in an unstructured format) and outputs (e.g., responses in a structured format). Although the specification states that the first AI model transforms unstructured data into structured data (Paragraph 0090), the claim and specification do not include any specific details about how the IA model operates (e.g., how the data is transformed). Thus, the analysis step of the model is a black box, which is merely claiming the idea of a solution or outcome (MPEP 2106.05f). Also, the computer is considered “field of use” since it’s just used to receive assessment data for an anomaly analysis, but the technology is not improved (MPEP 2106.05h). Lastly, the step of “generating recommendations” is merely outputting data (MPEP 2106.05g). Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B - The claim does not include additional elements that are sufficient to amount significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claims describe how to generally “apply” the concept of managing risk of an entity based on one or more rules (e.g., by evaluating responses of the entity). The specification shows that the computer-implemented method includes receiving assessment data associated with a first entity. The assessment data includes at least a set of questions and a first set of responses for the set of questions. The set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity. The computer-implemented method further includes detecting a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules. The set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, an accuracy of the assessment data, and compliance of the assessment data. Further, the computer-implemented method includes generating anomaly data associated with the first set of anomalies based on the assessment data and the first set of anomalies. The anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data. The computer-implemented method further includes generating a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data. The set of recommendations is generated to resolve the first set of anomalies. The computer-implemented method further includes outputting the anomaly data and the set of recommendations (Paragraph 0003). The first AI uses a text processing unit configured to analyze and transform raw and unstructured assessment data into a structured format that can be further processed and utilized within the system (Paragraph 0090). Also, the steps of “receiving one or more inputs based on the outputting of the set of recommendations to update the assessment data” and “updating the assessment data based on the one or more inputs” are considered conventional computer functions of “receiving and transmitting over a network” and “performing repetitive calculations” (MPEP 2106.05d). Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Independent claim 12 is directed to a system at step 1, which is a statutory category. Claim 12 recites similar limitations as claim 1 and is rejected for the same reasons at step 2a, prong one; step 2a, prong 2; and step 2b. Claim 12 further recites: a computer-readable storage media; and a processor – which are treated as just an explicit “processor/computer” for executing the operations and are treated under MPEP 2106.05f in the same manner as claim 1. Accordingly, these limitations are viewed as “apply it on a computer” at step 2a, prong 2 and step 2b. The claim is not patent eligible.
Independent claim 20 is directed to an article of manufacture at step 1, which is a statutory category. Claim 20 recites similar limitations as claim 1 and is rejected for the same reasons at step 2a, prong one; step 2a, prong 2; and step 2b. Claim 20 further recites: a computer-readable storage media – which is treated as just an explicit “processor/computer” for executing the operations and is treated under MPEP 2106.05f in the same manner as claim 1. Accordingly, these limitations are viewed as “apply it on a computer” at step 2a, prong 2 and step 2b. The claim is not patent eligible.
Dependent claims 2 and 13 are not directed to any additional claim elements. Rather, these claims offer further descriptive functions of elements found in the independent claims and addressed above - such as: applying a first set of logic rules of the set of logic rules to the assessment data; and detecting the first set of anomalies in the assessment data based on the one or more risk parameters and the applying of the first set of logic rules to the assessment data. In this case, the main functions are merely used for: collecting data (e.g., assessment data) and analyzing the data (e.g., detecting anomalies). Those are functions that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception (see MPEP 2106.05(h)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Dependent claims 3-10 and 14-18 are directed to an additional element such as: a second Artificial Intelligence (AI) model. The first AI model is merely used for generating first anomaly data (Paragraph 0023). The second AI model is merely used for generating a first set of recommendations of the set of recommendations based on the on the first anomaly data (Paragraph 0026). Merely stating that the step is performed by a computer component (e.g., AI models) results in “apply it” on a computer (MPEP 2106.05f) being applicable at both Step 2A, Prong 2 and Step 2B. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. In this case, the claims do not provide any specific details about how the AI models operate (e.g., how the recommendations are generated). Also, the plain meaning of the “providing response data feedback” step is merely describing how the AI models are receiving continuous data to iteratively learn about anomalies in the response data (e.g., validating the data based on an accuracy/performance score). See 2024 AI Guidance, example 47, claim 2. Further, the step of “receiving updated assessment data” is considered a well-understood, routing, and conventional function since it's just “performing repetitive calculations” and “receiving or transmitting data over a network” (MPEP 2106.05d). Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Dependent claims 11 and 19 are not directed to any additional claim elements. Rather, these claims offer further descriptive limitations of the abstract idea mentioned above - such as: wherein the one or more risk parameters comprise financial stability indicators of the first entity, a regulatory compliance status of the first entity, security posture metrics of the first entity, and an incident history of the first entity. These processes are similar to the abstract idea noted in the independent claim because they further the limitations of the independent claim which are directed to certain methods of organizing human activity which include managing personal behavior (e.g., managing risk of an entity). In addition, there are no additional elements to consider at Step 2A Prong 2 and Step 2B. Therefore, the claims still recite an abstract idea that can be grouped into certain methods of organizing human activity.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-8 and 11-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sabourin et al. (US 2022/0335136 A1).
Regarding claim 1 (Currently Amended), Sabourin et al. discloses a computer-implemented method, comprising (Paragraph 0005, In general, various aspects of the present disclosure provide methods, apparatuses, systems, computing devices, computing entities, and/or the like for addressing a modified risk rating identifying a risk to a first entity of having computer-implemented functionality provided by a vendor integrated with a computing system of the first entity):
receiving, by a computer, assessment data associated with a first entity, wherein the assessment data comprises at least a set of questions and a first set of responses for the set of questions, and the set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity (Paragraph 0036, To combat this problem, many entities will perform an assessment of a vendor when considering integrating computer-implemented functionality provided by the vendor with a computing system of the entity. For example, an entity may have the vendor complete a questionnaire that includes questions on various attributes of the computer-implemented functionality that may influence an amount of risk that may be involved in using the functionality. The entity may then evaluate the vendor's answers to the questions to determine a risk (e.g., risk rating) associated with integrating the computer-implemented functionality with the entity's computing system; Paragraph 0062, The risk assessments may be in the form of a questionnaire that asks the vendor certain questions with respect to the computer-implemented functionality. The risk assessments are generally used in evaluating the risk of integrating the computer-implemented functionality with the entity computing systems 190. For example, a questionnaire may ask the vendor to indication whether the vendor is using certain access controls such as two-factor authentication for the computer-implemented functionality. Accordingly, the use or lack of use of the access controls can indicate to an entity a level (e.g., amount) of risk that is involved with integrating the computer-implemented functionality with an entity computing system 190; Examiner interprets the “vendor” as the “first entity.” Also, the “entity providing the evaluation/questionnaire to the vendor” as the “second entity”);
detecting, by the computer, a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules, wherein the set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, an accuracy of the assessment data, and compliance of the assessment data (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0079, In various aspects, the assessment can include multiple question/answer pairings that are related to any particular attribute. For example, the assessment may include a first question asking the personnel whether the vendor has implemented an encryption process for encrypting data that is transferred from an entity (e.g., transferred from an entity computing system 190) to the vendor (e.g., to a vendor computing system 195) to be used in a service provided by the vendor. The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0083, At Operation 435, the dynamic assessment module 120 determines whether the answer to the question contains an inconsistency. If so, then the dynamic assessment module 120 determines whether to address the inconsistency in Operation 440. In various aspects, the dynamic assessment module 120 performs this particular operation by using a decision engine to determine a relevance of a particular identified inconsistency. In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes);
applying, by the computer, a first artificial intelligence (Al) model on the assessment data and the first set of anomalies; generating, by the computer, anomaly data associated with the first set of anomalies based on the applying of the first Al model on the assessment data and the first set of anomalies, wherein the generating of the anomaly data comprises: evaluating, by the first Al model, free-format text associated with the assessment data based on a set of instructions; transforming, by the first Al model, unstructured assessment data including the free-format text into a structured format including key extracted information; and generating, by the first Al model, the anomaly data associated with the first set of anomalies based on the structured format (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0080, Therefore, in Operation 430, the dynamic assessment module 120 compares the answers for the question/answering pairings related to the one or more attributes. The dynamic assessment module 120 may perform this particular operation using various functionality depending on the form of the answers. For example, the answers may be a freeform text format. Therefore, the dynamic assessment module 120 can use one or more natural language processing techniques to compare the answers of the question/answer pairings; Paragraph 0081, For example, the dynamic assessment module 120 may compare a first answer string from a first question/answer pairing with a second answer string from a second question/answer pairing by first utilizing the natural language processing techniques to generate embedded vector representations (e.g., tokenized representations) of the first answer string and the second answer string. The dynamic assessment module 120 can then compare the embedded vector representations for the first and second answer strings to identify whether the first answer string contains an inconsistency; Examiner notes that the NLP is a subfield of artificial intelligence. Also, Examiner interprets “transforming the freeform text format into an embedded vector representation” as “transforming unstructured assessment data into a structured format), and the anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Examiner interprets “information on the inconsistencies” as the “set of reasons for an occurrence” such as inconsistencies in the answers as to whether the vendor encrypts data);
generating, by the computer, a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data, wherein the set of recommendations is generated to resolve the first set of anomalies (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); As stated in Paragraph 0104 of Applicant’s specification, the recommendation may include at least to provide additional supporting documents);
outputting, by the computer, the anomaly data and the set of recommendations on a first electronic device associated with the first entity (Paragraph 0006, In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the first entity that identifies the inconsistency and the attribute for the computer-implemented functionality. In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the vendor that identifies the inconsistency and the attribute for the computer-implemented functionality; Paragraph 0076, The personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.));
receiving, by the computer, one or more inputs based on the outputting of the set of recommendations on the first electronic device associated with the first entity to update the assessment data; and updating, by the computer, the assessment data based on the one or more inputs, wherein the updated assessment data comprises at least one updated response associated with the first set of responses (Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); Paragraph 0085, The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100).
Regarding claim 12 (Currently Amended), Sabourin et al. discloses a computer system, comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media, the program instructions executable by the processor set to cause the processor set to (Paragraph 0005, In general, various aspects of the present disclosure provide methods, apparatuses, systems, computing devices, computing entities, and/or the like for addressing a modified risk rating identifying a risk to a first entity of having computer-implemented functionality provided by a vendor integrated with a computing system of the first entity; Paragraph 0124, A computer program product may include a non-transitory computer-readable storage medium storing applications; Paragraph 0135, The one or more modules 1222 may also reside, completely or at least partially, within main memory 1204 and/or within the processor 1202 during execution thereof by the hardware device 1200- main memory 1204 and processor 1202 also constituting computer-accessible storage media):
receive assessment data associated with a first entity, wherein the assessment data comprises at least a set of questions and a first set of responses for the set of questions, and the set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity (Paragraph 0036, To combat this problem, many entities will perform an assessment of a vendor when considering integrating computer-implemented functionality provided by the vendor with a computing system of the entity. For example, an entity may have the vendor complete a questionnaire that includes questions on various attributes of the computer-implemented functionality that may influence an amount of risk that may be involved in using the functionality. The entity may then evaluate the vendor's answers to the questions to determine a risk (e.g., risk rating) associated with integrating the computer-implemented functionality with the entity's computing system; Paragraph 0062, The risk assessments may be in the form of a questionnaire that asks the vendor certain questions with respect to the computer-implemented functionality. The risk assessments are generally used in evaluating the risk of integrating the computer-implemented functionality with the entity computing systems 190. For example, a questionnaire may ask the vendor to indication whether the vendor is using certain access controls such as two-factor authentication for the computer-implemented functionality. Accordingly, the use or lack of use of the access controls can indicate to an entity a level (e.g., amount) of risk that is involved with integrating the computer-implemented functionality with an entity computing system 190; Examiner interprets the “vendor” as the “first entity.” Also, the “entity providing the evaluation/questionnaire to the vendor” as the “second entity”);
detect a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules, wherein the set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, accuracy of the assessment data, and compliance of the assessment data (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0079, In various aspects, the assessment can include multiple question/answer pairings that are related to any particular attribute. For example, the assessment may include a first question asking the personnel whether the vendor has implemented an encryption process for encrypting data that is transferred from an entity (e.g., transferred from an entity computing system 190) to the vendor (e.g., to a vendor computing system 195) to be used in a service provided by the vendor. The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0083, At Operation 435, the dynamic assessment module 120 determines whether the answer to the question contains an inconsistency. If so, then the dynamic assessment module 120 determines whether to address the inconsistency in Operation 440. In various aspects, the dynamic assessment module 120 performs this particular operation by using a decision engine to determine a relevance of a particular identified inconsistency. In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes);
apply a first artificial intelligence (Al) model on the assessment data and the first set of anomalies; generating, by the computer, anomaly data associated with the first set of anomalies based on the applying of the first Al model on the assessment data and the first set of anomalies, wherein the generating of the anomaly data comprises: evaluating, by the first Al model, free-format text associated with the assessment data based on a set of instructions; transforming, by the first Al model, unstructured assessment data including the free-format text into a structured format including key extracted information; and generating, by the first Al model, the anomaly data associated with the first set of anomalies based on the structured format (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0080, Therefore, in Operation 430, the dynamic assessment module 120 compares the answers for the question/answering pairings related to the one or more attributes. The dynamic assessment module 120 may perform this particular operation using various functionality depending on the form of the answers. For example, the answers may be a freeform text format. Therefore, the dynamic assessment module 120 can use one or more natural language processing techniques to compare the answers of the question/answer pairings; Paragraph 0081, For example, the dynamic assessment module 120 may compare a first answer string from a first question/answer pairing with a second answer string from a second question/answer pairing by first utilizing the natural language processing techniques to generate embedded vector representations (e.g., tokenized representations) of the first answer string and the second answer string. The dynamic assessment module 120 can then compare the embedded vector representations for the first and second answer strings to identify whether the first answer string contains an inconsistency; Examiner notes that the NLP is a subfield of artificial intelligence. Also, Examiner interprets “transforming the freeform text format into an embedded vector representation” as “transforming unstructured assessment data into a structured format), and the anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Examiner interprets “information on the inconsistencies” as the “set of reasons for an occurrence” such as inconsistencies in the answers as to whether the vendor encrypts data);
generate a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data, wherein the set of recommendations is generated to resolve the first set of anomalies (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); As stated in Paragraph 0104 of Applicant’s specification, the recommendation may include at least to provide additional supporting documents);
output the anomaly data and the set of recommendations on a first electronic device associated with the first entity (Paragraph 0006, In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the first entity that identifies the inconsistency and the attribute for the computer-implemented functionality. In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the vendor that identifies the inconsistency and the attribute for the computer-implemented functionality; Paragraph 0076, The personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.));
receive one or more inputs based on the output of the set of recommendations on the first electronic device associated with the first entity to update the assessment data; and update the assessment data based on the one or more inputs, wherein the updated assessment data comprises at least one updated response associated with the first set of responses (Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); Paragraph 0085, The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100).
Regarding claim 20 (Currently Amended), Sabourin et al. discloses a computer program product for a determination and a resolution of a first set of anomalies in assessment data associated with a first entity, the computer program product comprising: one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to perform operations comprising (Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0124, A computer program product may include a non-transitory computer-readable storage medium storing applications; Paragraph 0135, The one or more modules 1222 may also reside, completely or at least partially, within main memory 1204 and/or within the processor 1202 during execution thereof by the hardware device 1200- main memory 1204 and processor 1202 also constituting computer-accessible storage media):
receiving assessment data associated with a first entity, wherein the assessment data comprises at least a set of questions and a first set of responses for the set of questions, and wherein the set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity (Paragraph 0036, To combat this problem, many entities will perform an assessment of a vendor when considering integrating computer-implemented functionality provided by the vendor with a computing system of the entity. For example, an entity may have the vendor complete a questionnaire that includes questions on various attributes of the computer-implemented functionality that may influence an amount of risk that may be involved in using the functionality. The entity may then evaluate the vendor's answers to the questions to determine a risk (e.g., risk rating) associated with integrating the computer-implemented functionality with the entity's computing system; Paragraph 0062, The risk assessments may be in the form of a questionnaire that asks the vendor certain questions with respect to the computer-implemented functionality. The risk assessments are generally used in evaluating the risk of integrating the computer-implemented functionality with the entity computing systems 190. For example, a questionnaire may ask the vendor to indication whether the vendor is using certain access controls such as two-factor authentication for the computer-implemented functionality. Accordingly, the use or lack of use of the access controls can indicate to an entity a level (e.g., amount) of risk that is involved with integrating the computer-implemented functionality with an entity computing system 190; Examiner interprets the “vendor” as the “first entity.” Also, the “entity providing the evaluation/questionnaire to the vendor” as the “second entity”);
detecting the first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules, wherein the set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, accuracy of the assessment data, and compliance of the assessment data (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0079, In various aspects, the assessment can include multiple question/answer pairings that are related to any particular attribute. For example, the assessment may include a first question asking the personnel whether the vendor has implemented an encryption process for encrypting data that is transferred from an entity (e.g., transferred from an entity computing system 190) to the vendor (e.g., to a vendor computing system 195) to be used in a service provided by the vendor. The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0083, At Operation 435, the dynamic assessment module 120 determines whether the answer to the question contains an inconsistency. If so, then the dynamic assessment module 120 determines whether to address the inconsistency in Operation 440. In various aspects, the dynamic assessment module 120 performs this particular operation by using a decision engine to determine a relevance of a particular identified inconsistency. In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes);
applying a first artificial intelligence (Al) model on the assessment data and the first set of anomalies; generating anomaly data associated with the first set of anomalies based on the applying of the first Al model on the assessment data and the first set of anomalies, wherein the generating of the anomaly data comprises: evaluating, by the first Al model, free-format text associated with the assessment data based on a set of instructions; transforming, by the first Al model, unstructured assessment data including the free-format text into a structured format including key extracted information; and generating, by the first Al model, the anomaly data associated with the first set of anomalies based on the structured format (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0080, Therefore, in Operation 430, the dynamic assessment module 120 compares the answers for the question/answering pairings related to the one or more attributes. The dynamic assessment module 120 may perform this particular operation using various functionality depending on the form of the answers. For example, the answers may be a freeform text format. Therefore, the dynamic assessment module 120 can use one or more natural language processing techniques to compare the answers of the question/answer pairings; Paragraph 0081, For example, the dynamic assessment module 120 may compare a first answer string from a first question/answer pairing with a second answer string from a second question/answer pairing by first utilizing the natural language processing techniques to generate embedded vector representations (e.g., tokenized representations) of the first answer string and the second answer string. The dynamic assessment module 120 can then compare the embedded vector representations for the first and second answer strings to identify whether the first answer string contains an inconsistency; Examiner notes that the NLP is a subfield of artificial intelligence. Also, Examiner interprets “transforming the freeform text format into an embedded vector representation” as “transforming unstructured assessment data into a structured format), and the anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0076, Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Examiner interprets “information on the inconsistencies” as the “set of reasons for an occurrence” such as inconsistencies in the answers as to whether the vendor encrypts data);
generating a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data, wherein the set of recommendations is generated to resolve the first set of anomalies (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); As stated in Paragraph 0104 of Applicant’s specification, the recommendation may include at least to provide additional supporting documents);
outputting the anomaly data and the set of recommendations on a first electronic device associated with the first entity (Paragraph 0006, In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the first entity that identifies the inconsistency and the attribute for the computer-implemented functionality. In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the vendor that identifies the inconsistency and the attribute for the computer-implemented functionality; Paragraph 0076, The personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.));
receiving one or more inputs based on the outputting of the set of recommendations on the first electronic device associated with the first entity to update the assessment data; and updating the assessment data based on the one or more inputs, wherein the updated assessment data comprises at least one updated response associated with the first set of responses (Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.); Paragraph 0085, The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100).
Regarding claims 2 and 13 (Currently Amended), which are dependent of claims 1 and 12, Sabourin et al. discloses all the limitations in claims 1 and 12. Sabourin et al. further discloses applying, by the computer, a first set of logic rules of the set of logic rules to the assessment data; and detecting, by the computer, the first set of anomalies in the assessment data based on the one or more risk parameters and the applying of the first set of logic rules to the assessment data (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0079, In various aspects, the assessment can include multiple question/answer pairings that are related to any particular attribute. For example, the assessment may include a first question asking the personnel whether the vendor has implemented an encryption process for encrypting data that is transferred from an entity (e.g., transferred from an entity computing system 190) to the vendor (e.g., to a vendor computing system 195) to be used in a service provided by the vendor. The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0083, At Operation 435, the dynamic assessment module 120 determines whether the answer to the question contains an inconsistency. If so, then the dynamic assessment module 120 determines whether to address the inconsistency in Operation 440. In various aspects, the dynamic assessment module 120 performs this particular operation by using a decision engine to determine a relevance of a particular identified inconsistency. In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes).
Regarding claims 3 and 14 (Currently Amended), which are dependent of claims 2 and 13, Sabourin et al. discloses all the limitations in claims 2 and 13. Sabourin et al. further discloses generating, by the computer, first anomaly data of the anomaly data based on the application of the first Al model on the assessment data and the first set of anomalies, wherein the first anomaly data is indicative of at least a first reason of the set of reasons for the occurrence of each anomaly within the first set of anomalies (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0079, The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0080, Therefore, in Operation 430, the dynamic assessment module 120 compares the answers for the question/answering pairings related to the one or more attributes. The dynamic assessment module 120 may perform this particular operation using various functionality depending on the form of the answers. For example, the answers may be a freeform text format. Therefore, the dynamic assessment module 120 can use one or more natural language processing techniques to compare the answers of the question/answer pairings; Paragraph 0081, For example, the dynamic assessment module 120 may compare a first answer string from a first question/answer pairing with a second answer string from a second question/answer pairing by first utilizing the natural language processing techniques to generate embedded vector representations (e.g., tokenized representations) of the first answer string and the second answer string. The dynamic assessment module 120 can then compare the embedded vector representations for the first and second answer strings to identify whether the first answer string contains an inconsistency; Examiner interprets “information on the inconsistencies” as the “set of reasons for an occurrence such as inconsistencies in the answers as to whether the vendor encrypts data”).
Regarding claims 4 and 15 (Original), which are dependent of claims 3 and 14, Sabourin et al. discloses all the limitations in claims 3 and 14. Sabourin et al. further discloses wherein the first reason is associated with at least one of an absence of at least one response in a first set of responses, an occurrence of incorrect data formatting in the first set of responses, or an occurrence of a set of errors associated with a first set of criteria for the first set of responses (Paragraph 0070, (5) Any other suitable inconsistency between the value of any attribute for the particular vendor that is specified in the newly completed assessment dataset and the corresponding value for the attribute for the particular vendor that is specified in the selected assessment dataset (e.g., different numeric responses, different text responses, responses in the selected assessment dataset for which there is no corresponding response in the newly completed assessment dataset as a result of a blank response, etc.; It can be noted that the claim language is written in alternative form. The limitation taught by Sabourin et al. is based on “an absence of at least one response in a first set of responses")
Regarding claims 5 and 16 (Currently Amended), which are dependent of claims 3 and 14, Sabourin et al. discloses all the limitations in claims 3 and 14. Sabourin et al. further discloses applying, by the computer, a second Al model on the first anomaly data, the first set of anomalies, and the assessment data; and generating, by the computer, a first set of recommendations of the set of recommendations based on the applying of the second Al model on the first anomaly data, the first set of anomalies, and the assessment data, wherein the first set of recommendations is generated to resolve the first set of anomalies (Paragraph 0085, In some aspects, the dynamic assessment module 120 may use a machine-learning model to determine the one or more actions to take to address the inconsistency. For example, the machine-learning model may be a trained model such as a multi-label classification model that processes the particular question/answer pairing and/or the related question/answer pairings and generates a data representation having a set of predictions (e.g., values) in which each prediction is associated with a particular action to take to address the inconsistency. The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information).
Regarding claims 6 and 17 (Currently Amended), which are dependent of claims 5 and 16, Sabourin et al. discloses all the limitations in claims 5 and 16. Sabourin et al. further discloses applying, by the computer, a second set of logic rules of the set of logic rules to the updated assessment data (Paragraph 0083, In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes; Paragraph 0085, In some aspects, the dynamic assessment module 120 may use a machine-learning model to determine the one or more actions to take to address the inconsistency. For example, the machine-learning model may be a trained model such as a multi-label classification model that processes the particular question/answer pairing and/or the related question/answer pairings and generates a data representation having a set of predictions (e.g., values) in which each prediction is associated with a particular action to take to address the inconsistency. The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Examiner notes that the updated assessment data (e.g., follow up questions) is dynamically evaluated based on logic rules);
detecting, by the computer, a second set of anomalies in the updated assessment data based on the applying of the second set of logic rules to the updated assessment data; and outputting, by the computer, the second set of anomalies (Paragraph 0083, At Operation 435, the dynamic assessment module 120 determines whether the answer to the question contains an inconsistency. If so, then the dynamic assessment module 120 determines whether to address the inconsistency in Operation 440. In various aspects, the dynamic assessment module 120 performs this particular operation by using a decision engine to determine a relevance of a particular identified inconsistency. In some aspects, the decision engine may entail a rules-based model that uses a set of rules in determining whether the inconsistency should be addressed. For example, the set of rules may include rules that apply to the level (degree) of inconsistency in determining whether the inconsistency should be address. As a specific example, the set of rules may include a rule that if the inconsistency involves numerically different answers that satisfy a threshold, then the inconsistency should be addressed. In addition, the set of rules may include rules that apply to certain attributes and/or certain types of attributes; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100; Examiner notes that the system is dynamically detecting anomalies for subsequent submissions).
Regarding claims 7 and 18 (Currently Amended), which is dependent of claims 6 and 17, Sabourin et al. discloses all the limitations in claims 6 and 17. Sabourin et al. further discloses applying, by the computer, the first AI model on the updated assessment data and the second set of anomalies; generating, by the computer, second anomaly data of the anomaly data based on the applying of the first AI model on the updated assessment data and the second set of anomalies, wherein the second anomaly data is indicative of at least a second reason of the set of reasons for the occurrence of each anomaly within the second set of anomalies; applying, by the computer, the second AI model on the second anomaly data, the second set of anomalies, and the updated assessment data; generating, by the computer, a second set of recommendations of the set of recommendations based on the applying of the second AI model on the second anomaly data, the second set of anomalies, and the updated assessment data, wherein the second set of recommendations is generated to resolve the second set of anomalies (Paragraph 0085, In some aspects, the dynamic assessment module 120 may use a machine-learning model to determine the one or more actions to take to address the inconsistency. For example, the machine-learning model may be a trained model such as a multi-label classification model that processes the particular question/answer pairing and/or the related question/answer pairings and generates a data representation having a set of predictions (e.g., values) in which each prediction is associated with a particular action to take to address the inconsistency. The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100; Examiner notes that the machine learning is receiving feedback of actions taken by the user to continuously learn the best way to resolve the inconsistencies. The system uses the updated responses/actions to further retrain the machine learning, wherein the retrained machine learning is used for generating a second set of actions/recommendations);
and outputting, by the computer, the second anomaly data, and the second set of recommendations (Paragraph 0006, In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the first entity that identifies the inconsistency and the attribute for the computer-implemented functionality. In some aspects, the action comprises sending, by the computing hardware, an electronic notification to personnel of the vendor that identifies the inconsistency and the attribute for the computer-implemented functionality; Paragraph 0084, If the dynamic assessment module 120 determines the inconsistency should be addressed, then the dynamic assessment module 120 addresses the inconsistency in Operation 445. Accordingly, the dynamic assessment module 120 may determine one or more actions to take to address the inconsistency based on, for example: (1) a number of attributes that are mapped to the particular question/answer pairing; (2) a type of each of the one or more attributes that are mapped to the particular question/answer pairing; (3) one or more past responses by users to the question/answering paring being flagged or similar question/answering pairings being flagged (e.g., one or more past responses to a question/answer pairing with similar mapped attributes as the particular question/answer pairing); and/or (4) any other suitable factors. Accordingly, the dynamic assessment module may identify one or more actions such as, for example: (1) prompting the personnel to provided supporting information and/or documentation to address the inconsistency; (2) provide the personnel with a follow up question related to the inconsistency; (3) request the personnel to readdress the particular question/answer pairing involved in the inconsistency; and/or (4) take any other suitable action related to the inconsistency (e.g., provide an indication that the inconsistency is acceptable, ignore the flagged response, etc.)).
Regarding claim 8 (Original), which is dependent of claim 7, Sabourin et al. discloses all the limitations in claim 7. Sabourin et al. further discloses wherein the second reason comprises at least one of an occurrence of one or more logical flaws in the at least one updated response or an absence of at least one section of data in the at least one updated response (Paragraph 0070, (5) Any other suitable inconsistency between the value of any attribute for the particular vendor that is specified in the newly completed assessment dataset and the corresponding value for the attribute for the particular vendor that is specified in the selected assessment dataset (e.g., different numeric responses, different text responses, responses in the selected assessment dataset for which there is no corresponding response in the newly completed assessment dataset as a result of a blank response, etc.; Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0086, At this point, the dynamic assessment module 120 determines whether the assessment includes another question to ask the personnel in Operation 450. If so, then the dynamic assessment module 120 returns to Operation 410, selects the next question, and performs the operations just discussed for the newly selected question. Once the dynamic assessment module 120 has processed all of the questions for the assessment, the dynamic assessment module 120 records the assessment in Operation 455. For example, the dynamic assessment module 120 may record the assessment by submitting the assessment as a newly completed assessment dataset to the vendor risk management computing system 100; It can be noted that the claim language is written in alternative form. The limitation taught by Sabourin et al. is based on “an occurrence of one or more logical flaws in the at least one updated response." Examiner interprets providing different numeric responses and/or different text responses as the one or more logical flaws in the at least one updated response).
Regarding claims 11 and 19 (Original), which are dependent of claims 1 and 12, Sabourin et al. discloses all the limitations in claims 1 and 12. Sabourin et al. further discloses wherein the one or more risk parameters comprise financial stability indicators of the first entity, a regulatory compliance status of the first entity, security posture metrics of the first entity, and an incident history of the first entity (Paragraph 0004, However, an entity's integration of the service with its computing system can expose the computing system to significant risk. For example, installing the API within the computing system can provide a channel for a nefarious third-party to gain access to the computing system through the vendor's service. Such access can expose the computing system to experiencing a data breach, being hacked, having malware (e.g., a virus and/or ransomware) installed within the computing system, and/or the like. Therefore, any entity that is onboarding computer-implemented functionality provided through a vendor with a computing system of the entity must be able to recognize, manage, and mitigate risks associated with such integration in an effective manner to ensure integrity of the computing system is maintained; Paragraph 0044, In addition, the questionnaire may include one or more questions on security controls that the vendor has put into place with respect to sensitive data that the service may use that is collected through the first entity's computing system; It can be noted that the claim language is written in alternative form. The limitation taught by Sabourin et al. is based on “security posture metrics of the first entity").
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Sabourin et al. (US 2022/0335136 A1), in view of Samuel et al. (US 2026/0023976 A1).
Regarding claim 9 (Currently Amended), which is dependent of claim 7, Sabourin et al. discloses all the limitations in claim 7. Sabourin et al. further discloses transmitting, by the computer, the assessment data to a second electronic device associated with the second entity, wherein the second entity is responsible for the evaluation of the first entity about the operational activity (Paragraph 0036, To combat this problem, many entities will perform an assessment of a vendor when considering integrating computer-implemented functionality provided by the vendor with a computing system of the entity. For example, an entity may have the vendor complete a questionnaire that includes questions on various attributes of the computer-implemented functionality that may influence an amount of risk that may be involved in using the functionality. The entity may then evaluate the vendor's answers to the questions to determine a risk (e.g., risk rating) associated with integrating the computer-implemented functionality with the entity's computing system; Paragraph 0062, The risk assessments may be in the form of a questionnaire that asks the vendor certain questions with respect to the computer-implemented functionality. The risk assessments are generally used in evaluating the risk of integrating the computer-implemented functionality with the entity computing systems 190. For example, a questionnaire may ask the vendor to indication whether the vendor is using certain access controls such as two-factor authentication for the computer-implemented functionality. Accordingly, the use or lack of use of the access controls can indicate to an entity a level (e.g., amount) of risk that is involved with integrating the computer-implemented functionality with an entity computing system 190);
obtaining, by the computer, response data from the second entity based on the transmitting of the assessment data to the second electronic device, wherein the response data is indicative of an occurrence of one or more anomalies in the assessment data (Paragraph 0076, the personnel for the vendor may be executing the dynamic assessment module 120 on a vendor computing system 195 associated with the vendor. Accordingly, as the personnel provides answers to the questions presented for the assessment, the dynamic assessment module 120 detects inconsistencies in the assessment in real time and in response, flags one or more responses related to the inconsistencies for additional action; Paragraph 0079, In various aspects, the assessment can include multiple question/answer pairings that are related to any particular attribute. For example, the assessment may include a first question asking the personnel whether the vendor has implemented an encryption process for encrypting data that is transferred from an entity (e.g., transferred from an entity computing system 190) to the vendor (e.g., to a vendor computing system 195) to be used in a service provided by the vendor. The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute);
determining, by the computer, a … with the first AI model based on the response data and the first set of anomalies, … (Paragraph 0074, In particular aspects, the modification module 110 may also output the modified risk rating by displaying the modified risk rating on a graphical user interface, storing the modified risk rating, sending an electronic communication such as an email with the modified risk rating to personnel of an entity and/or the vendor, and/or the like. The modification module 110 may also include information on the inconsistencies that resulted in the modified risk rating in the output; Paragraph 0079, The assessment may also include a second, different question asking the personnel whether the vendor has put any controls in place for ensuring secure data transfers. Therefore, both the first and second question/answering pairings touch on the same attribute as to whether the vendor encrypts data that is transfer from an entity to the vendor. Accordingly, the assessment is configured in this manner to require the personnel to provides multiple answers related to the same attribute to ensure the personnel is providing complete, consistent, and accurate answers (information) related to the attribute; Paragraph 0080, Therefore, in Operation 430, the dynamic assessment module 120 compares the answers for the question/answering pairings related to the one or more attributes. The dynamic assessment module 120 may perform this particular operation using various functionality depending on the form of the answers. For example, the answers may be a freeform text format. Therefore, the dynamic assessment module 120 can use one or more natural language processing techniques to compare the answers of the question/answer pairings; Paragraph 0081, For example, the dynamic assessment module 120 may compare a first answer string from a first question/answer pairing with a second answer string from a second question/answer pairing by first utilizing the natural language processing techniques to generate embedded vector representations (e.g., tokenized representations) of the first answer string and the second answer string. The dynamic assessment module 120 can then compare the embedded vector representations for the first and second answer strings to identify whether the first answer string contains an inconsistency); …
Although Sabourin et al. discloses applying a first AI model for the detection of the first set of anomalies in the assessment data (e.g., a natural language processing to detect inconsistencies in the data such as different text responses), Sabourin et al. does not specifically disclose how the performance of the first AI is evaluated (e.g., accuracy of the model).
However, Samuel et al. discloses transmitting, by the computer, the assessment data to a second electronic device associated with the second entity, wherein the second entity is responsible for the evaluation of the first entity about the operational activity (Paragraph 0002, Every year, millions of people, businesses, and organizations around the world use software applications for building and processing electronic datasets. For example, a given software application may be used to complete and submit datasets such as forms, tax returns, product orders, job applications, and/or the like);
obtaining, by the computer, response data from the second entity based on the transmitting of the assessment data to the second electronic device, wherein the response data is indicative of an occurrence of one or more anomalies in the assessment data (Paragraph 0003, However, creating a computing system that allows for seamless automated submission and processing of datasets presents many technical challenges. User submissions may, for instance, contain errors, omissions, incorrectly formatted data, and/or the like that prevent the software application from processing the submission or cause the software application to incorrectly process the submission. To prevent such errors, a software application may, for example, contain manually-written software code that defines acceptable ranges, formats, etc. for a submission and does not allow a user to submit a dataset until the submission complies with the requirements; Paragraph 0015, According to certain embodiments, one or more rules for datasets may be provided, and embedding representations of the rules may be created. The embedding representations may be stored in a vector store. A user may submit an electronic dataset, and a machine learning model may identify one or more rules (e.g., from the vector store) that are relevant to the dataset. The same or a different machine learning model may evaluate the dataset based on the embedding representations of the rules to determine whether the dataset complies with the rules. One or more actions may be taken based on the evaluation; Paragraph 0017, For example, a rule may specify a range within which a value within a dataset should fall. As another example, a rule may specify a format for either a dataset or a component of a dataset (e.g., a file format for an attachment that is included with the submission). Rules may also specify additional information that is required or recommended based on information within a dataset; Paragraph 0038, The first machine learning model 230 (or a different machine learning model) may use the identified rules to evaluate the dataset 120. For example, the first machine learning model 230 may interpret an embedding representation of a rule and evaluate the dataset 120 based on the interpretation (e.g., determine whether the dataset 120 satisfies the identified rule));
determining, by the computer, a performance score associated with the first AI model based on the response data and the first set of anomalies, wherein the performance score is associated with a performance of the first AI model for the detection of the first set of anomalies in the assessment data (Paragraph 0017, For example, a rule may specify a range within which a value within a dataset should fall. As another example, a rule may specify a format for either a dataset or a component of a dataset (e.g., a file format for an attachment that is included with the submission). Rules may also specify additional information that is required or recommended based on information within a dataset; Paragraph 0038, The first machine learning model 230 (or a different machine learning model) may use the identified rules to evaluate the dataset 120. For example, the first machine learning model 230 may interpret an embedding representation of a rule and evaluate the dataset 120 based on the interpretation (e.g., determine whether the dataset 120 satisfies the identified rule); Paragraph 0039, The embedding model 210 and/or the first machine learning model 230 may be trained through a supervised learning process. Supervised learning techniques generally involve providing training inputs to a machine learning model. The machine learning model processes the training inputs and outputs predictions based on the training inputs. The predictions are compared to the known labels associated with the training inputs to determine the accuracy of the machine learning model, and parameters of the machine learning model are iteratively adjusted until one or more conditions are met. For instance, the one or more conditions may relate to an objective function (e.g., a cost function or loss function) for optimizing one or more variables (e.g., model accuracy); Paragraph 0040, The training data for the supervised learning process involving the embedding model 210 may include datasets that are labeled based on whether the datasets satisfy a set of rules. Embedding representations of the rules may be generated. If a variance exists between a ground truth label and a prediction as to whether a rule was satisfied (e.g., a prediction made by the first machine learning model 230 based on an embedding representation of the rule generated by embedding model 210), the embedding model 210 may be retrained. For example, if a ground truth label indicates that a rule was satisfied, but the first machine learning model 230 determines that the rule was not satisfied based on the embedding representation of the rule, this may indicate that the embedding representation contains errors or otherwise needs refinement. Thus, the embedding model 210 may be retrained and/or one or more parameters of the embedding model 210 may be updated);
validating, by the computer, the performance of the first AI model based on the performance score and a threshold performance score; and training, by the computer, the first AI model based on the response data and the first set of anomalies upon the validation of the performance of the first AI model (Paragraph 0017, For example, a rule may specify a range within which a value within a dataset should fall. As another example, a rule may specify a format for either a dataset or a component of a dataset (e.g., a file format for an attachment that is included with the submission). Rules may also specify additional information that is required or recommended based on information within a dataset; Paragraph 0038, The first machine learning model 230 (or a different machine learning model) may use the identified rules to evaluate the dataset 120. For example, the first machine learning model 230 may interpret an embedding representation of a rule and evaluate the dataset 120 based on the interpretation (e.g., determine whether the dataset 120 satisfies the identified rule); Paragraph 0039, The embedding model 210 and/or the first machine learning model 230 may be trained through a supervised learning process. Supervised learning techniques generally involve providing training inputs to a machine learning model. The machine learning model processes the training inputs and outputs predictions based on the training inputs. The predictions are compared to the known labels associated with the training inputs to determine the accuracy of the machine learning model, and parameters of the machine learning model are iteratively adjusted until one or more conditions are met. For instance, the one or more conditions may relate to an objective function (e.g., a cost function or loss function) for optimizing one or more variables (e.g., model accuracy); Paragraph 0040, The training data for the supervised learning process involving the embedding model 210 may include datasets that are labeled based on whether the datasets satisfy a set of rules. Embedding representations of the rules may be generated. If a variance exists between a ground truth label and a prediction as to whether a rule was satisfied (e.g., a prediction made by the first machine learning model 230 based on an embedding representation of the rule generated by embedding model 210), the embedding model 210 may be retrained. For example, if a ground truth label indicates that a rule was satisfied, but the first machine learning model 230 determines that the rule was not satisfied based on the embedding representation of the rule, this may indicate that the embedding representation contains errors or otherwise needs refinement. Thus, the embedding model 210 may be retrained and/or one or more parameters of the embedding model 210 may be updated).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the first AI used for a determination of a first set of anomalies in assessment data associated with a first entity of the invention of Sabourin et al. to further specify how the performance of the first AI is evaluated of the invention of Samuel et al. because doing so would allow the first AI to iteratively adjust parameters until one or more conditions are met (see Samuel et al., Paragraph 0039). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 10 (Original), which is dependent of claim 9, Sabourin et al. discloses all the limitations in claim 9. Sabourin et al. further discloses obtaining, by the computer, one or more recommendations from the second electronic device to resolve the first set of anomalies upon the transmission of the assessment data to the second electronic device; validating, by the computer, … of the second AI model based on the one or more recommendations and the first set of recommendations; and training, by the computer, the second AI model based on the one or more recommendations and the first set of recommendations upon the validation of the performance of the second AI model (Paragraph 0085, In some aspects, the dynamic assessment module 120 may use a machine-learning model to determine the one or more actions to take to address the inconsistency. For example, the machine-learning model may be a trained model such as a multi-label classification model that processes the particular question/answer pairing and/or the related question/answer pairings and generates a data representation having a set of predictions (e.g., values) in which each prediction is associated with a particular action to take to address the inconsistency. The machine-learning model may be trained, for example, using training data derived from users' responses to follow up requests previously provided for inconsistencies detected in one or more assessment question/answer pairings such as: (1) whether the user ignored the flagged question or related follow up action; (2) a particular type of action the user took in response to the flagged question and/or related follow up action (e.g., providing support for the response, implementing a remediating action, modifying one or more attributes, etc.); (3) one or more frameworks and/or standards that are mapped to the flagged question; and/or (4) any other suitable information; Examiner notes that the machine learning is receiving feedback of actions taken by the user to continuously learn the best way to resolve the inconsistencies).
Although Sabourin et al. discloses applying a second AI model for generating one or more recommendations to resolve the first set of anomalies (e.g., a machine-learning model to determine the one or more actions to take to address the inconsistency), Sabourin et al. does not specifically disclose how the performance of the second AI is evaluated (e.g., accuracy of the model).
However, Samuel et al. discloses obtaining, by the computer, one or more recommendations from the second electronic device to resolve the first set of anomalies upon the transmission of the assessment data to the second electronic device; validating, by the computer, a performance of the second AI model based on the one or more recommendations and the first set of recommendations; and training, by the computer, the second AI model based on the one or more recommendations and the first set of recommendations upon the validation of the performance of the second AI model (Paragraph 0042, The results of the evaluation (e.g., an indication of one or more rules that were not satisfied) may be provided to a second machine learning model 240. The second machine learning model 240 may be a language processing machine learning model such as a Large Language Model (LLM). The second machine learning model 240 may be trained and/or otherwise configured to generate an evaluation summary 250 based on the evaluation. The evaluation summary 250 may comprise natural language instructions, suggestions, indications, and/or the like that help a user understand and/or correct problems with the dataset 120. For example, if the dataset 120 does not comply with a rule, the evaluation summary 250 may tell a user that the dataset 120 does not comply with the rule (e.g., the evaluation summary 250 may indicate one or more rules that the dataset 120 violates) and/or provide the user with instructions and/or tips for correcting the dataset 120; Paragraph 0043, As discussed above with respect to FIG. 1, user feedback may be received based on the results of the evaluation (e.g., from server-side user, or from a client-side user based on an evaluation summary 250). The feedback may comprise natural language feedback, a selection of a multiple choice answer to a question regarding the accuracy of the dataset evaluation engine 150, and/or the like. One or more machine learning models (e.g., embedding model 210, first machine learning model 230, and/or second machine learning model 240) may be retrained based on the user feedback. For example, the user feedback may be used as a ground truth label in a supervised learning process as described above).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the second AI used for generating one or more recommendations to resolve the first set of anomalies of the invention of Sabourin et al. to further specify how the performance of the second AI is evaluated of the invention of Samuel et al. because doing so would allow the second AI to be retrained based on the user feedback (see Samuel et al., Paragraph 0043). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Response to Arguments
Applicant's arguments filed on 07/01/2026 (related to the 103 Rejection) have been fully considered but are moot in view of new grounds of rejection. Applicant's amendments necessitated the new ground(s) of rejection presented in this Office action. Rejection based on a newly cited reference(s) follows.
Applicant's arguments filed on 07/01/2026 (related to the 112 Rejection) have been fully considered but they are not persuasive.
Applicant states, on pages 16-17, that the Specification explicitly discloses that the computer readable storage medium is not to be construed as being transitory signals per se (see ¶ 0043).
Examiner respectfully disagrees with Applicant. Although the specification recites the exclusion of transitory signals (see ¶ 0043), the computer readable-medium as claimed does not explicitly exclude transitory media. Examiner recommends to change the language from “computer readable medium” to “non-transitory computer readable medium.”
Applicant's arguments filed on 07/01/2026 (related to the 101 Rejection) have been fully considered but they are not persuasive.
Applicant states, on pages 18-26, that the Applicant respectfully traverses the Examiner's finding under MPEP 2106.04(a)(2) and submits that, the claimed features are not directed to a certain methods of organizing human activity but to a computer-implemented processing of assessment data, including anomaly detection, Al-based evaluation of free-format text, transformation of unstructured data into a structured format, and generation of anomaly data and recommendations, thereby analyzing and transforming unstructured free-format text to generate anomaly data indicative of reasons for occurrences of identified anomalies in the assessment data and generating recommendations to resolve the identified anomalies by updating the assessment data based on the recommendations. Hence, the claimed operations do not fall within the "certain methods of organizing human activity" grouping under MPEP § 2106.04(a)(2).
Also, the introduction of the system 202 now provides the first entity 302 with real-time feedback for updating the assessment data. This enhancement allows the first entity 302 to improve the set of responses in the assessment data, ensuring they are complete, correct, and compliant. The system 202 includes a logical engine 310, the Al engine 312, and a response data repository 314 ... [t]his proactive correction mechanism alleviates the processing load on the system 202 by decreasing the volume of erroneous submissions that require re-evaluation. Further, the proactive correction mechanism also enhances overall efficiency of the system 202 and accelerates data handling, leading to faster decision-making and improved operational responsiveness ... [t]he system's use of LLMs ensures a higher accuracy rate in evaluating free-text responses. This advanced analysis allows for a thorough understanding of context, nuances, and intent within the submissions, which ... increases the likelihood of prompt approvals". See the as-filed Specification at ¶¶ [0018]-[0020], [0081] and [0143].
The Applicant respectfully traverses the Examiner's findings under MPEP § 2106.05(h) and submits that the recited computer is not merely a field-of-use limitation or a generic computer used to receive assessment data. Rather, the claimed invention is applied in a practical technological context by improving the manner in which a computer-based risk assessment platform processes assessment data containing free-format text and unstructured responses. Unlike conventional systems that allow inaccurate, inconsistent, or incomplete assessment data to propagate through subsequent processing stages, the claimed method detects anomalies, evaluates free-format text using an Al model, transforms unstructured assessment data into a structured format, generates anomaly data explaining the detected anomalies, generates recommendations for resolving the detected anomalies.
Further, by identifying and resolving logical flaws, information gaps, and inconsistencies at an earlier stage of the processing pipeline, the claimed method reduces the volume of erroneous assessment data requiring re-evaluation, decreases processing load on the system, improves data-handling efficiency, and accelerates assessment-data evaluation. Furthermore, the Al-based evaluation of free-format text enables more accurate processing of contextual information contained in assessment submissions, thereby improving the quality, reliability, and efficiency of assessment-data processing within the computer-based risk assessment platform.
Therefore, the recited computer does not constitute a mere "field of use" limitation under MPEP § 2106.05(h) instead integrates any alleged judicial exception into a practical application.
Examiner respectfully disagrees with Applicant. Step 2A, Prong One: Claim 1 limitations are still considered to be abstract ideas because they are directed to “certain methods of organizing human activity” which include “managing personal behavior.” In this case, evaluating risk parameters associated with an entity based on one or more rules is merely following rules or instructions (see MPEP 2106.04(a)(2)). Also, the limitations recite “collecting information, analyzing it, and displaying certain results of the collection and analysis," where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind (see MPEP 2106.04(a)). Examiner notes that the limitations of: receiving assessment data …; transforming unstructured data into structured data …; and generating the anomaly data … based on the structured format are merely describing evaluation of assessment data to identify anomalies, which is considered a mental process. Although the claim further requires “applying a first artificial intelligence model” and “generating recommendations based on the anomaly data,” the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process (see MPEP 2106.04(a), a claim that requires a computer may still recite a mental process). If a claim limitation, under its broadest reasonable interpretation, covers evaluations, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A, Prong Two: Claim 1 includes additional elements: a computer; and a first artificial intelligence (Al) model. The computer-implemented method includes receiving assessment data associated with a first entity. The assessment data includes at least a set of questions and a first set of responses for the set of questions. The set of questions is associated with an evaluation of the first entity conducted by a second entity about an operational activity. The computer-implemented method further includes detecting a first set of anomalies in the assessment data based on one or more risk parameters associated with the first entity and a set of logic rules. The set of logic rules is rules associated with the assessment data to evaluate an integrity of the assessment data, an accuracy of the assessment data, and compliance of the assessment data. Further, the computer-implemented method includes generating anomaly data associated with the first set of anomalies based on the assessment data and the first set of anomalies. The anomaly data is indicative of a set of reasons for an occurrence of each anomaly of the first set of anomalies in the assessment data. The computer-implemented method further includes generating a set of recommendations based on the anomaly data, the first set of anomalies, and the assessment data. The set of recommendations is generated to resolve the first set of anomalies. The computer-implemented method further includes outputting the anomaly data and the set of recommendations (Paragraph 0003). The first AI uses a text processing unit configured to analyze and transform raw and unstructured assessment data into a structured format that can be further processed and utilized within the system (Paragraph 0090). These elements of “computer” and “first AI model” are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer element (MPEP 2106.05f). In this case, the first AI model includes inputs (e.g., responses in an unstructured format) and outputs (e.g., responses in a structured format). Although the specification states that the first AI model transforms unstructured data into structured data (Paragraph 0090), the claim and specification do not include any specific details about how the IA model operates (e.g., how the data is transformed). Thus, the analysis step of the model is a black box, which is merely claiming the idea of a solution or outcome (MPEP 2106.05f). Also, the step of “generating recommendations” is merely outputting data (MPEP 2106.05g).
Step 2B: As discussed in Step 2A, Prong Two above, the recitation of a computer and a first AI model amounts to no more than mere instructions to apply the exception using a generic computer component. Also, the steps of “receiving one or more inputs based on the outputting of the set of recommendations to update the assessment data” and “updating the assessment data based on the one or more inputs” are considered conventional computer functions of “receiving and transmitting over a network” and “performing repetitive calculations” (MPEP 2106.05d). Therefore, claim 1 is similar to example 47, claim 2 of July 2024 AI Subject Matter Eligibility.
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. The claim is ineligible.
Independent claims 12 and 20 recite similar features and therefore are rejected for the same reasons as independent claim 1. Claims 2-11 and 13-18 are rejected for having the same deficiencies as those set forth with respect to the claims that they depend from, independent claims 1 and 12.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Baggett et al. (US 6,925,443 B1) – discloses step 304 outputs deficiency statements based on the analysis of user responses, vulnerabilities and/or other considerations. Deficiency statements can be directed to technical and/or non-technical issues. Deficiency statements can include, without limitation, lists of identified vulnerabilities, deficiencies, critical deficiencies, and risks. Example embodiments of this process are described below. Deficiency statements can also include suggested corrective actions. Other example types of deficiency statements are found throughout this specification (see at least Column 15, lines 23-31).
Bolukbas et al. (WO 2024/167782 A1) – discloses a cyber-risk assessment system that cross-correlates various cybersecurity standards and frameworks along with user- submitted questionnaires and security policies to estimate compliance level of an entity to various standards, frameworks, and regulations with non-intrusive data gathering and risk scoring according to the present teaching (see at least Paragraph 0006).
Glas (US 2024/0275809 A1) – discloses the machine learning model may be trained using any suitable supervised learning approach, and the machine learning model may be trained on a dataset of known questionnaire responses. Various algorithms, like decision trees or neural networks, can be used to handle the textual data and learn patterns. The trained model may then be able to score new responses based on these learned patterns. The scoring algorithm and the scores generated by the trained model may be validated against manually scored responses for accuracy. In some embodiments, the machine learning model may be continually improved through a feedback loop that incorporates user feedback and regularly updates the model with new training data to maintain relevance and accuracy (see at least Paragraph 0079).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARJORIE PUJOLS-CRUZ whose telephone number is (571)272-4668. The examiner can normally be reached Mon-Thru 7:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia H Munson can be reached at (571)270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.P./Examiner, Art Unit 3624 /PATRICIA H MUNSON/Supervisory Patent Examiner, Art Unit 3624
Prosecution Insights