Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is in response to an amendment filed on September 23rd, 2025. The amendments have been entered. Claims 1-20 are pending in the current application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 5, 6, 9, 11, 14, 15, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nallapati (U.S Patent No. US 11604794 B1) in view of Xiaofei Ma (Contrastive Fine-tuning Improves Robustness for Neural Rankers (NPL)) in further view of Cicero Nogueira dos Santos. (Herein referred to as Cicero) (Beyond [CLS] through Ranking by Generation (NPL)) and in further view of Yu Gu. (Herein referred to as Gu) (Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases (NPL))
Regarding claim 1, Nallapati teaches a method of knowledge base question answering, the method comprising: receiving, via a communication interface, a training dataset comprising a question and a corresponding logical form, (“Candidate entity linkages may be determined between an entity recognized in the natural language query and columns in data sets… template queries may be generated and prompted as auto completion for natural language queries, which may guide users to submitting a natural language query that can be understood with a high confidence using the machine learning models from whose training data the template queries have been derived… Entity recognition model 410 may take as input a natural language query, as well as any user-specified entity linkages as discussed above”, pg. 1, Abstract; column 4, lines 12-17; column 16, lines 21-23) (The “template queries” correspond to training data, with a query, comprising a question, and entity linkages between a query and column in data sets corresponding to logical forms.) generating, by accessing a knowledge base, a set of candidate logical forms, (“Linkable entities (e.g., unlinked and/or conflicting entities) identified in natural language query 306 determined by entity detection model 410 may be provided to entity linkage candidate index search 420, in some embodiments. Entity linkage candidate index search 420 may retrieve a list of linking candidates for each entity. For example, for entities of type “column name”, candidates may be retrieved from a column name index. The column name index, which may be generated from metadata provided (as discussed) above for the various available data sets, may be used to identify column names using known column names, column aliases or column “friendly names” associated with a column.”, columns 16 and 17, lines 65-68 for column 16 and lines 1-9 for column 17) (The next quote is what was discussed above in regards to metadata.) (“manually specified metadata can augment database schemas with metadata that can be used by algorithms (e.g., various machine learning models natural language query processing pipeline stages discussed below) to improve question answering…Data set metadata management 340 may take the candidate data sets 404 and obtain those data sets metadata (e.g., from automated and manually specified sources) to generate metadata snapshot 406. Metadata snapshot 406 may be used by various stages of natural language query processing 216 to inform analysis.”, column 8, lines 55-60; column 10, lines 7-12) and generating, by the trained ranking model, a ranked list of candidate logical forms from the set of candidate logical forms based on similarity scores between the question and the set of candidate logical forms, respectively. (“a machine learning model may be used to rank the candidate entity linkages according to respectively determined confidence scores, in some embodiments.”, column 23, lines 25-28) (The confidence score correspond to similarity scores)
However, Nallapati does not explicitly teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms, nor training a generation model based on a loss objective using the generated ranked list as training data.
Xiaofei Ma teaches training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms. (“Contrastive fine-tuning for neural rankers. During fine-tuning, a batch of positive and negative samples from different queries is fed into a neural encoder. The embeddings of query-document pairs from the same query are used to generate ranking scores, which are employed to compute the ranking loss. In parallel, the embeddings of all pairs are used to compute the contrastive loss.”, Figure 1 (See the figure below))
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of knowledge base question answering, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma. One of ordinary skill in the art would have been motivated to combine the teachings as fine-tuning a model with contrastive loss improves the robustness and efficiency of a model, as disclosed by Xiaofei Ma, (“As shown in Table 5, adding contrastive loss improves model robustness against all types of perturbations we tested.” pg. 6, right column, under Section 5.3, “Robustness to Query Perturbations”)
However, Nallapati, as modified by Xiaofei Ma, still does not teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; nor training a generation model based on a loss objective using the generated ranked list as training data.
Cicero teaches training a generation model based on a loss objective using the generated ranked list as training data. (“In this task, given an input question and a set of candidate passages, the goal is to rank the candidate passages so that passages containing the correct answer appear at the top of the ranked list… We fine-tune the LM using the following loss function… A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models.”, pg. 1, right column, second paragraph; pg. 2, right column, bottom of the page; pg. 5, directly under section 3.5 “Question and Passage Generation”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of knowledge base question answering, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero. One of ordinary skill in the art would have been motivated to combine the teachings of Nallapati as modified by Xiaofei Ma, and Cicero, as using a ranked list as training data for a generative model allows for generated outputs to be using as additional training data, improving models, as disclosed by Cicero. (“A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models such as BERT-PR.” pg. 5, directly under section 3.5 “Question and Passage Generation”)
However, the combination does not explicitly teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base;
Gu teaches functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; (“We employ a simple yet effective strategy to generate candidate logical forms: We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question”, pg. 8, left column, under “4.4 Inference”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of knowledge base question answering, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero, with the strategy to generate candidate logical forms, as disclosed by Gu. One of ordinary skill in the art would have been motivated to combine the teachings, prior to the current application, as the strategy aids in inference for a model, as disclosed by Gu. (“We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question. The recall is 80% on GrailQA. Questions with superlatives and comparatives often do not have a topic entity and are therefore not covered. Ranking can prune the search space… Our Ranking model achieves an F1 of 25.0%, which significantly outperforms the prior art SPARQA [37] by 3.5 percent.”, pg. 8, left column, under “4.4 Inference”)
PNG
media_image1.png
360
793
media_image1.png
Greyscale
Image: Xiaofei Ma’s Figure 1
Regarding claim 11, Nallapati teaches a system of knowledge base question answering, the system comprising: a communication interface receiving a training dataset comprising a question and a corresponding logical form, (In Nallapati’s disclosure, linkages/linkable entities are the logical forms, as there are in a form that has logic associated with it.) (“template queries may be generated and prompted as auto completion for natural language queries, which may guide users to submitting a natural language query that can be understood with a high confidence using the machine learning models from whose training data the template queries have been derived… Entity recognition model 410 may take as input a natural language query, as well as any user-specified entity linkages as discussed above”, column 4, lines 12-17; column 16, lines 21-23) a memory storing a plurality of processor-executable instruction; (“System memory 1020 may store program instructions and/or data accessible by processor 1010.”, column 26, lines 26-27) a processor reading and executing the plurality of processor-executable instructions to perform operations, (“System memory 1020 may store program instructions and/or data accessible by processor 1010.”, column 26, lines 26-27) generating, by accessing a knowledge base, a set of candidate logical forms, (“Linkable entities (e.g., unlinked and/or conflicting entities) identified in natural language query 306 determined by entity detection model 410 may be provided to entity linkage candidate index search 420, in some embodiments. Entity linkage candidate index search 420 may retrieve a list of linking candidates for each entity. For example, for entities of type “column name”, candidates may be retrieved from a column name index. The column name index, which may be generated from metadata provided (as discussed) above for the various available data sets, may be used to identify column names using known column names, column aliases or column “friendly names” associated with a column.”, columns 16 and 17, lines 65-68 for column 16 and lines 1-9 for column 17) (The next quote is what was discussed above in regards to metadata.) (“manually specified metadata can augment database schemas with metadata that can be used by algorithms (e.g., various machine learning models natural language query processing pipeline stages discussed below) to improve question answering…Data set metadata management 340 may take the candidate data sets 404 and obtain those data sets metadata (e.g., from automated and manually specified sources) to generate metadata snapshot 406. Metadata snapshot 406 may be used by various stages of natural language query processing 216 to inform analysis.”, column 8, lines 55-60; column 10, lines 7-12) and generating, by the trained ranking model, a ranked list of candidate logical forms from the set of candidate logical forms based on similarity scores between the question and the set of candidate logical forms, respectively. (“a machine learning model may be used to rank the candidate entity linkages according to respectively determined confidence scores, in some embodiments.”, column 23, lines 25-28) (The confidence score correspond to similarity scores)
However, Nallapati does not teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms, nor training a generation model based on a loss objective using the generated ranked list as training data.
Xiaofei Ma teaches training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms. (“Contrastive fine-tuning for neural rankers. During fine-tuning, a batch of positive and negative samples from different queries is fed into a neural encoder. The embeddings of query-document pairs from the same query are used to generate ranking scores, which are employed to compute the ranking loss. In parallel, the embeddings of all pairs are used to compute the contrastive loss.”, Figure 1 (See the figure above))
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the knowledge base question answering system, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma. One of ordinary skill in the art would have been motivated to combine the teachings as fine-tuning a model with contrastive loss improves the robustness and efficiency of a model, as disclosed by Xiaofei Ma, (“As shown in Table 5, adding contrastive loss improves model robustness against all types of perturbations we tested.” pg. 6, right column, under Section 5.3, “Robustness to Query Perturbations”)
However, Nallapati, as modified by Xiaofei Ma, still does not teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; training a generation model based on a loss objective using the generated ranked list as training data.
Cicero teaches training a generation model based on a loss objective using the generated ranked list as training data. (“In this task, given an input question and a set of candidate passages, the goal is to rank the candidate passages so that passages containing the correct answer appear at the top of the ranked list… We fine-tune the LM using the following loss function… A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models.”, pg. 1, right column, second paragraph; pg. 2, right column, bottom of the page; pg. 5, directly under section 3.5 “Question and Passage Generation”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the knowledge base question answering system, as disclosed by Nallapati and modified by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero. One of ordinary skill in the art would have been motivated to combine the teachings as using a ranked list as training data for a generative model allows for generated outputs to be using as additional training data, improving models, as disclosed by Cicero. (“A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models such as BERT-PR.” pg. 5, directly under section 3.5 “Question and Passage Generation”)
However, the combination does not explicitly teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base;
Gu teaches functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; (“We employ a simple yet effective strategy to generate candidate logical forms: We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question”, pg. 8, left column, under “4.4 Inference”) (With the logical forms in the database of Nallapati, Gu’s strategy could be easily configured to function with the method of Nallapati to generate a set of candidate logical forms.)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of knowledge base question answering, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero, with the strategy to generate candidate logical forms, as disclosed by Gu. One of ordinary skill in the art would have been motivated to combine the teachings, prior to the current application, as the strategy aids in inference for a model, as disclosed by Gu. (“We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question. The recall is 80% on GrailQA. Questions with superlatives and comparatives often do not have a topic entity and are therefore not covered. Ranking can prune the search space… Our Ranking model achieves an F1 of 25.0%, which significantly outperforms the prior art SPARQA [37] by 3.5 percent.”, pg. 8, left column, under “4.4 Inference”)
Regarding claim 20, Nallapati teaches a processor-readable non-transitory storage medium storing a plurality of processor- executable instructions for knowledge base question answering, the instructions being executed by one or more processors to perform operations, (“In some embodiments, instructions stored on a non-transitory, computer-accessible medium separate from computer system 1000 may be transmitted to computer system 1000 via transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as a network and/or a wireless link.”, column 28, lines 6-12) receiving, via a communication interface, a training dataset comprising a question and a corresponding logical form, (In Nallapati’s disclosure, linkages/linkable entities are the logical forms, as there are in a form that has logic associated with it.) (“template queries may be generated and prompted as auto completion for natural language queries, which may guide users to submitting a natural language query that can be understood with a high confidence using the machine learning models from whose training data the template queries have been derived… Entity recognition model 410 may take as input a natural language query, as well as any user-specified entity linkages as discussed above”, column 4, lines 12-17; column 16, lines 21-23) generating, by accessing a knowledge base, a set of candidate logical forms (“Linkable entities (e.g., unlinked and/or conflicting entities) identified in natural language query 306 determined by entity detection model 410 may be provided to entity linkage candidate index search 420, in some embodiments. Entity linkage candidate index search 420 may retrieve a list of linking candidates for each entity. For example, for entities of type “column name”, candidates may be retrieved from a column name index. The column name index, which may be generated from metadata provided (as discussed) above for the various available data sets, may be used to identify column names using known column names, column aliases or column “friendly names” associated with a column.”, columns 16 and 17, lines 65-68 for column 16 and lines 1-9 for column 17) (The next quote is what was discussed above in regards to metadata.) (“manually specified metadata can augment database schemas with metadata that can be used by algorithms (e.g., various machine learning models natural language query processing pipeline stages discussed below) to improve question answering…Data set metadata management 340 may take the candidate data sets 404 and obtain those data sets metadata (e.g., from automated and manually specified sources) to generate metadata snapshot 406. Metadata snapshot 406 may be used by various stages of natural language query processing 216 to inform analysis.”, column 8, lines 55-60; column 10, lines 7-12) and generating, by the trained ranking model, a ranked list of candidate logical forms from the set of candidate logical forms based on similarity scores between the question and the set of candidate logical forms, respectively. (“a machine learning model may be used to rank the candidate entity linkages according to respectively determined confidence scores, in some embodiments.”, column 23, lines 25-28) (The confidence score correspond to similarity scores)
However, Nallapati does not teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms, nor training a generation model based on a loss objective using the generated ranked list as training data.
Xiaofei Ma teaches training a ranking model based on a contrastive loss using the corresponding logical form as a positive sample and negative samples from the generated set of candidate logical forms. (“Contrastive fine-tuning for neural rankers. During fine-tuning, a batch of positive and negative samples from different queries is fed into a neural encoder. The embeddings of query-document pairs from the same query are used to generate ranking scores, which are employed to compute the ranking loss. In parallel, the embeddings of all pairs are used to compute the contrastive loss.”, Figure 1 (See the figure above))
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the processor-readable non-transitory storage medium containing a knowledge base question answering system, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma. One of ordinary skill in the art would have been motivated to combine the teachings as fine-tuning a model with contrastive loss improves the robustness and efficiency of a model, as disclosed by Xiaofei Ma, (“As shown in Table 5, adding contrastive loss improves model robustness against all types of perturbations we tested.” pg. 6, right column, under Section 5.3, “Robustness to Query Perturbations”)
However, Nallapati, as modified by Xiaofei Ma, still does not teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; nor training a generation model based on a loss objective using the generated ranked list as training data.
Cicero teaches training a generation model based on a loss objective using the generated ranked list as training data. (“In this task, given an input question and a set of candidate passages, the goal is to rank the candidate passages so that passages containing the correct answer appear at the top of the ranked list… We fine-tune the LM using the following loss function… A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models.”, pg. 1, right column, second paragraph; pg. 2, right column, bottom of the page; pg. 5, directly under section 3.5 “Question and Passage Generation”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the processor-readable non-transitory storage medium containing a knowledge base question answering system, as disclosed by Nallapati, and modified by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero. One of ordinary skill in the art would have been motivated to combine the teachings as using a ranked list as training data for a generative model allows for generated outputs to be using as additional training data, improving models; as disclosed by Cicero. (“A good side effect of using generative models to perform ranking is that we can use the trained model to generate new questions given a passage and vice-versa (depending on the conditioning context used for fine-tuning). This type of synthetically generated data could be used as additional training data to improve discriminative models such as BERT-PR.” pg. 5, directly under section 3.5 “Question and Passage Generation”)
However, the combination does not explicitly teach functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base;
Gu teaches functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base; (“We employ a simple yet effective strategy to generate candidate logical forms: We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question”, pg. 8, left column, under “4.4 Inference”) (With the logical forms in the database of Nallapati, Gu’s strategy could be easily configured to function with the method of Nallapati to generate a set of candidate logical forms.)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of knowledge base question answering, as disclosed by Nallapati, with the training method of a ranking model, as disclosed by Xiaofei Ma, and the utilization of a ranked list for training generation models, as disclosed by Cicero, with the strategy to generate candidate logical forms, as disclosed by Gu. One of ordinary skill in the art would have been motivated to combine the teachings, prior to the current application, as the strategy aids in inference for a model, as disclosed by Gu. (“We enumerate all logical forms, optionally with a count function, within 2 hops starting from each entity identified in the question. The recall is 80% on GrailQA. Questions with superlatives and comparatives often do not have a topic entity and are therefore not covered. Ranking can prune the search space… Our Ranking model achieves an F1 of 25.0%, which significantly outperforms the prior art SPARQA [37] by 3.5 percent.”, pg. 8, left column, under “4.4 Inference”)
Regarding claims 4 and 14, Nallapati, as modified by Cicero, Xiaofe Ma, and Gu, teaches the cross-entropy loss is computed by: generating, by the ranking model, a first logit representing a first similarity score between the question and the positive sample ("Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate and at most Y (e.g., 63) negative candidates.”, column 18, lines 1-4 and 16-20 (Nallapati)) generating a plurality of logits representing similarity scores between the question and the plurality of negative samples, respectively, ("Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate and at most Y (e.g., 63) negative candidates.”, column 18, lines 1-4 and 16-20 (Nallapati)) and generating a plurality of logits representing similarity scores between the question and the plurality of negative samples, respectively ("Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score)… Finally, a softmax may be used on the logit scores of all datasets, and compute scores for each candidate dataset.”, column 18, lines 1-4 and 16-20; column 19, lines 29-31 (Nallapati);
However, Nallapati, as modified by Cicero and Gu, does not teach contrastive loss nor computing the contrastive loss based on the first logit and the plurality of logits.
Xiaofei Ma teach contrastive loss, (“The contrastive loss can be applied to the representations from a variety of encoders… In this work, we explore contrastive fine-tuning for both BERT (Devlin et al., 2018) and BART (Lewis et al.) models”, pg.3, right column, under “2.2 Contrastive Loss” (Xiaofei Ma)) and with the contrastive loss applying to BERT models (according to Xiaofei Ma), one could combine Xiaofei Ma’s contrastive loss with Nallapati’s logits (which use a BERT score) to compute the contrastive loss based on the first logit and the plurality of logits.
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine Nallapati, as modified by Cicero and Gu, with Xiaofei Ma’s contrastive loss fine-tuning. One of ordinary skill in the art would have been motivated to combine the teachings as fine-tuning a model with contrastive loss improves the robustness and efficiency of a model, as disclosed by Xiaofei Ma, (“As shown in Table 5, adding contrastive loss improves model robustness against all types of perturbations we tested.” pg. 6, right column, under Section 5.3, “Robustness to Query Perturbations”)
Regarding claims 5 and 15, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the method and system of claims 1 and 11 respectively, as well as training the generation model comprises: generating, by the generation model, a first target logical form from the generated ranked list of candidate logical forms at a current training step. (“During training of data set selection model 520, negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates …the target embeddings consists of: (1) Grammar representation, (2) Column representation, and (3) Entity representation. These target embeddings corresponds to action vocabularies, and they represent the possible actions that the decoder can take at each step.”, column 19, lines 30-35; column 20, lines 25-29 (Nallapati))
Regarding claims 6 and 16, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the method and system of claims 5 and 15 respectively, as well as generating, by the generation model, a second target logical form from the generated ranked list of candidate logical forms at a next training step, (“During training of data set selection model 520, negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates …the target embeddings consists of: (1) Grammar representation, (2) Column representation, and (3) Entity representation. These target embeddings correspond to action vocabularies, and they represent the possible actions that the decoder can take at each step.”, column 19, lines 30-35; column 20, lines 25-29 (Nallapati)) and computing a cross-entropy loss between the second target logical form and the first target logical form as ground truth. (“A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate and at most Y (e.g., 63) negative candidates. The positive candidate may come from the annotated ground truth, and the negative candidates may come from index search results.”, column 18, lines 16-22 (Nallapati))
Regarding claim 9, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches determining, for a first entity mentioned in the question, a first set of candidate entities in the knowledge base that match the first entity, ("Linkable entities (e.g., unlinked and/or conflicting entities) identified in natural language query 306 determined by entity detection model 410 may be provided to entity linkage candidate index search 420, in some embodiments. Entity linkage candidate index search 420 may retrieve a list of linking candidates for each entity.”, columns 16 and 17, lines 65-68 for column 16, and lines 1-3 for column 17 (Nallapati)) and determining linking relations between a second entity mentioned in the question and the first set of candidate entities. (“Entity linkage candidate index search 420 may allow for fuzzy queries based on the Levenshtein distance or other metric for measuring the difference between two sequences… Candidate entity linkages (e.g., a mapping between an entity and column or cell value) returned by entity linkage candidate index search 420 may be limited, in some embodiments, to a top X number of results per entity. Query input bundle 408 may include the natural language query (402), user-specified entity linkages 404, and per-entity linkage candidates determined for recognized entities (e.g., limited to the top X number of linkage candidates).”, column 17, lines 15-17, 26-34 (Nallapati))
Claim(s) 2, 7, 8, 10, 12, 17, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nallapati (U.S Patent No. US 11604794 B1) in view of Xiaofei Ma (Contrastive Fine-tuning Improves Robustness for Neural Rankers (NPL)) in further view of Cicero (Beyond [CLS] through Ranking by Generation (NPL)) in further view of Yu Gu. (Herein referred to as Gu) (Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases (NPL)) and in even further view of G. Maheshwari et al. (“Learning to Rank Query Graphs for Complex Question Answering over Knowledge Graphs” from “the Semantic Web” (NPL))
Regarding claim 2, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the method of claim 1, as well as the ranking model trained by: randomly sampling a subset of negative samples from the set of candidate logical forms, (“negative random samples”, column 19, line 32 (Nallapati)) forming a positive input of the question and the corresponding logical form, ("negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates.", column 19, lines 32-35 (Nallapati)) forming a plurality of negative inputs from the question and the subset of negative samples, (“Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate…”, column 18, lines 1-4 and 16-19 (Nallapati))
However, the combination does not teach training the ranking model using the positive input and the plurality of negative inputs for a number of epochs at a beginning of training.
Maheshwari teaches training the ranking model using the positive input and the plurality of negative inputs for a number of epochs at a beginning of training. ("Our models are trained with negative sampling, where we sample 100 negative core chains per question, along with the correct one…We train our models for a maximum of 300 epochs, using a 70-10-20 split as train, validation and test data…" pg. 497, Paragraph labeled "Training")
Therefore, it would have been obvious to someone with ordinary skill in the art to combine the method of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the training of a model with epochs, as taught by Maheshwari. One would be motivated to combine the teachings, prior to the filing date of the current application, as training a model for a number of epochs allows a model to reach its best performance by minimizing error. (“When using training algorithms, the number of epochs can reach thousands, and the process is programmed to continue until the model error is suitably minimized. Typically, tutorials and examples employ figures such as 10, 100, 1000, or even higher.”, Under “What is an epoch in ML?” (Deepchecks))
Regarding claim 7, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, teaches the method of claim 1, but does not teach receiving a testing question, nor generating, by the trained ranking model and the trained generation model, a target logical form for the testing question.
Maheshwari teaches receiving a testing question, (“QALD-7 [Question Answering over Linked Data model version 7] is a gold-standard dataset having 220 and 43 training and test questions respectively along with their corresponding SPARQL queries.” pg. 496, last paragraph) and generating, by the trained ranking model and the trained generation model, a target logical form for the testing question. (“We treat KGQA as the task of generating and ranking query graph candidates w.r.t. [with respect to] a given NLQ.", pg. 492, first paragraph; "We now have all the information required to construct the query graph, and the corresponding executable query.”, pg. 496, paragraph 3)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the method of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the testing data, as taught by Maheshwari. One would be motivated to combine the teachings as, according to Maheshwari, testing data is useful for error analysis. and one might the combination effective for the purpose of providing an improved technique. ("Error Analysis: We now illustrate the effect of different characteristics of the test data on the model performance…An error analysis shows that the model performs especially well on smaller candidate sets and for longer questions which highlights its high potential for answering complicated questions.", pg. 498 under Error Analysis; pg. 502, under “Conclusion and Future Work” (Maheshwari))
Regarding claim 8, Nallapati, as modified by Xiaofei Ma, Cicero, Gu, and Maheshwari teaches the method of claim 7, further comprising: generating an answer to the testing question by applying the target logical form on the knowledge base. (“We treat KGQA as the task of generating and ranking query graph candidates w.r.t. [with respect to] a given NLQ...An error analysis shows that the model performs especially well on smaller candidate sets and for longer questions which highlights its high potential for answering complicated questions.", pg. 492, First Paragraph; pg. 502, under Conclusion and Future Work (Maheshwari))
Regarding claim 10, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the method of claim 9, further comprising: forming a positive input pair based on the question and a corresponding linking relation, ("During training of data set selection model 520, negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates...As indicted at 506, linkages with selected data sets 506 may be provided to intermediate representation generation model 530 along with data set metadata snapshot 406 to provide an intermediate representation 508 of the natural language query" Paragraphs 63 and 64; FIG. 5, "data set selection model" 520, "linkage with selected data sets" 506, " intermediate representation generation model" 530 (Nallapati)) and forming a plurality of negative input pairs based on the question and the determined linking relations, ("negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates...As indicted at 506, linkages with selected data sets 506 may be provided to intermediate representation generation model 530 along with data set metadata snapshot 406 to provide an intermediate representation 508 of the natural language query.", Paragraphs 63 and 64; FIG. 5, "data set selection model" 520, "linkage with selected data sets" 506, " intermediate representation generation model" 530 (Nallapati))
However, the combination does not teach re-training the trained ranking model based on the positive input pair and the plurality of negative input pairs.
Maheshwari teaches fine-tuning the trained ranking model based on the positive input pair and the plurality of negative input pairs. ("We find that fine-tuning a ranking model trained over LC-QuAD [Large-Scale Complex Question Answering Dataset] leads to a substantial (∼11%) increase in performance on QALD-7 [Question Answering over Linked Data model version 7] compared to non-pretrained models." Pg. 501, Paragraph 3) While fine-tuning is not the same thing as re-training, fine-tuning is a type of retraining. (“There are several steps involved in fine-tuning a model. First, you need to select a pre-trained model that is well-suited to your task. Next, you need to prepare your sample data and fine-tune the model on this data. Finally, you need to iterate on your model to improve its performance… Preparing your sample data involves cleaning and preprocessing your data to make it suitable for training. You should also split your data into training and validation sets to evaluate the performance of your model. The format of your data should match the format expected by the pre-trained model you are using.”, Microsoft, pg. 1, under “Introduction”; pg. 2 under “Prepare your sample data”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to the filing date of this application to combine the linking relations of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the fine-tuning method as taught by Maheshwari. One of ordinary skill in the art would be motivated to combine the teachings, as fine-tuning a model leads to better performance for the model specific task, as disclosed by Maheshwari. (“We find that fine-tuning a ranking model trained over LC-QuAD [Large-Scale Complex Question Answering Dataset] leads to a substantial (∼11%) increase in performance on QALD-7 [Question Answering over Linked Data model version 7] compared to non-pretrained models." Pg. 501, Paragraph 3)
Regarding claim 12, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, teaches the system of claim 11, as well as the ranking model trained by: randomly sampling a subset of negative samples from the set of candidate logical forms, (“negative random samples”, column 19, line 32 (Nallapati)) forming a positive input of the question and the corresponding logical form, ("negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates.", column 19, lines 32-35 (Nallapati)) forming a plurality of negative inputs from the question and the subset of negative samples, (“Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate…”, column 18, lines 1-4 and 16-19 (Nallapati))
However, the combination does not teach training the ranking model using the positive input and the plurality of negative inputs for a number of epochs at a beginning of training.
Maheshwari teaches training the ranking model using the positive input and the plurality of negative inputs for a number of epochs at a beginning of training. ("Our models are trained with negative sampling, where we sample 100 negative core chains per question, along with the correct one…We train our models for a maximum of 300 epochs, using a 70-10-20 split as train, validation and test data…" pg. 497, Paragraph labeled "Training")
Therefore, it would have been obvious to someone with ordinary skill in the art to combine the system of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the training of a model with epochs, as taught by Maheshwari. One would be motivated to combine the teachings, as training a model for a number of epochs allows a model to reach its best performance by minimizing error. (“When using training algorithms, the number of epochs can reach thousands, and the process is programmed to continue until the model error is suitably minimized. Typically, tutorials and examples employ figures such as 10, 100, 1000, or even higher.”, Under “What is an epoch in ML?” (Deepchecks))
Regarding claim 17, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the system of claim 11, but does not teach receiving a testing question, nor generating, by the trained ranking model and the trained generation model, a target logical form for the testing question.
Maheshwari teaches receiving a testing question, (“QALD-7 [Question Answering over Linked Data model version 7] is a gold-standard dataset having 220 and 43 training and test questions respectively along with their corresponding SPARQL queries.” pg. 496, last paragraph) and generating, by the trained ranking model and the trained generation model, a target logical form for the testing question. (“We treat KGQA as the task of generating and ranking query graph candidates w.r.t. [with respect to] a given NLQ.", pg. 492, First Paragraph; "We now have all the information required to construct the query graph, and the corresponding executable query.”, pg. 496, paragraph 3)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to this application’s filing date, to combine the system of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the testing data, as taught by Maheshwari. One would be motivated to combine the teachings as, according to Maheshwari, testing data is useful for error analysis and one might the combination effective for the purpose of providing an improved technique. ("Error Analysis: We now illustrate the effect of different characteristics of the test data on the model performance…An error analysis shows that the model performs especially well on smaller candidate sets and for longer questions which highlights its high potential for answering complicated questions.", pg. 498 under Error Analysis; pg. 502, under “Conclusion and Future Work” (Maheshwari))
Regarding claim 18, Nallapati, as modified by Xiaofei Ma, Cicero, Gu, and Maheshwari teaches the system of claim 17 as well as generating an answer to the testing question by applying the target logical form on the knowledge base. (“We treat KGQA as the task of generating and ranking query graph candidates w.r.t. [with respect to] a given NLQ...An error analysis shows that the model performs especially well on smaller candidate sets and for longer questions which highlights its high potential for answering complicated questions.", pg. 492, First Paragraph; pg. 502, under Conclusion and Future Work (Maheshwari))
Regarding claim 19, Nallapati, as modified by Xiaofei Ma, Cicero, and Gu teaches the system of claim 11, as well as determining, for a first entity mentioned in the question, a first set of candidate entities in the knowledge base that match the first entity, ("Linkable entities (e.g., unlinked and/or conflicting entities) identified in natural language query 306 determined by entity detection model 410 may be provided to entity linkage candidate index search 420, in some embodiments. Entity linkage candidate index search 420 may retrieve a list of linking candidates for each entity.”, columns 16 and 17, lines 65-68 for column 16, and lines 1-3 for column 17 (Nallapati)) and determining linking relations between a second entity mentioned in the question and the first set of candidate entities. (“Entity linkage candidate index search 420 may allow for fuzzy queries based on the Levenshtein distance or other metric for measuring the difference between two sequences… Candidate entity linkages (e.g., a mapping between an entity and column or cell value) returned by entity linkage candidate index search 420 may be limited, in some embodiments, to a top X number of results per entity. Query input bundle 408 may include the natural language query (402), user-specified entity linkages 404, and per-entity linkage candidates determined for recognized entities (e.g., limited to the top X number of linkage candidates).”, column 17, lines 15-17, 26-34 (Nallapati)) forming a positive input pair based on the question and a corresponding linking relation; ("During training of data set selection model 520, negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates...As indicted at 506, linkages with selected data sets 506 may be provided to intermediate representation generation model 530 along with data set metadata snapshot 406 to provide an intermediate representation 508 of the natural language query" Paragraphs 63 and 64; FIG. 5, "data set selection model" 520, "linkage with selected data sets" 506, " intermediate representation generation model" 530 (Nallapati)) and forming a plurality of negative input pairs based on the question and the determined linking relations; ("negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates...As indicted at 506, linkages with selected data sets 506 may be provided to intermediate representation generation model 530 along with data set metadata snapshot 406 to provide an intermediate representation 508 of the natural language query.", Paragraphs 63 and 64; FIG. 5, "data set selection model" 520, "linkage with selected data sets" 506, " intermediate representation generation model" 530 (Nallapati))
However, the combination, does not teach re-training the trained ranking model based on the positive input pair and the plurality of negative input pairs.
Maheshwari teaches fine-tuning the trained ranking model based on the positive input pair and the plurality of negative input pairs. ("We find that fine-tuning a ranking model trained over LC-QuAD [Large-Scale Complex Question Answering Dataset] leads to a substantial (∼11%) increase in performance on QALD-7 [Question Answering over Linked Data model version 7] compared to non-pretrained models." Pg. 501, Paragraph 3) While fine-tuning is not the same thing as re-training, fine-tuning is a type of retraining. (As stated by Microsoft) (“There are several steps involved in fine-tuning a model. First, you need to select a pre-trained model that is well-suited to your task. Next, you need to prepare your sample data and fine-tune the model on this data. Finally, you need to iterate on your model to improve its performance… Preparing your sample data involves cleaning and preprocessing your data to make it suitable for training. You should also split your data into training and validation sets to evaluate the performance of your model. The format of your data should match the format expected by the pre-trained model you are using.”, Microsoft, pg. 1, under “Introduction”; pg. 2 under “Prepare your sample data”)
Therefore, it would have been considered obvious to one of ordinary skill in the art, prior to the filing date of this application to combine the linking relations of Nallapati, as modified by Xiaofei Ma, Cicero, and Gu, with the fine-tuning method as taught by Maheshwari. One of ordinary skill in the art would be motivated to combine the teachings, as fine-tuning a model leads to better performance for the model specific task, as disclosed by Maheshwari. (“We find that fine-tuning a ranking model trained over LC-QuAD [Large-Scale Complex Question Answering Dataset] leads to a substantial (∼11%) increase in performance on QALD-7 [Question Answering over Linked Data model version 7] compared to non-pretrained models." Pg. 501, Paragraph 3)
Claim(s) 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Nallapati (U.S. Patent No. US 11604794 B1) in view of Xiaofei Ma (Contrastive Fine-tuning Improves Robustness for Neural Rankers (NPL)) in further view of Cicero (Beyond [CLS] through Ranking by Generation (NPL)) in further view of Yu Gu. (Herein referred to as Gu) (Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases) and in further view of Yixuan Su (“Dialogue Response Selection with Hierarchical Curriculum Learning”, relevant pages: pg. 1-3 (NPL))
Regarding claims 3 and 13, Nallapati, as modified by Xiaofei Ma, Cicero, Gu, and Maheshwari, teaches the method and system of claims 2 and 12 respectively, as well as, forming a set of negative inputs by pairing the question and the one or more negative samples; (“Entity linkage model 512 may be implemented as a deep learning model, utilizing a neural network trained to identify and rank entity linkages in a given query text string and other input data... A linear layer may then be applied on the classification token [CLS] to produce a logit score (BERT score). During training, cross-entropy loss may be calculated on all the linking candidates, including one positive candidate", column 18, lines 1-4 and 16-19 (Nallapati)) and training the ranking model using the positive input and the set of negative inputs at a later stage of training. ("During training of data set selection model 520, negative random samples (e.g., four) may be gathered for each positive training example, and cross-entropy loss may be used to boost the scores of the positive candidates.", column 19, lines 31-35 (Nallapati))
However, the combination does not teach selecting one or more negative samples that are similar to the positive sample from the subset of negative samples
Yixuan Su teaches selecting one or more negative samples that are similar to the positive sample from the subset of negative samples (“we select the positive context-response pairs according to the corpus-level pacing function pcc(t). Then, for each instance in the selected batch, we sample its associated negative examples according to the instance”, pg. 4, right column, under “3.4 Hierarchical Curriculum Learning “; See also pg. 3, under “3.2 Corpus-Level Curriculum”)
Therefore, it would have been considered obvious to one of ordinary skill of the art, prior to the filing date of this application, to combine the teachings of Nallapati, Xiaofei Ma, Cicero, Gu, and Maheshwari with the difficulty function of Yixuan Su. One could be motivated to combine the teachings, as the difficulty function allows for the ability to measure the difficulty of each training pair. (“To measure the difficulty of each training context-response pair, we adopt a pre-trained ranking model to calculate its relevance score…”, pg. 3, under “3.2 Corpus-Level Curriculum (Yixuan Su))
Response to Arguments
Applicant's arguments, filed September 23rd, 2025, have been fully considered, but are not fully persuasive. Applicant's claim amendments have overcome the 35 USC 112(b) rejections of the previous office action.
The applicant argues in substance
Argument 1: Regarding claim 1, Nallapati does not teach “similarity scores between the question and the set of candidate logical forms, respectively" specifically “the discussion of a BERT score does not describe any comparison to logical forms, but rather only to entities of a database”
Examiner respectfully disagrees. As stated in this action and given further explanation here, the confidence scores of Nallapati is interpreted to be a metric of similarity between the expected question and corresponding set of candidate logical forms, and the actual question and corresponding set of candidate logical forms. The better the confidence, the less ambiguous that the link between the question and the columns in data sets is a proper linkage. The rejection of claim 1 as amended does not rely on Nallapati’s BERT score for teaching similarity between candidate logical forms and the question. Instead, as set forth in the office action, Nallapati’s confidence scores corresponds to the claimed similarity scores as explained above. Therefore, the rejection is maintained.
Argument 2: Regarding claim 1, Nallapati does not teach "generating, by accessing a knowledge base, a set of candidate logical forms including functions operating on set-based semantics, by starting from each entity of the set of entities and searching for paths reachable within two hops of each entity in a knowledge base."
A new ground of rejection has been put forth in this action, rendering the argument moot under the new ground of rejection. The new reference Gu, teaches this limitation (pg. 8, left column, under “4.4 Inference”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tyler E Iles whose telephone number is (571)272-5442. The examiner can normally be reached 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.E.I./Patent Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights