Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claims 1, 9, 15 objected to because of the following informalities: Claim 1 recites the system operable to produce “a respective confidence differences between the first confidence score and the respective confidence score for each text element;” Examiner will construe the recited “a” as omitted. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-17 rejected under 35 U.S.C. 103 as being unpatentable over Shenoy: 10896295 hereinafter She further in view of Ding: 20190251164 and further in view of Ribeiro: ““Why Should I Trust You?””(copy provided by Examiner, copyright 2016, and hereinafter Rib).
Regarding claim 1
She teaches:
A computer-implemented method comprising:
obtaining a target semantic object and a first text sequence in a natural language, the first text sequence comprising a plurality of text elements (She: Col 23:35-23:60, 37:23-37:33; Fig 4A: assistant system analyzes input text with respect to a target semantic object(s), candidate entities therein, confidence score(s) thereof, distance vector(s) therebetween, etc.);
determining using a trained object prediction model a first confidence score of the target semantic object being mentioned in the first text sequence , the trained object prediction model being configured to output a confidence score of a semantic object being mentioned in an input text sequence; (She: ¶ Col 11:5-11:64, 16:22-16:49, 25:10-25:53; Fig 4A: such as by determining candidate objects in a textual unit by assignment of candidate scores thereto; in the fig Amundsen, as represented by selectable user interface element 440; further each/any named entity may bear a confidence score, likelihood, etc. that the entity is referenced by a corresponding message slot);
determining using the trained object prediction model, a second confidence score of the target semantic object being mentioned in the first text sequence with a first text element being ignored from the first text sequence (She: ¶ Col 11:5-11:64, 25:17-25:25; Fig 4A: such as by determining candidate objects in a textual unit by assignment of candidate scores thereto; ; in the figure Scott, as represented by selectable user interface element 450; the calculation of confidence score for Scott is independent from the calculation of a confidence score for Amundsen, they are next compared such as based on a threshold, numerical distance, etc.); and
training (She: Col 16:27-16:49, 39:5-39:30; Fig 4A: system is trained based on user profile, actions, and feedback; particularly with respect to a textual input and target semantic objects, entities, etc. thereof/therein, said entities resulting in an object linking model which presents entities based on target distances, candidate scores, differences of weightings therebetween) at least based on a first confidence difference between the first confidence score and the second confidence score, the first text sequence, and the target semantic object (She: Col 19:19-19:25, 26:46-27:8, 36:35-36-67, etc.; Fig 4A: entities, objects, etc. derived based on user intent represented as vectors such that difference scores determined among the vectors representing the objects operate to disambiguate and/or allow a user to disambiguate results thereof which are linked to or otherwise represented to a user based on differences among first, second etc. confidence scores of the first, second, etc. candidates; said links, confidence score, difference score, firs/second objects, etc. represented to the user by selectable user interface elements);
the object linking model being configured to determine whether the target semantic object is linked to one of the plurality of text elements (She: 30:8-30-30, etc.; Figs 4, 5, 6, etc.: such as by operating to provide links to information with respect to each of the determined , entities, objects, etc. and determination of user behavior with respect thereto, such as by a user clicking/not clicking upon the provided links).
Thus in She the object linking model is considered the linking of entities determined in the text to information borne at a provided link attendant thereto (She: such as in figure 4A, 4B where a determined entity, object, etc., i.e. Amundsen, resolves a linked which is displayed based on user selection). She does not explicitly teach an object linking model trained in concert with receiving training data comprising a target and a text sequence; determining a second confidence score based on automatically ignoring an element in a first text sequence; and training based on the training data and the difference. That is She does not teach a linking model based on a difference between/among confidence scores computed before and after ignoring a text element.
She does not teach iterating over text elements in a manner sufficient to determine
for each text element of the plurality of text elements, determining, using the trained object prediction model, a second confidence score of the target semantic object being mentioned in the first text sequence with each text element of the plurality of text elements being ignored from the first text sequence to produce a respective confidence differences between the first confidence score and the respective confidence score for each text element; thereby generating a confidence difference sequence for the plurality of text elements based on the respective confidence differences¸ nor training using the confidence difference score, sequence, etc. as a training feature.
In a related field of endeavor Ding teaches a system and method for training an object linking model separate from the prediction model such that the linking model is configured to determine linking between a target semantic object and one of the plurality of text elements (Ding: ¶ 27, 28 such as training a language model and topic model to output a matching degree between an unambiguous entity and a context of a text based on output results of the language mode, topic model and training data)
wherein the training data comprises an entity word, text, sequence, etc. unambiguously linked to an entity and training an unambiguous entity recognition model to output a matching, confidence, etc. probability based thereon suitable to provide a link confidence score (Ding: Abstract); such as using a trained object prediction model to output of a matching probability, confidence score, etc. between an entity word and an unambiguous entity said probability signifying a confidence level that the word refers to or otherwise matches an unambiguous entity (Ding: ¶ 21, 22, etc.; Fig 1, etc.) wherein the relevance thereof is output by the topic model when greater than a second relevance threshold (Ding: ¶ 34), such as for training based on confidence difference sequences, a first text sequence and a target semantic object (Ding: ¶ 30, 44).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize training data such as that of Ding to improve object linking in the She system and method for at least the purpose of disambiguating objects in a more streamlined or automatic fashion such as to obviate user reliant disambiguation when necessary, appropriate, or desired, such as based on a confidence threshold, confidence difference, etc. of a plurality of objects; one of ordinary skill in the art would have expected only predictable results therefrom.
It may be that She in view of Ding does not explicitly teach the broadest reasonable interpretation of the recited confidence difference material in a way requiring explicit masking or removal of tokens, words, etc. in the text rather than utilizing the absence of a first term to obviate conditions of ambiguity among token sequences such as for performing the confidence difference score computation generative of a confidence difference sequency of score differences based on positional data.
In a related field of endeavor Rib teaches a system and method for explainable classification (Rib: Abstract) wherein for each text element the system determines a second confidence score wherein each text element is conditionally ignored to produce respective confidence difference scores (Rib: § 3.2, 3.3, 3.4, 4: system determines a probability that an instance x belongs to a certain class such that an ignored element from the first text sequence generates a confidence score with the text element present and with the text element ignored and generative of a perturbed input and records the present, absent versions thereof thereby producing a respective confidence difference such as explanation matrix W representative of the local importance of interpretable components for each instance) thereby generating confidence difference sequence for the plurality of text elements based on respective confidence differences (Rib: § 4: each column of the matrix represent a specific word and each entry encodes a value of the corresponding words effect on the prediction score, this is considered a relative confidence difference which when taken over a plurality of word positions comprises a sequence of per element confidence difference aligned with the text and generative of global importance scores for each/any word position) operative for training an object linking model separate from the prediction model based on the confidence difference scores, the first text sequence and the target semantic object (Rib: § 3.4; Algorithm 1)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to improve the She in view of Ding taught system and method by incorporation of the Rib taught perturbations of the confidence vector by inclusion/disclusion of worder thereof to generate a confidence difference sequence for at least the purpose of providing a importance matrix of elements and global importance scores thereof; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 2
She in view of Ding in view of Rib teaches or suggests:
The method of claim 1, wherein a trained object prediction model (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: output of a score which predicts a matching degree between a word and an entity); (Rib: § 5.4, 6.3, etc.: such as for training and improving a classifier) is used to determine the first confidence score and the second confidence score, respectively (please see claim 1 supra; She: 19:19-19:25, 26:46-27:8, 36:35-36-67, etc.; Fig 4-6: the provided model, trained at least on prior input text data, determined entities therein, etc. to generate first, second, etc. confidence values), the method further comprising:
obtaining training data for the object prediction model, the training data comprising a second text sequence, a semantic object, and supervision information for the semantic object indicating whether the semantic object is mentioned in the second text sequence (She: Col 36; Fig 8 etc. system trained on first, second, plural, etc. instances of n-grams of terms therein and provided or accreted as a dictionary and further trained to generate or amend a dictionary); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: input text matched to particular entity(s)); and
training the object prediction model based on the training data (She: Col 36, etc.: system is iteratively trained based on emergent data); (Ding: Abstract; ¶ 5, 21, 22, etc.; Fig 1, etc.: such as to probabilistically predict linkages among a particular word and entity and thereby iteratively train a recognition model). The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 3
She in view of Ding in view of Rib teaches or suggests:
The method of claim 1, wherein training the object linking model comprises:
determining a first linking score for the first text element based on the first confidence difference, the first linking score indicating a probability of the target semantic object being linked to the first text element (She: ¶ 25:25-25:30, 26:2-26:35; Figs 5: such as by providing a highest ranked candidate in a first slot and a second candidate in a second slot wherein the slots are determined based on differences among the confidence scores); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: output of a score which predicts a matching degree between a word and an entity); (Rib: § 3.2, 3.3, 3.4, 5.4, 6.3: system determines present/absent scores for each/any first, second, etc. text elements based on contextual data thereof)
determining, using the object linking model, a second linking score for the first text element based on the first text sequence and the target semantic object, the second linking score indicating a probability of the target semantic object being linked to the first text element (She: ¶ 25:25-25:30, 26:2-26:35; Figs 5: such as by providing a highest ranked candidate in a first slot and a second candidate in a second slot wherein the slots are determined based on differences among the confidence scores); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: output of a score which predicts a matching degree between a word and an entity); (Rib: § 3.2, 3.3, 3.4, 5.4, 6.3: such as by adjustment of term importance to optimize consistency among classifier outputs of determined entity types);
constructing a training objective function for the object linking model based on the first and second linking scores, the training objective function being based on an increase of a combined score of the first and second linking scores (She: 16:27-16:49, 28:53-29:5, 36:24-36:27, 39:5-39:30: system manages and updates a ranking function as part of training objectives such as with respect to a provided dictionary and/or accreted dictionary); (Rib: § 3.2, 3.3, 3.4, 5.4, 6.3: such as by iteratively training the system to rank, rerank, etc. particular context and position based word values to determine difference based scores thereof)
updating a parameter value of the object linking model based on the training objective function (She: 16:27-16:49, 28:53-29:5, 30:7-30:40, 39:5-39:30; Figs 5: system updates user parameters based on user selection, such as of provided links); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: linking model constructed to improve matching performance of a recognition model); (Rib: § 3.2, 3.3, 3.4, 5.4, 6.3: such as by iteratively training the system to rank, rerank, etc. particular context and position based word values to determine difference based scores thereof). The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 4
She in view of Ding in view of Rib teaches or suggests:
The method of claim 3, wherein determining the first linking score comprises: obtaining the supervision information for the target semantic object indicating whether the target semantic object is mentioned in the first text sequence (She: 16:27-16:49, 28:53-29:5, 30:7-30:40, 39:5-39:30; Figs 5: such as the first, second, etc. confidence score);
in accordance with a determination that the supervision information for the target semantic object indicates that the target semantic object is mentioned in the first text sequence, calculating the first linking score based on the first confidence difference (She: 16:27-16:49, 28:53-29:5, 30:7-30:40, 39:5-39:30; Figs 5: such as by determination of confidence and display of targets in slots based thereon); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: such as by linking a term mention to an entity);
and
in accordance with a determination that the supervision information for the target semantic object indicates that the target semantic object is not mentioned in the first text sequence, determining the first linking score to indicate that the target semantic object is not linked to the first text element (She: 16:27-16:49, 28:53-29:5, 30:7-30:40, 39:5-39:30; Figs 5: such as by update of the first or second parameter score dependent on which of the presented options is clicked upon or not); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: such as by not linking a term mention to an entity in the absence of a word which may be linked to an unambiguous entity). The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 5
She in view of Ding in view of Rib teaches or suggests:
The method of claim 1, wherein training the object linking model comprises: determining a third confidence score of the target semantic object being mentioned in the first text sequence with a second text element ignored from the first text sequence; and training the object linking model further based on a second confidence difference between the first confidence score and the third confidence score (please see claim 1 supra and additionally She: 16:27-16:49, 28:53-29:5, 30:7-30:40, 39:5-39:30; Figs 5: such as by iterating the method when the distance between three or more candidates falls with a threshold; Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: output of a score which predicts a matching degree between a word and an entity and/or Rib: § 3.2, 3.3, 3.4, 5.4, 6.3: such as by adjustment of term importance to optimize consistency among classifier outputs of determined entity types). The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 6
She in view of Ding in view of Rib teaches or suggests:
The method of claim 5, wherein training the object linking model further based on the second confidence difference comprises: determining a third linking score for the second text element based on the second confidence difference, the third linking score indicating a probability of the target semantic object being linked to the second text element; determining, using the object linking model, a fourth linking score for the second text element based on the first text sequence and the target semantic object, the fourth linking score indicating a probability of the target semantic object being linked to the second text element; constructing a training objective function for the object linking model based on the third and fourth linking scores, the training objective function being based on an increase of a combined score of the third and fourth linking scores; and updating a parameter value of the object linking model based on the training objective function. While not explicit in discussion of the further scores being determined in this way doing so is considered obvious such as by applying the discussed scoring method to inter-relations among three or more candidates. That is, She in view of Ding in view of Rib discusses the base method of presenting linkages based on a threshold distance among two candidates in the manner claimed (please see claim 1 supra: She: 19:19-19:25, 26:46-27:8, 36:35-36-67, etc.; Fig 4-6), and Ding discusses plurality of scores comprising matching degrees of a plurality of input words to a plurality of entities (please see claim 1 supra: Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: output of a score which predicts a matching degree between a word and an entity) and Rib discusses ranking and iteratively re-ranking scores for each/any of plural terms with respect to textual data (please see claim 1 supra: Rib: § 3.2, 3.3, 3.4, 5.4, 6.3). As the She in view of Ding in view of Rib taught technique of doing so is pliant to the expansion of the base method and such an expansion to three or more candidates would have been expected by the average skilled practitioner to yield no more than predictable results while improving the reifying of candidates overall. As such, the claim is considered obvious over She in view of Ding in view of Rib such as a by iterating the known techniques of She in view of Ding in view of Rib for at least the purpose of scoring plural strings of text sequences with respect to three or more semantic objects. The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 7
She in view of Ding in view of Rib teaches or suggests:
The method of claim 1, wherein determining the second confidence score comprises: replacing the first text element with a predetermined text symbol; and determining the second confidence score based on text elements among the plurality of text elements other than the first text element, the predetermined text symbol, and the target semantic object. Examiner has taken official notice which Applicant has failed to timely and explicitly traverse and it is thus accepted as Admitted Prior Art (APA: please see MPEP 2144.03) that BERT methods of training natural language understanding systems by substituting symbols, blanks, or decoy words to improve scores thereof would have comprised an obvious inclusion for at least the purpose of improving the compute time, accuracy, etc. of a model; one of ordinary skill in the art would have expected only predictable results therefrom. The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 8
She in view of Ding in view of Rib teaches or suggests:
The method of claim 1, wherein determining the first confidence score comprises: extracting, using a pre-trained language model (PLM) , a plurality of text feature representations of the plurality of text elements and a first object feature representation of the target semantic object, the PLM being included in the object linking model (She: Col 23:35-23:60, 37:23-37:33; Fig 4A: at time of use the model trained based on input text with respect to a target semantic object(s), candidate entities therein, confidence score(s) thereof, distance vector(s) therebetween, etc. is considered pre-trained); (Ding: Abstract; ¶ 21, 22, etc.; Fig 1, etc.: system improves operation of pre-trained models); and
determining the first confidence score based on the first object feature representation, and wherein determining the second confidence score comprises: extracting, using the PLM, text feature representations of text elements among the plurality of text elements other than the first text element and a second object feature representation of the target semantic object; and determining the second confidence score based on the second object feature representation (please see claim 1 supra). The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 9, 15—the claims recite substantially similar subject matter to that of claim 1 supra and are similarly rejected.
Regarding claim 10—the claim recites substantially similar subject matter to that of claim 2 supra and is similarly rejected.
Regarding claim 11—the claim recites substantially similar subject matter to that of claim 3 supra and is similarly rejected.
Regarding claim 12—the claim recites substantially similar subject matter to that of claim 4 supra and is similarly rejected.
Regarding claim 13—the claim recites substantially similar subject matter to that of claim 5 supra and is similarly rejected.
Regarding claim 14—the claim recites substantially similar subject matter to that of claim 6 supra and is similarly rejected.
Regarding claims 16, 17
She in view of Ding in view of Rib teaches or suggests:
The method of claim of claim i wherein the target semantic object and a first text sequence in a natural language are separated from each other, such as by a separator symbol. Examiner has taken official notice which Applicant has failed to timely and explicitly traverse and it is thus accepted as Admitted Prior Art (APA: please see MPEP 2144.03) that separations in a training set, corpus, such as to differentiate between elements in a set, such as a target object, entity, etc. and a text sequence, were well known in the art before the effective filing date of the instant application to comprise annotation such as with a separator symbol, such as a [SEP] token or the like for at least the purpose of delineating between portions of text intended to be held separate and as such would have comprised an obvious inclusion. The claim is considered obvious over She as modified by Ding and Rib as addressed in the base claim as it would have been obvious to apply the further teaching of She, Ding, and/or Rib to the modified device of She, Ding, and Rib; one of ordinary skill in the art would have expected only predictable results therefrom.
Response to Arguments
Applicant’s arguments, see Remarks and Claims, filed 3/24/26, with respect to the rejection(s) of claim(s) 1-17 under 35 USC 103 over Shenoy, Ding, and Qi have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Shenoy, Ding, and Ribeiro. Applicant’s arguments are considered addressed by the discussion provided in the art rejection supra.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL C MCCORD whose telephone number is (571)270-3701. The examiner can normally be reached 730-630 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CAROLYN EDWARDS can be reached at (571) 270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL C MCCORD/Primary Examiner, Art Unit 2692
Prosecution Insights