DETAILED ACTION
This communication is in response to the amendment filed 5/27/26 in which claims 1, 2, 4, 7, 11, 12, 14, 15, 17, 20 were amended. Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant argues that the amended claims are not directed to an abstract idea because a text corpus comprises a large body of text requiring computational processing. App. Resp. 10-11. Examiner respectfully disagrees. A ‘text corpus’ may be interpreted as having a small or large size and, therefore, does not preclude mental evaluation. Further, ‘small’ or ‘large’ size is a term of degree and, therefore, is subjective. Applicant further argues practical integration of the judicial exception. App. Resp. 11-12. However, as explained in the rejection, the additional elements of the claim describe a combination of generic computer components, data inputting/outputting, and instructions to apply the exception. Such limitations fail to integrate the exception or provide an inventive concept. With respect to the amended limitations, they do not describe how the transformation of the text corpus occurs. This is unlike Enfish and McRO in which the claims detailed the specific steps by which the solution and outcome is achieved.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
A computer-implemented method for accuracy evaluation of a concept expansion systems, the computer-implemented method comprising:
[1] automatically generating a test set of concepts and an expected set of concepts that are compatible with the concept expansion system from a text corpus to a domain without requiring manual labelling;
[2] receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system;
[3] performing, by the computer, a comparison between an expected set of concepts and the returned set of related concepts corresponding to the test set of concepts to identify intersection;
[4] generating, by the computer, an accuracy metric corresponding to the concept expansion system based on the intersection between the expected set of concepts and the returned set of related concepts corresponding to the test set of concepts;
[5] determining, by the computer, whether the accuracy metric is greater than a minimum accuracy threshold level; and
[6] performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level, wherein the concept expansion system is a computational or AI-based framework configured to receive a set of input concepts corresponding to the domain and return a set of related concepts augmenting the set of input concepts within the domain.
Step 1: YES. The claim is directed to a method and, therefore, a statutory category.
Step 2A Prong 1: YES. Limitations [1] and [3]-[5] recite generating a test set of concepts and an expected set of concepts from a text corpus to a domain, performing a comparison between an expected set of concepts and the returned set of related concepts to identify an intersection of concepts, generating an accuracy metric based on the intersection of the concepts, and determining whether the accuracy metric is greater than a minimum threshold. Each of these limitations may be performed mentally by human evaluation and judgment and, therefore, falls under the Mental Processes grouping of abstract ideas. As described in the specification, the test and expected set of concepts are obtained by partitioning the set of compatible concepts, something that may be performed by opinion, evaluation, and judgment. The accuracy metric may be calculated by simple pen and paper calculation and may also be performed as a Mental Process. Under a broadest reasonable interpretation, the accuracy metric may involve merely assigning a metric value to the intersection based on the number of overlapping concepts and, therefore, may not even require any calculation.
Step 2A Prong 2: NO. The additional elements fail to integrate the judicial exception into a practical application.
Limitation [2] recites “receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system.” Data inputting constitutes insignificant extra-solution activity and, therefore, does not practically integrate the judicial exception. The type and/or source of data (e.g., returned set of related concepts corresponding to a test set of concepts from a concept expansion system) also does not cause the data gathering activity to integrate the judicial exception into a practical application. Further, the recitation of a generic computer component (e.g., a computer) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Limitation [6] recites “performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level.” Performing an action step, under a broadest reasonable interpretation, is mere instruction to apply the exception and, therefore, fails to integrate the exception into a practical application. The recitation of a generic computer component (e.g., by a computer) also constitutes mere instruction to “apply” the exception and does not integrate the exception into a practical application.
Limitation [6] further recites “wherein the concept expansion system is a computational or AI-based framework configured to receive a set of input concepts corresponding to the domain and return a set of related concepts augmenting the set of input concepts within the domain.” The description of the concept expansion system as a computational or AI-based framework generally links the judicial exception to a particular technological environment and, therefore, does not integrate the exception into a practical application. Receiving input concepts and returning related concepts is data inputting/outputting and, therefore, is insignificant extra-solution activity.
Step 2B: NO. The additional elements do not provide an inventive concept.
Limitation [2] recites “receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system.” Receiving information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Further, the recitation of a generic computer component (e.g., a computer) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Limitation [6] recites “performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level.” Performing an action step, under a broadest reasonable interpretation, is mere instruction to apply the exception and, therefore, fails to integrate the exception into a practical application. The recitation of a generic computer component (e.g., by a computer) also constitutes mere instruction to “apply” the exception and does not provide an inventive concept.
Limitation [6] further recites “wherein the concept expansion system is a computational or AI-based framework configured to receive a set of input concepts corresponding to the domain and return a set of related concepts augmenting the set of input concepts within the domain.” The description of the concept expansion system as a computational or AI-based framework generally links the judicial exception to a particular technological environment and, therefore, does not integrate the exception into a practical application. Receiving input concepts and returning related concepts is data inputting/outputting and, therefore, is insignificant extra-solution activity. Under 2B this insignificant extra solution activity is well understood routine and conventional activity. See “Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362.”
Accordingly, claim 1 is ineligible.
Claim 11 is a system claim corresponding to claim 1 and, therefore, is similarly ineligible. Claim 11 additionally recites:
A computer system for accuracy evaluation of concept expansion systems, the computer system comprising:
a communication fabric;
a storage device connected to the communication fabric, wherein the storage device stores program instructions; and
a processor connected to the communication fabric, wherein the processor executes the program instructions to:
Step 2A Prong 2/Step 2B: NO. The additional elements of claim 11 fail to integrate the exception into a practical application or provide an inventive concept.
Recitation of generic computer components (e.g., a communication fabric, storage device, processor) at a high level of generality constitutes mere instruction to “apply” the exception.
Claim 14 is a computer program product claim corresponding to claim 1 and, therefore, is similarly ineligible.
Claim 14 additionally recites:
A computer program product for accuracy evaluation of concept expansion systems, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
Recitation of generic computer components (e.g., a computer-readable storage medium, program instructions, computer) at a high level of generality constitutes mere instruction to “apply” the exception.
Claim 2
The computer-implemented method of claim 1, further comprising:
[1] receiving, by the computer, the text corpus corresponding to the domain to evaluate accuracy of the concept expansion system from a client device of a domain expert via a network;
[2] annotating, by the computer, using an annotator corresponding to the domain, text of the text corpus to form annotated text; and
[3] extracting, by the computer, using the annotator corresponding to the domain, the annotated text from the text corpus.
Step 2A Prong 1: YES. Limitation [2] recites annotating text of a text corpus using an annotator corresponding to a particular domain. Annotation of text is a step which may be performed by a domain expert (e.g., a doctor in a healthcare setting) and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Limitation [1] recites receiving a text corpus corresponding to a particular domain (e.g., medicine) from a client device of a domain expect. Mere data gathering is insignificant extra-solution activity and, therefore, does not integrate the judicial exception into a practical application. The type or source of data (e.g., text corpus corresponding to a particular domain from a client device of a domain expert via a network) does not cause the data gathering to practically integrate the exception. Recitation of a generic computer component (e.g., by a computer, via a network) at a high level of abstraction constitutes mere instruction to “apply” the exception. Limitation [3] recites extracting the annotated text from the text corpus and is also data gathering.
Step 2B: NO.
Receiving information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). Accordingly, limitations [1] and [3] do not provide an inventive concept.
Further, the recitation of a generic computer component (e.g., a computer, a network) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Accordingly, claim 2 is ineligible.
Claim 12 is a system claim corresponding to claim 2 and, therefore, is similarly ineligible.
Claim 15 is a CRM claim corresponding to claim 2 and, therefore, is similarly ineligible.
Claim 3
The computer-implemented method of claim 2, further comprising:
[1] mapping, by the computer, using an annotation mapper, the annotated text extracted from the text corpus to a set of concepts used by the concept expansion system; and
[2] generating, by the computer, using the annotation mapper, a set of compatible concepts that is compatible with the concept expansion system as input based on the mapping of the annotated text extracted from the text corpus to the set of concepts used by the concept expansion system.
Step 2A Prong 1: YES.
Limitation [1] recites mapping annotated text to a set of concepts. Mapping, in the context of the claim and under a broadest reasonable interpretation, encompasses a mental evaluation and mapping of annotated text to a set of concepts. Limitation [2] recites generating a set of compatible concepts to the set of concepts used by the concept expansion system. “Generating” encompasses a user mentally evaluating the set of concepts and coming up with a set of compatible concepts, something which may be performed by a human and, therefore, also falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Recitation of generic computer components (e.g., by the computer, using an annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception and does not integrate the judicial exception into a practical application.
Step 2B: NO.
Recitation of generic computer components (e.g., by the computer, using an annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception and does not provide an inventive concept.
Accordingly, claim 3 is ineligible.
Claim 13 is a system claim corresponding to claim 3 and, therefore, is similarly ineligible.
Claim 16 is a CRM claim corresponding to claim 3 and, therefore, is similarly ineligible.
Claim 4
The computer-implemented method of claim 3, wherein the annotation mapper maps the annotated text extracted from the text corpus to a set of concepts used by the concept expansion system based on retrieved domain knowledge corresponding to the particular domain of the text corpus.
Step 2A Prong 1: YES.
Mapping annotated text to a set of concepts based on retrieved domain knowledge corresponding to a particular domain (e.g., medicine) may be performed by evaluation, opinion, and judgement, and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2/Step 2B: NO.
Recitation of generic computer components (e.g., annotation mapper, concept expansion system) at a high level of generality constitutes mere instruction to “apply” the exception and, therefore, does not integrate the exception into a practical application or provide an inventive concept.
Accordingly, claim 4 is ineligible.
Claim 17 is a CRM claim corresponding to claim 4 and, therefore, is similarly ineligible.
Claim 5
The computer-implemented method of claim 3, further comprising:
[1] generating, by the computer, using the annotation mapper, a report regarding the set of compatible concepts that includes at least one of statistics corresponding to frequency of distribution of the set of compatible concepts, information corresponding to how the set of compatible concepts was generated, and provenance corresponding to where each of the set of compatible concepts was found in the text corpus;
[2] sending, by the computer, the report regarding the set of compatible concepts to the client device of the domain expert via the network; and
[3] receiving, by the computer, feedback regarding the set of compatible concepts from the client device of the domain expert via the network.
Step 2A Prong 2: NO.
Limitation [1] recites generating a report regarding the compatible concepts and other information as to how the compatible concepts were generated. Data outputting is considered insignificant extra-solution activity and, therefore, does not integrate the exception into a practical application. Limitation [2] also describes data outputting step and, therefore, does not integrate the exception into a practical application. Limitation [3] describes a data gathering step (receiving feedback from the domain expert) and constitutes insignificant extra-solution activity.
Recitation of generic computer components (e.g., by the computer, using the annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception.
Step 2B: NO.
Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Recitation of generic computer components (e.g., by the computer, using the annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception.
Accordingly, claim 5 is ineligible.
Claim 18 is a CRM claim corresponding to claim 5 and, therefore, is similarly ineligible.
Claim 6
The computer-implemented method of claim 5, further comprising:
[1] determining, by the computer, whether the feedback received from the domain expert includes corrections to the set of compatible concepts; and
[2] implementing, by the computer, the corrections in the set of compatible concepts in response to the computer determining that the feedback received from the domain expert does include corrections to the set of compatible concepts.
Step 2A Prong 1: YES.
Limitation [1] describes determining whether the feedback from the domain expert includes corrections. This may be performed mentally by evaluation, opinion, and judgment of a human and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Limitation [2] recites implementing the corrections in the set of compatible concepts in response to determining that the feedback includes corrections. The description of a solution or outcome without detail as to how it is achieved constitutes mere instruction to “apply” the exception. Recitation of generic computer components (e.g., by the computer) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 6 is ineligible.
Claim 19 is a CRM claim corresponding to claim 6 and, therefore, is similarly ineligible.
Claim 7
The computer-implemented method of claim 6, further comprising:
[1] splitting, by the computer, using a set splitter, the set of compatible concepts into the test set of concepts and the expected set of concepts corresponding to the text corpus based on a splitting strategy; and
[2] inputting, by the computer, the test set of concepts into the concept expansion system, wherein splitting of the set of compatible concepts constitutes the automatically generating of the test set of concepts and the expected set of concepts.
Step 2A Prong 1: YES.
Limitation [1] describes splitting a set of concepts into a test set and an expected set of concepts based on a splitting strategy. Under a broadest reasonable interpretation of “splitting,” this encompasses a user mentally separating a set of concepts into two subsets using opinion, evaluation, and judgment and, therefore, falls under the Mental Processes grouping of abstract ideas. As discussed above, generating the test set and expected set of concepts from the compatible concepts may be performed mentally.
Step 2A Prong 2: NO.
Limitation [2] recites a data inputting step which constitutes insignificant extra-solution activity. The type of data does not cause the data inputting to integrate the exception into a practical application. Recitation of generic computer components (e.g., by the computer, using a set splitter) at a high level of generality also constitutes mere instruction to “apply” the exception.
Step 2B: NO.
Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Recitation of generic computer components (e.g., by the computer, using a set splitter) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 7 is ineligible.
Claim 8
The computer-implemented method of claim 7, wherein the splitting strategy is one of a random concept sampling strategy, a most recently used concept strategy, or a user-specified concept splitting strategy.
Step 2A Prong 1: YES. A user is mentally able to implement a random concept sampling strategy, a most recently used concept strategy, or a user-specified concept splitting strategy.
Accordingly, claim 8 is ineligible.
Claim 20 is a CRM claim corresponding to claim 8 and, therefore, is similarly ineligible.
Claim 9
The computer-implemented method of claim 1, wherein the computer generates the accuracy metric corresponding to the concept expansion system by calculating at least one of precision, recall, or F1-score.
Step 2A Prong 1: YES. The claim recites performing a calculation and, therefore, falls under the Mathematical Concepts grouping of abstract ideas.
Step 2A Prong 1/Step 2B: Recitation of generic computer components (e.g., the computer) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 9 is ineligible.
Claim 10
The computer-implemented method of claim 1, wherein the set of action steps includes at least one of sending a notification to a domain expert regarding accuracy of the concept expansion system, sending the expected set of concepts to the domain expert for review, and generating a new set of compatible concepts to reevaluate the accuracy of the concept expansion system.
Step 2A Prong 1: YES. The limitation “generating a new set of compatible concepts…” recites a mental process and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO. The claim recites sending a notification to a domain expert and sending the expected set of concepts to the domain expert for review. These constitute insignificant extra-solution activity. The type of data does not cause the data outputting to integrate the exception into a practical application.
Step 2B: NO. Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHID KHAN whose telephone number is (571)270-0419. The examiner can normally be reached M-F, 9-5 est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571)272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHID K KHAN/ Primary Examiner, Art Unit 2146
Prosecution Insights