DETAILED ACTION
This communication is in response to the application filed 6/13/23 in which claims 1-20 were presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 6/13/23 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 2/5/26 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
A computer-implemented method for accuracy evaluation of concept expansion systems, the computer-implemented method comprising:
[1] receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system;
[2] performing, by the computer, a comparison between an expected set of concepts and the returned set of related concepts corresponding to the test set of concepts to identify intersection;
[3] generating, by the computer, an accuracy metric corresponding to the concept expansion system based on the intersection between the expected set of concepts and the returned set of related concepts corresponding to the test set of concepts;
[4] determining, by the computer, whether the accuracy metric is greater than a minimum accuracy threshold level; and
[5] performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level.
Step 1: YES. The claim is directed to a method and, therefore, a statutory category.
Step 2A Prong 1: YES. Limitations [2]-[4] recite performing a comparison between an expected set of concepts and the returned set of related concepts to identify an intersection of concepts, generating an accuracy metric based on the intersection of the concepts, and determining whether the accuracy metric is greater than a minimum threshold. Each of these limitations may be performed mentally by human evaluation and judgment and, therefore, falls under the Mental Processes grouping of abstract ideas. The accuracy metric may be calculated by simple pen and paper calculation and may also be performed as a Mental Process. Under a broadest reasonable interpretation, the accuracy metric may involve merely assigning a metric value to the intersection based on the number of overlapping concepts and, therefore, may not even require any calculation.
Step 2A Prong 2: NO. The additional elements fail to integrate the judicial exception into a practical application.
Limitation [1] recites “receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system.” Data inputting constitutes insignificant extra-solution activity and, therefore, does not practically integrate the judicial exception. The type and/or source of data (e.g., returned set of related concepts corresponding to a test set of concepts from a concept expansion system) also does not cause the data gathering activity to integrate the judicial exception into a practical application. Further, the recitation of a generic computer component (e.g., a computer) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Limitation [5] recites “performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level.” Performing an action step, under a broadest reasonable interpretation, is mere instruction to apply the exception and, therefore, fails to integrate the exception into a practical application. The recitation of a generic computer component (e.g., by a computer) also constitutes mere instruction to “apply” the exception and does not integrate the exception into a practical application.
Step 2B: NO. The additional elements do not provide an inventive concept.
Limitation [1] recites “receiving, by a computer, a returned set of related concepts corresponding to a test set of concepts from a concept expansion system.” Receiving information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Further, the recitation of a generic computer component (e.g., a computer) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Limitation [5] recites “performing, by the computer, a set of action steps in response to the computer determining that the accuracy metric is not greater than the minimum accuracy threshold level.” Performing an action step, under a broadest reasonable interpretation, is mere instruction to apply the exception and, therefore, fails to integrate the exception into a practical application. The recitation of a generic computer component (e.g., by a computer) also constitutes mere instruction to “apply” the exception and does not provide an inventive concept.
Accordingly, claim 1 is ineligible.
Claim 11 is a system claim corresponding to claim 1 and, therefore, is similarly ineligible. Claim 11 additionally recites:
A computer system for accuracy evaluation of concept expansion systems, the computer system comprising:
a communication fabric;
a storage device connected to the communication fabric, wherein the storage device stores program instructions; and
a processor connected to the communication fabric, wherein the processor executes the program instructions to:
Step 2A Prong 2/Step 2B: NO. The additional elements of claim 11 fail to integrate the exception into a practical application or provide an inventive concept.
Recitation of generic computer components (e.g., a communication fabric, storage device, processor) at a high level of generality constitutes mere instruction to “apply” the exception.
Claim 14 is a computer program product claim corresponding to claim 1 and, therefore, is similarly ineligible.
Claim 14 additionally recites:
A computer program product for accuracy evaluation of concept expansion systems, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
Recitation of generic computer components (e.g., a computer-readable storage medium, program instructions, computer) at a high level of generality constitutes mere instruction to “apply” the exception.
Claim 2
The computer-implemented method of claim 1, further comprising:
[1] receiving, by the computer, a text corpus corresponding to a particular domain to evaluate accuracy of the concept expansion system from a client device of a domain expert via a network;
[2] annotating, by the computer, using an annotator corresponding to the particular domain, text of the text corpus to form annotated text; and
[3] extracting, by the computer, using the annotator corresponding to the particular domain, the annotated text from the text corpus.
Step 2A Prong 1: YES. Limitation [2] recites annotating text of a text corpus using an annotator corresponding to a particular domain. Annotation of text is a step which may be performed by a domain expert (e.g., a doctor in a healthcare setting) and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Limitation [1] recites receiving a text corpus corresponding to a particular domain (e.g., medicine) from a client device of a domain expect. Mere data gathering is insignificant extra-solution activity and, therefore, does not integrate the judicial exception into a practical application. The type or source of data (e.g., text corpus corresponding to a particular domain from a client device of a domain expert via a network) does not cause the data gathering to practically integrate the exception. Recitation of a generic computer component (e.g., by a computer, via a network) at a high level of abstraction constitutes mere instruction to “apply” the exception. Limitation [3] recites extracting the annotated text from the text corpus and is also data gathering.
Step 2B: NO.
Receiving information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). Accordingly, limitations [1] and [3] do not provide an inventive concept.
Further, the recitation of a generic computer component (e.g., a computer, a network) constitutes a mere instruction to “apply” the exception and, therefore, fails to integrate the judicial exception into a practical application.
Accordingly, claim 2 is ineligible.
Claim 12 is a system claim corresponding to claim 2 and, therefore, is similarly ineligible.
Claim 15 is a CRM claim corresponding to claim 2 and, therefore, is similarly ineligible.
Claim 3
The computer-implemented method of claim 2, further comprising:
[1] mapping, by the computer, using an annotation mapper, the annotated text extracted from the text corpus to a set of concepts used by the concept expansion system; and
[2] generating, by the computer, using the annotation mapper, a set of compatible concepts that is compatible with the concept expansion system as input based on the mapping of the annotated text extracted from the text corpus to the set of concepts used by the concept expansion system.
Step 2A Prong 1: YES.
Limitation [1] recites mapping annotated text to a set of concepts. Mapping, in the context of the claim and under a broadest reasonable interpretation, encompasses a mental evaluation and mapping of annotated text to a set of concepts. Limitation [2] recites generating a set of compatible concepts to the set of concepts used by the concept expansion system. “Generating” encompasses a user mentally evaluating the set of concepts and coming up with a set of compatible concepts, something which may be performed by a human and, therefore, also falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Recitation of generic computer components (e.g., by the computer, using an annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception and does not integrate the judicial exception into a practical application.
Step 2B: NO.
Recitation of generic computer components (e.g., by the computer, using an annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception and does not provide an inventive concept.
Accordingly, claim 3 is ineligible.
Claim 13 is a system claim corresponding to claim 3 and, therefore, is similarly ineligible.
Claim 16 is a CRM claim corresponding to claim 3 and, therefore, is similarly ineligible.
Claim 4
The computer-implemented method of claim 3, wherein the annotation mapper maps the annotated text extracted from the text corpus to a set of concepts used by the concept expansion system based on retrieved domain knowledge corresponding to the particular domain of the text corpus.
Step 2A Prong 1: YES.
Mapping annotated text to a set of concepts based on retrieved domain knowledge corresponding to a particular domain (e.g., medicine) may be performed by evaluation, opinion, and judgement, and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2/Step 2B: NO.
Recitation of generic computer components (e.g., annotation mapper, concept expansion system) at a high level of generality constitutes mere instruction to “apply” the exception and, therefore, does not integrate the exception into a practical application or provide an inventive concept.
Accordingly, claim 4 is ineligible.
Claim 17 is a CRM claim corresponding to claim 4 and, therefore, is similarly ineligible.
Claim 5
The computer-implemented method of claim 3, further comprising:
[1] generating, by the computer, using the annotation mapper, a report regarding the set of compatible concepts that includes at least one of statistics corresponding to frequency of distribution of the set of compatible concepts, information corresponding to how the set of compatible concepts was generated, and provenance corresponding to where each of the set of compatible concepts was found in the text corpus;
[2] sending, by the computer, the report regarding the set of compatible concepts to the client device of the domain expert via the network; and
[3] receiving, by the computer, feedback regarding the set of compatible concepts from the client device of the domain expert via the network.
Step 2A Prong 2: NO.
Limitation [1] recites generating a report regarding the compatible concepts and other information as to how the compatible concepts were generated. Data outputting is considered insignificant extra-solution activity and, therefore, does not integrate the exception into a practical application. Limitation [2] also describes data outputting step and, therefore, does not integrate the exception into a practical application. Limitation [3] describes a data gathering step (receiving feedback from the domain expert) and constitutes insignificant extra-solution activity.
Recitation of generic computer components (e.g., by the computer, using the annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception.
Step 2B: NO.
Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Recitation of generic computer components (e.g., by the computer, using the annotation mapper) at a high level of generality constitutes mere instruction to “apply” the exception.
Accordingly, claim 5 is ineligible.
Claim 18 is a CRM claim corresponding to claim 5 and, therefore, is similarly ineligible.
Claim 6
The computer-implemented method of claim 5, further comprising:
[1] determining, by the computer, whether the feedback received from the domain expert includes corrections to the set of compatible concepts; and
[2] implementing, by the computer, the corrections in the set of compatible concepts in response to the computer determining that the feedback received from the domain expert does include corrections to the set of compatible concepts.
Step 2A Prong 1: YES.
Limitation [1] describes determining whether the feedback from the domain expert includes corrections. This may be performed mentally by evaluation, opinion, and judgment of a human and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Limitation [2] recites implementing the corrections in the set of compatible concepts in response to determining that the feedback includes corrections. The description of a solution or outcome without detail as to how it is achieved constitutes mere instruction to “apply” the exception. Recitation of generic computer components (e.g., by the computer) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 6 is ineligible.
Claim 19 is a CRM claim corresponding to claim 6 and, therefore, is similarly ineligible.
Claim 7
The computer-implemented method of claim 6, further comprising:
[1] splitting, by the computer, using a set splitter, the set of compatible concepts into the test set of concepts and the expected set of concepts corresponding to the text corpus based on a splitting strategy; and
[2] inputting, by the computer, the test set of concepts into the concept expansion system.
Step 2A Prong 1: YES.
Limitation [1] describes splitting a set of concepts into a test set and an expected set of concepts based on a splitting strategy. Under a broadest reasonable interpretation of “splitting,” this encompasses a user mentally separating a set of concepts into two subsets using opinion, evaluation, and judgment and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO.
Limitation [2] recites a data inputting step which constitutes insignificant extra-solution activity. The type of data does not cause the data inputting to integrate the exception into a practical application. Recitation of generic computer components (e.g., by the computer, using a set splitter) at a high level of generality also constitutes mere instruction to “apply” the exception.
Step 2B: NO.
Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Recitation of generic computer components (e.g., by the computer, using a set splitter) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 7 is ineligible.
Claim 8
The computer-implemented method of claim 7, wherein the splitting strategy is one of a random concept sampling strategy, a most recently used concept strategy, or a user-specified concept splitting strategy.
Step 2A Prong 1: YES. A user is mentally able to implement a random concept sampling strategy, a most recently used concept strategy, or a user-specified concept splitting strategy.
Accordingly, claim 8 is ineligible.
Claim 20 is a CRM claim corresponding to claim 8 and, therefore, is similarly ineligible.
Claim 9
The computer-implemented method of claim 1, wherein the computer generates the accuracy metric corresponding to the concept expansion system by calculating at least one of precision, recall, or F1-score.
Step 2A Prong 1: YES. The claim recites performing a calculation and, therefore, falls under the Mathematical Concepts grouping of abstract ideas.
Step 2A Prong 1/Step 2B: Recitation of generic computer components (e.g., the computer) at a high level of generality also constitutes mere instruction to “apply” the exception.
Accordingly, claim 9 is ineligible.
Claim 10
The computer-implemented method of claim 1, wherein the set of action steps includes at least one of sending a notification to a domain expert regarding accuracy of the concept expansion system, sending the expected set of concepts to the domain expert for review, and generating a new set of compatible concepts to reevaluate the accuracy of the concept expansion system.
Step 2A Prong 1: YES. The limitation “generating a new set of compatible concepts…” recites a mental process and, therefore, falls under the Mental Processes grouping of abstract ideas.
Step 2A Prong 2: NO. The claim recites sending a notification to a domain expert and sending the expected set of concepts to the domain expert for review. These constitute insignificant extra-solution activity. The type of data does not cause the data outputting to integrate the exception into a practical application.
Step 2B: NO. Receiving and transmitting information is well understood routine and conventional and, therefore, does not provide an inventive concept. See MPEP 2106.05(d)(II) (Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rivas, Andreia Rodrıguez, Eva Lorenzo Iglesias, and L. Borrajo. "Study of query expansion techniques and their application in the biomedical information retrieval." The Scientific World Journal 2014.1 (2014): 132158.
Zeng, Qing T., et al. "Synonym, topic model and predicate-based query expansion for retrieving clinical documents." AMIA Annual Symposium Proceedings. Vol. 2012. 2012.
Kholghi, Mahnoosh, et al. "Active learning: a step towards automating medical concept extraction." Journal of the American Medical Informatics Association 23.2 (2016): 289-296.
El-Hasnony, Ibrahim M., et al. "Multi-label active learning-based machine learning model for heart disease prediction." Sensors 22.3 (2022): 1184.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHID KHAN whose telephone number is (571)270-0419. The examiner can normally be reached M-F, 9-5 est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571)272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHID K KHAN/ Primary Examiner, Art Unit 2146
Prosecution Insights