DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-10 are pending and examined herein. No claims are canceled.
Priority
As detailed on the 25 July 2022 filing receipt, the application claims priority as early as 18 May 2022. At this point in examination, all claims have been interpreted as being accorded this priority date as the effective filing date.
Information Disclosure Statement
An information disclosure statement (IDS) was filed on 19 July 2023. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the reference is being considered by the examiner.
Claim Objections
Claim 1 is objected to because of the following informalities: one type of modalities” should read “one type of modality” and contains multiple elements not set at the same indentation. That is, the obtaining steps are indented but the inputting and modifying steps are not. MPEP 608.01(i) pertains.
Claim 5 is objected to because it does not end with a period.
Appropriate correction is required.
35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 USC § 101 because the claimed inventions are directed to an abstract idea without significantly more. "Claims directed to nothing more than abstract ideas (such as a mathematical formula or equation), natural phenomena, and laws of nature are not eligible for patent protection" (MPEP 2106.04 § I). Abstract ideas include mathematical concepts, and procedures for evaluating, analyzing or organizing information, which are a type of mental process (MPEP 2106.04(a)(2)). The claims as a whole, considering all claim elements individually and in combination, are directed to a judicial exception at Step 2A, Prong 2, and the additional elements of the claims, considered individually and in combination, do not provide significantly more at Step 2B than the abstract idea of predicting Alzheimer’s disease.
MPEP 2106 organizes JE analysis into Steps 1, 2A (Prong One & Prong Two), and 2B as analyzed below.
Step 1: Are the claims directed to a process, machine, manufacture, or composition of matter (MPEP 2106.03)?
Step 2A, Prong One: Do the claims recite a judicially recognized exception, i.e., a law of
nature, a natural phenomenon, or an abstract idea (MPEP 2106.04(a-c))?
Step 2A, Prong Two: If the claims recite a judicial exception under Prong One, then is the judicial exception integrated into a practical application by an additional element (MPEP 2106.04(d))?
Step 2B: Do the claims recite a non-conventional arrangement of elements in addition to any identified judicial exception(s) (MPEP 2106.05)?
Step 1: Are the claims directed to a 101 process, machine, manufacture, or composition of matter (MPEP 2106.03)?
The claims are directed to a method (claims 1-7) and a computer system (claims 8-10), each of which falls within one of the categories of statutory subject matter. [Step 1: Yes]
Step 2A, Prong One: Do the claims recite a judicially recognized exception, i.e., a law of nature, a natural phenomenon, or an abstract idea (MPEP 2106.04(a-c))?
With respect to Step 2A, Prong One, the claims recite judicial exceptions in the form of abstract ideas. MPEP § 2106.04(a)(2) further explains that abstract ideas are defined as:
• mathematical concepts (mathematical formulas or equations, mathematical relationships
and mathematical calculations) (MPEP 2106.04(a)(2)(I));
• certain methods of organizing human activity (fundamental economic principles or practices, managing personal behavior or relationships or interactions between people) (MPEP 2106.04(a)(2)(II)); and/or
• mental processes (concepts practically performed in the human mind, including observations, evaluations, judgments, and opinions) (MPEP 2106.04(a)(2)(III)).
Claim 1 recites obtaining standard models of according to every modality. Obtaining the model is disclosed as a relationship between the patient samples’ factors and a prediction or modality (pg. 16-17, paragraph [28]) and the model itself reading on mathematical concepts algorithmically by parameter adjustment and optimization (pg. 7, paragraph [29]). Therefore, the obtaining of the standard models is interpreted as a mathematical concept.
Claim 1 recites obtaining modality combinations related to multi-modal standard models, which are interpreted a mathematical concepts for similar reasons as the single-modality models above.
Claim 1 recites modifying the prediction model using the standard models to obtain a trained model. The standard models are mathematical concepts as explained above. The prediction model is disclosed as trained using a loss function and gradient descent method (pg. 24, paragraph [40]), which are verbal descriptions of mathematical concepts. The predicting itself may also be interpreted as a mental process, where the human mind is practically equipped to evaluate data.
Claim 2 recites additional information about the prediction model, including outputting predictions and calculating a loss function, and thus is also directed to a mathematical concept.
Claims 3-5 recite additional description of the loss function, and thus is also directed to a mathematical concept.
Claim 6 recites a teacher-student training architecture, which is a relationship between the mathematical concepts and thus also considered to be a mathematical concept.
Claim 7 recites training using a loss function and parameterization of the models, and thus also considered to be a mathematical concept.
Claim 8 recites generating a prediction result. The prediction result is interpreted a judgment based on the model results, where judgment is a step the human mind is practically equipped to perform.
Claim 9 recites selecting modalities, where selecting a step the human mind is practically equipped to perform.
Claim 10 recites additional information regarding the characterization information. The characterization information is data, and information or “data per se” is not being directed to any statutory category and thus abstract. MPEP 2106.03(I) pertains.
It is remarked that a mathematical relationship may be expressed in words and there is no particular word or set of words that indicates a claim recites a mathematical calculation. That is, a claim does not have to recite the word "calculating" in order to be considered a mathematical calculation. For example, a step of "determining" a variable or number using mathematical methods or "performing" a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation. MPEP 2106.04(a)(2) pertains.
Hence, the claims explicitly recite numerous elements that, individually and in combination,
constitute abstract ideas. The claims must therefore be examined further to determine whether they
integrate that abstract idea into a practical application (MPEP 2106.04(d)). [Step 2A: Yes]
Step 2A, Prong Two: If the claims recite a judicial exception under Prong One, then is the judicial exception integrated into a practical application by an additional element (MPEP 2106.04(d))?
The following elements are recited in addition to the abstract ideas: a computer comprising at least one processor and storage unit (claims 1 and 8) and inputting or importing data (claims 1 and 8-9).
Regarding the computer comprising a processor and memory/storage, the claims state nothing more than that a generic computer performs the functions that constitute the abstract idea. Hence, these are mere instructions to apply the abstract idea using a computer, and therefore the claim does not integrate that abstract idea into a practical application (see MPEP 2106.04(d) § I; and MPEP 2106.05(f)).
Regarding inputting or importing data, these steps are considered to be data gathering steps required to perform the mathematical steps of the algorithm and mental step of evaluating and predicting Alzheimer’s status. Data gathering is insignificant extra-solution activity and does not integrate the abstract ideas into a practical application (MPEP 2106.05(g)). [Step 2A Prong Two: No]
Step 2B: Do the claims recite a non-conventional arrangement of elements in addition to any identified judicial exception(s) (MPEP 2106.05)?
Claims found to be directed to a judicial exception are then further evaluated to determine if the claims recite an inventive concept that provides significantly more than the judicial exception itself. Step 2B of 101 analysis determines whether the claims contain additional elements that amount to an inventive concept, and an inventive concept cannot be furnished by an abstract idea itself (MPEP 2106.05). The following elements are recited in addition to the abstract ideas: a computer comprising at least one processor and storage unit (claims 1 and 8) and inputting or importing data (claims 1 and 8-9).
Storing data on a computer is a conventional computer function (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93). Performing calculations on the computer is a conventional computer function (Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012)). The courts have found that receiving and outputting data are well-understood, routine, and conventional functions of a computer when claimed in a merely generic manner or as insignificant extra-solution activity (see Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information), buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network), Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015), and OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93, as discussed in MPEP 2106.05(d)(II)(i)). MPEP 2106.05(d)) pertains.
Therefore, the recited additional elements, alone or in combination with the judicial exceptions, do not appear to provide an inventive concept. [Step 2B: No]
Conclusion: Claims are Directed to Non-statutory Subject Matter
For these reasons, the claims, when the limitations are considered individually and as a whole,
are directed to an abstract idea and lack an inventive concept. Hence, the claimed invention does not
constitute significantly more than the abstract idea, so the claims are rejected under 35 USC § 101 as
being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102(a)(1)
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 7-8 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Liu (Medical Image Analysis 69(101953): 11 pgs., 2021; newly cited).
Claim 1 recites a prediction model, performed by a computer, wherein the computer comprises at least one processor and at least one storage unit coupled to the processor, the storage unit comprises multiple samples, each of the samples comprises at least one type of modalities, the samples have N types of modalities in total, and some of the samples each has N types of modalities simultaneously, wherein N is a positive integer not less than 3.
Liu teaches a classifier based on multi-modal data where 3 modalities are exemplified (Fig. 1).
Claim 1 recites obtaining pre-established single-modality standard models according to every single type of the modalities in the samples.
Liu teaches single modal data (pg. 2, col. 1, second paragraph).
Claim 1 recites obtaining modality combinations each having m types of the modalities from the samples having multiple types of the modalities to establish corresponding multi-modalities standard models, wherein m is a combination of positive integers not greater than N-1 and not less than 2.
Liu teaches combinations of modalities where a two modalities are known per subject (Fig. 1), where the recited sample is interpreted as reading on the subjects.
Claim 1 recites obtaining modality combinations including single modalities, incomplete modalities (more than one but less than complete), and complete modalities, inputting the models into a to-be-trained model, and obtaining a trained prediction model.
Liu teaches single, incomplete, and the concept of complete modalities for subjects/samples (Fig. 1; pg. 2, col. 2, second paragraph) to train a model based on the features (Fig. 2).
A computer with a processor and memory is not explicitly taught by Liu, but Liu teaches “computer-aided research” (pg. 1, col. 1, first paragraph), where a computer performing analysis using machine learning algorithms based on data and images may be reasonably interpreted as inherently having a processor and memory.
Claim 7 recite a process of establishing the multi-modalities standard models, the multi-modalities standard models are trained by using a loss function and a gradient descent method, a parameter of a corresponding one of the multi-modalities standard models is modified in each epoch to minimize the loss function, and the trained multi-modalities standard models are established after specified epochs are completed.
Liu teaches a loss function (pg. 5, col. 2, Section 4.2) and gradient descent (pg. 5, col. 2, Section 4.1). The functions are performed iteratively (pg. 5, col. 2, Section 4), where each iteration improving the model is interpreted as an epoch.
Claim 8 recites a prediction system comprising memory and storage performing the modeling steps of any of one claims 1-8 to generate a prediction result, which is taught by Liu as obtaining modality information from the subjects, inputting the data into a learning algorithm, and outputting a classification for the data (Fig. 2).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Liu as applied to claims 1 and 7-8 in the rejection under 35 USC 102 above and further in view of Zheng (Medical Imaging 12033: 5 pgs., 2022; newly cited).
Claim 2 recites each of the samples which are used to be the training data has a corresponding ground truth and when the inputted training data is the single-modality training data or the multi-modalities training data, a ground truth of a corresponding one of the samples is imported, each of the single types of the modalities or each of the multiple types of the modalities is calculated by using the to-be-trained prediction model to output a prediction result, a corresponding training data is inputted into a corresponding one of the single-modality standard models or a corresponding one of the multi-modalities standard models to generate a corresponding standard result, and a corresponding loss function is calculated by using the prediction result, the ground truth and the standard result; and when the inputted training data is the complete-modalities training data, each modality in the complete modalities training data is calculated by the to-be-trained prediction model to output a prediction result, a ground truth of a corresponding one of the samples is imported, and a corresponding loss function is calculated by using the prediction result and the ground truth.
Liu teaches a cluster label and ground truth label of each sample (pg.7 , col. 2, Equation 20). Liu teaches single modal data (pg. 3, col. 1, second paragraph) and otherwise incomplete data (Fig. 2), and classifications based on the single modality and multi-modality datasets (Fig. 4). Liu is not interpreted as clearly teaching training the individual single-modality datasets.
Zheng is interpreted as teaching training of individual datasets (Fig. 1), where training occurs on the high accessibility data and low accessibility data (abstract).
Claim 6 recites a prediction model according claim 2, wherein based on a teacher-student model training architecture, the single-modality standard models and the multi-modalities standard models are
teacher models, and the prediction model is a student model.
Liu teaches single modality and multi-modality models (Fig. 2) but not a teacher-student model.
Zheng teaches a knowledge distillation process in which knowledge is transferred from a teacher model to a student model (pg. 1, last paragraph).
Combining Liu and Zheng
The claimed invention is directed to combining multiple data types or modalities to produce a prediction. Prior art Liu teaches multiple data types in the form of MRI and PET imaging. Prior art Zheng teaches using clinical data in addition to imaging data (abstract) and a student teacher model (pg. 1, last paragraph). An advantage of this architecture taught by Zheng is the student model is less complex with fewer parameters, and is expected to have better performance than the teacher while at the same time being computationally efficient (pg.1, last paragraph). The applied prior art is directed to multi-model prediction of disease and thus a shared field of endeavor. Therefore, the invention is prima facie obvious.
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Liu as applied to claims 1 and 7-8 in the rejection under 35 USC 102 above and further in view of Azad and Bi (arXiv 2203: 21 pgs., 2022; newly cited).
Claim 3 recites the inputted training data is the single-modality training data or the multi-modalities training data, the loss function is defined by a classification loss and a distillation loss; the corresponding classification loss is calculated by using a corresponding prediction result and a corresponding a ground truth; and the corresponding distillation loss is calculated by using a corresponding prediction result and a corresponding standard result.
Liu teaches a loss function (pg. 5, col. 2, Section 4.2) but not clearly classification loss and distillation loss.
Azad teaches classification loss and distillation loss in a student-teacher framework (Fig. 14).
Claim 4 recites the inputted training data is the complete-modalities training data, the loss function is defined by a classification loss; and the corresponding classification loss is calculated by using a corresponding prediction result and a corresponding ground truth.
Liu teaches training using complete modalities (Fig. 2) and a loss function (pg. 5, col. 2, Section 4.2) but not clearly classification loss and distillation loss.
Azad teaches training using full and single modalities (Fig. 14) and classification loss and distillation loss in a student-teacher framework (Fig. 14).
Combining Liu and Azad
An invention would have been obvious to one of ordinary skill in the art if some motivation in the prior art would have led that person to modify prior art reference teachings to arrive at the claimed invention prior to the effective filing date of the invention. One would have been motivated to combine the work of Azad, which teaches different loss metrics (Fig. 14), with the work of Liu because classification loss is a metric in student-teacher models to minimize loss between a student prediction and ground truth while distillation loss is designed to minimize loss between student and teacher, which together are necessary to learn correct labels. Liu and Azad are both directed to shared field of endeavor of training a model with full/complete and missing modalities (Azad: Fig. 16) and thus the invention is considered prima facie obvious.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Liu as applied to claims 1 and 7-8 in the rejection under 35 USC 102 above and further in view of Zheng and Bi (Brain Imaging and Behavior 15: 1986-1996, 2021; newly cited).
Claim 9 recites a prognostic system for Alzheimer's disease wherein the multiple types of modalities of interest in the input information are selected from any three types of a clinical factor modality, a brain image modality, an electroencephalography modality, an environment air pollution modality and a gene modality and each of the multiple types of modalities of interest is generated by inputting a corresponding characterization information into a corresponding one of the single-modality standard models.
Liu teaches MRI and PET scan data (pg. 2, col. 2, third paragraph), which are both image data, but not other types of data.
Zheng teaches combination with demographic and cognitive measures (abstract), interpreted as clinical features.
Bi teaches combining imaging and gene data (abstract).
Claim 10 recites the clinical factor characterization information comprises at least one of an information of age and gender, a history of related diseases and comorbidity, brain cognitive functions and mental behavior symptoms of a patient; the brain image characterization information comprises a magnetic resonance image or a computed tomography image of a brain of a patient; the electro-encephalography characterization information includes at least one feature of electro-encephalography with a specific frequency and localization; the environment air pollution characterization information is air pollution data of a life place of a patient, wherein the air pollution data comprises at least one of concentration of suspended particulates, concentration of fine suspended particulates, concentration of nitrogen oxide, concentration of nitrogen monoxide, concentration of nitrogen dioxide, concentration of carbon monoxide, concentration of carbon dioxide and concentration of ozone; and the gene characterization information is single nucleotide polymorphism at a specific gene site and/or a length of nucleotide.
Zheng teaches gender and age as clinical information (abstract).
Liu teaches MRI and PET scan data (pg. 2, col. 2, third paragraph). Bi teaches single nucleotide polymorphism data of patients (pg. 1987, col. 2, last paragraph).
Combining Liu, Zheng, and Bi
The claimed invention is directed to combining multiple data types or modalities to produce a prediction. Prior art Liu teaches multiple data types in the form of MRI and PET imaging. Prior art Zheng teaches using clinical data in addition to imaging data (abstract) and prior art Bi teaches genetic data in addition to imaging data (abstract). One could have combined the elements as claimed by known methods, and that in combination, each element merely would have performed the same function as it did separately; furthermore one of ordinary skill in the art would have recognized that the results of the combination were predictable. MPEP 2143(A) pertains. The applied prior art is directed to multi-model prediction of disease and thus a shared field of endeavor. Therefore, the invention is prima facie obvious.
Claims Considered Free of the Prior Art
Claim 5 recites the mathematical structure of the loss function in the student-teacher architecture. Similar art, such as Zheng and Azad, do not clearly teach this mathematical concept and so claim 5 is considered free of the prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Robert J Kallal whose telephone number is (571)272-6252. The examiner can normally be reached Monday through Friday 8 AM - 4 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Olivia M. Wise can be reached at (571) 272-2249. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Robert J. Kallal/Examiner, Art Unit 1685