DETAILED ACTION
This action is in response to the application field on 12/21/2023. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/21/2023. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Subject Matter of Eligibility Analysis Step 1:
Claim 1 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
extracting features from the data to determine a plurality of training examples, each training example being associated with a different physician (this limitation is a mental process as it encompasses a human mentally selecting features from a data set).
determining ground truth labels for one or more of the training examples to generate a plurality of labeled training examples, each ground truth label comprising a taxonomy associated with a physician (this limitation is a mental process as it encompasses a human mentally determining ground truth labels for a data set).
segregating the plurality of labeled training examples into training data, validation data, and test data (this limitation is a mental process as it encompasses a human mentally separating data into different data sets).
Therefore, claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
receiving data associated with a plurality of physicians (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
training a machine learning model to predict a taxonomy associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
tuning hyperparameters of the model based on the validation data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 1 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
receiving data associated with a plurality of physicians is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
training a machine learning model to predict a taxonomy associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
tuning hyperparameters of the model based on the validation data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 1 is subject-matter ineligible.
Regarding claim 2:
Subject Matter of Eligibility Analysis Step 1:
Claim 2 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 2 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 2 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 2 further recites additional elements of
the model comprises a random forest architecture (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 2 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 2 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the model comprises a random forest architecture recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 2 is subject-matter ineligible.
Regarding claim 3:
Subject Matter of Eligibility Analysis Step 1:
Claim 3 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 3 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 3 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 3 further recites additional elements of
the model comprises a deep neural network (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 3 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 3 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the model comprises a deep neural network recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 3 is subject-matter ineligible.
Regarding claim 4:
Subject Matter of Eligibility Analysis Step 1:
Claim 4 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
extracting features from the data comprises determining a number of times that each of the physicians has performed each of the different medical procedures with a predetermined time period (this limitation is an abstract as it encompasses a human mentally selecting features from a data set that determines the number of times a physician performs different medical procedures within a time period).
Therefore, claim 4 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 4 further recites additional elements of
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 4 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 4 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 4 is subject-matter ineligible.
Regarding claim 5:
Subject Matter of Eligibility Analysis Step 1:
Claim 5 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 5 recites
selecting the model having the best performance among the plurality of models (this limitation is a mental process as it encompasses a human mentally choosing the performing model).
Therefore, claim 5 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 5 further recites additional elements of
training a plurality of models to predict a taxonomy associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
testing a performance of each of the models based on the test data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 5 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
training a plurality of models to predict a taxonomy associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
testing a performance of each of the models based on the test data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 5 is subject-matter ineligible.
Regarding claim 6:
Subject Matter of Eligibility Analysis Step 1:
Claim 6 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 6 is dependent on claim 5, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 5 is applied here. Therefore claim 6 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 6 further recites additional elements of
testing the performance of each of the models by determining F-scores associated with outputs of each of the models (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 6 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 6 do not provide significantly more than the abstract idea itself, taken alone and in combination because
testing the performance of each of the models by determining F-scores associated with outputs of each of the models is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 6 is subject-matter ineligible.
Regarding claim 7:
Subject Matter of Eligibility Analysis Step 1:
Claim 7 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 7 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 7 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 7 further recites additional elements of
training the model to determine a confidence level of the predicted taxonomy (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 7 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 7 do not provide significantly more than the abstract idea itself, taken alone and in combination because
training the model to determine a confidence level of the predicted taxonomy is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 7 is subject-matter ineligible.
Regarding claim 8:
Subject Matter of Eligibility Analysis Step 1:
Claim 8 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 8 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 8 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of
training a first stage of the model to predict a specialty associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
training a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 8 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 8 do not provide significantly more than the abstract idea itself, taken alone and in combination because
training a first stage of the model to predict a specialty associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
training a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 8 is subject-matter ineligible.
Regarding claim 9:
Subject Matter of Eligibility Analysis Step 1:
Claim 9 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 9 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 9 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
training a stacked autoencoder using unlabeled data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
adding a plurality of fully connected layers to the stacked autoencoder (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
after training the stacked autoencoder, training the fully connected layers based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
after training the fully connected layers, training the model comprising the stacked autoencoder and the fully connected layers based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 9 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
training a stacked autoencoder using unlabeled data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
adding a plurality of fully connected layers to the stacked autoencoder is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
after training the stacked autoencoder, training the fully connected layers based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
after training the fully connected layers, training the model comprising the stacked autoencoder and the fully connected layers based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 9 is subject-matter ineligible.
Regarding claim 10:
Subject Matter of Eligibility Analysis Step 1:
Claim 10 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 10 recites
determining ground truth labels for one or more of the unlabeled training examples based on the similarity to generate supplemental labeled training examples (this limitation is a mental process as it encompasses a human mentally determining ground truth labels for a dataset)
combining the labeled training examples and the supplemental labeled training examples to generate expanded labeled training examples (this limitation is a mental process as it encompasses a human mentally combining two data sets into one).
segregating the expanded labeled training examples into training data, validation data, and test data (this limitation is a mental process as it encompasses a human mentally separating data into different data sets).
Therefore, claim 10 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of
using unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 10 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
using unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 10 is subject-matter ineligible.
Regarding claim 11:
Subject Matter of Eligibility Analysis Step 1:
Claim 11 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 11 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 11 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 11 further recites additional elements of
determining a relative amount that one or more of the features contribute to one or more taxonomies output by the model using Shapley Additive Explanations (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 11 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 11 do not provide significantly more than the abstract idea itself, taken alone and in combination because
determining a relative amount that one or more of the features contribute to one or more taxonomies output by the model using Shapley Additive Explanations is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 11 is subject-matter ineligible.
Regarding claim 12:
Subject Matter of Eligibility Analysis Step 1:
Claim 12 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 12 recites
extracting target features from the unlabeled data (this limitation is an abstract idea as it encompasses a human mentally selecting features from a data set).
assigning a taxonomy to the target physician based on the trained model (this limitation is an abstract idea as it encompasses a human mentally assigning a taxonomy to a physician based on an output).
Therefore, claim 12 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 12 further recites additional elements of
receiving unlabeled data associated with a target physician (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
inputting the target features into the trained model (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 12 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 12 do not provide significantly more than the abstract idea itself, taken alone and in combination because
receiving unlabeled data associated with a target physician is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
inputting the target features into the trained model is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 12 is subject-matter ineligible.
Regarding claim 13:
Subject Matter of Eligibility Analysis Step 1:
Claim 13 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 13 recites
assigning the taxonomy to the target physician comprises selecting the taxonomy having the highest probability value output by the trained model (this limitation is an abstract idea as it encompasses a human mentally assigning a taxonomy to a physician based on an output).
Therefore, claim 13 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
the trained model outputs a probability value that the target physician is associated with each of a plurality of taxonomies (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 13 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the trained model outputs a probability value that the target physician is associated with each of a plurality of taxonomies is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 13 is subject-matter ineligible.
Regarding claim 14:
Subject Matter of Eligibility Analysis Step 1:
Claim 14 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 14 is dependent on claim 12, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 12 is applied here. Therefore claim 14 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
determining a relative amount that one or more of the target features contribute to the taxonomy assigned to the target physician using Shapley Additive Explanations (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 14 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 14 do not provide significantly more than the abstract idea itself, taken alone and in combination because
determining a relative amount that one or more of the target features contribute to the taxonomy assigned to the target physician using Shapley Additive Explanations is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 14 is subject-matter ineligible.
Regarding claim 15:
Subject Matter of Eligibility Analysis Step 1:
Claim 15 recites an apparatus, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 15 recites
extract features from the data to determine a plurality of training examples, each training example being associated with a different physician (this limitation is a mental process as it encompasses a human mentally selecting features from a data set).
determine ground truth labels for one or more of the training examples to generate a plurality of labeled training examples, each ground truth label comprising a taxonomy associated with a physician (this limitation is a mental process as it encompasses a human mentally determining ground truth labels for a data set).
segregate the plurality of labeled training examples into training data, validation data, and test data (this limitation is a mental process as it encompasses a human mentally separating data into different data sets).
Therefore, claim 15 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements of
An apparatus comprising a controller (this element does not integrate the abstract idea into a practical application because it recites a generic computing component on which to perform the abstract idea (see MPEP 2106.05(b))).
receive data associated with a plurality of physicians (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
train a machine learning model to predict a taxonomy associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
tune hyperparameters of the model based on the validation data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 15 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
An apparatus comprising a controller is a generic component used to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(b)).
receive data associated with a plurality of physicians is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
train a machine learning model to predict a taxonomy associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
tune hyperparameters of the model based on the validation data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 15 is subject-matter ineligible.
Regarding claim 16:
Subject Matter of Eligibility Analysis Step 1:
Claim 16 recites an apparatus, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 16 recites
extract features from the data comprises determining a number of times that each of the physicians has performed each of the different medical procedures with a predetermined time period (this limitation is an abstract as it encompasses a human mentally selecting features from a data set that determines the number of times a physician performs different medical procedures within a time period).
Therefore, claim 16 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 16 further recites additional elements of
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 16 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 16 do not provide significantly more than the abstract idea itself, taken alone and in combination because
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 16 is subject-matter ineligible.
Regarding claim 17:
Subject Matter of Eligibility Analysis Step 1:
Claim 17 recites an apparatus, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 17 recites
select the model having the best performance among the plurality of models (this limitation is a mental process as it encompasses a human mentally choosing the performing model).
Therefore, claim 17 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 17 further recites additional elements of
train a plurality of models to predict a taxonomy associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
test a performance of each of the models based on the test data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 17 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 17 do not provide significantly more than the abstract idea itself, taken alone and in combination because
train a plurality of models to predict a taxonomy associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
test a performance of each of the models based on the test data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 17 is subject-matter ineligible.
Regarding claim 18:
Subject Matter of Eligibility Analysis Step 1:
Claim 18 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 18 is dependent on claim 15, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 15 is applied here. Therefore claim 18 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 18 further recites additional elements of
train a first stage of the model to predict a specialty associated with a physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
train a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 18 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 18 do not provide significantly more than the abstract idea itself, taken alone and in combination because
train a first stage of the model to predict a specialty associated with a physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
train a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 18 is subject-matter ineligible.
Regarding claim 19:
Subject Matter of Eligibility Analysis Step 1:
Claim 19 recites an apparatus, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 19 recites
determine ground truth labels for one or more of the unlabeled training examples based on the similarity to generate supplemental labeled training examples (this limitation is a mental process as it encompasses a human mentally determining ground truth labels for a dataset)
combine the labeled training examples and the supplemental labeled training examples to generate expanded labeled training examples (this limitation is a mental process as it encompasses a human mentally combining two data sets into one).
segregate the expanded labeled training examples into training data, validation data, and test data (this limitation is a mental process as it encompasses a human mentally separating data into different data sets).
Therefore, claim 19 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 19 further recites additional elements of
use unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 19 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 19 do not provide significantly more than the abstract idea itself, taken alone and in combination because
use unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 19 is subject-matter ineligible.
Regarding claim 20:
Subject Matter of Eligibility Analysis Step 1:
Claim 20 recites an apparatus, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 20 recites
extract target features from the unlabeled data (this limitation is an abstract idea as it encompasses a human mentally selecting features from a data set).
assign a taxonomy to the target physician based on the trained model (this limitation is an abstract idea as it encompasses a human mentally assigning a taxonomy to a physician based on an output).
Therefore, claim 20 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 20 further recites additional elements of
receive unlabeled data associated with a target physician (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
input the target features into the trained model (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 20 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 20 do not provide significantly more than the abstract idea itself, taken alone and in combination because
receive unlabeled data associated with a target physician is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
input the target features into the trained model is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 20 is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 2, 12, 13, 15, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove et al. (Using Machine Learning to Predict Primary Care and Advance Workforce Research) (hereafter referred to as Wingrove) in view of Burnett et al. (US 12040062 B2) (hereafter referred to as Burnett).
Regarding claim 1, Wingrove teaches
receiving data associated with a plurality of physicians (Wingrove, Data Sources Section, “For this cross sectional study, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Part D Prescriber Public Use Files to identify prescriptions.19 These data sets include information regarding beneficiaries enrolled in Medicare Part D (70% of all beneficiaries), information about providers (eg, NPI and self-reported specialty), and prescriptions (except for over-the-counter drugs). To identify procedures, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Physician and Other Supplier Public Use Files.20 In this Medicare Part B data set, procedures were identified with Healthcare Common Procedure Coding System codes”).
extracting features from the data to determine a plurality of training examples, each training example being associated with a different physician (Wingrove, Variables Section, “We assigned physicians from specialties with a low number of physicians or for which multiple specialties practice in similar ways into 1 of 27 larger specialties (eg, internal medicine or family medicine were relabeled as primary care). To avoid rare drugs or procedures, we restricted the analysis to the 850 most common prescriptions and 1,500 most common procedure codes and excluded items that did not appear in all 3 years. For each year, we characterized physicians by whether they prescribed or performed each of the 2,350 prescriptions/procedures”).
determining ground truth labels for one or more of the training examples to generate a plurality of labeled training examples, each ground truth label comprising a taxonomy associated with a physician (Wingrove, Variables Section, “To assess the same cohort of physicians, the analysis was restricted to nonpediatric physicians appearing in all 3 years (though they only needed to appear in either the procedure or prescription data sets for a given year). To maintain consistency, physicians were only included if they self-reported the same specialty across all 3 years”).
training a machine learning model to predict a taxonomy associated with a physician based on the training data (Wingrove, Deriving the Algorithm Section, “To begin, we trained a separate random forest model (the combined model, consisting of both prescription and procedure data) for each year. Each random forest consisted of 200 trees and had a pool of 100 possible variables at each node”).
Wingrove does not teach, but Burnett does teach
segregating the plurality of labeled training examples into training data, validation data, and test data (Burnett, paragraph 0038, “Turning to the training risk model training process, as described above, the historical patient data is divided, by the data selecting module 216, into at least three subsets of model data 244. The at least three subsets of model data 244, training data 302; validation data 304; and test data 306, are used by the machine-learning training module 220 to generate the one or more risk models”).
tuning hyperparameters of the model based on the validation data (Burnett, paragraph 0040, “Hypermeters, on the other hand, are external to the risk model and has a value that cannot be estimated from the training data 302...In some embodiments, the one or more hyperparameters are tuned for a risk model and help identify one or more parameters that cannot be estimated from the training data 302, but contribute to the performance of the risk model in a significant way. As discussed below, the machine-learning training module 220 uses the validation data 304 to determine (e.g., tune) one or more hyperparameters for a risk model”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of dividing a dataset and tuning hyperparameters to machine learning model. One of the ordinary skill in the art would have known to apply the known technique of splitting a dataset and tune hyperparameters. Therefore, applying Burnett’s technique would yield the predicable result of preventing overfitting and accurately measure the performance of the machine learning model (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 2, Wingrove and Burnett teach the method of claim 1, Wingrove further teaches
the model comprises a random forest architecture (Wingrove, Deriving the Algorithm Section, “To begin, we trained a separate random forest model (the combined model, consisting of both prescription and procedure data) for each year. Each random forest consisted of 200 trees and had a pool of 100 possible variables at each node”).
Regarding claim 12, Wingrove and Burnett teach the method of claim 1, Wingrove further teaches
receiving unlabeled data associated with a target physician (Wingrove, Data Sources Section, “For this crosssectional study, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Part D Prescriber Public Use Files to identify prescriptions.19 These data sets include information regarding beneficiaries enrolled in Medicare Part D (70% of all beneficiaries), information about providers (eg, NPI and self-reported specialty), and prescriptions (except for over-the-counter drugs). To identify procedures, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Physician and Other Supplier Public Use Files.20 In this Medicare Part B data set, procedures were identified with Healthcare Common Procedure Coding System codes”).
assigning a taxonomy to the target physician based on the trained model (Wingrove, Aggregate Analysis, “We then categorized physicians according to whether their self-reported specialties did or did not match the predictions”. Examiner notes that categorizing physicians to specialties maps to assigning a taxonomy to the physician).
Wingrove does not teach, but Burnett does teach
receiving unlabeled data associated with a target physician (Burnett, paragraph 0044, “In some embodiments, the data selecting module 216 identifies recently collected patient data the patient database…The current patient data 308 is distinct from the training data 302, validation data 304, and test data 306”. Examiner notes that recently collected data shows that data is unlabeled and has not been used since it is not a part of the training, validating, or testing data sets).
extracting target features from the unlabeled data (Burnett, paragraph 0048, “the risk feature engineering module 218 generates one or more features from the current patient data 308 such that the selected risk model receives expected inputs (e.g., each patient's data is adjusted or formatted to fit the generated one or more features)”).
inputting the target features into the trained model (Burnett, paragraph 0047, “the risk model application module 224 receives the one or more risk features generated by the risk feature engineering module 218 and uses the one or more risk features with the risk models selected by the risk model selecting module 222”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of creating features and using them as input for the machine learning model for unlabeled data. One of the ordinary skill in the art would have known to apply the known technique of machine learning models using unlabeled data. Therefore, applying Burnett’s technique would yield the predicable result of using a larging data set to improve generalization and reduce overfitting (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 13, Wingrove and Burnett teach the method of claim 12, Wingrove further teaches
the trained model outputs a probability value that the target physician is associated with each of a plurality of taxonomies (Wingrove, Abstract Section, “Comparing the predicted specialty to self-report, we assessed performance with F1 scores and area under the receiver operating characteristic curve (AUROC) values”)
assigning the taxonomy to the target physician comprises selecting the taxonomy having the highest probability value output by the trained model (Wingrove, Aggregate Analysis, “We then categorized physicians according to whether their self-reported specialties did or did not match the predictions”. Examiner notes that categorizing physicians to specialties maps to assigning a taxonomy to the physician).
Regarding claim 15, Wingrove teaches
receive data associated with a plurality of physicians (Wingrove, Data Sources Section, “For this cross sectional study, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Part D Prescriber Public Use Files to identify prescriptions.19 These data sets include information regarding beneficiaries enrolled in Medicare Part D (70% of all beneficiaries), information about providers (eg, NPI and self-reported specialty), and prescriptions (except for over-the-counter drugs). To identify procedures, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Physician and Other Supplier Public Use Files.20 In this Medicare Part B data set, procedures were identified with Healthcare Common Procedure Coding System codes”).
extract features from the data to determine a plurality of training examples, each training example being associated with a different physician (Wingrove, Variables Section, “We assigned physicians from specialties with a low number of physicians or for which multiple specialties practice in similar ways into 1 of 27 larger specialties (eg, internal medicine or family medicine were relabeled as primary care). To avoid rare drugs or procedures, we restricted the analysis to the 850 most common prescriptions and 1,500 most common procedure codes and excluded items that did not appear in all 3 years. For each year, we characterized physicians by whether they prescribed or performed each of the 2,350 prescriptions/procedures”).
determine ground truth labels for one or more of the training examples to generate a plurality of labeled training examples, each ground truth label comprising a taxonomy associated with a physician (Wingrove, Variables Section, “To assess the same cohort of physicians, the analysis was restricted to nonpediatric physicians appearing in all 3 years (though they only needed to appear in either the procedure or prescription data sets for a given year). To maintain consistency, physicians were only included if they self-reported the same specialty across all 3 years”).
train a machine learning model to predict a taxonomy associated with a physician based on the training data (Wingrove, Deriving the Algorithm Section, “To begin, we trained a separate random forest model (the combined model, consisting of both prescription and procedure data) for each year. Each random forest consisted of 200 trees and had a pool of 100 possible variables at each node”).
Wingrove does not teach, but Burnett does teach
An apparatus comprising a controller (Burnett, paragraph 0017, “In accordance with some embodiments, an electronic device (e.g., a server system, a computer system, a client device, etc.) includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for performing the operations of the method described above. In accordance with some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by an electronic device, cause the server system to perform the operations of the method described above”).
segregate the plurality of labeled training examples into training data, validation data, and test data (Burnett, paragraph 0038, “Turning to the training risk model training process, as described above, the historical patient data is divided, by the data selecting module 216, into at least three subsets of model data 244. The at least three subsets of model data 244, training data 302; validation data 304; and test data 306, are used by the machine-learning training module 220 to generate the one or more risk models”).
tune hyperparameters of the model based on the validation data (Burnett, paragraph 0040, “Hypermeters, on the other hand, are external to the risk model and has a value that cannot be estimated from the training data 302...In some embodiments, the one or more hyperparameters are tuned for a risk model and help identify one or more parameters that cannot be estimated from the training data 302, but contribute to the performance of the risk model in a significant way. As discussed below, the machine-learning training module 220 uses the validation data 304 to determine (e.g., tune) one or more hyperparameters for a risk model”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of dividing a dataset and tuning hyperparameters to machine learning model. One of the ordinary skill in the art would have known to apply the known technique of splitting a dataset and tune hyperparameters. Therefore, applying Burnett’s technique would yield the predicable result of preventing overfitting and accurately measure the performance of the machine learning model (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 20, Wingrove and Burnett teach the apparatus of claim 15, Wingrove further teaches
receive unlabeled data associated with a target physician (Wingrove, Data Sources Section, “For this crosssectional study, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Part D Prescriber Public Use Files to identify prescriptions.19 These data sets include information regarding beneficiaries enrolled in Medicare Part D (70% of all beneficiaries), information about providers (eg, NPI and self-reported specialty), and prescriptions (except for over-the-counter drugs). To identify procedures, we used the 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Physician and Other Supplier Public Use Files.20 In this Medicare Part B data set, procedures were identified with Healthcare Common Procedure Coding System codes”).
assign a taxonomy to the target physician based on the trained model (Wingrove, Aggregate Analysis, “We then categorized physicians according to whether their self-reported specialties did or did not match the predictions”. Examiner notes that categorizing physicians to specialties maps to assigning a taxonomy to the physician).
Wingrove does not teach, but Burnett does teach
receive unlabeled data associated with a target physician (Burnett, paragraph 0044, “In some embodiments, the data selecting module 216 identifies recently collected patient data the patient database…The current patient data 308 is distinct from the training data 302, validation data 304, and test data 306”. Examiner notes that recently collected data shows that data is unlabeled and have not been used since it is not a part of the training, validating, or testing data sets).
extract target features from the unlabeled data (Burnett, paragraph 0048, “the risk feature engineering module 218 generates one or more features from the current patient data 308 such that the selected risk model receives expected inputs (e.g., each patient's data is adjusted or formatted to fit the generated one or more features)”).
input the target features into the trained model (Burnett, paragraph 0047, “the risk model application module 224 receives the one or more risk features generated by the risk feature engineering module 218 and uses the one or more risk features with the risk models selected by the risk model selecting module 222”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of creating features and using them as input for the machine learning model for unlabeled data. One of the ordinary skill in the art would have known to apply the known technique of machine learning models using unlabeled data. Therefore, applying Burnett’s technique would yield the predicable result of using a larging data set to improve generalization and reduce overfitting (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett and Chung et al. (US 10614361 B2) (hereafter referred to as Chung).
Regarding claim 3, Wingrove and Burnett teach the method of claim 1, Wingrove and Burnett do not teach, but Chung does teach
the model comprises a deep neural network (Chung, paragraph 0022, “In one particular embodiment, the model of FIG. 2 can be implemented in the context of a cost-sensitive deep neural network (CSDNN), although other deep learning environments may be used as well, and the present disclosure is not intended to be limited to any particular type.”).
Wingrove, Burnett, and Chung are considered analogous to the claimed invention because they deal with machine learning models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use a deep neural network from Chung. Chung teaches “a pre-trained cost-sensitive auto-encoder can be used in combination with a training (fine-tuning) stage for cost-sensitive deep learning (e.g., a neural network classifier with multiple hidden layers, as will be explained below). Thus, cost information is effectively combined with deep learning by modifying the objective function in the pre-training phase. By minimizing the modified objective function, the auto-encoder not only tries to capture the underlying pattern in the data, it further “learns” the cost information and “stores” it in the structure. By later fine-tuning at the training stage, the classification system yields improved performance (lower cost) than a typical classification system that does not take cost information into account during pre-training” (Chung, paragraph 012) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Claim(s) 4 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett and Yoo et al. (US 10691407 B2) (hereafter referred to as Yoo).
Regarding claim 4, Wingrove and Burnett teach the method of claim 1, Wingrove and Burnett do not teach, but Yoo does teach
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians; and extracting features from the data comprises determining a number of times that each of the physicians has performed each of the different medical procedures with a predetermined time period (Yoo, paragraph 0114, “In one embodiment, the analysis engine 202 accesses a database 206 to determine the total number of procedures of a particular type performed by the plurality of professionals. In another embodiment, the analysis engine 202 determines the total number of procedures of the type performed during a retrospective time period (e.g., the previous day, month, quarter, year, and so on)”).
Wingrove, Burnett, and Yoo are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to determine features such as the type of procedures performed as well as the number of said procedures performed from a professional from Yoo. One of the ordinary skill in the art would have known to apply the known technique of extracting count-based frequency features. Therefore, applying Yoo’s technique would yield the predicable result of providing a stronger feature representation for the model to use to train on (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 16, Wingrove and Burnett teach the apparatus of claim 15, Wingrove and Burnett do not teach, but Yoo does teach
the data associated with the plurality of physicians comprises data about different medical procedures performed by the physicians; and the controller is configured to extract features from the data comprises determining a number of times that each of the physicians has performed each of the different medical procedures with a predetermined time period (Yoo, paragraph 0114, “In one embodiment, the analysis engine 202 accesses a database 206 to determine the total number of procedures of a particular type performed by the plurality of professionals. In another embodiment, the analysis engine 202 determines the total number of procedures of the type performed during a retrospective time period (e.g., the previous day, month, quarter, year, and so on)”).
Wingrove, Burnett, and Yoo are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to determine features such as the type of procedures performed as well as the number of said procedures performed from a professional from Yoo. One of the ordinary skill in the art would have known to apply the known technique of extracting count-based frequency features. Therefore, applying Yoo’s technique would yield the predicable result of providing a stronger feature representation for the model to use to train on (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 5, 6, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett and Wubbels et al. (US 10810512 B1) (hereafter referred to as Wubbels).
Regarding claim 5, Wingrove and Burnett teach the method of claim 1, Wingrove teaches
training a plurality of models to predict a taxonomy associated with a physician based on the training data (Wingrove, Abstract “We used 2014-2016 prescription and procedure Medicare data to train 3 sets of random forest classifiers (prescription only, procedure only, and combined) to predict specialty…Physicians were assigned to testing and training cohorts, and random forest models were trained and then applied to 2014-2016 data sets for the testing cohort to generate a series of specialty predictions.”)
testing a performance of each of the models based on the test data (Wingrove, Validating the Algorithm Section, “To assess consistency, we applied each of the 3 random forest models to each of the 3 years of Test data, giving 9 sets of predictions based on the physicians in the Test group. The 9 sets of predictions were compared with self-reported specialty to generate an F1 score (harmonic mean of precision [positive predictive value] and recall [sensitivity]) for each specialty, and a macro F1 score, calculated on the average precision and recall of all specialties”).
Wingrove and Burnett do not teach, but Wubbels does teach
selecting the model having the best performance among the plurality of models (Wubbels, paragraph 0048, “In the end, the training module 410 can select only the substantially optimal machine learning models, namely those with the highest accuracy and lowest latency”)
Wingrove, Burnett, and Wubbels are considered analogous to the claimed invention because they deal with medical data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to select the machine learning model with the highest accuracy and lowest latency from Wubbels. One of the ordinary skill in the art would have known to apply the known technique of choosing the most optimal model to use. Therefore, applying Wubbels’ technique would yield the predicable result of using the machine model with the best performance (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 6, Wingrove, Burnett, and Wubbels teach the method of claim 5, Wingrove further teaches
testing the performance of each of the models by determining F-scores associated with outputs of each of the models (Wingrove, Prescription- and Procedure-Only Subanalyses Section, “We then generated an F1 score for each specialty and macro F1 scores of the prescription-only and procedure-only sets of predictions”)
PNG
media_image1.png
860
319
media_image1.png
Greyscale
Regarding claim 17, Wingrove and Burnett teach the apparatus of claim 15, Wingrove teaches
train a plurality of models to predict a taxonomy associated with a physician based on the training data (Wingrove, Abstract, “We used 2014-2016 prescription and procedure Medicare data to train 3 sets of random forest classifiers (prescription only, procedure only, and combined) to predict specialty…Physicians were assigned to testing and training cohorts, and random forest models were trained and then applied to 2014-2016 data sets for the testing cohort to generate a series of specialty predictions.”)
test a performance of each of the models based on the test data (Wingrove, Validating the Algorithm Section, “To assess consistency, we applied each of the 3 random forest models to each of the 3 years of Test data, giving 9 sets of predictions based on the physicians in the Test group. The 9 sets of predictions were compared with self-reported specialty to generate an F1 score (harmonic mean of precision [positive predictive value] and recall [sensitivity]) for each specialty, and a macro F1 score, calculated on the average precision and recall of all specialties”).
Wingrove and Burnett do not teach, but Wubbels does teach
select the model having the best performance among the plurality of models (Wubbels, paragraph 0048, “In the end, the training module 410 can select only the substantially optimal machine learning models, namely those with the highest accuracy and lowest latency”)
Wingrove, Burnett, and Wubbels are considered analogous to the claimed invention because they deal with medical data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to select the machine learning model with the highest accuracy and lowest latency from Wubbels. One of the ordinary skill in the art would have known to apply the known technique of choosing the most optimal model to use. Therefore, applying Wubbels’ technique would yield the predicable result of using the machine model with the best performance (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett and Breiman et al. (Random Forests) (hereafter referred to as Breiman).
Regarding claim 7, Wingrove and Brunett teach the method of claim 1, Wingrove and Brunett do not teach, but Breiman does teach
training the model to determine a confidence level of the predicted taxonomy (Breiman, Section 2.1, “Given an ensemble of classifiers h1(x),h2(x),...,hK(x), and with the training set drawn at random from the distribution of the random vector Y, X,define the margin function as mg(X,Y) = avkI(hk(X) = Y)−maxj̸=Y avkI(hk(X) = j). where I(·) is the indicator function. The margin measures the extent to which the average number of votes at X, Y for the right class exceeds the average vote for any other class. The larger the margin, the more confidence in the classification. The generalization error is given by PE∗ = PX,Y(mg(X,Y)<0) where the subscripts X, Y indicate that the probability is over the X, Y space)”. Examiner notes that Wingrove references Breiman and that it uses random forest, therefore Breiman’s confidence apply to Wingrove’s random forest).
Wingrove, Brunett, and Breiman are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use the random forest from Breiman. One of the ordinary skill in the art would have known to apply the known technique of determining a confidence level in classification. Therefore, applying Breiman’s technique would yield the predictable result of determining the accuracy and reliability of a model (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett and Chennupati et al. (Adaptive Hierarchical Decomposition of Large Deep Networks) (hereafter referred to as Chennupati).
Regarding claim 8, Wingrove and Brunett teach the method of claim 1, Wingrove and Brunett do not teach, but Chennupati does teach
training a first stage of the model to predict a specialty associated with a physician based on the training data (Chennupati, Abstract, “This paper introduces a framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network. Class similarities guide the creation of a family from course to fine classifiers which solve categorical problems more effectively than a single large classifier” and “The hierarchical model consists of a hierarchical classifier which makes coarse category predictions and a class assignment classifier that predicts the final class category. In a simple hierarchical model, a sample with its coarse category label is sent to a hierarchical classifier which learns the coarse category representations present in the dataset” (Chennupati, Section 4.1.1) Examiner notes that the coarse category maps to the specialty).
training a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data (Chennupati, Section 4.1.1, “Later, the sample is sent to one of the class assignment classifiers based on the fine category label”).
Wingrove, Burnett, and Chennupati are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use the deep networks structure from Chennupati. Chennupati teaches “framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network. Class similarities guide the creation of a family from course to fine classifiers which solve categorical problems more effectively than a single large classifier. The resulting smaller networks are highly scalable, parallel and more practical to train, and achieve higher classification accuracy”(Chennupati, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 18, Wingrove and Brunett teach the apparatus of claim 15, Wingrove and Brunett do not teach, but Chennupati does teach
train a first stage of the model to predict a specialty associated with a physician based on the training data (Chennupati, Abstract, “This paper introduces a framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network. Class similarities guide the creation of a family from course to fine classifiers which solve categorical problems more effectively than a single large classifier” and “The hierarchical model consists of a hierarchical classifier which makes coarse category predictions and a class assignment classifier that predicts the final class category. In a simple hierarchical model, a sample with its coarse category label is sent to a hierarchical classifier which learns the coarse category representations present in the dataset” (Chennupati, Section 4.1.1) Examiner notes that the coarse category maps to the specialty).
train a second stage of the model to predict a taxonomy within the specialty associated with the physician based on the training data (Chennupati, Section 4.1.1, “Later, the sample is sent to one of the class assignment classifiers based on the fine category label”).
Wingrove, Burnett, and Chennupati are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use the deep networks structure from Chennupati. Chennupati teaches “framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network. Class similarities guide the creation of a family from course to fine classifiers which solve categorical problems more effectively than a single large classifier. The resulting smaller networks are highly scalable, parallel and more practical to train, and achieve higher classification accuracy”(Chennupati, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett, Chung and Chollet et al. (Building Powerful Image Classification Models Using Very Little Data) (hereafter referred to as Chollet).
Regarding claim 9, Wingrove and Burnett teach the method of claim 1, Wingrove and Burnett do not teach, but Chung does teach
training a stacked autoencoder using unlabeled data (Chung, paragraph 0027, “A conventional auto-encoder generally maps input x ... to a hidden representation h ... and then maps h back to z ... The goal is to learn a set of hidden variables h for reconstructing the input x with minimized reconstruction error”).
adding a plurality of fully connected layers to the stacked autoencoder (Chung, paragraph 0029, “Once the auto-encoder is trained, the decoder layer (W′) can be removed, and the encoded layer (W) is used as input for stacking the next auto-encoder. By stacking K auto-encoders, a deep neural network with K layers is pre-trained. In other words, the K-th layer may be trained based on the previously trained K-1 layers because the latent representation from the previous layer can be computed” and “Fine-tuning may be accomplished, for example, by replacing the last decoder layer with an extra softmax regression layer (e.g., the s(x) sigmoid function) at the end of the deep neural network” (Chung, paragraph 0044)).
Wingrove, Burnett, and Chung are considered analogous to the claimed invention because they deal with machine learning models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use a deep neural network from Chung. Chung teaches “a pre-trained cost-sensitive auto-encoder can be used in combination with a training (fine-tuning) stage for cost-sensitive deep learning (e.g., a neural network classifier with multiple hidden layers, as will be explained below). Thus, cost information is effectively combined with deep learning by modifying the objective function in the pre-training phase. By minimizing the modified objective function, the auto-encoder not only tries to capture the underlying pattern in the data, it further “learns” the cost information and “stores” it in the structure. By later fine-tuning at the training stage, the classification system yields improved performance (lower cost) than a typical classification system that does not take cost information into account during pre-training” (Chung, paragraph 012) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Chung does not teach, but Chollet does teach
after training the stacked autoencoder, training the fully connected layers based on the training data (Chollet, Fine-tuning the top layer of a pre-trained network Section, “in order to perform finetuning, all layers should start with properly trained weights: for instance you should not slap a randomly initialized fully connected network on top of a pretrained convolutional base. This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights”).
after training the fully connected layers, training the model comprising the stacked autoencoder and the fully connected layers based on the training data (Chollet, Fine-tuning the top layer of a pre-trained network Section, “we can try to "finetune" the last convolutional block of the VGG16 model alongside the toplevel classifier. Finetuning consist in starting from a trained network, then retraining it on a new dataset using very small weight updates”).
Wingrove, Burnett, Chung, and Chollet are considered analogous to the claimed invention because they deal with machine learning models. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove, Burnett, and Chung to include the auto-encoder from Chollet. Chollet teaches that “in order to perform fine-tuning, all layers should start with properly trained weights: for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base. This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base” (Chollet, Fine-tuning the top layer of a pre-trained network Section) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Claim(s) 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett, and Damerau et al. (US 6697998 B1) (hereafter referred to as Damerau).
Regarding claim 10, Wingrove and Burnett teach the method of claim 1, Wingrove and Burnett do not teach, but Damerau does teach
using unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples (Damerau, Abstract, “Members of the answer set are converted to vectors representing centroids of unknown groups of unlabeled text data. Unlabeled text data are clustered relative to the centroids by a nearest neighbor algorithm”. Examiner notes that the nearest neighbor clustering algorithm is an unsupervised learning technique that calculates similarity).
determining ground truth labels for one or more of the unlabeled training examples based on the similarity to generate supplemental labeled training examples (Damerau, Claim 1, “assigning an ID to each said centroid; and labeling each of the unlabeled text data documents with said ID of the answer in the cluster to which the unlabeled text data document has been assigned by said clustering step”. Examiner notes that the centroid ID maps to the ground truth labels)
combining the labeled training examples and the supplemental labeled training examples to generate expanded labeled training examples (Damerau, Claim 2, “training a supervised machine learning algorithm on the newly labeled data; and outputting a classifier for assigning labels to new text data”)
Wingrove, Burnett, and Damerau are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to apply Damerau’s technique of labeling unlabeled data through unsupervised learning and using the labeled data. One of the ordinary skill in the art would have known to apply the known technique of turning unlabeled data to labeled data to use for supervised learning. Therefore, applying Damerau’s technique would yield the predicable result of providing clear predictions to test and train supervised models (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Damerau does not teach, but Burnett does teach
segregating the expanded labeled training examples into training data, validation data, and test data(Burnett, paragraph 0038, “Turning to the training risk model training process, as described above, the historical patient data is divided, by the data selecting module 216, into at least three subsets of model data 244. The at least three subsets of model data 244, training data 302; validation data 304; and test data 306, are used by the machine-learning training module 220 to generate the one or more risk models”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of dividing a dataset. One of the ordinary skill in the art would have known to apply the known technique of splitting a dataset. Therefore, applying Burnett’s technique would yield the predicable result of preventing overfitting and accurately measure the performance of the machine learning model (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 19, Wingrove and Burnett teach the apparatus of claim 15, Wingrove and Burnett do not teach, but Damerau does teach
use unsupervised learning techniques to determine a similarity between unlabeled training examples and the labeled training examples based on the features of the labeled training examples and the features of the unlabeled training examples (Damerau, Abstract, “Members of the answer set are converted to vectors representing centroids of unknown groups of unlabeled text data. Unlabeled text data are clustered relative to the centroids by a nearest neighbor algorithm”. Examiner notes that the nearest neighbor clustering algorithm is an unsupervised learning technique).
determine ground truth labels for one or more of the unlabeled training examples based on the similarity to generate supplemental labeled training examples (Damerau, Claim 1, “assigning an ID to each said centroid; and labeling each of the unlabeled text data documents with said ID of the answer in the cluster to which the unlabeled text data document has been assigned by said clustering step”. Examiner notes that the centroid ID maps to the ground truth labels)
combine the labeled training examples and the supplemental labeled training examples to generate expanded labeled training examples (Damerau, Claim 2, “training a supervised machine learning algorithm on the newly labeled data; and outputting a classifier for assigning labels to new text data”)
Wingrove, Burnett, and Damerau are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to apply Damerau’s technique of labeling unlabeled data through unsupervised learning and using the labeled data. One of the ordinary skill in the art would have known to apply the known technique of turning unlabeled data to labeled data to use for supervised learning. Therefore, applying Damerau’s technique would yield the predicable result of providing clear predictions to test and train supervised models (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Damerau does not teach, but Burnett does teach
segregate the expanded labeled training examples into training data, validation data, and test data(Burnett, paragraph 0038, “Turning to the training risk model training process, as described above, the historical patient data is divided, by the data selecting module 216, into at least three subsets of model data 244. The at least three subsets of model data 244, training data 302; validation data 304; and test data 306, are used by the machine-learning training module 220 to generate the one or more risk models”).
Wingrove and Burnett are considered analogous to the claimed invention because they deal with physician data. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove to apply Burnett’s technique of dividing a dataset. One of the ordinary skill in the art would have known to apply the known technique of splitting a dataset. Therefore, applying Burnett’s technique would yield the predicable result of preventing overfitting and accurately measure the performance of the machine learning model (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 11 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wingrove in view of Burnett, and Lundberg et al. (A Unified Approach to Interpreting Model Predictions) (hereafter referred to as Lundberg).
Regarding claim 11, Wingrove and Burnett teach the method of claim 1, Wingrove and Burnett do not teach, but Lundberg does teach
determining a relative amount that one or more of the features contribute to one or more taxonomies output by the model using Shapley Additive Explanations (Lundberg, Abstract, “SHAP [Shapely Additive Explanations] assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties”).
Wingrove, Burnett, and Lundberg are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use SHAP from Lundberg. Lundberg teaches “The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches” (Lundberg, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Regarding claim 14, Wingrove and Burnett teach the method of claim 12, Wingrove and Burnett do not teach, but Lundberg does teach
determining a relative amount that one or more of the target features contribute to the taxonomy assigned to the target physician using Shapley Additive Explanations (Lundberg, Simple Properties Uniquely Determine Additive Feature Attributions Section, “When approximating the original model f for a specific input x, local accuracy requires the explanation model to at least match the output of f for the simplified input x (which corresponds to the original input x) ... The explanation model g(x) matches the original model f(x) when x = hx(x), where φ0 =f(hx(0)) represents the model output with all simplified inputs toggled off (i.e. missing)”. Examiner notes that the specific input x maps to the target physician).
Wingrove, Burnett, and Lundberg are considered analogous to the claimed invention because they deal with classification. It would have been obvious to one having ordinary skill in the art prior to the effective filling date to have modified Wingrove and Burnett to use SHAP from Lundberg. Lundberg teaches “The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches” (Lundberg, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Grifno et al. (US 20170185723 A1) discloses analyzing updated set of provider data, the updated set of claims data, and updated set of absence data and generate, assign, and output an updated ranking number for each of healthcare providers based on the redefined set of rules. Powell et al. (US 10887188 B2) discloses extracting features to predict malpractice risk of physicians. Baker et al. (US 20060161456 A1) discloses presenting information to patients that yield easily understood, yet statistically valid, rankings, in multiple scoring domains, in response to those patients' desire to find or compare medical providers associated with their medical condition or treatment needs, as well as other criteria
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN VO whose telephone number is (571)272-9622. The examiner can normally be reached Monday - Friday from 7-3 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.V./ Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights