DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on October 3, 2025 has been entered.
Response to Amendment
In the amendment filed on October 3, 2025, the following has occurred: claim(s) 1, 13, 16-22 have been amended, claim(s) 24 have been added, and claim(s) 4 have been cancelled. Now, claim(s) 1-3, 5, 7-24 are pending.
Claim Objections
Claim 18 objected to because of the following informalities: “…that the the difference…”. This appears to be a typographical error. Appropriate correction is required. For examination purposes, the Examiner will interpret the claimed portion as “…that the difference…”.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-3, 5, 7-24 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-3, 5, 7-15, 23-24: Step 2A Prong One
Claim 1 recite(s):
process a physical in vitro biological sample, and
determine an analytical testing result based on processing of the physical in vitro biological sample; and
obtain analytical testing data comprising:
the analytical testing result determined by the one or more analyzer instruments; and
metadata associated with the analytical testing result;
validate the analytical testing result, wherein:
the training analytical testing data comprises at least one training analytical
testing result and training metadata, and
generates a validation determination indicating that the analytical testing result, is valid or invalid;
determine a difference level between a live data set and the first training data set,
wherein:
the live data set comprises the analytical testing data, and
the difference level is determined based on a comparison of distribution
characteristics of the live data set and the first training data set
These limitations, as drafted, given the broadest reasonable interpretation, but for the recitation of generic computer components, encompass managing personal behavior or relationships or interactions between people (including social activities, teaching, and following
rules or instructions) which is a subgrouping of Certain Methods of Organizing Human Activity. That is, other than reciting, “one or more analyzer instruments configured for…”, “a monitoring system configured to:”, “…, determined by the one or more analyzer instruments…”, “the validation algorithm comprises a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data,”, “the re-trained algorithm” to perform these functions, nothing in the claim precludes the limitations from practically being performed by a person following rules or instructions to analyze a physical in vitro biological sample. For example, the claim encompasses a user following instructions to process a physical in vitro biological sample to determine an analytical testing result, a user following instructions to obtain an analytical testing result, a user following instructions to analyze analytical testing data, a user following instructions to validate an analytical testing result by generating a validation determination, a user following instructions to determine a difference level between a live data set and the first training data set, and a user following instructions to validate the analytical testing result. These steps could be carried out by a user following rules or instructions.
Claims 2-3, 5, 7-15, 23-24 incorporate the abstract idea identified above and recite additional limitations that expand on the abstract idea. For example, claim 2 describes a second difference level. Similarly, claim 3 further describes the second training data set. Similarly, claim 5 describes notifying another user of a possible error associated with the analyzing testing process. Similarly, claims 7-8, 11-12 further describe analysis of the data sets. Similarly, claims 9-10 further describe the metadata. Similarly, claims 13-15, 23 describe performance of validation. Finally, claim 24 further describes the one or more analyzer instruments. Such steps, but for generic computer components, encompass a Certain Method of Organizing Human Activity.
Claims 1-3, 5, 7-15, 23-24: Step 2A Prong Two
This judicial exception is not integrated into a practical application because the remaining
elements amount to no more than general purpose computer components programmed to perform
the abstract idea.
Claims 1-5, 7-15, 23-24, directly or indirectly, recite the following generic computer components configured to implement the abstract idea: “one or more analyzer instruments configured for…”, “a monitoring system configured to:”, “the validation algorithm comprises a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data,”, “the re-trained algorithm” (e.g., “According to some embodiments, at least one of the one or more analyzer instruments (10) is a biological sample analyzer designed for processing biological samples and providing an analytical testing result associated with the biological sample.” in Specification Paragraph [0012], “According to some embodiments, the validation algorithm is a quality control algorithm and/or is comprised in a quality control algorithm. The quality control algorithm can e.g. be designed for detecting errors, e.g. systematical errors, based on the validation presumptions for the analytical testing results made by the validation algorithm.” in Specification Paragraph [0050], “The above described process and processing, for example, the process 200, can also be performed by the processing unit 701. For example, in some embodiments, the process 200 may be implemented as a computer software program being tangibly included in the machine-readable medium, for example, the storage unit 708. In some embodiments, the computer program may be partially or fully loaded and/or mounted to the device 700 via the ROM 702 and/or communication unit 709. When the computer program is loaded to the RAM 703 and executed by the CPU 701, one or more steps of the above described methods or processes can be implemented.” in Specification in Paragraph [00179]) As set forth in the 2019 Eligibility Guidance, 84 Fed. Reg. at 55 “merely including instructions to implement an abstract idea on a computer” is an example of when an abstract idea has not been integrated into a practical application.
Additionally, the claims recite “…using a validation algorithm,…”, “…, determined by the one or more analyzer instruments…”, “re-train the validation algorithm using a second training data set if the difference level between the live data set and the first training data set is greater than a first threshold”, “use the re-trained validation algorithm for validation of the analytical testing result or a second analytical testing result” at a high degree of generality, amount no more than generally linking the abstract idea to a particular technical environment. The recitation is also similar to adding the words “apply it” to the abstract idea. As set forth in MPEP 2106.05(f), merely reciting the words “apply it” or an equivalent, is an example of when an abstract idea has not been integrated into a practical application.
Claims 1-3, 5, 7-15, 23-24: Step 2B
The claim (s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of
the abstract idea into a practical application, the additional elements of using a computer
configured to perform above identified functions amounts to no more than mere instructions to
apply the exception using generic computer components. Mere instructions to apply an exception
using a generic computer component cannot provide an inventive concept. See Alice 573 U.S. at
223 (“mere recitation of a generic computer cannot transform a patent-ineligible abstract idea
into a patent-eligible invention.”)
Additionally, generally linking the abstract idea to a particular technological environment does not amount to significantly more than the abstract idea (See MPEP 2016.05(h) and Affinity Labs of Texas v. DirectTV, LLC, 838 F.3d 1253, 120 USP12d 1201 (Fed. Cir. 2016)). The claims are not patent eligible.
Claims 19-20 recite the same functions as claim 1, but in system and method form.
Therefore, whether considered alone or in combination, the additional elements do not
amount to significantly more than the abstract idea.
Claim 16: Step 2A Prong One
Claim 16 recite(s):
receiving a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples, wherein the analytical testing data indicates:
an analytical testing result of the one or more physical in vitro biological samples, and
metadata associated with the analytical testing result;
validating the analytical testing result of the live data set by generating, a validation determination indicating whether the analytical testing result is valid or invalid, wherein:
the training analytical testing data comprises a training analytical testing result and training metadata;
determining a difference level between the live data set and the first training data set, based on comparison of distribution characteristics of the live data set and the first training data set;
determining that the difference level is greater than a threshold.
These limitations, as drafted, given the broadest reasonable interpretation, but for the recitation of generic computer components, encompass concepts, encompass managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) which is a subgrouping of Certain Methods of Organizing Human Activity. That is, other than reciting, “…by one or more analyzer instruments…”, “a validation algorithm”, “…determined by the one or more analyzer instruments based on…” to perform these functions, nothing in the claim precludes the limitations from practically being performed by a person following rules or instructions to analyze a physical in vitro biological sample. For example, the claim encompasses a user following instructions to receive a live data set, a user following instructions to obtain an analytical testing result, a user following instructions to validate the analytical testing result of the live data set, a user following instructions to determine a difference level between the live data set of and the first training data set, and a user following instructions to making an evaluation that the difference level is greater than a threshold. These steps could be carried out by a user following rules or instructions.
Claim 16: Step 2A Prong Two
This judicial exception is not integrated into a practical application because the remaining elements amount to no more than general purpose computer components programmed to perform the abstract idea.
Claim 16, directly or indirectly, recite the following generic computer components configured to implement the abstract idea: “…by one or more analyzer instruments…”, “a validation algorithm” (e.g., “According to some embodiments, the validation algorithm is a quality control algorithm and/or is comprised in a quality control algorithm. The quality control algorithm can e.g. be designed for detecting errors, e.g. systematical errors, based on the validation presumptions for the analytical testing results made by the validation algorithm.” in Specification Paragraph [0050]) As set forth in the 2019 Eligibility Guidance, 84 Fed. Reg. at 55 “merely including instructions to implement an abstract idea on a computer” is an example of when an abstract idea has not been integrated into a practical application.
Additionally, the claims recite “…determined by the one or more analyzer instruments based on…”, “…using a validation algorithm, the validation algorithm comprising a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data,…”, “re-training the validation algorithm using a second training data set based on determining that the difference level between the live data set and the first training set is greater than the threshold” at a high degree of generality, amount no more than generally linking the abstract idea to a particular technical environment. The recitation is also similar to adding the words “apply it” to the abstract idea. As set forth in MPEP 2106.05(f), merely reciting the words “apply it” or an equivalent, is an example of when an abstract idea has not been integrated into a practical application.
Claim 16: Step 2B
The claim(s) does/do not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to integration of
the abstract idea into a practical application, the additional elements of using a computer
configured to perform above identified functions amounts to no more than mere instructions to
apply the exception using generic computer components. Mere instructions to apply an exception
using a generic computer component cannot provide an inventive concept. See Alice 573 U.S. at
223 (“mere recitation of a generic computer cannot transform a patent-ineligible abstract idea
into a patent-eligible invention.”)
Additionally, generally linking the abstract idea to a particular technological environment does not amount to significantly more than the abstract idea (See MPEP 2016.05(h) and Affinity Labs of Texas v. DirectTV, LLC, 838 F.3d 1253, 120 USP12d 1201 (Fed. Cir. 2016)). The claims are not patent eligible.
Claims 17-18, 21-22 recite the same functions as claim 16, but in method, system, system, and computer-readable medium form.
The addition of “determining, by testing physical in vitro biological samples with one or more analyzer instruments, a plurality of analytical testing results associated with the testing of the physical in vitro biological samples; providing a live data set comprising: the plurality of analytical testing results, and metadata associated with the plurality of analytical testing results;” in claim 17 given the broadest reasonable interpretation, but for the recitation of generic computer components, recite a user following instructions to determine a plurality of analytical testing results, a user following instructions to provide a plurality of analytical testing data. These steps could be carried out by a user following rules or instructions.
The addition of “one or more analyzer instruments programmed and configured to determine analytical testing results associated with testing of one or more physical in vitro biological samples;” in claim 18 given the broadest reasonable interpretation, but for the recitation of generic computer components, recite a user following instructions to determine analytical testing results. This step could be carried out by a user following rules or instructions.
The addition of “a processing unit; and a memory coupled to the processing unit and having instructions stored thereon that, when executed by the processing unit, cause an electronic device…” in claim 21 is recited at a high level of generality that amount to no more than generic computer components.
Therefore, whether considered alone or in combination, the additional elements do not
amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5, 7-11, 13-15, 19-20, 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Venkatraman et al. (U.S. Patent Pre-Grant Publication No. 2022/0054008) in view of Edgar (U.S. Patent Pre-Grant Publication No. 2021/0201190) in view of Hughes (U.S. Patent Pre-Grant Publication No. 2019/0156241) in further view of Von Hoff et al. (U.S. Patent Pre-Grant Publication No. 2010/0304989).
As per independent claim 1, Venkatraman discloses a diagnostic analyzing system comprising:
a monitoring system configured to: validate the analytical testing result using a validation algorithm (See Paragraphs [0053]-[0055]: The training set (e.g., type of data and size of the training set) may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, and patient biological data that is recorded and transmitted to the computing system may be processed on its own, and/or along with patient data previously transmitted by the patient, or it may be processed (e.g., compared) with control samples and/or patient data transmitted by other patients, which the Examiner is interpreting the validation to encompass validate the analytical testing result, and interpreting algorithm to encompass a validation algorithm), wherein:
the validation algorithm comprises a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data (See Paragraphs [0052]-[0055]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain expert to encompass a first training data set comprising training analytical testing data),
the training analytical testing data comprises at least one training analytical testing result and training metadata (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain experts to encompass training analytical testing result and the biological sensor data may be tagged along with additional data such as patient metadata ([0097]) to encompass training metadata), and the validation algorithm generates a validation determination indicating that the analytical testing result, determined by the one or more analyzer instruments, is valid or invalid (See Paragraph [0053]: The training set may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, which the Examiner is interpreting output having a predetermined specificity to encompass a validation determination indicating that the analytical testing result is valid or invalid as the specificity could be validity when combined with Von Hoff);
determine a difference level between a live data set and the first training data set (See Paragraphs [0052]-[0055], [0146]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, which the Examiner is interpreting ongoing live stream data to encompass a live data set, and the trained algorithm may compare recorded data to at least pre-existing data to encompass determine a difference level), wherein:
the live data set comprises the analytical testing data (See Paragraphs [0131]-[0133], [0138]: The data streaming engine may have access to one or more memories that may be used for temporarily storing live stream patient data (e.g., a memory buffer register on a hard drive) during the process of streaming patient data to a receiving device, and the trained algorithm may be a neural network or similar artificial intelligence (AI) system that takes biological sensor data as input and processes it in relation to training data in order to output a result, which the Examiner is interpreting patient data to encompass the analytical testing data), and
the difference level is determined based on a comparison of distribution characteristics of the live data set and the first training data set (See Paragraphs [0047], [0050]-[0054]: Analytics engine may compare ECG data generated by ECG sensors at consultations held every month for a period of a year, in order to determine whether an arrhythmia is stable or getting worse, or whether a patient's heart disease is evolving in response to a treatment, and the trained AI algorithm may compare and identify a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data and/or audio data to a recorded ECG or audio data, which the Examiner is interpreting comparing a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data to encompass the difference level being determined based on a comparison of distribution characteristics of the live data set and the first training data set.)
While Venkatraman teaches the system as described above, Venkatraman may not explicitly teach a monitoring system configured to: re-train the validation algorithm using a second training data set if the difference level, between the live data set and the first training data set, is greater than a first threshold; and use the re-trained validation algorithm for validation at least one of the analytical testing result or a second analytical testing result.
Edgar teaches a system a monitoring system configured to:
re-train the validation algorithm using a second training data set if the difference level, between the live data set and the first training data, set is greater than a first threshold (See Paragraph [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the system updates the machine learning model based on re-training the machine learning model using the additional data samples to encompass re-train the validation algorithm using a second training data set if the difference level between the live data set and the first training data set is greater than a first threshold when combined with the data of Venkatraman); and
use the re-trained validation algorithm for validation at least one of the analytical testing result or a second analytical testing result (See Paragraphs [0089]-[0090]: The machine learning model development and optimization ensures performance validation and data sufficiency for regulatory approval, which the Examiner is interpreting to encompass validation of an analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include re-train the validation algorithm using a second training data set if the difference level, between the live data set and the first training data set, is greater than a first threshold; and use the re-trained validation algorithm for validation at least one of the analytical testing result or a second analytical testing result as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
While Venkatraman/Edgar teaches the system as described above, Edgar/Venkatraman may not explicitly teach a monitoring system configured to: obtain analytical testing data comprising: the analytical testing result determined by the one or more analyzer instruments; and metadata associated with the analytical testing result.
Hughes teaches a system for a monitoring system configured to:
obtain analytical testing data (See Paragraphs [0046], [0061]: The analysis service results might be in an arbitrary format defined by that analysis service vendor, as such the analysis integration module may convert that result from a non-standard format to one of a more standardized set of formats, such as, in the case of medical imaging data to DICOM presentation state, which the Examiner is interpreting medical imaging equipment to encompass one or more analyzer instruments) comprising:
the analytical testing result determined by the one or more analyzer instruments (See Paragraphs [0061]-[0062]: The acquired data may be provided to the data integration module, which persistently saves the acquired data to the data sources, and the data may be retrieved for processing and checks of metadata to determine if the data can be analyzed, which the Examiner is interpreting the data may be retrieved for processing and checks of metadata to determine if the data can be analyzed to encompass the analytical testing result determined by the one or more analyzer instruments); and
metadata associated with the analytical testing result (See Paragraphs [0061]-[0062]: The acquired data may be provided to the data integration module, which persistently saves the acquired data to the data sources, and the data may be retrieved for processing and checks of metadata to determine if the data can be analyzed, which the Examiner is interpreting the data may be retrieved for processing and checks of metadata to determine if the data can be analyzed to encompass metadata associated with the analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman/Edgar to include a monitoring system configured to: obtain analytical testing data comprising: the analytical testing result determined by the one or more analyzer instruments; and metadata associated with the analytical testing result as taught by Hughes. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar with Hughes with the motivation of improving time and cost efficiency (See Background of Hughes in Paragraph [0003]).
While Venkatraman/Edgar/Hughes teaches the system as described above, Edgar/Venkatraman/Hughes may not explicitly teach one or more analyzer instruments configured to: process a physical in vitro biological sample, and determine an analytical testing result based on the processing of the physical in vitro biological sample.
Von Hoff teaches a system for one or more analyzer instruments (See Paragraph [0311]: RNA was specifically bound and then eluted. The RNA was tested for integrity by assessing the ratio of 28S to 18S ribosomal RNA on an Agilent BioAnalyzer, which the Examiner is interpreting an Agilent BioAnalyzer to encompass one or more analyzer instruments) configured to:
process a physical in vitro biological sample (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting a biological sample to encompass a physical in vitro biological sample ([0082]: Such samples include blood and blood fractions or products (e.g., serum, buffy coat, plasma, platelets, red blood cells, and the like), sputum, cheek cells tissue, cultured cells (e.g., primary cultures, explants, and transformed cells), stool, urine, other biological or bodily fluids (e.g., prostatic fluid, gastric fluid, intestinal fluid, renal fluid, lung fluid, cerebrospinal fluid, and the like), etc.)), and
determine an analytical testing result based on the processing of the physical in vitro biological sample (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting molecular profiling to encompass an analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman/Edgar/Hughes to include one or more analyzer instruments configured to: process a physical in vitro biological sample, and determine an analytical testing result based on the processing of the physical in vitro biological sample as taught by Von Hoff. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar/Hughes with Von Hoff with the motivation of improved patient care and enhanced treatment outcomes (See Background of Von Hoff in Paragraph [0006]).
Claim 19 mirrors claim 1 but for reciting the system as “A monitoring system for diagnostic analytical testing” instead of “A diagnostic analyzing system”, and is rejected for the same reason as claim 1.
Claim 20 mirrors claim 1 only within a different statutory category and the addition of “determining whether to re-train the validation algorithm based on a second comparison of the difference level against a threshold” is encompassed by the combination of Venkatraman/Edgar/Hughes/Von Hoff in Edgar in Paragraphs [0097]-[0098] as “Such retraining may be performed as described above using additional data sets.” and “The revised set of parameters at the second point in time may include an updated set of weights and bias values where one or more of these values may have been updated in comparison to those of the first set.”, which the Examiner is interpreting an updated set of weights to encompass a second comparison of the difference level against a threshold, and is rejected for the same reason as claim 1.
As per claim 2, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein a second difference level between the live data set and the second training data set is lower than a second threshold.
Edgar teaches a system wherein a second difference level between the live data set and the second training data set is lower than a second threshold (See Paragraphs [0085]-[0086]: The system determines subgroup performance measures (lower confidence interval values) for subgroups of the data samples respectively associated with different metadata factors, which the Examiner is interpreting the lower confidence interval values to encompass the second training data set is lower than a second threshold when combined with Venkatraman’s disclosure of determination of difference levels ([0052]-[0055], [0146]).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include a second difference level between the live data set and the second training data set is lower than a second threshold as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 3, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman further teaches wherein the second training data set comprises the live data set (See Paragraph [0052]: A training data set for supervised learning problem may be biological sensor data reviewed and labeled by domain experts, which the Examiner is interpreting the biological sensor data to encompass the live data set.)
As per claim 5, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is configured to present output indicative of a possible error associated with an analyzing testing process based on the validation determination indicating that the analytical testing result is invalid (See Paragraph [0065]: MAAP thresholds for regulatory approval of SaDMs can be based on one or more risk/mitigation valuations that measure one or more of the following: algorithmic risk, contribution to care, risk timeliness, and ability to detect and mitigate (e.g., respond to system error), which the Examiner is interpreting algorithmic risk to encompass a possible error associated with the analyzing testing process based on the analysis when combined with Venkatraman’s teachings of “output having a predetermined specificity” ([0053]).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the monitoring system is configured to present output indicative of a possible error associated with an analyzing testing process based on the validation determination indicating that the analytical testing result is invalid as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 7, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman further teaches wherein the distribution characteristics are determined based on the at least one training analytical testing results indicated by the first training data set and the analytical testing result indicated by the live data set. (See Paragraphs [0050]-[0055], [0146]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, which the Examiner is interpreting ongoing live stream data to encompass a live data set, and interpreting comparing a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data to encompass the distribution characteristics.)
As per claim 8, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman further teaches wherein the distribution characteristics are determined based on the training metadata indicated by the first training data set and the metadata associated with analytical testing result indicated by the live data set (See Paragraphs [0050]-[0055], [0096], [0138]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, and the biological sensor data may be tagged along with additional data such as patient metadata, which the Examiner is interpreting patient metadata used as patient data as input for the trained algorithm to encompass the distribution characteristics are determined based on the training metadata.)
As per claim 9, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the metadata indicates at least one of an age of a patient associated with the analytical testing result, a gender of the patient, a type of sourcing of the patient, a ward of the patient, or a health diagnosis of the patient.
Edgar teaches a system wherein the metadata indicates at least one of:
an age of a patient associated with the analytical testing result,
a gender of the patient (See Paragraphs [0039]-[0040]: Patient-related factors can include demographic factors such as gender, age, diseases and state, implants, medical history factors, etc.),
a type of sourcing of the patient,
a ward of the patient, or
a health diagnosis of the patient.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the metadata indicates at least one of an age of a patient associated with the analytical testing result, a gender of the patient as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 10, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is programmed and configured to: determine a first characteristic value based on the metadata associated with the live data set; determine a second characteristic value based on the training metadata associated with the first training data set; and determine the difference level using the first characteristic value and the second characteristic value.
Edgar teaches a system wherein the monitoring system is programmed and configured to:
determine a first characteristic value based on the metadata associated with the live data set (See Paragraphs [0056]-[0057]:The criteria used to define the subgroups can be predefined based on one or more measurable factors (e.g., demographics, diagnosis codes, etc.) that may be a source of a variance in the performance of the model, which the Examiner is interpreting a measurable factor to encompass a first characteristic value based on the metadata associated with the live data set when combined with the live stream data of Venkatraman);
determine a second characteristic value based on the training metadata associated with the first training data set (See Paragraphs [0056]-[0057]: The criteria used to define the subgroups can be predefined based on one or more measurable factors (e.g., demographics, diagnosis codes, etc.) that may be a source of a variance in the performance of the model, which the Examiner is interpreting one or more measurable factor to encompass a second characteristic value based on the training metadata associated with the first training data set when combined with the training data of Venkatraman); and
determine the difference level using the first characteristic value and the second characteristic value (See Paragraphs [0056]-[0057], [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting to encompass the claimed portion.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include determine a first characteristic value based on the metadata associated with the live data set; determine a second characteristic value based on the training metadata associated with the first training data set; and determine the difference level using the first characteristic value and the second characteristic value as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 11, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is programmed and configured to: determine a first association between a first feature of the live data set and a first set of ground-truth labels associated with the live data set, the first set of ground-truth labels being indicative of first validity values respectively corresponding to analytical testing data instances included in the live data set; determine a second association between a second feature of the first training data set and a second set of ground-truth labels associated with the first training data set, the second set of ground-truth labels being indicative of second validity values respectively corresponding to training analytical testing data instances included in the first training data set; and determine the difference level using the first association and the second association.
Edgar teaches a system wherein the monitoring system is programmed and configured to:
determine a first association between a first feature of the live data set and a first set of ground-truth labels associated with the live data set, the first set of ground-truth labels being indicative of first validity values respectively corresponding to analytical testing data instances included in the live data set (See Paragraphs [0057]-[0059]: The grouping component can identify potential relevant grouping factors for the subgroups in metadata associated with the validation data samples, the grouping component can employ automated clustering/grouping algorithms to facilitate identifying and generating the relevant subgroups based on identified distinguishing features associated with different subgroups of the data samples, which the Examiner is interpreting to encompass the claimed portion when combined with the live stream data of Venkatraman);
determine a second association between a second feature of the first training data set and a second set of ground-truth labels associated with the first training data set, the second set of ground-truth labels being indicative of second validity values respectively corresponding to training analytical testing data instances included in the first training data set (See Paragraphs [0057]-[0059]: The grouping component can identify potential relevant grouping factors for the subgroups in metadata associated with the validation data samples, the grouping component can employ automated clustering/grouping algorithms to facilitate identifying and generating the relevant subgroups based on identified distinguishing features associated with different subgroups of the data samples, which the Examiner is interpreting to encompass the claimed portion); and
determine the difference level using the first association and the second association (See Paragraphs [0057]-[0059], [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting to encompass the claimed portion.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the monitoring system is programmed and configured to: determine a first association between a first feature of the live data set and a first set of ground-truth labels associated with the live data set, the first set of ground-truth labels being indicative of first validity values respectively corresponding to analytical testing data instances included in the live data set; determine a second association between a second feature of the first training data set and a second set of ground-truth labels associated with the first training data set, the second set of ground-truth labels being indicative of second validity values respectively corresponding to training analytical testing data instances included in the first training data set; and determine the difference level using the first association and the second association as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 13, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is programmed and configured to: determine a first performance associated with a first version of the validation algorithm; determine a second performance associated with the re-trained validation algorithm by processing a testing data set with the re-trained validation algorithm, wherein the testing data set comprises a plurality of analytical testing data; and in response to determining that the second performance is better than the first performance, using the re-trained validation algorithm for validation of the at least one of the analytical testing result or the second analytical testing result.
Edgar teaches a system wherein the monitoring system is programmed and configured to:
determine a first performance associated with a first version of the validation algorithm (See Paragraphs [0045]-[0047]: Training curve reflects the estimated mean performance of the model/algorithm, training curve reflect an estimated performance of the model/algorithm at a lower performance bound (LPB) (e.g., wherein the LPB corresponds to the lower confidence bound for a 95 % confidence ), training curve reflects an estimated performance of the model/algorithm at a LPB, and training curve corresponds to the current performance of the model / algorithm at a LPB, which the Examiner is interpreting an estimated performance of the model/algorithm to encompass a first performance associated with the first version of the validation algorithm);
determine a second performance associated with the re-trained validation algorithm by processing a testing data set with the re-trained validation algorithm, wherein the testing data set comprises a plurality of analytical testing data (See Paragraphs [0045]-[0047]: Training curve reflects the estimated mean performance of the model/algorithm, training curve reflect an estimated performance of the model/algorithm at a lower performance bound (LPB) (e.g., wherein the LPB corresponds to the lower confidence bound for a 95 % confidence), training curve reflects an estimated performance of the model/algorithm at a LPB, and training curve corresponds to the current performance of the model/algorithm at a LPB, which the Examiner is interpreting an estimated performance of the model/algorithm to encompass a second performance associated with the re-trained validation algorithm); and
in response to determining that the second performance is better than the first performance, using the re-trained validation algorithm for validation of the at least one of the analytical testing result or the second analytical testing result (See Paragraphs [0054], [0089]-[0090]: The machine learning model development and optimization ensures performance validation and data sufficiency for regulatory approval, which the Examiner is interpreting to encompass validation of an analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the monitoring system is programmed and configured to: determine a first performance associated with a first version of the validation algorithm; determine a second performance associated with the re-trained validation algorithm by processing a testing data set with the re-trained validation algorithm, wherein the testing data set comprises a plurality of analytical testing data; and in response to determining that the second performance is better than the first performance, using the re-trained validation algorithm for validation of the at least one of the analytical testing result or the second analytical testing result as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 14, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the testing data set is processed by the re-trained validation algorithm according to an order, and wherein the monitoring system is programmed and configured to: determine a first number of analytical testing results that have been processed by the re-trained validation algorithm before a faulty invalidation is made by the re-trained validation algorithm according to the order; determine a second number of the analytical testing results before one of the analytical testing results that is labeled as invalid is processed according to the order; and determine the second performance using the first number and the second number.
Edgar further teaches wherein the testing data set is processed by the re-trained validation algorithm according to an order, and wherein the monitoring system is programmed and configured to:
determine a first number of analytical testing results that have been processed by the re-trained validation algorithm before a faulty invalidation is made by the re-trained validation algorithm according to the order (See Paragraphs [0065]-[0066]: MAAP thresholds for regulatory approval of SaDms can be based one or more risk/mitigation valuations that measure one algorithmic risk, the algorithm risk can measure risk of patient harm based on algorithm error, such as an incorrect prediction from ground truth or another algorithm prediction error, which the Examiner is interpreting MAAP thresholds based on algorithmic prediction error to encompass a first number of analytical testing results that have been processed by the re-trained validation algorithm before a faulty invalidation is made by the re-trained validation algorithm according to the order);
determine a second number of the analytical testing results before one of the analytical testing results that is labeled as invalid is processed according to the order (See Paragraphs [0065]-[0066]: MAAP thresholds for regulatory approval of SaDms can be based one or more risk/mitigation valuations that measure one algorithmic risk, the algorithm risk can measure risk of patient harm based on algorithm error, such as an incorrect prediction from ground truth or another algorithm prediction error, which the Examiner is interpreting MAAP thresholds based on algorithmic prediction error to encompass a second number of the analytical testing results before one of the analytical testing results that is labeled as invalid is processed according to the order); and
determine the second performance using the first number and the second number (See Paragraphs [0084], [0089]-[0090]: The system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting updating the machine learning model using additional data samples to encompass the claimed portion.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the testing data set is processed by the re-trained validation algorithm according to an order, and wherein the monitoring system is programmed and configured to: determine a first number of analytical testing results that have been processed by the re-trained validation algorithm before a faulty invalidation is made by the re-trained validation algorithm according to the order; determine a second number of the analytical testing results before one of the analytical testing results that is labeled as invalid is processed according to the order; and determine the second performance using the first number and the second number as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 15, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claims 1 and 13 as described above. Venkatraman may not explicitly teach wherein the monitoring system is programmed and configured to: determine a number of at least one of faulty validation predictions or faulty invalidation predictions by the re-trained validation algorithm based on the testing data set; and determine the second performance using the number of the at least one of the faulty validation predictions or the faulty invalidation predictions by the re-trained validation algorithm.
Edgar teaches a system wherein the monitoring system is programmed and configured to:
determine a number of at least one of faulty validation predictions or faulty invalidation predictions by the re-trained validation algorithm based on the testing data set (See Paragraphs [0065]-[0066], [0090]: MAAP thresholds for regulatory approval of SaDms can be based one or more risk/mitigation valuations that measure one algorithmic risk, the algorithm risk can measure risk of patient harm based on algorithm error, such as an incorrect prediction from ground truth or another algorithm prediction error, and the system updates the machine learning model based on re-training the machine learning model using the additional data samples); and
determine the second performance using the number of the at least one of the faulty validation predictions or the faulty invalidation predictions by the re-trained validation algorithm (See Paragraphs [0084], [0089]-[0090]: The system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting updating the machine learning model using additional data samples to encompass the claimed portion.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include wherein the monitoring system is programmed and configured to: determine a number of at least one of faulty validation predictions or faulty invalidation predictions by the re-trained validation algorithm based on the testing data set; and determine the second performance using the number of the at least one of the faulty validation predictions or the faulty invalidation predictions by the re-trained validation algorithm as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 23, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is configured to: validate the analytical testing result using a first version of the validation algorithm and the re-trained validation algorithm, associate the analytical testing result with a first level of warning based on the analytical testing result being determined to be invalid by the first version of the validation algorithm and being determined to be valid by the re-trained validation algorithm, and associate the analytical testing result with a second level of warning based on the analytical testing result being determined to be invalid by both the first version of the validation algorithm and the re-trained validation algorithm.
Edgar teaches a system wherein the monitoring system is configured to:
validate the analytical testing result using a first version of the validation algorithm and the re-trained validation algorithm (See Paragraphs [0045]-[0047]: Training curve reflects the estimated mean performance of the model/algorithm, training curve reflect an estimated performance of the model/algorithm at a lower performance bound (LPB) (e.g., wherein the LPB corresponds to the lower confidence bound for a 95 % confidence ), training curve reflects an estimated performance of the model/algorithm at a LPB, and training curve corresponds to the current performance of the model / algorithm at a LPB, which the Examiner is interpreting an estimated performance of the model/algorithm to encompass validate the analytical testing result using a first version of the validation algorithm and the re-trained validation algorithm),
associate the analytical testing result with a first level of warning based on the analytical testing result being determined to be invalid by the first version of the validation algorithm and being determined to be valid by the re-trained validation algorithm (See Paragraphs [0052], [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the different subgroups can respectively be associated with different, measurable factors associated with the data samples (e.g., the training, test and validation data samples) to encompass associate the analytical testing result with a first level of warning based on the analytical testing result being determined to be invalid by the first version of the validation algorithm and being determined to be valid by the re-trained validation algorithm the level of granularity of the different subgroups can vary), and
associate the analytical testing result with a second level of warning based on the analytical testing result being determined to be invalid by both the first version of the validation algorithm and the re-trained validation algorithm (See Paragraphs [0052], [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the different subgroups can respectively be associated with different, measurable factors associated with the data samples (e.g., the training, test and validation data samples) to encompass associate the analytical testing result with a second level of warning based on the analytical testing result being determined to be invalid by both the first version of the validation algorithm and the re-trained validation algorithm as the level of granularity of the different subgroups can vary.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include the monitoring system is configured to: validate the analytical testing result using a first version of the validation algorithm and the re-trained validation algorithm, associate the analytical testing result with a first level of warning based on the analytical testing result being determined to be invalid by the first version of the validation algorithm and being determined to be valid by the re-trained validation algorithm, and associate the analytical testing result with a second level of warning based on the analytical testing result being determined to be invalid by both the first version of the validation algorithm and the re-trained validation algorithm as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
As per claim 24, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman/Edgar/Hughes may not explicitly teach wherein the one or more analyzer instruments are configured to use at least one of chemical, biological, physical, or optical procedures to process the physical in vitro biological sample.
Von Hoff teaches a system wherein the one or more analyzer instruments are configured to use at least one of chemical, biological, physical, or optical procedures to process the physical in vitro biological sample (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting molecular profiling to encompass use at least one of chemical, biological, physical, or optical procedures to process the physical in vitro biological sample.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman/Edgar/Hughes to the one or more analyzer instruments are configured to use at least one of chemical, biological, physical, or optical procedures to process the physical in vitro biological sample as taught by Von Hoff. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar/Hughes with Von Hoff with the motivation of improved patient care and enhanced treatment outcomes (See Background of Von Hoff in Paragraph [0006]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Venkatraman et al. (U.S. Patent Pre-Grant Publication No. 2022/0054008) in view of Edgar (U.S. Patent Pre-Grant Publication No. 2021/0201190) in view of Hughes (U.S. Patent Pre-Grant Publication No. 2019/0156241) in view of Von Hoff et al. (U.S. Patent Pre-Grant Publication No. 2010/0304989) in further view of McCourt, Jr. (U.S. Patent Pre-Grant Publication No. 2020/0019884).
As per claim 12, Venkatraman/Edgar/Hughes/Von Hoff discloses the system of claim 1 as described above. Venkatraman may not explicitly teach wherein the monitoring system is programmed and configured to: determine the difference level using the first percentage and the second percentage.
Edgar teaches a system wherein the monitoring system is programmed and configured to: determine the difference level using the first percentage and the second percentage (See Paragraphs[0057]-[0059], [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting to encompass the claimed portion when combined with McCourt, Jr. as described below.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman to include determine the difference level using the first percentage and the second percentage as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
While Venkatraman/Edgar/Hughes/Von Hoff discloses the system as described above, Venkatraman/Edgar/Hughes/Von Hoff may not explicitly teach wherein the monitoring system is programmed and configured to: determine a first percentage of analytical testing results of the live data set that are labeled as invalid; determine a second percentage of training analytical testing results of the first training data set that are labeled as invalid.
McCourt, Jr. teaches a system wherein the monitoring system is programmed and configured to:
determine a first percentage of analytical testing results of the live data set that are labeled as invalid (See Paragraph [0031]-[0032]: The tolerator is also input with or identifies a correct priori probability P (I.sub.i is correct) that the set of training data entries have correctly labeled output labels and an incorrect priori probability P (I.sub.i is wrong) that the set of training data entries have incorrectly labeled output labels, which the Examiner is interpreting to encompass the claimed portion when combined with the live stream data of Venkatraman);
determine a second percentage of training analytical testing results of the first training data set that are labeled as invalid (See Paragraph [0031]-[0032]: The tolerator is also input with or identifies a correct priori probability P (I.sub.i is correct) that the set of training data entries have correctly labeled output labels and an incorrect priori probability P (I.sub.i is wrong) that the set of training data entries have incorrectly labeled output labels, which the Examiner is interpreting to encompass the claimed portion when combined with the training data of Venkatraman.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the system of Venkatraman/Edgar/Hughes/Von Hoff to include determine a first percentage of analytical testing results of the live data set that are labeled as invalid; determine a second percentage of training analytical testing results of the first training data set that are labeled as invalid as taught by McCourt, Jr. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar/Hughes/Von Hoff with McCourt, Jr. with the motivation of provide a much more accurate and efficient device and/or process for determining whether training data is incorrect for training a binary signal classifier (See Detailed Description of McCourt, Jr.in Paragraph [0159]).
Claims 16-18, 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Venkatraman et al. (U.S. Patent Pre-Grant Publication No. 2022/0054008) in view of Edgar (U.S. Patent Pre-Grant Publication No. 2021/0201190) in further view of Von Hoff et al. (U.S. Patent Pre-Grant Publication No. 2010/0304989).
As per independent claim 16, Venkatraman discloses a computer-implemented method for quality control monitoring of diagnostic analytical testing, comprising:
receiving a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples by one or more analyzer instrument (See Paragraph [0055]: Patient biological data that is recorded and transmitted to the computing system may be processed on its own, and/or along with patient data previously transmitted by the patient, which the Examiner is interpreting the patient biological data to encompass one or more physical in vitro biological samples when combined with Von Hoff described below), wherein the analytical testing data indicates:
metadata associated with the analytical testing result (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting the biological sensor data may be tagged along with additional data such as patient metadata ([0097]) to encompass metadata associated with the analytical testing result);
validating the analytical testing result of the live data set by generating, using a validation algorithm (See Paragraphs [0053]-[0055]: The training set (e.g., type of data and size of the training set) may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, and patient biological data that is recorded and transmitted to the computing system may be processed on its own, and/or along with patient data previously transmitted by the patient, or it may be processed (e.g., compared) with control samples and/or patient data transmitted by other patients, which the Examiner is interpreting the validation to encompass validating the analytical testing result, and interpreting algorithm to encompass a validation algorithm), a validation determination indicating whether the analytical testing result is valid or invalid (See Paragraph [0053]: The training set may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, which the Examiner is interpreting output having a predetermined specificity to encompass a validation determination indicating that the analytical testing result is valid or invalid as the specificity could be validity), wherein:
the validation algorithm comprises a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data (See Paragraphs [0052]-[0055]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain expert to encompass a first training data set comprising training analytical testing data), and
the training analytical testing data comprises a training analytical testing result and training metadata (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain experts to encompass training analytical testing result and the biological sensor data may be tagged along with additional data such as patient metadata ([0097]) to encompass training metadata);
determining a difference level between the live data set and the first training data set (See Paragraphs [0052]-[0055], [0146]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, which the Examiner is interpreting ongoing live stream data to encompass a live data set, and the trained algorithm may compare recorded data to at least pre-existing data to encompass determine a difference level), based on comparison of distribution characteristics of the live data set and the first training data set (See Paragraphs [0047], [0050]-[0054]: Analytics engine may compare ECG data generated by ECG sensors at consultations held every month for a period of a year, in order to determine whether an arrhythmia is stable or getting worse, or whether a patient's heart disease is evolving in response to a treatment, and the trained AI algorithm may compare and identify a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data and/or audio data to a recorded ECG or audio data, which the Examiner is interpreting comparing a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data to encompass the difference level being determined based on comparison of distribution characteristics of the live data set and the first training data set.)
While Venkatraman teaches the method as described above, Venkatraman may not explicitly teach determining that the difference level is greater than a threshold; and re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold.
Edgar teaches a method for determining that the difference level is greater than a threshold (See Paragraph [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the system updates the machine learning model based on re-training the machine learning model using the additional data samples to encompass determining that the difference level between the live data set and the first training data set is greater than a first threshold when combined with the data of Venkatraman); and
re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold (See Paragraphs [0089]-[0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the system updates the machine learning model based on re-training the machine learning model using the additional data samples to encompass re-train the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold when combined with the data of Venkatraman.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the method of Venkatraman to include determining that the difference level is greater than a threshold; and re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
While Venkatraman/Edgar teaches the method as described above, Venkatraman/Edgar may not explicitly teach receiving a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples by one or more analyzer instruments, wherein the analytical testing data indicates: an analytical testing result determined by the one or more analyzer instruments based on processing of the one or more physical in vitro biological samples.
Von Hoff teaches a method for receiving a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting a biological sample to encompass a physical in vitro biological sample ([0082]: Such samples include blood and blood fractions or products (e.g., serum, buffy coat, plasma, platelets, red blood cells, and the like), sputum, cheek cells tissue, cultured cells (e.g., primary cultures, explants, and transformed cells), stool, urine, other biological or bodily fluids (e.g., prostatic fluid, gastric fluid, intestinal fluid, renal fluid, lung fluid, cerebrospinal fluid, and the like), etc.), and interpreting the information from the molecular testing to encompass a live data set) by one or more analyzer instruments (See Paragraph [0311]: RNA was specifically bound and then eluted. The RNA was tested for integrity by assessing the ratio of 28S to 18S ribosomal RNA on an Agilent BioAnalyzer, which the Examiner is interpreting an Agilent BioAnalyzer to encompass one or more analyzer instruments), wherein the analytical testing data indicates:
an analytical testing result determined by the one or more analyzer instruments based on processing of the one or more physical in vitro biological samples (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting molecular profiling to encompass an analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the method of Venkatraman/Edgar to include receiving a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples by one or more analyzer instruments, wherein the analytical testing data indicates: an analytical testing result determined by the one or more analyzer instruments based on processing of the one or more physical in vitro biological samples as taught by Von Hoff. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar/Hughes with Von Hoff with the motivation of improved patient care and enhanced treatment outcomes (See Background of Von Hoff in Paragraph [0006]).
Claim 18 mirrors claim 16 but for reciting in system form, and is rejected for the same reason as claim 16. The addition of “one or more analyzer instruments programmed and configured to determine analytical testing results associated with testing of one or more physical in vitro biological samples” is encompassed by Von Hoff in Paragraph [0311]: RNA was specifically bound and then eluted. The RNA was tested for integrity by assessing the ratio of 28S to 18S ribosomal RNA on an Agilent BioAnalyzer.
Claim 21 mirrors claim 16 but for reciting in system form, and is rejected for the same reason as claim 16. The additions of “a monitoring system for diagnostic analytical testing, comprising: a processing unit; and a memory coupled to the processing unit and having instructions stored thereon that, when executed by the processing unit,” are encompassed by Venkatraman in Paragraph [0093]: “Instructions for carrying out method 400 may be executed by a processing system of the transmitting device (e.g., the processor of the transmitting device 132 shown in FIG. 1 and described above) based on instructions stored in non-transitory memory of the transmitting device (e.g., memory 144 shown in FIG. 1 and described above) in conjunction with input provided to the transmitting device via a user interface (such as user interface device 148 shown by FIG. 1 and described above) by a user of the transmitting device (e.g., a patient, or an attending caregiver).”
Claim 22 mirrors claim 16 but for reciting in non-transitory computer-readable medium form, and is rejected for the same reason as claim 16. The additions of “when executed by one or more processors of a monitoring system” are encompassed by Venkatraman in Paragraph [0093]: “Instructions for carrying out method 400 may be executed by a processing system of the transmitting device (e.g., the processor of the transmitting device 132 shown in FIG. 1 and described above) based on instructions stored in non-transitory memory of the transmitting device (e.g., memory 144 shown in FIG. 1 and described above) in conjunction with input provided to the transmitting device via a user interface (such as user interface device 148 shown by FIG. 1 and described above) by a user of the transmitting device (e.g., a patient, or an attending caregiver).”
As per independent claim 17, Venkatraman discloses a method for monitoring of diagnostic analytical testing, comprising:
providing a live data set comprising:
the plurality of analytical testing results (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting a health condition to encompass an analytical testing result), and
metadata associated with the plurality of analytical testing results (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting the biological sensor data may be tagged along with additional data such as patient metadata ([0097]) to encompass metadata associated with the analytical testing result);
validating at least one analytical testing result, of the plurality of analytical testing results, indicated by the live data set by generating, using a validation algorithm (See Paragraphs [0053]-[0055]: The training set (e.g., type of data and size of the training set) may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, and patient biological data that is recorded and transmitted to the computing system may be processed on its own, and/or along with patient data previously transmitted by the patient, or it may be processed (e.g., compared) with control samples and/or patient data transmitted by other patients, which the Examiner is interpreting the validation to encompass validating the analytical testing result, and interpreting algorithm to encompass a validation algorithm), a validation determination indicating that the at least one analytical testing result determined by the one or more analyzer instruments is valid or invalid (See Paragraph [0053]: The training set may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity, which the Examiner is interpreting output having a predetermined specificity to encompass a validation determination indicating that the at least one analytical testing result determined by the one or more analyzer instruments is valid or invalid when combined with Von Hoff), wherein:
the validation algorithm comprises a machine learning algorithm that has been trained using a first training data set comprising training analytical testing data (See Paragraphs [0052]-[0055]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain expert to encompass a first training data set comprising training analytical testing data), and
the training analytical testing data comprises at least one training analytical testing result and training metadata (See Paragraphs [0052]-[0055], [0096]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), which the Examiner is interpreting biological sensor data reviewed and labeled by domain experts to encompass training analytical testing result and the biological sensor data may be tagged along with additional data such as patient metadata ([0097]) to encompass training metadata);
determining a difference level between the live data set and the first training data set (See Paragraphs [0052]-[0055], [0146]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, which the Examiner is interpreting ongoing live stream data to encompass a live data set, and the trained algorithm may compare recorded data to at least pre-existing data to encompass determine a difference level), based on comparison of distribution characteristics of the live data set and the first training data set (See Paragraphs [0047], [0050]-[0054]: Analytics engine may compare ECG data generated by ECG sensors at consultations held every month for a period of a year, in order to determine whether an arrhythmia is stable or getting worse, or whether a patient's heart disease is evolving in response to a treatment, and the trained AI algorithm may compare and identify a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data and/or audio data to a recorded ECG or audio data, which the Examiner is interpreting comparing a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data to encompass the difference level being determined based on comparison of distribution characteristics of the live data set and the first training data set.)
While Venkatraman teaches the method as described above, Venkatraman may not explicitly teach determining that the difference level is greater than a threshold; re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold; and using the re-trained validation algorithm for validation of a second analytical testing result indicated by at least one of the live data set or a second live data set.
Edgar teaches a method for determining that the difference level is greater than a threshold (See Paragraph [0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the system updates the machine learning model based on re-training the machine learning model using the additional data samples to encompass determining that the difference level between the live data set and the first training data set is greater than a first threshold when combined with the data of Venkatraman);
re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold (See Paragraphs [0089]-[0090]: The system determines underperforming subgroups of the subgroups based on the subgroup performance measures associated with the underperforming subgroups failing to satisfy a threshold performance measure, the system updates the machine learning model based on re-training the machine learning model using the additional data samples, which the Examiner is interpreting the system updates the machine learning model based on re-training the machine learning model using the additional data samples to encompass re-train the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold when combined with the data of Venkatraman); and
using the re-trained validation algorithm for validation of a second analytical testing result indicated by at least one of the live data set or a second live data set (See Paragraphs [0089]-[0090]: The machine learning model development and optimization ensures performance validation and data sufficiency for regulatory approval, which the Examiner is interpreting to encompass validation of an analytical testing result when combined with the live data of Venkatraman.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the method of Venkatraman to include determining that the difference level is greater than a threshold; re-training the validation algorithm using a second training data set based on determining that the difference level, between the live data set and the first training data set, is greater than the threshold; and using the re-trained validation algorithm for validation of a second analytical testing result indicated by at least one of the live data set or a second live data set as taught by Edgar. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman with Edgar with the motivation of providing a clinical benefit (See Detailed Description of Edgar in Paragraph [0070]).
While Venkatraman/Edgar teaches the method as described above, Venkatraman/Edgar may not explicitly teach determining, by testing physical in vitro biological samples with one or more analyzer instruments, a plurality of analytical testing results associated with the testing of the physical in vitro biological samples.
Von Hoff teaches a method for determining, by testing physical in vitro biological samples with one or more analyzer instruments (See Paragraph [0311]: RNA was specifically bound and then eluted. The RNA was tested for integrity by assessing the ratio of 28S to 18S ribosomal RNA on an Agilent BioAnalyzer, which the Examiner is interpreting an Agilent BioAnalyzer to encompass one or more analyzer instruments), a plurality of analytical testing results associated with the testing of the physical in vitro biological samples (See Fig. 2, Paragraphs [0082]-[0083], [0251]: In order to determine a medical intervention for a particular disease state using molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient in step 52, target is defined as any molecular finding that may be obtained from molecular testing, which the Examiner is interpreting molecular profiling to encompass an analytical testing result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed to modify the method of Venkatraman/Edgar to include determining, by testing physical in vitro biological samples with one or more analyzer instruments, a plurality of analytical testing results associated with the testing of the physical in vitro biological samples as taught by Von Hoff. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to modify Venkatraman/Edgar with Von Hoff with the motivation of improved patient care and enhanced treatment outcomes (See Background of Von Hoff in Paragraph [0006]).
Response to Arguments
In the Remarks filed on October 3, 2025, the Applicant argues that the newly amended and/or added claims overcome the 35 U.S.C. 101 rejection(s) and 35 U.S.C. 103 rejection(s). The Examiner does not acknowledge that the newly added and/or amended claims overcome the newly added Claim Objection(s), 35 U.S.C. 101 rejection(s), and 35 U.S.C. 103 rejection(s).
The Applicant argues that:
(1) Applicant respectfully disagrees that the claims recite a "mental process" abstract idea, and notes that claims "do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations." See MPEP § 2106.04(a)(2), subsection III.A. Here, Applicant respectfully submits that it is not practical to perform the elements of Applicant's claims in the human mind. Id Accordingly, Applicant respectfully submits that the claims do not recite a "mental process" abstract idea. As an example, Applicant respectfully submits that it is not practical, or possible, for a human to mentally process a "physical in vitro biological sample," for example as amended claim 1 recites. For instance, Applicant respectfully submits that a human cannot mentally process or test a blood sample, or any other physical in vitro biological sample. The Office indicates that "the claim encompasses a user manually processing a biological sample." See Office Action, page 4. However, even if a human could manually process a physical in vitro biological sample with the assistance of laboratory equipment, Applicant respectfully submits that a human is not capable of processing a physical in vitro biological sample entirely in the human mind, or even with the assistance of pen and paper. Accordingly, Applicant respectfully submits that such claim elements do not recite a "mental process" abstract idea. As another example, the August 2025 USPTO Memo reiterates indications in USPTO Example 39 that a claim element such as "'training the neural network in a first stage using the first training set' ... does not recite a judicial exception." See August 2025 USPTO Memo, page 3. Accordingly, for the reasons discussed in Example 39 and the August 2025 USPTO Memo, Applicant respectfully submits that similar elements of Applicant's claims, such as a validation algorithm comprising a "machine learning algorithm that has been trained using a first training data set" and re-training "the validation algorithm using a second training data set," for example as amended claim 1 recites, are elements that do not recite a mental process or any other judicial exception. For at least these reasons, Applicant respectfully submits that claims 1-3, 5, and 7-23 do not recite an abstract idea. Accordingly, Applicant respectfully requests that the § 101 rejections of claims 1-3, 5, and 7-23 be withdrawn;
(2) As discussed above, Applicant respectfully submits that multiple elements of the claims are not practical to perform in the human mind. As such, Applicant respectfully submits that those elements are "additional elements," beyond the alleged "mental process" abstract idea, that are to be considered at Prong Two of Step 2A and at Step 2B of the subject matter eligibility analysis. As an example, Applicant respectfully submits that the "one or more analyzer instruments" recited in claim 1 and other independent claims, which as claim I recites may be configured to "process a physical in vitro biological sample" and determine a corresponding "analytical testing result," are "additional elements" beyond the alleged "mental process" abstract idea because it is not practical for a human to mentally process a physical in vitro biological sample. As another example, because training of a machine learning model is not a mental process or any other judicial exception according to Example 39 and the August 2025 USPTO Memo, Applicant respectfully submits that the initial training and subsequent re-training of a machine learning "validation algorithm," for example as recited in amended claim 1 and other independent claims, are also "additional elements" beyond the alleged "mental process" abstract idea. The MPEP notes that additional elements recited in a claim demonstrate that the claim as a whole integrates a judicial exception into a practical application when the judicial exception is implemented with, or is used in conjunction with, a "particular machine or manufacture that is integral to the claim." See MPEP § 2106.04(d)(I) (emphasis added). Here, as discussed during the interview, Applicant respectfully submits that the claims do implement the alleged abstract idea with, or use the alleged abstract idea in conjunction with, particular machines that are integral to the claims - the recited "analyzer instruments" that process a "physical in vitro biological sample," for example as claim I recites. Applicant respectfully submits that such "analyzer instruments" are not generic computing elements, as there is no indication that generic computing elements are capable of processing or testing blood samples or other physical in vitro biological samples. Accordingly, Applicant respectfully submits that the recited "analyzer instruments" are particular machines that are integral to the claims, and thereby demonstrate that the alleged "mental process" abstract idea is integrated into a practical application such that the claims are patent eligible;
(3) the MPEP also notes that additional elements recited in a claim demonstrate that the claim, as a whole, integrates a judicial exception into a practical application if "the claimed invention improves ... another technology or technical field." Id The MPEP also states that "the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement." Id Here, Applicant respectfully submits that the additional elements recited in the claims lead to technical improvements that demonstrate integration of the alleged "mental process" abstract idea into a practical application, and that Applicant's specification describes those technical improvements. For example, Applicant's specification notes that "[a]nalytical testing can be subject to errors that can compromise the analytical testing results," for instance due to "mishandling, misconfiguration, and/or tear of an analyzer." See Specification, paragraph [0004]. Accordingly, "validation procedures are important to ensure validity of analytical testing results that are generated in various diagnostic tests." See Specification, paragraph [0043]. For instance, "an accumulation of invalidations of analytical testing results" may be "an indication of a systemic error in the analytical testing process." See Specification, paragraph [0047]. However, in some previous validation procedures, a validation algorithm may be "trained using a specific training data set." See Specification, paragraph [0057]. Such previous validation procedures may achieve "good performance when an overall characteristic of input analytical testing results is close to the training analytical testing results included in the training data set." Id If instead "an overall characteristic of the input analytical testing results" provided to the trained the validation algorithm "is significantly different from the training analytical testing results" that had previously been used to train the validation algorithm, the validation algorithm may generate "faulty validation predictions or faulty invalidation predictions." See Specification, paragraph [0058] (emphasis added). As one example, "if the validation algorithm is trained using analytical testing results generated in summer, the validation algorithm may be error prone when processing analytical testing results generated in a different season, e.g., winter." Id (emphasis added). To improve the accuracy of such a validation algorithm, and avoid situations in which different characteristics of a training data set relative to a live data set may cause a validation algorithm (previously trained on the training data set) to be error prone when attempting to validate testing results within the live data set, the specification indicates that the validation algorithm may be "re-trained using a second training set." See Specification, paragraph [0059]. Such re-training of the validation algorithm on a second training set, instead of the original training data set that was significantly different than the live data set that is currently being evaluated, may thus "significantly improve accuracy and quality of validation of analytical testing data." Id (emphasis added). As an example, if a difference between distribution characteristics of a training data set (previously used to train the validation algorithm) and a live data set is greater than a threshold, "the live data set" may be "greatly different from the training data set." See Specification, paragraph [00136]. Accordingly, application of the current version of the validation algorithm (that had been trained on the original training data set) may be "error prone" when applied to the live data set. Id (emphasis added). However, the likelihood of such errors may be reduced by re-training the validation algorithm using a different training data set. See Specification, paragraphs [00135] and [00136]. The re-training of the validation algorithm may, for instance, be a based on a "second training data set" that has a smaller difference with the live data set, relative to the difference between the live data set and the first training data set, such that the "re-trained validation algorithm may have a better performance." See Specification, paragraph [00140]. Applicant respectfully submits that the claims reflect the technical improvements discussed in the specification. For instance, amended claim 1 recites determining a "difference level" between: 1) a "live data set" that "comprises the analytical testing data" (including "the analytical testing result determined by the one or more analyzer instruments" and "metadata associated with the analytical testing result"); and 2) the "first training data set" upon which the "validation algorithm" has been trained, where the "difference level is determined based on a comparison of the distribution characteristics of the live data set and the first training data set" (emphasis added). Amended claim 1 also recites that if that "difference level" between the current "live data set" and the "first training data set" is "greater than a first threshold" (such that the validation algorithm may be error prone with respect to the live data set as the specification describes), the system may "re-train the validation algorithm using a second training data set" (and thereby increase the accuracy of the validation algorithm as the specification describes). Overall, Applicant respectfully submits that the claims recite additional elements, beyond the alleged abstract idea, that are implemented with and/or are used in conjunction with particular machines, and that provide technical improvements, as described above. Accordingly, Applicant respectfully submits that the claims show that the alleged abstract idea is integrated into a practical application, and that the claims are therefore not, as a whole, directed to the alleged abstract idea. See MPEP § 2106.04(d). As such, Applicant respectfully requests that the Office withdraw the § 101 rejection of claims 1-3, 5, and 7-22 for at least this additional reason;
(4) at Step 2B, additional elements recited in a claim can show that the claim includes "significantly more" than an identified abstract idea if an additional element, or a combination of elements, indicate that the identified abstract idea is applied "with, or by use of, a particular machine." See MPEP § 2106.05. As discussed above, Applicant respectfully submits that here, the claims do recite that the alleged "mental process" abstract idea is being applied "with, or by use of, a particular machine," the "analyzer instruments" that process or test physical in vitro biological samples, as recited in the claims. As discussed above, Applicant respectfully submits that the "analyzer instruments" are not generic computing elements, and should accordingly be given weight at Step 2B. Because the claims apply or use the alleged abstract idea in association with a "particular machine," Applicant respectfully submits that the claims include significantly more than the alleged abstract idea. As another example, at Step 2B, additional elements recited in a claim can show that the claim includes "significantly more" than an identified abstract idea if an additional element, or a combination of elements, show improvements to a "technology or technical field." See MPEP § 2106.05. As described above with respect to Step 2A, Applicant respectfully submits that the Applicant's claims demonstrate technical improvements, such as accuracy improvements relative to previous procedures for validating testing results generated via processing of physical in vitro biological samples, where a validation algorithm is re-trained to be more accurate if the previous data set used to train the validation algorithm has different distribution characteristics than the current live data set that is being evaluated by the validation algorithm. Accordingly, for at least this reason, Applicant respectfully submits that the claims include significantly more than the alleged abstract idea. As yet another example, at Step 2B, additional elements recited in a claim can show that the claim includes "significantly more" than an identified abstract idea if an additional element, or a combination of elements, are elements that are not "well-understood, routine, conventional activity in the field." See MPEP § 2106.05. The MPEP notes that "an examiner should determine that an element (or combination of elements) is well-understood, routine, conventional activity only when the examiner can readily conclude, based on their expertise in the art, that the element is widely prevalent or in common use in the relevant industry." See MPEP § 2106.05(d)(I) (emphasis added). Here, Applicant respectfully submits that there is no indication that the elements recited in Applicant's claims are "well-understood, routine, conventional activity in the field." Id For example, Applicant respectfully submits that there is no indication that it is well-understood, routine, or conventional in the field to: 1) use a "validation algorithm," comprising a machine learning algorithm trained using a first training data set, to validate an analytical testing result determined by "one or more analyzer instruments" that process a physical in vitro biological sample; 2) determine a difference between the "first training data set" and a "live data set" of analytical testing data (including the analytical testing result determined by "one or more analyzer instruments") based on a comparison of "distribution characteristics" of those two data sets; and 3) "re-train the validation algorithm using a second training data set if the difference between the live data set and the first training data set is greater than a first threshold," as amended claim 1 recites. As discussed above, the specification indicates that such re-training of a validation algorithm is not well-understood, routine, or conventional in the field, and that a "validation algorithm is typically trained" only on one "specific training data set" even if the training data set had significantly different characteristics than the data set that is being processed via the trained validation algorithm. See Specification, paragraphs [0057] and [0058]. Overall, Applicant respectfully submits that the combination of additional elements recited in the claims provides "significantly more" than the alleged abstract idea alone, and demonstrates that the claims are patent-eligible. Accordingly, for at least this additional reason, Applicant respectfully requests that the Office withdraw the§ 101 rejections of claims 1-3, 5, and 7-22;
(5) Applicant respectfully submits that the combination of Venkatraman, Edgar, and Hughes does not render claim 1 unpatentable. Venkatraman describes systems for "real-time biological sensor data transmission and analysis," where "biological sensor data" may be transmitted in real-time from a patient's device to a clinician's device. See Venkatraman, Abstract. Edgar describes systems for ensuring "performance validation and data sufficiency for regulatory approval," including determining whether a "machine learning model meets an acceptable level of performance for deployment in a field environment." See Edgar, Abstract. Hughes describes an "analysis collaboration platform (ACP) that provides for the connection and management of one or more analysis services with input data and sources of data." See Hughes, Abstract. However, Applicant respectfully submits that the combination of Venkatraman, Edgar, and Hughes does not teach or suggest all of the elements that amended claim 1 recites. As an example, the Office argues that, in the proposed combination, Venkatraman describes "analyzer instruments" that process a biological sample, as claim 1 recites, because "the Examiner is interpreting ... patient biological data to encompass a biological sample." See Office Action, pages 11 and 12. However, Applicant respectfully notes that Venkatraman describes a "telehealth system" that provides remote evaluation of "sensor data" (such as blood pressure monitoring data, pulse data, etc.) that was separately collected by a home patient monitoring device. See Venkatraman, paragraphs [0026]-[0029]. Accordingly, Applicant respectfully submits that Venkatraman does not describe "one or more analyzer instruments" that are configured to "process a physical in vitro biological sample," and "determine an analytical testing result based on processing of the physical in vitro biological sample," as amended claim 1 recites. As another example, the Office argues that, in the proposed combination, Venkatraman describes a "validation algorithm" that "generates a validation determination indicating that the analytical testing result ... is valid or invalid," as claim 1 recites. See Office Action, page 13. Applicant respectfully disagrees. As discussed during the interview, Venkatraman discusses training of an algorithm based on a training set that has been "selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity," such as being at least 90% accurate "when tested on a validation ... sample." See Venkatraman, paragraph [0053]. However, Applicant respectfully submits that the "validation" Venkatraman describes is a validation of the training of the algorithm itself, to validate that the algorithm has been sufficiently trained and is likely to generate accurate output. Id Applicant respectfully submits that validation of an algorithm's own training, as described in Venkatraman, is unrelated to determining whether an "analytical testing result" separately "determined by the one or more analyzer instruments" is "valid or invalid," as amended claim 1 recites (emphasis added). As yet another example, the Office argues that, in the proposed combination, Venkatraman describes determining a "difference level between a live data set and the first training data set" that had been used to train the validation algorithm, where the "difference level is determined based on a comparison of distribution characteristics of the live data set and the first training data set," as claim 1 recites. See Office Action, pages 13 and 14. For support, the Office points to Venkatraman' s comparison of patient data, for the same patient, over time in order to identify and monitor patterns. See Office Action, page 4 (citing paragraphs [0047] and [0050]-[0055] of Venkatraman). However, although Venkatraman may monitor data for a single patient over time to identify differences or disease progressions in association with that single patient, Applicant respectfully submits that there is no indication that older data for the same single patient was a "first training data set" that had been used to train a validation algorithm. Accordingly, Applicant respectfully submits that Venkatraman does not teach or suggest comparing "distribution characteristics" of a live data set and a first training data set (that had previously been used to train the validation algorithm), as amended claim 1 recites. Applicant respectfully submits that Edgar and Hughes fail to remedy the deficiencies of Venkatraman described above. Accordingly, for at least the reasons presented herein, Applicant respectfully submits that amended claim 1 would not have been obvious in view of Venkatraman, Edgar, and Hughes. Accordingly, Applicant respectfully requests that the Office withdraw the § 103 rejection of claim 1;
(6) claims 2, 3, 5, 7-15, and 23 ultimately depend from independent claim 1. As discussed above, Applicant respectfully submits that claim 1 is allowable over the cited documents. Therefore, Applicant respectfully submits that claims 2, 3, 5, 7-15, and 23 are allowable over the cited documents of record for at least their dependency from an allowable base claim, and also for the additional features that each recites. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejections of claims 2, 3, 5, 7-15, and 23;
(7) the Office rejects claim 19 based on the same rationale used to reject claim 1. See Office Action, page 17. To the extent that claims 1 and 19 recite similar subject matter, Applicant respectfully submits that amended claim 19 is not obvious in view of Venkatraman, Edgar, and Hughes, for at least reasons similar to those presented above with respect to claim 1. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 19. The Office rejects claim 20 based on the same rationale used to reject claim 1. Id To the extent that claims 1 and 20 recite similar subject matter, Applicant respectfully submits that amended claim 20 is not obvious in view of Venkatraman, Edgar, and Hughes, for at least reasons similar to those presented above with respect to claim 1. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 20;
(8) claim 12 stands rejected under 35 U.S.C. § 103 as allegedly being obvious over a combination of Venkatraman, Edgar, Hughes and McCourt, Jr. Applicant respectfully traverses the rejection. Claim 12 depends from independent claim 1. As discussed above, Applicant respectfully submits that claim 1 is allowable over the cited documents. Therefore, Applicant respectfully submits that claim 12 is allowable over the cited documents of record for at least its dependency from an allowable base claim, and also for the additional features that it recites. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 12;
(9) claims 16-18, 21, and 22 stand rejected under 35 U.S.C. § 103 as allegedly being obvious over a combination of Edgar and Venkatraman. Applicant respectfully traverses the rejection. Nevertheless, solely in the interest of expediting allowance, Applicant herein amends claims 16-18, 21, and 22 as shown above. Applicant respectfully requests reconsideration in light of the amendments presented herein;
(10) the Office rejects claim 16 based on a rationale that is similar to the rationale used to reject claim 1, based on a combination of Edgar and Venkatraman that omits Hughes. See Office Action, pages 35-39. However, to the extent that claims 1 and 16 recite similar subject matter, Applicant respectfully submits that amended claim 16 is not obvious in view of Edgar and Venkatraman, for at least reasons similar to those presented above with respect to claim 1. For example, for the reasons discussed above with respect to claim 1, Applicant respectfully submits that, in the proposed combination of Edgar and Venkatraman, Venkatraman does not teach or suggest "a live data set comprising analytical testing data associated with testing of one or more physical in vitro biological samples by one or more analyzer instruments," "generating, using a validation algorithm, a validation determination indicating whether the analytical testing result is valid or invalid," or determining a difference level between the live data set and the first training data set, based on comparison of distribution characteristics of the live data set and the first training data set," as amended claim 16 recites. Applicant respectfully submits that Edgar fails to remedy these deficiencies of Venkatraman. Accordingly, for at least the reasons described herein, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 16;
(11) the Office rejects claim 17 based on a rationale similar to the rationale used to reject claim 16. See Office Action, pages 40-44. To the extent that claims 1 and 17 recite similar subject matter, Applicant respectfully submits that amended claim 17 is not obvious in view of Edgar and Venkatraman, for at least reasons similar to those presented above with respect to claim 16. Accordingly, Applicant respectfully requests that the Office withdraw the § 103 rejection of claim 17;
(12) the Office rejects claim 18 based on the same rationale used to reject claim 16. See Office Action, page 39. To the extent that claims 1 and 18 recite similar subject matter, Applicant respectfully submits that amended claim 18 is not obvious in view of Edgar and Venkatraman, for at least reasons similar to those presented above with respect to claim 16. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 18;
(13) the Office rejects claim 21 based on the same rationale used to reject claim 16. See Office Action, pages 39 and 40. To the extent that claims 1 and 21 recite similar subject matter, Applicant respectfully submits that amended claim 21 is not obvious in view of Edgar and Venkatraman, for at least reasons similar to those presented above with respect to claim 16. Accordingly, Applicant respectfully requests that the Office withdraw the § 103 rejection of claim 21;
(14) the Office rejects claim 22 based on the same rationale used to reject claim 16. See Office Action, page 40. To the extent that claims 1 and 22 recite similar subject matter, Applicant respectfully submits that amended claim 22 is not obvious in view of Edgar and Venkatraman, for at least reasons similar to those presented above with respect to claim 16. Accordingly, Applicant respectfully requests that the Office withdraw the§ 103 rejection of claim 22;
(15) Applicant presents new claim 24 as dependent from independent claim 1. As discussed above, Applicant respectfully submits that independent claim 1 is allowable. Therefore, Applicant respectfully submits that claim 24, as presented, is also allowable for at least its dependency from an allowable base claim, and also for the additional features that it recites. New claim 24 is supported at least at paragraph [0045] of the originally filed application, and as such, Applicant respectfully submits that new claim 24 does not introduce new matter. Accordingly, Applicant respectfully requests that the Office enter and examine new claim 24.
In response to argument (1), the Examiner does not find the Applicant’s argument(s) persuasive. The Examiner maintains that the newly amended claims recite a Certain Method of Organizing Human Activity and does not recite significantly more. The Examiner maintains that the additional elements of “…using a validation algorithm,…”, “…, determined by the one or more analyzer instruments…”, “re-train the validation algorithm using a second training data set if the difference level between the live data set and the first training data set is greater than a first threshold”, “use the re-trained validation algorithm for validation of the analytical testing result or a second analytical testing result” at a high degree of generality, amount no more than generally linking the abstract idea to a particular technical environment. The recitation is also similar to adding the words “apply it” to the abstract idea. As set forth in MPEP 2106.05(f), merely reciting the words “apply it” or an equivalent, is an example of when an abstract idea has not been integrated into a practical application. The 35 U.S.C. 101 rejection(s) stand.
In response to argument (2), the Examiner does not find the Applicant’s argument(s) persuasive. The Examiner maintains that the newly amended claims recite a Certain Method of Organizing Human Activity and does not recite significantly more. The Examiner does not acknowledge that the additional elements are beyond the abstract idea. The Examiner maintains that the abstract idea is not integrated into a practical application because the remaining elements amount to no more than general purpose computer components programmed to perform the abstract idea. The Examiner does not acknowledge that the “analyzer instruments” are particular machines as the “determined by the one or more analyzer instruments” are recited at a level of generality that amounts to no more than generally linking the abstract idea to a particular technical environment, and the recitation is also similar to adding the words “apply it” to the abstract idea. The 35 U.S.C. 101 rejection(s) stand.
In response to argument (3), the Examiner does not find the Applicant’s argument(s) persuasive. The Examiner does not acknowledge that the newly amended claims recite an improvement to a technology or technical field. The Examiner maintains that the Applicant’s claims are similar to “iii. Gathering and analyzing information using conventional techniques and displaying the result, TLI Communications, 823 F.3d at 612-13, 118 USPQ2d at 1747-48” which the courts have indicated may not be sufficient to show an improvement to technology (See MPEP 2106.05(a)(II)). The 35 U.S.C. 101 rejection(s) stand.
In response to argument (4), the Examiner does not find the Applicant’s argument(s) persuasive. The Examiner maintains that the newly amended claims recite an abstract idea that is not integrated into a practical application or recites additional elements that are sufficient to amount to significantly more than the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer configured to perform above identified functions amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See Alice 573 U.S. at 223 (“mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.”) Additionally, generally linking the abstract idea to a particular technological environment does not amount to significantly more than the abstract idea (See MPEP 2016.05(h) and Affinity Labs of Texas v. DirectTV, LLC, 838 F.3d 1253, 120 USP12d 1201 (Fed. Cir. 2016)). The 35 U.S.C. 101 rejection(s) stand.
In response to argument (5), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner has added Von Hoff et al. (U.S. Patent Pre-Grant Publication No. 2010/0304989) to the 35 U.S.C. 103 rejection(s). The Examiner has relied upon Van Hoff to teach “one or more analyzer instruments configured to: process a physical in vitro biological sample, and determine an analytical testing result based on the processing of the physical in vitro biological sample” in independent claim 1 as described above in the 35 U.S.C. 103 rejection(s). The Examiner maintains that Venkatraman’s disclosure of “the training set (e.g., type of data and size of the training set) may be selected such that, in validation, the algorithm yields an output having a predetermined accuracy, sensitivity and/or specificity (e.g., an accuracy of at least 90% when tested on a validation or test sample independent of the training set)” ([0053]) encompasses the “validation algorithm” as the algorithm of Venkatraman could be selected to yield an output having a predetermined sensitivity and/or specify to identify valid (or true positives). The Examiner maintains that Venkatraman encompasses “difference level between a live data set and the first training data set” in Paragraphs [0052]-[0055], [0146]: A training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient as the algorithm is a trained artificial intelligence algorithm trained on a training data set for supervised learning problem may biological sensor data reviewed and labeled by domain experts. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (6), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claims 2, 3, 5, 7-15, and 23 remain rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (7), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claims 19-20 remain rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (8), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 12 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (9), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claims 16-18, 21, and 22 remain rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (10), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 16 remains rejected under 35 U.S.C. 103 when combined with Von Hoff et al. as Venkatraman discloses in Paragraphs [0052]-[0055], [0146]: A training data set for supervised learning problem may be biological sensor data reviewed and labeled by domain experts (e.g., physicians, clinicians, etc.), and a remote clinician may run an analytics program on ongoing live stream data, to determine a health condition of the patient, and in Paragraphs [0047], [0050]-[0054]: Analytics engine may compare ECG data generated by ECG sensors at consultations held every month for a period of a year, in order to determine whether an arrhythmia is stable or getting worse, or whether a patient's heart disease is evolving in response to a treatment, and the trained AI algorithm may compare and identify a number of examples from the annotated data sets that are closest in terms of a plurality of features of ECG data and/or audio data to a recorded ECG or audio data. The Examiner maintains that when combined with Von Hoff’s identification of molecular profiling that is independent of disease lineage diagnosis (i.e. not single disease restricted), at least one test is performed for at least one target from a biological sample of a diseased patient to encompass a live data set when combined with Venkatraman. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (11), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 17 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (12), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 18 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (13), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 21 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (14), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 22 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
In response to argument (15), the Examiner does find the Applicant’s argument(s) persuasive. The Examiner maintains that claim 24 remains rejected under 35 U.S.C. 103. The 35 U.S.C. 103 rejection(s) stand.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tran et al. (U.S. Patent Pre-Grant Publication No. 2005/0071143), describes methods, computer systems, and computer readable medium for testing a plurality of models in order to classify a biological specimen.
Vaughan et al. (U.S. Patent No. 11,972,336), describes a machine learning platform and system for data analysis including for purposes of providing digital evaluations and therapeutics.
Homayouni et al. (“ADQuaTe: An Automated Data Quality Test Approach for Constraint Discovery and Fault Detection”), describes ADQuaTe, which is an automated data quality test approach that uses an unsupervised machine learning technique to discover constraints that may have been missed by experts.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bennett S Erickson whose telephone number is (571)270-3690. The examiner can normally be reached Monday - Friday: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert Morgan can be reached at (571) 272-6773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Bennett Stephen Erickson/Primary Examiner, Art Unit 3683
Prosecution Insights