DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the Application filed on 07/15/2023. Claims 1-20 are pending in the case. Claims 1 and 11 are independent claims.
Specification
The disclosure is objected to because of the following informalities: paragraph 50, with reference to Figure 2, recites “a system monitoring module 240, and a data store comprising global join data 240.” However, Figure 2 only shows a system monitor 240 and does not appear to depict any sort of data store.
Appropriate correction is required.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-10 are directed towards the statutory category of a process.
With respect to claim 1:
2A Prong 1: This claim is directed to a judicial exception.
processing a data set, within a secure computing node, with the algorithm to generate an algorithm output (mental process – running the algorithm on data inside a safe, isolated computer to get results);
generating a performance model by regression modeling the algorithm output (mental process and/or mathematical concept – making a statistical model to show how well the algorithm works based on its results);
identifying surface regions of the performance model under a configured threshold (mental process – finding areas in the performance report where the algorithm’s performance is below a certain level); and
identifying algorithm inputs associated with the identified surface regions (mental process – figuring out which data caused the weak performance in those areas: manual analysis).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
routing the performance model to an algorithm developer (adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g)); and
performing active learning on the identified algorithm inputs (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
routing the performance model to an algorithm developer (MPEP 2106.05(d) indicates that merely “storing and retrieving information in memory” and/or "receiving or transmitting data over a network" are well‐understood, routine, conventional functions when they are claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer); and
performing active learning on the identified algorithm inputs (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
With respect to claim 2:
2A Prong 1: This claim is directed to a judicial exception.
the performance model models at least one of algorithm accuracy, F1 score accuracy, precision, recall, dice score, ROC (receiver operator characteristic) curve/area, log loss, Jaccard index, error, R2 or by some combination thereof (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 3:
2A Prong 1: This claim is directed to a judicial exception.
the regression modeling includes linear least squares, logistic regression, deep learning or some combination thereof (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 4:
2A Prong 1: This claim is directed to a judicial exception.
smoothing the performance model by identifying portions of the performance model which are highly variable (mental process and/or mathematical concept).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 5:
2A Prong 1: This claim is directed to a judicial exception.
the smoothing includes best fit transform, moving averages and application of filters, Loess smoothing, kernel smoothing, wavelets, splines or some combination thereof (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 6:
2A Prong 1: This claim is directed to a judicial exception.
the smoothing weights the data points of the raw performance model by instances of the algorithm’s input variables (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
2A Prong 1: This claim is directed to a judicial exception.
the algorithm developer receives multiple performance models from the algorithm operating on a plurality of data sets (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 8:
2A Prong 1: This claim is directed to a judicial exception.
identifying at least one perturbation in the multiple performance models (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 9:
2A Prong 1: This claim is directed to a judicial exception.
the active learning includes providing feedback to a data steward to generate more training data for the identified algorithm inputs (mental process).
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 10:
2A Prong 1: This claim is directed to a judicial exception.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
performing training on the algorithm in response to the more training data (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
performing training on the algorithm in response to the more training data (merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f)).
Additionally, claims 11-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter. During examination, the claims must be interpreted as broadly as their terms reasonably allow. In re American Academy of Science Tech Center, 367 F.3d 1359, 1369, 70 U.S.P.Q.2d 1827, 1834 (Fed. Cir. 2004). Independent claim 11 recites a “system,” which is not comprehensively defined by the specification. The broadest reasonable interpretation of a claim drawn to a system covers software per se in view of the ordinary and customary meaning of system, particularly when the specification is silent. Software per se is not a “process,” a “machine,” a “manufacture,” or a “composition of matter” as defined in 35 U.S.C. § 101. Dependent claims 12-20 inherit this deficiency.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Callcut et al. (U.S. Pat. App. Pub. No. 2020/0311300, hereinafter Callcut) in view of Giurgica-Tiron et al. (U.S. Pat. App. Pub. No. 2019/0228533, hereinafter Giurgica-Tiron).
As to independent claim 1, Callcut teaches
A computerized method of active algorithm training in a sequestered computing node comprising (a computer method for modeling the performance of a model (algorithm) in a secure computing framework; paragraph [0007]):
processing a data set, within a secure computing node, with the algorithm to generate an algorithm output (data processing system implemented within a secure computing framework to generate model outputs; Fig. 1, paragraph [0018], [0054]);
generating a performance model by… modeling the algorithm output (computing performance of each instance (raw) of the algorithm by minimizing a loss function; paragraph [0011], [0104]);
routing the performance model to an algorithm developer (training results, such as the parameters and/or training gradients, are transmitted (routed) to the master algorithm module (algorithm developer); paragraph [0105], [0107]);
identifying surface regions of the performance model under a configured threshold (model is determined to have been validated with a specific ground truth data (configured threshold) using area under a curve analysis is used to visualize the performance of the model; paragraph [0111]);
identifying algorithm inputs associated with the identified surface regions (features associated with area under a curve analysis (surface regions) are ranked according to validation criteria; paragraph [0111]); and
performing active learning on the identified algorithm inputs (learned (active) model parameters are chosen based on validation sets (inputs) for the models; paragraph [0110]).
Callcut does not appear to expressly teach regression modeling.
Giurgica-Tiron teaches regression modeling (the output layer of the model is a statistical model operated as a regression-based classifier; paragraph [0084], [0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 2, Callcut further teaches the performance model models at least one of algorithm accuracy, F1 score accuracy, precision, recall, dice score, ROC (receiver operator characteristic) curve/area, log loss, Jaccard index, error, R2 or by some combination thereof (evaluating the models performance includes the precision of the models; paragraph [0072]).
As to dependent claim 3, Giurgica-Tiron further teaches the regression modeling includes linear least squares, logistic regression, deep learning or some combination thereof (regression models uses deep neural network (learning) architecture; paragraph [0084], [0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 4, Giurgica-Tiron further teaches smoothing the performance model by identifying portions of the performance model which are highly variable (high variation outputs (raw performance model) may drastically change the state of the model so some type of smoothing may be used; paragraph [0054]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 5, Giurgica-Tiron further teaches the smoothing includes best fit transform, moving averages and application of filters, Loess smoothing, kernel smoothing, wavelets, splines or some combination thereof (applies filters to smooth out models; paragraph [0006]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 6, Callcut further teaches the smoothing weights the data points of the raw performance model by instances of the algorithm’s input variables (adjusts (smooths) predetermined weights of model instances according to the accuracy of the model in order to optimize the model; paragraph [0081]).
As to dependent claim 7, Callcut further teaches the algorithm developer receives multiple performance models from the algorithm operating on a plurality of data sets (master algorithm module receives a plurality trained instances (final performance models) trained on disjoint sets (plurality) of data assets; paragraph [0015]).
As to dependent claim 8, Callcut further teaches identifying at least one perturbation in the multiple performance models (identifying high variance (perturbation) in the models (multiple final) during the validation stage; paragraph [0068]).
As to dependent claim 9, Callcut further teaches the active learning includes providing feedback to a data steward to generate more training data for the identified algorithm inputs (performance evaluation includes determining if termination criteria is not met (portions of low performance in the model) then the progress status (feedback) of the algorithm training process is reported back to the algorithm developer (data steward) with reporting constraints (variables) and undergoes additional training; paragraph [0107]).
As to dependent claim 10, Callcut further teaches performing training on the algorithm in response to the more training data (if termination criteria is not met then a model undergoes further training; paragraph [0107]).
As to independent claim 11, Callcut teaches
A computerized system for active algorithm training comprising:
a sequestered computing node residing within a data steward’s computing environment, wherein the sequestered computing node remains inaccessible by the data steward, the sequestered computing node configured to process a data set with the algorithm to generate an algorithm output, generate a performance model by… modeling the algorithm output, and route the performance model to an algorithm developer (computing framework includes nodes that make up the infrastructure used by the algorithm developer (data steward); paragraph [0008], [0016]. securing data against unauthorized access which would include unauthorized algorithm developers; paragraph [0005]. data processing system implemented within a secure computing framework to generate model outputs; Fig.1, paragraph [0018], [0054]. computing performance of each instance of the algorithm by minimizing a loss function; paragraph [0011], [0104]. training results are transmitted (routed) to the master algorithm module (algorithm developer); paragraph [0105], [0107]);
a server within the algorithm developer configured to identify surface regions of the performance model under a configured threshold, and identify algorithm inputs associated with the identified surface regions (model is determined to have been validated with a specific ground truth data (configured threshold) using area under a curve analysis is used to visualize the performance of the model; paragraph [0111]. features associated with area under a curve analysis are ranked according to validation criteria; paragraph [0111]); and
the data steward configured to perform active learning on the identified algorithm inputs (algorithm developer learns (active) model parameters are based on validation sets (inputs) for the models; paragraph [0110]).
Callcut does not appear to expressly teach regression modeling.
Giurgica-Tiron teaches regression modeling (the output layer of the model is a statistical model operated as a regression-based classifier; paragraph [0084], [0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 12, Callcut further teaches the performance model models at least one of algorithm accuracy, F1 score accuracy, precision, recall, dice score, ROC (receiver operator characteristic) curve/area, log loss, Jaccard index, error, R2 or by some combination thereof (evaluating the models performance includes the precision of the models; paragraph [0072]).
As to dependent claim 13, Giurgica-Tiron further teaches the regression modeling includes linear least squares, logistic regression, deep learning or some combination thereof (regression models uses deep neural network (learning) architecture; paragraph [0084], [0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 14, Giurgica-Tiron further teaches the secure computing node is further configured to smooth the performance model by identifying portions of the performance model which are highly variable (high variation outputs (raw performance model) may drastically change the state of the model so some type of smoothing may be used; paragraph [0054]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 15, Giurgica-Tiron further teaches the smoothing includes best fit transform, moving averages and application of filters, Loess smoothing, kernel smoothing, wavelets, splines or some combination thereof (applies filters to smooth out models; paragraph [0006]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the privacy-preserving computing of Callcut to include the AI techniques of Giurgica-Tiron to provide a more realistic representation (see Giurgica-Tiron at paragraph 3).
As to dependent claim 16, Callcut further teaches the smoothing weights the data points of the raw performance model by instances of the algorithm’s input variables (adjusts (smooths) predetermined weights of model instances according to the accuracy of the model in order to optimize the model; paragraph [0081]).
As to dependent claim 17, Callcut further teaches the algorithm developer receives multiple performance models from the algorithm operating on a plurality of data sets (master algorithm module receives a plurality trained instances (final performance models) trained on disjoint sets (plurality) of data assets; paragraph [0015]).
As to dependent claim 18, Callcut further teaches the server further identifies at least one perturbation in the multiple performance models (identifying high variance (perturbation) in the models (multiple final) during the validation stage; paragraph [0068]).
As to dependent claim 19, Callcut further teaches the active learning includes providing feedback to a data steward to generate more training data for the identified algorithm inputs (performance evaluation includes determining if termination criteria is not met (portions of low performance in the model) then the progress status (feedback) of the algorithm training process is reported back to the algorithm developer (data steward) with reporting constraints (variables) and undergoes additional training; paragraph [0107]).
As to dependent claim 20, Callcut further teaches the sequestered computing node is further configured to train the algorithm in response to the more training data (if termination criteria is not met then a model undergoes further training; paragraph [0107]).
Citation of Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Bursell et al. (U.S. Pat. App. Pub. No. 2022/0171883) teaches receiving, by a first computing device, a request from a second computing device to establish a set of trusted execution environments (TEEs) in the first computing device; establishing a first TEE of the set of TEEs in the first computing device, wherein the trusted execution environment comprises an encrypted memory area and executable code; receiving, by the first TEE, cryptographic key data from the first computing device; establishing, by the first TEE, a second TEE of the set of TEEs in the first computing device, wherein the second TEE comprises a copy of the executable code; providing, by the first TEE, the cryptographic key data to the second TEE; and causing the executable code of the second TEE to communicate with the first computing device using the cryptographic key data.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Casey R. Garner whose telephone number is 571-272-2467. The examiner can normally be reached Monday to Friday, 8am to 5pm, Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Casey R. Garner/Primary Examiner, Art Unit 2123
Prosecution Insights