Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or
composition of matter, or any new and useful improvement thereof, may obtain a patent
therefor, subject to the conditions and requirements of this title.
Claims 1-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an
abstract idea without significantly more.
1. A computer-implemented method for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the method comprising:
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat;
determining, for each of the multiple methods, a metric which quantifies a quality and/or informative value of the method;
determining, for each of the multiple methods, a respective offer for allocating computing resources for the execution of the method, depending on the metric for the method;
and allocating the computing resources to the multiple methods depending on the respective offers.
Claim 1 Step 1:
Claim 1 is directed to a method for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the method comprising: a series of steps, and is therefore directed to a process, which is one of the four statutory categories.
Claim 1 Step 2A, Prong One:
Limitation(s) 1 b,c,d can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process. 2. Claim 1 Step 2A, Prong Two:
Limitation(s) 1 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer, which is not indicative of integration into a practical application. See MPEP 2106.04(d) and 2106.0S(f).
Claim 1 Step 2B:
The combination of these elements amount to a method comprising steps which can be performed mentally on a computer. Therefore, these elements, when considered individually and in combination, fail to add an inventive concept to the claim. Consequently, claim 1 as a whole does not amount to significantly more than the recited judicial exceptions and the claim is not eligible.
2. The method according to claim 1,
wherein each respective offer is taken from an available budget, or a budget is determined depending on the respective metric and the offer is taken from the budget determined depending on the metric.
Limitation(s) 2 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
3. The method according to claim 1,
wherein each metric is determined depending on a number of errors found with the method or a number of lines of code analyzed with the method.
Limitation(s) 3 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
4. The method according to claim 3,
wherein the number of errors or number of lines of code is normalized using a normalization parameter and the metric represents the number normalized using the parameter.
Limitation(s) 4 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
5. The method according to claim 2,
the respective offer for each method is determined with a function parameterized with at least one parameter, which maps the metric to the respective offer depending on the at least one parameter or which maps the metric and the budget to the respective offer depending on the at least one parameter, wherein the at least one parameter is statically specified or is learned.
Limitation(s) 5 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
6. The method according to claim 5,
wherein the at least one parameter is learned depending on the metric.
Limitation(s) 6 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
7. The method according to claim 1,
characterized in that the metric and the respective offer are determined for each method in iterations and the computing resources are repeatedly allocated in iterations depending on the metrics or offers determined for the respective iteration.
Limitation(s) 7 a can be performed in the human mind through observation, evaluation, judgement and opinion, with the aid of pen and paper, and is/are therefore reciting a mental process.
An integration into a practical application is not substantiated from the above.
8. The method according to claim 1, wherein the computing resources include:
time slots of a computing time or memory areas of a memory or bandwidth of a data line.
Limitation(s) 8 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
9. The method according to claim 1,
wherein the computing resources are provided on a server.
Limitation(s) 9 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
10. The method according to claim 1,
wherein the computer-aided development environment is provided in the distributed development process on distributed devices.
Limitation(s) 10 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
11. The method according to claim 10,
wherein at least one of the multiple methods for identifying the damage and/or the threat is carried out on one of the distributed devices to which the at least one of the multiple methods is assigned.
Limitation(s) 11 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
12.
An apparatus for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the apparatus comprising: at least one processor; and at least one memory, wherein the at least one processor is configured to carry out instructions, upon execution of which by the at least one processor the apparatus carries out the the following steps:
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat,
determining, for each of the multiple methods, a metric which quantifies a quality and/or informative value of the method,
determining, for each of the multiple methods, a respective offer for allocating computing resources for the execution of the method, depending on the metric for the method,
and allocating the computing resources to the multiple methods depending on the respective offers;
and wherein the at least one memory stores the instructions.
Limitations 12 b-e correspond to those of claim 1 and therefore inherit the same rejections.
Limitation(s) 12 a,f provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
13.
A non-transitory medium on which is stored a computer program including computer-executable instructions for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the instructions, when executed by a computer, causing the compter to perform the following stepsskill:
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat;
determining, for each of the multiple methods, a metric which quantifies a quality and/or informative value of the method;
determining, for each of the multiple methods, a respective offer for allocating computing resources for the execution of the method, depending on the metric for the method;
and allocating the computing resources to the multiple methods depending on the respective offers.
Limitations 13 b-e correspond to those of claim 1 and therefore inherit the same rejections.
Limitation(s) 13 a provide(s) mere instructions to implement the limitations which can be performed in the human mind, i.e., the judicial exception, on a computer.
An integration into a practical application is not substantiated from the above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20150379427 A1, Dirac, 2015-12-31.
1. A computer-implemented method for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the method comprising:
Dirac teaches the following.
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat;
[0080] the MLS (machine learning service) control plane may be responsible for generating processing plans corresponding to each of the job objects generated in response to client requests in at least some embodiments.
[0081] In the illustrated scenario, MLS job queue 142 comprises five jobs (…) Corresponding to job J1, an input data cleansing plan 422 may be generated, and the plan may be executed using resource set RS1. The input data cleansing plan may include operations to read and validate the contents of a specified data source, fill in missing values, identify and discard (or otherwise respond to) input records containing errors, and so on.
[0234] Best practices developed over years of experience with different data cleansing approaches, transformation types, parameter settings for transformations as well as models may be incorporated into the programmatic interfaces (such as easy-to learn and easy-to-use APIs) of the MLS,
determining, (...), a metric which quantifies a quality and/or informative value (…);
[0205] A variety of quality metrics 4012 may be determined in different embodiments and for different types of models,
for each of the multiple methods
[0234] Best practices developed over years of experience with different data cleansing approaches, transformation types, parameter settings for transformations as well as models may be incorporated into the programmatic interfaces (such as easy-to learn and easy-to-use APIs) of the MLS,
determining, for (…), a respective offer for allocating computing resources for the execution (…),
depending on the metric for (…);
[0524] one or more computing devices configured to: determine, via one or more programmatic interactions with a client of a machine learning service of a provider network, (a) one or more target variables to be predicted using a specified training data set, (b) one or more prediction quality metrics including a particular prediction quality metric, and (c) one or more prediction run-time goals including a particular prediction run-time goal; identify a set of candidate feature processing transformations to derive a first set of processed variables from one or more input variables of the specified data set, wherein at least a subset of the first set of processed variables is usable to train a machine learning model to predict the one or more target variables, and wherein the set of candidate feature processing transformations includes a particular feature processing transformation; determine (a) a quality estimate indicative of an effect, on the particular prediction quality metric, of implementing the particular candidate feature processing transformation, and (b) a cost estimate indicative of an effect, on a particular run-time performance metric associated with the particular prediction run-time goal, of implementing the particular candidate feature processing transformation; generate, based at least in part on the quality estimate and at least in part on the cost estimate, a feature processing proposal to be provided to the client for approval, wherein the feature processing proposal includes a recommendation to implement the particular feature processing transformation; and in response to an indication of approval from the client, execute a machine learning model trained using a particular processed variable obtained from the particular feature processing transformation.
and allocating the computing resources to (…) depending on the respective offers.
[0524] one or more computing devices configured to: determine, via one or more programmatic interactions with a client of a machine learning service of a provider network, (a) one or more target variables to be predicted using a specified training data set, (b) one or more prediction quality metrics including a particular prediction quality metric, and (c) one or more prediction run-time goals including a particular prediction run-time goal; identify a set of candidate feature processing transformations to derive a first set of processed variables from one or more input variables of the specified data set, wherein at least a subset of the first set of processed variables is usable to train a machine learning model to predict the one or more target variables, and wherein the set of candidate feature processing transformations includes a particular feature processing transformation; determine (a) a quality estimate indicative of an effect, on the particular prediction quality metric, of implementing the particular candidate feature processing transformation, and (b) a cost estimate indicative of an effect, on a particular run-time performance metric associated with the particular prediction run-time goal, of implementing the particular candidate feature processing transformation; generate, based at least in part on the quality estimate and at least in part on the cost estimate, a feature processing proposal to be provided to the client for approval, wherein the feature processing proposal includes a recommendation to implement the particular feature processing transformation; and in response to an indication of approval from the client, execute a machine learning model trained using a particular processed variable obtained from the particular feature processing transformation.
Executing a model to carry out some function would necessarily entail allocating computing resources.
Dirac comprises each of the above elements as the present application. And while Dirac does not explicitly describe a method being selected among a plurality of methods, it does identify that a plurality of methods may be incorporated into the MLS (machine learning service). Given this, it would have been obvious to one skilled in the art at the time of this application’s filing to modify the design of Dirac such that multiple methods would be considered at the same time.
2. The method according to claim 1,
Dirac teaches the following.
wherein each respective offer is taken from an available budget, or a budget is determined depending on the respective metric and the offer is taken from the budget determined depending on the metric.
[0205] The goals 4016 may include elapsed time goals for producing predictions on a data set of a specified size, goals for an amount of memory not to be exceeded when making such predictions, budget goals regarding the maximum billing costs per prediction, and so on. In some embodiments, the FP manager may also be provided with a set of training phase goals, such as the maximum amount of time to be consumed to train the model, a budget not to be exceeded for training the model, or a time or budget limit for the MLS to provide a feature processing proposal to the client.
3. The method according to claim 1,
Dirac teaches the following.
wherein each metric is determined depending on a number of errors found with the method or a number of lines of code analyzed with the method.
[0226] Any combination of a variety of prediction quality metrics may be identified by the MLS component for different types of machine learning problems, such as an AUC (area under curve) metric, an accuracy metric, a recall metric, a sensitivity metric, a true positive rate, a specificity metric, a true negative rate, a precision metric, a false positive rate, a false negative rate, an F1 score, a coverage metric, an absolute percentage error metric, or a squared error metric.
4. The method according to claim 3,
Dirac teaches the following.
wherein the number of errors or number of lines of code is normalized using a normalization parameter and the metric represents the number normalized using the parameter.
[0226] Any combination of a variety of prediction quality metrics may be identified by the MLS component for different types of machine learning problems, such as an AUC (area under curve) metric, an accuracy metric, a recall metric, a sensitivity metric, a true positive rate, a specificity metric, a true negative rate, a precision metric, a false positive rate, a false negative rate, an F1 score, a coverage metric, an absolute percentage error metric, or a squared error metric.
Within the context of the art, normalized metrics are those whose values are scaled to a range such as 0 to 100. As such, the above values which correspond to ratios qualify as normalized.
5. The method according to claim 2,
Dirac teaches the following.
the respective offer for each method is determined with a function parameterized with at least one parameter, which maps the metric to the respective offer depending on the at least one parameter
or which maps the metric and the budget to the respective offer depending on the at least one parameter,
[0524] one or more computing devices configured to: determine, via one or more programmatic interactions with a client of a machine learning service of a provider network, (a) one or more target variables to be predicted using a specified training data set, (b) one or more prediction quality metrics including a particular prediction quality metric, and (c) one or more prediction run-time goals including a particular prediction run-time goal; identify a set of candidate feature processing transformations to derive a first set of processed variables from one or more input variables of the specified data set, wherein at least a subset of the first set of processed variables is usable to train a machine learning model to predict the one or more target variables, and wherein the set of candidate feature processing transformations includes a particular feature processing transformation; determine (a) a quality estimate indicative of an effect, on the particular prediction quality metric, of implementing the particular candidate feature processing transformation, and (b) a cost estimate indicative of an effect, on a particular run-time performance metric associated with the particular prediction run-time goal, of implementing the particular candidate feature processing transformation; generate, based at least in part on the quality estimate and at least in part on the cost estimate, a feature processing proposal to be provided to the client for approval, wherein the feature processing proposal includes a recommendation to implement the particular feature processing transformation; and in response to an indication of approval from the client, execute a machine learning model trained using a particular processed variable obtained from the particular feature processing transformation.
[0533] The system as recited in any of clauses 1-4, wherein the particular feature processing transformation comprises a use of one or more of: (a) a quantile bin function, (b) a Cartesian product function, (c) a bi-gram function, (d) an n-gram function, (e) an orthogonal sparse bigram function, (f) a calendar function, (g) an image processing function, (h) an audio processing function, (i) a bio-informatics processing function, or (j) a natural language processing function.
Each of the above functions require inputs to operate, that is to say they are all parameterized with at least parameter.
wherein the at least one parameter is statically specified or is learned.
[0058] The interfaces may, for example, allow non-experts to rely on default settings or parameters for various aspects of the procedures used for building, training and using machine learning models, where the defaults are derived from the accumulated experience of other practitioners addressing similar types of machine learning problems.
6. The method according to claim 5,
Dirac teaches the following.
wherein the at least one parameter is learned depending on the metric.
[0112] Based on the collected metrics and/or feedback, respective sets of best practices for various phases of machine learning workflows may be identified (element 1057). Some of the best practices may be specific to particular problem domains, while others may be more generally applicable, and may therefore be used across problem domains. Representations or summaries of the best practices identified may be stored in a knowledge base of the MLS. Access (e.g., via a browser or a search tool) to the knowledge base may be provided to MLS users (element 1060). The MLS may also incorporate the best practices into the programmatic interfaces exposed to users—e.g., by introducing new APIs that are more likely to lead users to utilize best practices, by selecting default parameters based on best practices,
7. The method according to claim 1,
Dirac teaches the following.
characterized in that the metric and the respective offer are determined for each method in iterations and the computing resources are repeatedly allocated in iterations depending on the metrics or offers determined for the respective iteration.
[0095] In some embodiments, the MLS may support recurring scheduling of related jobs.
8. The method according to claim 1, wherein the computing resources include:
Dirac teaches the following.
time slots of a computing time or memory areas of a memory or bandwidth of a data line.
[0422] one or more computing devices configured to: generate consistency metadata to be used for one or more training-and-evaluation iterations of a machine learning model, wherein the consistency metadata comprises at least a particular initialization parameter value for a pseudo-random number source; sub-divide an address space of a particular data set of the machine learning model into a plurality of chunks, including a first chunk comprising a first plurality of observation records, and a second chunk comprising a second plurality of observation records; retrieve, from one or more persistent storage devices, observation records of the first chunk into a memory of a first server, and observation records of the second chunk into a memory of a second server,
9. The method according to claim 1,
Dirac teaches the following.
wherein the computing resources are provided on a server.
[0422] one or more computing devices configured to: generate consistency metadata to be used for one or more training-and-evaluation iterations of a machine learning model, wherein the consistency metadata comprises at least a particular initialization parameter value for a pseudo-random number source; sub-divide an address space of a particular data set of the machine learning model into a plurality of chunks, including a first chunk comprising a first plurality of observation records, and a second chunk comprising a second plurality of observation records; retrieve, from one or more persistent storage devices, observation records of the first chunk into a memory of a first server, and observation records of the second chunk into a memory of a second server,
10. The method according to claim 1,
Dirac teaches the following.
wherein the computer-aided development environment is provided in the distributed development process on distributed devices.
[0422] one or more computing devices configured to: generate consistency metadata to be used for one or more training-and-evaluation iterations of a machine learning model, wherein the consistency metadata comprises at least a particular initialization parameter value for a pseudo-random number source; sub-divide an address space of a particular data set of the machine learning model into a plurality of chunks, including a first chunk comprising a first plurality of observation records, and a second chunk comprising a second plurality of observation records; retrieve, from one or more persistent storage devices, observation records of the first chunk into a memory of a first server, and observation records of the second chunk into a memory of a second server,
It is implicit from the description of multiple servers working together that the environment is a distributed.
11. The method according to claim 10,
Dirac teaches the following.
wherein at least one of the multiple methods for identifying the damage and/or the threat is carried out on one of the distributed devices to which the at least one of the multiple methods is assigned.
[0074] A job scheduler 272 may retrieve a job from queue 142, e.g., after checking that the job's dependency requirements have been met, and identify one or more servers 262 from computing service 258 to execute the job's computational operations.
The following claim corresponds to claim 1 and therefore inherits the same rejection.
12. An apparatus for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats,
the apparatus comprising: at least one processor; and at least one memory, wherein the at least one processor is configured to carry out instructions, upon execution of which by the at least one processor the apparatus carries out the the following steps:
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat,
determining, for each of the multiple methods, a metric which quantifies a quality and/or informative value of the method,
determining, for each of the multiple methods, a respective offer for allocating computing resources for the execution of the method, depending on the metric for the method,
and allocating the computing resources to the multiple methods depending on the respective offers; and wherein the at least one memory stores the instructions.
The following claim corresponds to claim 1 and therefore inherits the same rejection.
13.
A non-transitory medium on which is stored a computer program including computer-executable instructions for allocating computing resources in a method for protecting a computer-aided development environment in a distributed development process from damage and threats, the instructions, when executed by a computer, causing the compter to perform the following stepsskill:
carrying out in the computer-aided development environment multiple methods for identifying damage and/or a threat;
determining, for each of the multiple methods, a metric which quantifies a quality and/or informative value of the method;
determining, for each of the multiple methods, a respective offer for allocating computing resources for the execution of the method, depending on the metric for the method;
and allocating the computing resources to the multiple methods depending on the respective offers.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Luke Absher whose telephone number is (571) 270-1057. The examiner can normally be reached M-F: 8:00 am - 4:00 pm. Examiner interviews are available via telephone, in person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the US PTO Automated Interview Request (AIR) at http:/ /www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Kevin Young can be reached at 571-270-3180.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https:/ /patentcenter.uspto.gov. Visit https:/ /www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786 9199 (IN USA OR CANADA) or 571-272-1000.
/LUCAS DONALD ABSHER/
Examiner, Art Unit 2194
/KEVIN L YOUNG/Supervisory Patent Examiner, Art Unit 2194
Prosecution Insights