DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This action is in response to the following communication: Amendment to application No. 16/537,215 filed on 09/08/2025.
3. Claims 1, 4-11 and 17-20 have been amended.
Claims 1-20 now remain pending.
Claims 1, 11 and 20 are independent claims.
Claim Objections
4: Claims 8 is objected to because of the following informalities:
Claim 8 grammatical error on line 3, “updating the of the one or more…”, examiner suggest using “updating
Appropriate correction is required.
Response to Arguments
5. Applicant’s arguments with respect to newly amended independent claims 1, 11 and 20 claims 2-10 and 12-19 on pages 7-12 of the response have been fully considered but they are not persuasive and are moot in view of the new ground(s) of rejection - see Gold (Art of record) and Hopkins (Art newly made of record) as applied below, as they further teach such use.
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1, 4, 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gold et al., US 2019/0121889 (hereinafter Gold) in view of Hopkins et al., U.S. Patent No. 10,735,299 (hereinafter Hopkins).
In regards to claim 1, Gold teaches:
A method, comprising: determining that a second version of one or more neural networks executed using a first set of hardware compute resources exceeds an improvement benchmark compared to a first version of the one or more neural networks executed using the first set of hardware compute resources (p. 51, [0371], see FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment… providing a recommendation that a particular user should switch from one machine learning model to another machine learning model in their production environment, by automatically rolling back to a previous version of a machine learning model if it is determined to be preferred over a subsequent version of the machine learning model, and so on. In the example method depicted in FIG. 29, identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models can include evaluating (2904) one or more machine learning models utilizing a predetermined model test. The predetermined model test may be embodied, for example, as an A/B testing model in which two machine learning models are compared) (emphasis added).
deploying the second version of the one or more neural networks to one or more web-based service clients based, at least in part, on the determination that the second version exceeds the improvement benchmark (p. 18, [0149], see exploring parameters and models, quickly testing with a smaller dataset, and iterating to converge on the most promising models to push into the production cluster), (p. 51, [0371], see the example method depicted in FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment), (p. 11, [0107], see storage clusters are connected to clients using Ethernet or fiber channel in some embodiments. If multiple storage clusters are configured into a storage grid, the multiple storage clusters are connected using the Internet) and (p. 14, [0128], see cloud services provider 302 may be configured to provide a variety of services to the storage system 306 and users of the storage system 306 … the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of an infrastructure as a service (‘IaaS’) service model where the cloud services provider 302 offers computing infrastructure such as virtual machines and other resources as a service to subscribers. In addition, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a platform as a service (‘PaaS’) service model where the cloud services provider 302 offers a development environment to application developers. Such a development environment may include, for example, an operating system, programming-language execution environment, database, web server, or other components that may be utilized by application developers to develop and run software solutions on a cloud platform. Furthermore, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a software as a service (‘SaaS’) service model where the cloud services provider 302 offers application software, databases, as well as the platforms that are used to run the applications to the storage system 306 and users of the storage system 306, providing the storage system 306 and users of the storage system 306 with on-demand software and eliminating the need to install and run the application on local computers) (emphasis added). It is noted that production clusters offered by cloud services providers connected to clients using Ethernet and lass and Paas service models offering execution environment, database, web server to clients is very much the same as such a web-based service.
Gold doesn’t explicitly teach:
and on a determination that the one or more web-based service clients are executed using a second set of hardware compute resources that have second one or more processors corresponding to the first set of hardware compute resources.
However, Hopkins teaches such use: (Abstract, see a mapping of application operation classifications to server characteristics suited to the application operation classifications is maintained. Multiple servers currently available to process at least a portion of the client software application may be monitored, and each of the multiple servers may be characterized according to their performance and resources. The classifications of operations of the analyzed application may be compared to the characteristics of the multiple servers currently available using the mapping, and a server may be selected based on the comparison), (column 15, lines 11-17, see the components of the computer system/server 512 may include, but are not limited to, one or more processors or processing units 516, a system memory 528, and a bus 518 that couples various system components including system memory 528 to processor 516), (column 2, lines 65-67, see FIG. 5 is a block diagram of an embodiment of a computer system or cloud server in which one or more aspects of the present invention may be implemented) and (column 21, lines 27-36, see Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings).
Gold and Hopkins are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold and Hopkins before him or her, to modify the system of Gold to include the teachings of Hopkins, as a system for management of client connections, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to handle client application with uniform performance, as suggested by Hopkins (column 21, lines 27-36, column 23, lines 26-28).
In regards to claim 4, Gold teaches:
the deployed second version of the one or more neural networks meet higher improvement benchmark thresholds than one or more neural networks already used by the one or more web-based service client (p. 51, [0371], see FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment… providing a recommendation that a particular user should switch from one machine learning model to another machine learning model in their production environment, by automatically rolling back to a previous version of a machine learning model if it is determined to be preferred over a subsequent version of the machine learning model, and so on. In the example method depicted in FIG. 29, identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models can include evaluating (2904) one or more machine learning models utilizing a predetermined model test. The predetermined model test may be embodied, for example, as an A/B testing model in which two machine learning models are compared).
In regards to claim 11, Gold teaches:
A non-transitory computer-readable medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: determining that a second version of one or more neural networks executed using a first set of hardware compute resources exceeds an improvement benchmark compared to a first version of the one or more neural networks executed using the first set of hardware compute resources (p. 51, [0371], see FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment… providing a recommendation that a particular user should switch from one machine learning model to another machine learning model in their production environment, by automatically rolling back to a previous version of a machine learning model if it is determined to be preferred over a subsequent version of the machine learning model, and so on. In the example method depicted in FIG. 29, identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models can include evaluating (2904) one or more machine learning models utilizing a predetermined model test. The predetermined model test may be embodied, for example, as an A/B testing model in which two machine learning models are compared) (emphasis added).
deploying the second version of the one or more neural networks to one or more web-based service clients based, at least in part, on the determination that the second version exceeds the improvement benchmark (p. 18, [0149], see exploring parameters and models, quickly testing with a smaller dataset, and iterating to converge on the most promising models to push into the production cluster), (p. 51, [0371], see the example method depicted in FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment), (p. 11, [0107], see storage clusters are connected to clients using Ethernet or fiber channel in some embodiments. If multiple storage clusters are configured into a storage grid, the multiple storage clusters are connected using the Internet) and (p. 14, [0128], see cloud services provider 302 may be configured to provide a variety of services to the storage system 306 and users of the storage system 306 … the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of an infrastructure as a service (‘IaaS’) service model where the cloud services provider 302 offers computing infrastructure such as virtual machines and other resources as a service to subscribers. In addition, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a platform as a service (‘PaaS’) service model where the cloud services provider 302 offers a development environment to application developers. Such a development environment may include, for example, an operating system, programming-language execution environment, database, web server, or other components that may be utilized by application developers to develop and run software solutions on a cloud platform. Furthermore, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a software as a service (‘SaaS’) service model where the cloud services provider 302 offers application software, databases, as well as the platforms that are used to run the applications to the storage system 306 and users of the storage system 306, providing the storage system 306 and users of the storage system 306 with on-demand software and eliminating the need to install and run the application on local computers) (emphasis added). It is noted that production clusters offered by cloud services providers connected to clients using Ethernet and lass and Paas service models offering execution environment, database, web server to clients is very much the same as such a web-based service.
Gold doesn’t explicitly teach:
and on a determination that the one or more web-based service clients are executed using a second set of hardware compute resources that have second one or more processors corresponding to the first set of hardware compute resources.
However, Hopkins teaches such use: (Abstract, see a mapping of application operation classifications to server characteristics suited to the application operation classifications is maintained. Multiple servers currently available to process at least a portion of the client software application may be monitored, and each of the multiple servers may be characterized according to their performance and resources. The classifications of operations of the analyzed application may be compared to the characteristics of the multiple servers currently available using the mapping, and a server may be selected based on the comparison), (column 15, lines 11-17, see the components of the computer system/server 512 may include, but are not limited to, one or more processors or processing units 516, a system memory 528, and a bus 518 that couples various system components including system memory 528 to processor 516), (column 2, lines 65-67, see FIG. 5 is a block diagram of an embodiment of a computer system or cloud server in which one or more aspects of the present invention may be implemented) and (column 21, lines 27-36, see Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings).
Gold and Hopkins are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold and Hopkins before him or her, to modify the system of Gold to include the teachings of Hopkins, as a system for management of client connections, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to handle client application with uniform performance, as suggested by Hopkins (column 21, lines 27-36, column 23, lines 26-28).
In regards to claim 20, Gold teaches:
A system, comprising: a memory storing instructions; and one or more processors that execute the instructions to perform a method comprising: determining that a second version of one or more neural networks executed using a first set of hardware compute resources exceeds an improvement benchmark compared to a first version of the one or more neural networks executed using the first set of hardware compute resources (p. 51, [0371], see FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment… providing a recommendation that a particular user should switch from one machine learning model to another machine learning model in their production environment, by automatically rolling back to a previous version of a machine learning model if it is determined to be preferred over a subsequent version of the machine learning model, and so on. In the example method depicted in FIG. 29, identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models can include evaluating (2904) one or more machine learning models utilizing a predetermined model test. The predetermined model test may be embodied, for example, as an A/B testing model in which two machine learning models are compared) (emphasis added).
deploying the second version of the one or more neural networks to one or more web-based service clients based, at least in part, on the determination that the second version exceeds the improvement benchmark (p. 18, [0149], see exploring parameters and models, quickly testing with a smaller dataset, and iterating to converge on the most promising models to push into the production cluster), (p. 51, [0371], see the example method depicted in FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment), (p. 11, [0107], see storage clusters are connected to clients using Ethernet or fiber channel in some embodiments. If multiple storage clusters are configured into a storage grid, the multiple storage clusters are connected using the Internet) and (p. 14, [0128], see cloud services provider 302 may be configured to provide a variety of services to the storage system 306 and users of the storage system 306 … the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of an infrastructure as a service (‘IaaS’) service model where the cloud services provider 302 offers computing infrastructure such as virtual machines and other resources as a service to subscribers. In addition, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a platform as a service (‘PaaS’) service model where the cloud services provider 302 offers a development environment to application developers. Such a development environment may include, for example, an operating system, programming-language execution environment, database, web server, or other components that may be utilized by application developers to develop and run software solutions on a cloud platform. Furthermore, the cloud services provider 302 may be configured to provide services to the storage system 306 and users of the storage system 306 through the implementation of a software as a service (‘SaaS’) service model where the cloud services provider 302 offers application software, databases, as well as the platforms that are used to run the applications to the storage system 306 and users of the storage system 306, providing the storage system 306 and users of the storage system 306 with on-demand software and eliminating the need to install and run the application on local computers) (emphasis added). It is noted that production clusters offered by cloud services providers connected to clients using Ethernet and lass and Paas service models offering execution environment, database, web server to clients is very much the same as such a web-based service.
Gold doesn’t explicitly teach:
and on a determination that the one or more web-based service clients are executed using a second set of hardware compute resources that have second one or more processors corresponding to the first set of hardware compute resources.
However, Hopkins teaches such use: (Abstract, see a mapping of application operation classifications to server characteristics suited to the application operation classifications is maintained. Multiple servers currently available to process at least a portion of the client software application may be monitored, and each of the multiple servers may be characterized according to their performance and resources. The classifications of operations of the analyzed application may be compared to the characteristics of the multiple servers currently available using the mapping, and a server may be selected based on the comparison), (column 15, lines 11-17, see the components of the computer system/server 512 may include, but are not limited to, one or more processors or processing units 516, a system memory 528, and a bus 518 that couples various system components including system memory 528 to processor 516), (column 2, lines 65-67, see FIG. 5 is a block diagram of an embodiment of a computer system or cloud server in which one or more aspects of the present invention may be implemented) and (column 21, lines 27-36, see Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings).
Gold and Hopkins are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold and Hopkins before him or her, to modify the system of Gold to include the teachings of Hopkins, as a system for management of client connections, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to handle client application with uniform performance, as suggested by Hopkins (column 21, lines 27-36, column 23, lines 26-28).
8. Claims 2, 3, 12, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Gold in view of Hopkins in view of Brueckner et al., US 2016/0078361 (hereinafter Brueckner).
In regards to claims 1 and 11, the rejections above are incorporated respectively.
In regards to claim 2, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the one or more neural networks are to be used to perform inferencing operations and to output inferenced data to a software application installed on a client computing device.
However, Brueckner teaches such use: (Fig. 1, see process arrow flow from Raw data 130, Input record handlers 160, Feature Processors 162, ML Algorithm implementations 166, Workload distribution 175, server pools 185, MlS artifact repository 120, Clients 164, Machine learning service 189) and (p. 6, [0064}, see the output 116 of the feature processing transformations may in turn be used as input for a selected machine learning algorithm 166, which may be executed in accordance with algorithm parameters 154 using yet another set of resources from pool 185. A wide variety of machine learning algorithms may be supported natively by the MLS libraries, including for example random forest algorithms, neural network algorithms, stochastic gradient descent algorithms, and the like).
Gold, Hopkins and Brueckner are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Brueckner before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Brueckner, as a system for training machine learning models, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data transformations to apply to data input, , as suggested by Brueckner (p. 6, [0064}, p 50, [0562-0564 ).
In regards to claim 3, Gold teaches:
Improvement benchmark is indicative of performance of the one or more neural networks (p. 51, [0371], see FIG. 29 also includes identifying (2902), from amongst a plurality of machine learning models, a preferred machine learning model. Identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models may be carried out, for example, by comparing a plurality of machine learning models to identify which machine learning model performed the best relative to a predetermined set of criteria (e.g., most accurate, quickest time to achieve a particular accuracy threshold, and so on. In such an example, a plurality of metrics may be used in a weighted or unweighted fashion to identify (2902) the preferred machine learning model. In such an example, the artificial intelligence infrastructure (2402) may be configured to use such information, for example, by automatically pushing the preferred machine learning model into a production environment… providing a recommendation that a particular user should switch from one machine learning model to another machine learning model in their production environment, by automatically rolling back to a previous version of a machine learning model if it is determined to be preferred over a subsequent version of the machine learning model, and so on. In the example method depicted in FIG. 29, identifying (2902) a preferred machine learning model from amongst a plurality of machine learning models can include evaluating (2904) one or more machine learning models utilizing a predetermined model test. The predetermined model test may be embodied, for example, as an A/B testing model in which two machine learning models are compared).
In regards to claim 12, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the one or more neural networks and a software application using the one or more neural networks are installed on a client computing device.
However, Brueckner teaches such use: (Fig. 1, see process arrow flow from Raw data 130, Input record handlers 160, Feature Processors 162, ML Algorithm implementations 166, Workload distribution 175, server pools 185, MlS artifact repository 120, Clients 164, Machine learning service 189) and (p. 6, [0064}, see the output 116 of the feature processing transformations may in turn be used as input for a selected machine learning algorithm 166, which may be executed in accordance with algorithm parameters 154 using yet another set of resources from pool 185. A wide variety of machine learning algorithms may be supported natively by the MLS libraries, including for example random forest algorithms, neural network algorithms, stochastic gradient descent algorithms, and the like).
Gold, Hopkins and Brueckner are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Brueckner before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Brueckner, as a system for training machine learning models, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data transformations to apply to data input, , as suggested by Brueckner (p. 6, [0064}, p 50, [0562-0564 ).
In regards to claim 17, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the one or more neural networks perform inferencing operations and provide inferenced data to a software application by at least: providing input data by the software application to the one or more neural networks, processing the input data by the one or more neural networks to generate the inferenced data, and outputting the inferenced data to the software application.
However, Brueckner teaches such use: (Fig. 1, see process arrow flow from Raw data 130, Input record handlers 160, Feature Processors 162, ML Algorithm implementations 166, Workload distribution 175, server pools 185, MlS artifact repository 120, Clients 164, Machine learning service 189) and (p. 6, [0064}, see the output 116 of the feature processing transformations may in turn be used as input for a selected machine learning algorithm 166, which may be executed in accordance with algorithm parameters 154 using yet another set of resources from pool 185. A wide variety of machine learning algorithms may be supported natively by the MLS libraries, including for example random forest algorithms, neural network algorithms, stochastic gradient descent algorithms, and the like).
Gold, Hopkins and Brueckner are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Brueckner before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Brueckner, as a system for training machine learning models, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data transformations to apply to data input, , as suggested by Brueckner (p. 6, [0064}, p 50, [0562-0564 ).
9. Claims 5-10, 15-16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Gold in view of Hopkins in view of Novielli et al., U.S. 2019/0278870 (hereinafter Novielli).
In regards to claims 1 and 11 the rejections above are incorporated respectively.
In regards to claim 5, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
further comprising generating the second version of the one or more neural networks by at least retraining the one or more neural networks.
However, Novielli teaches such use: (p. 3, [0050], see due to the number of queries processed each second by the search system 104 and the amount of feedback received by the machine learning model creation system 106, a sufficient amount of data to retrain the model can be received very quickly. Thus, the machine learning model can be retrained, a new model created… In one embodiment, the trigger event can be time so that the machine learning model is updated (i.e., an existing model retrained or a new model created) on a periodic or a peridoc schedule. In another embodiment, the trigger even can be machine learning model performance so that the machine learning model is updated when the performance falls below a threshold).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 6, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
automatically distributing the one or more updates to the one or more neural networks by at least distributing one or more updated parameters the one or more neural networks to a client computing device.
However, Novielli teaches such use: (p. 3, [0050] in yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used) and (p. 4, [0053], see the model creation process 122 can distribute a whole new model, updated coefficients for an existing model, or both… Coefficients in this context are the weights or other parameters that turn an untrained learning model into a trained learning model) and (p. 4, [0055], see various machine learning models that receive an input list and output an output list can be used in embodiments of the current disclosure … appropriate models are those that are able to classify and/or rank a list of input items, including supervised, unsupervised and semi-supervised models. Various models fall into the categories such as deep learning models, ensemble models, neural networks) (emphasis added).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 7, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the automatically distributing the one or more updates to the one or more neural networks comprises distributing one or more updated hyperparameter adjustments for the one or more neural networks to the client computing device.
However, Novielli teaches such use: (p. 3, [0050] in yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used) and (p. 4, [0053], see the model creation process 122 can distribute a whole new model, updated coefficients for an existing model, or both… Coefficients in this context are the weights or other parameters that turn an untrained learning model into a trained learning model).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 8, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the automatically distributing the one or more updates to the one or more neural networks comprises updating the one or more neural networks of the client computing device with at least one of a layer substitution, a layer fusing, or input stacking.
However, Novielli teaches such use: (p. 3, [0050] in yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 9, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
generating the second version of the one or more neural networks wherein automatically.
However, Novielli teaches such use: (p. 3, [0050] in yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used) and (p. 4, [0055], see various machine learning models that receive an input list and output an output list can be used in embodiments of the current disclosure… appropriate models are those that are able to classify and/or rank a list of input items, including supervised, unsupervised and semi-supervised models. Various models fall into the categories such as deep learning models, ensemble models, neural networks) (emphasis added).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 10, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
automatically distributing second version the one or more neural networks to the one or more web-based service client when an updated version of the one or more neural networks meets or exceeds one or more thresholds for improvement relating to at least one of accuracy, quality, or performance.
However, Novielli teaches such use: (p. 3, [0050] In another embodiment, the trigger even can be machine learning model performance so that the machine learning model is updated when the performance falls below a threshold such as when the number or rate of incorrect prefetches exceeds a threshold and/or the number or rate of correct prefetches falls below a threshold. In yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 15, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
a software application is a voice recognition application that uses the one or more neural networks.
However, Novielli teaches such use: (p. 4, [0063], see input is received in operation 306. As noted above, the input can be in any format such as text, voice, gesture, and so forth).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 16, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the one or more neural networks generates inferenced data that includes one or more audio-related inferences, the one or more audio-related inferences being at least one of a language translation or a voice recognized command.
However, Novielli teaches such use: (p. 4, [0063], see input is received in operation 306. As noted above, the input can be in any format such as text, voice, gesture, and so forth).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
In regards to claim 18, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
the second version of the one or more neural networks is received responsive to a trigger.
However, Novielli teaches such use: (p. 3, [0050] in yet another embodiment, performance of a test machine learning model triggers updating the machine learning model to a wider group of users (see the A/B testing discussion below). Thus, when a test machine learning model (i.e., model A) performance exceeds another deployed model (i.e., model B), the A model can be deployed to replace the B model. In other embodiments a combination of trigger events can be used) and (p. 4, [0055], see various machine learning models that receive an input list and output an output list can be used in embodiments of the current disclosure… appropriate models are those that are able to classify and/or rank a list of input items, including supervised, unsupervised and semi-supervised models. Various models fall into the categories such as deep learning models, ensemble models, neural networks) (emphasis added).
Gold, Hopkins and Novielli are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Novielli before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Novielli, as a system for loading search results using machine learning, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize data collected from user devices as feedback to train machine learning models, as suggested by Novielli (p. 3, [0050], p. 11, [0209]).
10. Claims 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Gold in view of Hopkins in view of Brueckner in view of Skuin et al., U.S. Patent No. 10,7135,43 (hereinafter Skuin).
In regards to claim 11 the rejections above are incorporated respectively.
In regards to claim 13, Gold, Hopkins and Brueckner, in particular Gold doesn’t explicitly teach:
the software application is a video game.
However, Skuin teaches such use: (column 11, lines 12-21, see to learn detail of these video game characters it may be beneficial for the machine learning models to have access to high resolution and/or large versions of the characters. Thus, the training data 112 may include the particular video game characters rendered at different scales. Additionally, the different scales may be beneficial as real-world gameplay (e.g., a broadcast of a hockey game) may include images captured by television cameras at different zoom levels).
Gold, Hopkins, Brueckner and Skuin are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins, Brueckner and Skuin before him or her, to modify the system of Gold, Hopkins and Brueckner, in particular Gold to include the teachings of Skuin, as a system for machine training, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize machine learning with a video game, as suggested by Skuin (column 11, lines 12-21, p. 29, lines 41-46).
In regards to claim 14, Gold, Hopkins and Brueckner, in particular Gold doesn’t explicitly teach:
the one or more neural networks are used to generate inferenced data that includes one or more image-related inferences, the one or more image-related inferences being at least one of an anti-aliased image, an images with upscaled resolution, or a denoised image.
However, Skuin teaches such use: (column 11, lines 12-21, see to learn detail of these video game characters it may be beneficial for the machine learning models to have access to high resolution and/or large versions of the characters. Thus, the training data 112 may include the particular video game characters rendered at different scales. Additionally, the different scales may be beneficial as real-world gameplay (e.g., a broadcast of a hockey game) may include images captured by television cameras at different zoom levels).
Gold, Hopkins, Brueckner and Skuin are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins, Brueckner and Skuin before him or her, to modify the system of Gold, Hopkins and Brueckner, in particular Gold to include the teachings of Skuin, as a system for machine training, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to utilize machine learning with a video game, as suggested by Skuin (column 11, lines 12-21, p. 29, lines 41-46).
11. Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Gold in view of Hopkins in view of Palladino et al., U.S. 2019/0037005 (hereinafter Palladino).
In regards to claim 11 the rejections above are incorporated respectively.
In regards to claim 19, Gold and Hopkins, in particular Gold doesn’t explicitly teach:
request is triggered as a result of a call to a feature application programming interface (API) that causes execution of the one or more neural networks.
However, Palladino teaches such use: (p. 10, 2nd column, 1st para., see the system of claim 17, wherein the executed auto-documentation code is further configured to update the machine learning model based on the content of the API request or response as observed).
Gold, Hopkins and Palladino are analogous art because they are from the same field of endeavor, AI model updates.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teaching of Gold, Hopkins and Palladino before him or her, to modify the system of Gold and Hopkins, in particular Gold to include the teachings of Palladino, as a system for processing machine learning API request, and accordingly it would enhance the system of Gold, which is focused on ensuring reproducibility in an artificial intelligence infrastructure, because that would provide Gold with the ability to update a machine learning model, as suggested by Palladino (p. 10, 2nd column, p. 9, [0093]).
Conclusion
12. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Patent Application Publications
Baptiste 11,537,378 teaches a sample set of cloud computing resources is dynamically selected for testing a software deployment. Telemetry data associated with the resources and customer support data associated with customers that utilize the resources are obtained. A subset of the customers is selected based on the customer support data, and a candidate subset of the resources is selected based on the selected subset of customers and the telemetry data. Criteria for the selection is based on usage patterns and is configurable.
Koblents 2019/0196853 obtaining, during runtime, from a compiler, two versions of a data parallel loop for an operation. The host computing system comprises includes a CPU and a GPU is accessible to the host. The processor(s) online profiles the two versions by asynchronously executing the first version, in a profile mode, with the GPU and executing the second version, in the profile mode, with the CPU. The processor(s) generates execution times for the first version and the second version. The processor(s) execute the operation with the CPU or the GPU based on the performance data.
13. Examiner, in light of the above submission maintains the previous rejections, and any new ground(s) of rejection is necessitated by Applicant’s amendment. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
14. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Correspondence Information
15. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Evral Bodden whose telephone number is 571-272-3455. The examiner can normally be reached on Monday to Friday from 9am to 5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do, can be reached at telephone number 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automatedinterview-request-air-form.
/EVRAL E BODDEN/Primary Examiner, Art Unit 2193
Prosecution Insights