DETAILED ACTION
This action is in response to the original filling on 05/16/2023. Claims 1-20 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/08/2023 is being considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1 and 11 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 8 of Copending Application No. 18/318,057 and in view of Babu et al (US 20190102700 A1, hereinafter Babu) and further in view of Khare et al. (US 11249827 B2, hereinafter Khare).
This is a provisional nonstatutory patenting rejection.
Although the claims are not identical, they are not patentably distinct from each other because the claims of the present invention are similar in scope to the claims of copending Application No. 18/318,057. For example, the table below shown similarity and different between the instant application and the copending application 18/318,057.
18/318,052 (Instant Application)
18/318,057
1. A method for application programming interface (API) based machine learning model publication, the method comprising: receiving, by a machine learning model management (MLMM) system from a client device through an API of the MLMM system, a request to publish a machine learning model trained using a third-party machine learning modeling application, the MLMM system having a processor and a non-transitory computer-readable medium, the request containing a machine learning model package; processing, by the MLMM system, the machine learning model package to obtain the machine learning model; converting, by the MLMM system, the machine learning model to a standard format supported by the MLMM system; generating, by the MLMM system, a docker image of the machine learning model in the standard format; and posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.
8. A method for format-agnostic publishing of a machine learning model, comprising: receiving a request to publish a machine learning model, the request comprising a machine learning model schema and a machine learning model package comprising one or more files; publishing the machine learning model based on the machine learning model schema and the machine learning model package, comprising: validating the machine learning model schema and the machine learning model package to determine whether the request to publish the machine learning model is valid; wherein, if the request to publish the machine learning model is valid, converting the machine learning model to a common machine learning model format and deploying the converted machine learning model; and if the request to publish the machine learning model is invalid, generating a response that the request to publish the machine learning model is invalid.
11. A system application programming interface (API) based machine learning model publication, the system comprising: a processor; a non-transitory computer-readable medium; and instructions stored on the non-transitory computer-readable medium and translatable by the processor for: receiving, from a client device through an API of the system, a request to publish a machine learning model trained using a third-party machine learning modeling application, the request containing a machine learning model package; processing the machine learning model package to obtain the machine learning model; converting the machine learning model to a standard format supported by the system; generating a docker image of the machine learning model in the standard format; and posting the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.
1. A system for format-agnostic publishing of a machine learning model, comprising: a processor; a non-transitory computer-readable medium; and stored instructions translatable by the processor for executing: receiving a request to publish a machine learning model, the request comprising a machine learning model schema and a machine learning model package comprising one or more files; publishing the machine learning model based on the machine learning model schema and the machine learning model package, comprising: validating the machine learning model schema and the machine learning model package to determine whether the request to publish the machine learning model is valid; wherein, if the request to publish the machine learning model is valid, converting the machine learning model to a common machine learning model format and deploying the converted machine learning model; and if the request to publish the machine learning model is invalid, generating a response that the request to publish the machine learning model is invalid.
The co-pending 18/318,057 did not teach application programming interface (API) based; machine learning model management (MLMM) system from a client device through an API of the MLMM system; generating, by the MLMM system, a docker image of the machine learning model in the standard format; and posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster as recited in claims 1 and 11 of the instant application.
However, Babu teaches: application programming interface (API) based (Paragraph [0064]: Model management and scoring platform 130 may import external models through an API 132, and convert the external models that have different schema into models that share a same schema as described in detail below. The created or imported models may be managed, evaluated, deployed, and updated by model management and scoring platform 130. For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134); machine learning model management (MLMM) system from a client device through an API of the MLMM system (Paragraph [0064]: For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134. Paragraph [0152]: Cloud services are generally provided on an on-demand self-service basis, subscription-based, elastically scalable, reliable, highly available, and secure manner. For example, a customer, via a subscription order, may order one or more services provided by cloud infrastructure system 1702. Cloud infrastructure system 1702 then performs processing to provide the services requested in the customer's subscription order. For example, a user may request the cloud infrastructure system to register an application, as described above, and provide services to the application per the application's specified requirements. Cloud infrastructure system 1702 may be configured to provide one or even multiple cloud services.); and generating, by the MLMM system, a docker image of the machine learning model in the standard format; (Paragraph [0059]: Existing solutions to these challenges may include containerizing everything using, for example, Docker. However, unless the developers stick with this approach during the whole development process, an image-creation development and maintenance task may need to be performed. Paragraph [0061]: The machine learning platform disclosed herein may manage different machine learning models or different versions of a machine learning model, and facilitate the evaluation of the machine learning models and the selection and deployment of the machine learning model for production. The machine learning platform may also collect and report information, such as scores of the model applied to the input data or statistics about the usage of a model, which may be used to improve the model or the selection of the model by a selector. In some cases, the machine learning platform may use the selector to select an appropriate model for a give dataset from a number of available models.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the concepts of an API based system to publish machine learning models, including generating a docker image, as suggested by the combination of Babu into the co-pending application 18/318,057 because these systems address the need for deploying machine learning models. Doing so would be beneficial to facilitate the evaluation of the machine learning models and the selection and deployment of the machine learning model for production (Babu Paragraph [0061]).
The combination of co-pending 18/318,057 and Babu did not teach posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.
In the same field of endeavor, Khare teaches: posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster. (Col. 2 37-59: In some embodiments, a producer provides a container to the web services model repository service 121 using a model/algorithm container registry 105. This container is shared as an image. In some embodiments, a model/algorithm container registry 105 is a fully-managed container registry that allows for storing, managing, and deploying of container images. (Col. 3 Lines 4-10: The publishing/listing agent 125 publishes received code or containers, lists containers, and responds to queries.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the concepts of an API based system to publish machine learning models, including generating a docker image, and posting the model to a managed cluster as suggested by Khare into the combination of co-pending 18/318057 and Babu because these systems all address the need of deploying machine learning models. Doing so would be beneficial to facilitate the evaluation of the machine learning models and the selection and deployment of the machine learning model for production (Babu Paragraph [0061]), not all programmers and/or system administrators have the time or requisite knowledge to produce this content or integrate it into a pipeline of actions (Khare Col. 2, Lines 3 - 6).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1 and 11
Step 1: Claims 1 and 11 recite a method and system, therefore, they are directed to the statutory categories of a method and machine.
Step 2A Prong 1: The claims recite, inter alia:
“processing, by the MLMM system, the machine learning model package to obtain the machine learning model; converting, by the MLMM system, the machine learning model to a standard format supported by the MLMM system;” Under its broadest reasonable interpretation in light of the specification, these limitation encompasses the mental process of evaluation or judgement that is practically capable of being performed in the human mind with the aid of pen and paper.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of “receiving, by a machine learning model management (MLMM) system from a client device through an API of the MLMM system,” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)). The additional elements of “a request to publish a machine learning model trained using a third-party machine learning modeling application,” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)). The additional elements of “the MLMM system having a processor and a non-transitory computer-readable medium (f),” amount to no more than mere instructions to apply an exception (see MPEP § 2106.05(f)). The claim invokes computers or other machinery merely as a tool to perform an existing process. The additional elements of “the request containing a machine learning model package (g);” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)). The additional elements of “generating, by the MLMM system, a docker image of the machine learning model in the standard format; and” amount to no more than a recitation of the words "apply it" (or an equivalent) or are no more than mere instructions to implement an abstract idea or other exception on a computer. (see MPEP § 2106.05(f)). “posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model,” and “wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)).
Step 2B: The claims do not contain significantly more than the judicial exception. The additional elements of “receiving, by a machine learning model management (MLMM) system from a client device through an API of the MLMM system,” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)). The additional elements of “a request to publish a machine learning model trained using a third-party machine learning modeling application,” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)). The additional elements of “the MLMM system having a processor and a non-transitory computer-readable medium (f),” amount to no more than mere instructions to apply an exception (see MPEP § 2106.05(f)). The claim invokes computers or other machinery merely as a tool to perform an existing process. The additional elements of “the request containing a machine learning model package (g);” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)). The additional elements of “generating, by the MLMM system, a docker image of the machine learning model in the standard format; and” amount to no more than a recitation of the words "apply it" (or an equivalent) or are no more than mere instructions to implement an abstract idea or other exception on a computer. (see MPEP § 2106.05(f)). “posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model,” and “wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)).
Claims 2-3, 12-13
Step 1: Claims recite a method and system; therefore, they are directed to the statutory categories of a method and machine.
Step 2A Prong 1: Claims 2-3 and 12-13 merely narrow the previously recited abstract limitations. For the reasons described above with respect to claims 1 and 11, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mental processes that are practically capable of being performed in the human mind with the assistance of pen and paper and mathematical concepts that are achievable through mathematical computation.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
Claims 2 and 12 recite the additional element of: “The method according to claim 1, wherein the API comprises an API wrapper, wherein the API wrapper calls a model conversion API to perform the converting and receive the machine learning model in the standard format and then calls an MLMM API with the machine learning model in the standard format to perform the generating.” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)).
Claims 3 and 13 recite “The method according to claim 2, wherein, prior to the converting, the API wrapper calls a model validation API for validating the machine learning model obtained from the machine learning model package, and wherein the API wrapper calls the model conversion API responsive to the machine learning model being valid.” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)) and is considered well-understood, routine, conventional activity similar to gathering statistics and presenting offers (see MPEP 2106.05(d)(II)).
Step 2B: The claims do not contain significantly more than the judicial exception.
Claims 4-6 and 14-16
Step 1: Claims recite a method, and a system; therefore, they are directed to the statutory categories of a method and machine.
Step 2A Prong 1: Claims 4 and 14 recite, inter alia: “The method according to claim 3, wherein validation of the machine learning model comprises at least one of: checking whether the machine learning model is of a valid supported model type; verifying whether the machine learning model is packaged correctly based on the valid supported model type; determining whether the machine learning model is free of malware; determining whether the machine learning model package contains any unwarranted file or system call; or where the machine learning model package is a zip file, validating a file name of the zip file.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluation or judgement that is practically capable of being performed in the human mind with the aid of pen and paper.
Claims 5 and 15 recite, inter alia: “The method according to claim 3, further comprising: responsive to the machine learning model being invalid, deleting the machine learning model from a temporary location in a file system of the MLMM system.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluation or judgement that is practically capable of being performed in the human mind with the aid of pen and paper.
Claims 6 and 16 recite, inter alia: “The method according to claim 3, further comprising: responsive to the machine learning model being invalid, generating an error code or message indicating that the machine learning model is invalid.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluation or judgement that is practically capable of being performed in the human mind with the aid of pen and paper.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claims are patent ineligible.
Claims 7 and 17
Step 1: Claims recite a method, and a system; therefore, they are directed to the statutory categories of a method and machine.
Step 2A Prong 1: Claims 7 and 17 recite, inter alia: “validating each of the third-party machine learning models; for each validated third-party machine learning model, converting the validated third-party machine learning model into the standard format;” Under its broadest reasonable interpretation in light of the specification, these limitations encompass the mental process of evaluation or judgement that is practically capable of being performed in the human mind with the aid of pen and paper.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional elements of “receiving requests to publish third-party machine learning models from disparate modeling applications where the machine learning models were trained, wherein the disparate modeling applications run on disparate computing environments;” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)). The additional elements of “generating a docker image for the third-party machine learning model in the standard format; and” and “storing the docker image for the third-party machine learning model in the docker registry.” amount to no more than a recitation of the words "apply it" (or an equivalent) or are no more than mere instructions to implement an abstract idea or other exception on a computer. (see MPEP § 2106.05(f)).
Step 2B: The claims do not contain significantly more than the judicial exception. The additional elements of “receiving requests to publish third-party machine learning models from disparate modeling applications where the machine learning models were trained, wherein the disparate modeling applications run on disparate computing environments;” These elements amount to no more than linking a particular exception to a field of use or technological environment, limited to a particular data source (see MPEP § 2106.05(h)). The additional elements of “generating a docker image for the third-party machine learning model in the standard format; and” and “storing the docker image for the third-party machine learning model in the docker registry.” amount to no more than a recitation of the words "apply it" (or an equivalent) or are no more than mere instructions to implement an abstract idea or other exception on a computer. (see MPEP § 2106.05(f)). Nothing in the claims provides significantly more than that abstract idea. As such, the claims are ineligible.
Claims 8-10 and 18-20
Step 1: Claims recite a method, and a system; therefore, they are directed to the statutory categories of a method and machine.
Step 2A Prong 1: Claims 8-10 and 18-20, merely narrow the previously recited abstract limitations. For the reasons described above with respect to claims 1 and 11, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mental processes that are practically capable of being performed in the human mind with the assistance of pen and paper and mathematical concepts that are achievable through mathematical computation.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
Claims 8 and 18 recite the additional element of: “The method according to claim 1, wherein the request further comprises a machine learning model input schema.”
Claims 9 and 19 recite the additional element of: “The method according to claim 1, wherein the machine learning model package comprises at least one of a file, a directory, or assets needed to host the machine learning model as a service.” amount to no more than mere data gathering and output (see MPEP § 2106.05(g)).
Claims 10 and 20 recite the additional element of: “The method according to claim 1, wherein the managed cluster comprises at least one of an on-prem managed cluster operating in an enterprise computing environment or a cloud-based cluster operating in a cloud computing environment.”
Step 2B: The claims do not contain significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-10, 11-12, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Babu et al (US 20190102700 A1, hereinafter Babu) in view of Khare et al (US 11249827 B2, hereinafter Khare).
Regarding Claim 1, Babu teaches A method for application programming interface (API) based machine learning model publication, the method comprising: (Paragraph [0064] Model management and scoring platform 130 may import external models through an API 132, and convert the external models that have different schema into models that share a same schema as described in detail below. The created or imported models may be managed, evaluated, deployed, and updated by model management and scoring platform 130. For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134)
receiving, by a machine learning model management (MLMM) system from a client device through an API of the MLMM system, a request to publish a machine learning model trained using a third-party machine learning modeling application, the MLMM system having a processor and a non-transitory computer-readable medium, the request containing a machine learning model package; (Paragraph [0064] For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134. Paragraph [0152] Cloud services are generally provided on an on-demand self-service basis, subscription-based, elastically scalable, reliable, highly available, and secure manner. For example, a customer, via a subscription order, may order one or more services provided by cloud infrastructure system 1702. Cloud infrastructure system 1702 then performs processing to provide the services requested in the customer's subscription order. For example, a user may request the cloud infrastructure system to register an application, as described above, and provide services to the application per the application's specified requirements. Cloud infrastructure system 1702 may be configured to provide one or even multiple cloud services.)
processing, by the MLMM system, the machine learning model package to obtain the machine learning model; (Paragraph [0156] As depicted in the example in FIG. 17, cloud infrastructure system 1702 may include infrastructure resources 1730 that are utilized for facilitating the provision of various cloud services offered by cloud infrastructure system 1702. Infrastructure resources 1730 may include, for example, processing resources, storage or memory resources, networking resources, and the like. In certain examples, the storage virtual machines that are available for servicing storage requested from applications may be part of cloud infrastructure system 1702. In other examples, the storage virtual machines may be part of different systems. (Could include processing of package) Paragraph [0120] In some cases, the developer may have subsequently come up with what is believed to be a better model. But the company's policy may be to treat these models as software that should be subject to different types of testing. The developer may save the new model to a file system and upload it to the platform. FIG. 12 illustrates an example user interface for uploading a new version of a model to an example machine learning platform according to certain embodiments. Again, the developer may choose to do this from the user interface or using code.) and
converting, by the MLMM system, the machine learning model to a standard format supported by the MLMM system; (Paragraph [0127] Optionally, at 1530, the computer system may convert a first ML model having a schema different from the common schema based on the common schema. For example, if the schema of the first ML model is congruent to the common schema of the model group, the datatype in the first ML model may be converted based on the datatype in the common schema. Two schemas are congruent if all feature vectors and datatypes of the feature vectors for the two models match or if the datatype of a feature in one model can be adapted to the datatype of a corresponding feature in another model. In some cases, a feature in the schema for the first ML model may be dropped based on determining that the feature has an importance level below a second threshold value.).
However, Babu fails to disclose: generating, by the MLMM system, a docker image of the machine learning model in the standard format; and posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, wherein the machine learning model published to the docker registry is available for deployment to a managed cluster.
In the same field of endeavor, Khare teaches: generating, by the MLMM system, a docker image of the machine learning model in the standard format; and (Col 2 Lines 51-59 In some embodiments, a producer provides a container to the web services model repository service 121 using a model/algorithm container registry 105. This container is shared as an image. In some embodiments, a model/algorithm container registry 105 is a fully-managed container registry that allows for storing, managing, and deploying of container images.)
posting, by the MLMM system, the docker image to a docker registry to thereby publish the machine learning model, (Col. 2 Lines 37-60 In some embodiments, a producer provides a container to the web services model repository service 121 using a model/algorithm container registry 105. This container is shared as an image. In some embodiments, a model/algorithm container registry 105 is a fully-managed container registry that allows for storing, managing, and deploying of container images. Col. 3 Lines 4-5 The publishing/listing agent 125 publishes received code or containers, lists containers, and responds to queries.) and
wherein the machine learning model published to the docker registry is available for deployment to a managed cluster. (Col. 3 Lines 4-5 The publishing/listing agent 125 publishes received code or containers, lists containers, and responds to queries. Col. 15 Lines 42-55 The illustrative environment includes at least one application server 1408 and a data store 1410. It should be understood that there can be several application servers, layers, or other elements, processes or components, which may be chained or otherwise configured, which can interact to perform tasks such as obtaining data from an appropriate data store. As used herein the term “data store” refers to any device or combination of devices capable of storing, accessing and retrieving data, which may include any combination and number of data servers, databases, data storage devices and data storage media, in any standard, distributed or clustered environment.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the concept of posting a machine learning model to a docker registry for publishing as suggested by Khare into Babu’s system because both of these systems are addressing the need to deploy machine learning models, as not all programmers and/or system administrators have the time or requisite knowledge to produce this content or integrate it into a pipeline of actions. (Khare Col. 2 Lines 3-18).
Regarding Claim 2, the combination of Babu and Khare teaches: wherein the API comprises an API wrapper, wherein the API wrapper calls a model conversion API to perform the converting and (Babu Paragraph [0063]: An ML model can be generated using model generator 120 of the ML platform by invoking a “create model” API. The data from data flow may be input into model generator 120 through a user interface (UI) 122. Feature vectors may then be extracted from the data and used to train the ML model. The trained model may be saved to model store 110. In some embodiments, model generator 120 may retrieve an existing model from model store 110, retrain the existing model using incoming data to generate a new model (e.g., a new version of a model), and save the new model to model store 110.) and
receive the machine learning model in the standard format and then calls an MLMM API with the machine learning model in the standard format to perform the generating. (Paragraph [0064]: Model management and scoring platform 130 may import external models through an API 132, and convert the external models that have different schema into models that share a same schema as described in detail below. The created or imported models may be managed, evaluated, deployed, and updated by model management and scoring platform 130. For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134.)
Regarding Claim 7, the combination of Babu and Khare teaches: receiving requests to publish third-party machine learning models from disparate modeling applications where the machine learning models were trained, wherein the disparate modeling applications run on disparate computing environments; (Babu Paragraph [0064] Model management and scoring platform 130 may import external models through an API 132, and convert the external models that have different schema into models that share a same schema as described in detail below. The created or imported models may be managed, evaluated, deployed, and updated by model management and scoring platform 130. For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134.)
validating each of the third-party machine learning models; (Paragraph [0073] At 430, the ML platform may manage the models (including versions) to, for example, search models, evaluate models with various test datasets to ensure that a model is ready for publishing for wider usage, compare versions of models in various dimensions (e.g., hyper-parameters of models, metrics of models, etc.). For example, the machine learning platform may allow users to: import/export models, evaluate and compare various models (prior to deployment for scoring), manage various versions of models, deploy/un-deploy models, and transparently retrain models with recent data to prevent model drift. In some embodiments, the machine learning platform may provide API(s) for retrieving a list of models based on several search criteria, which may support name-based search. In some embodiments, the machine learning platform may manage several versions of a model.)
for each validated third-party machine learning model, converting the validated third-party machine learning model into the standard format; (Paragraph [0064] Model management and scoring platform 130 may import external models through an API 132, and convert the external models that have different schema into models that share a same schema as described in detail below. The created or imported models may be managed, evaluated, deployed, and updated by model management and scoring platform 130. For example, model management and scoring platform 130 may selectively retrieve models from model store 110 and publish or deploy the selected model for analyzing online data. The scores may be feedback to model management and scoring platform 130 through a UI 134.)
In the same field of endeavor, Khare teaches: generating a docker image for the third-party machine learning model in the standard format; and storing the docker image for the third-party machine learning model in the docker registry. (Col 2 Lines 51-59 In some embodiments, a producer provides a container to the web services model repository service 121 using a model/algorithm container registry 105. This container is shared as an image. In some embodiments, a model/algorithm container registry 105 is a fully-managed container registry that allows for storing, managing, and deploying of container images.)
Regarding Claim 8, the combination of Babu and Khare teaches: wherein the request further comprises a machine learning model input schema. (Babu Paragraph [0012] In some embodiments, the method may also include reporting usage of the one or more ML models in the model group for the analyzing. In some embodiments, the method may include receiving a plurality of ML models, selecting the one or more ML models from the plurality of ML models, determining a common schema for the one or more ML models, converting a first ML model having a schema different from the common schema based on the common schema, and adding the converted first ML model to the model group. Determining the common schema for the one or more ML models may include determining the common schema that is a union of schemas for the one or more ML models, adding one of two congruent features in two respective schemas for two ML models to the common schema, or dropping a feature in a schema for a second ML model based on determining that the feature has an importance level below a second threshold value.)
Regarding Claim 9, the combination of Babu and Khare teaches: wherein the machine learning model package comprises at least one of a file, a directory, or assets needed to host the machine learning model as a service. (Khare Col 6 Lines 18-22 Received code is caused to be packaged at 503. There are many ways to perform this packaging, but the end result is a package that is a compressed file (such as a zip, tar, etc.) consisting of the received code and any dependencies in some embodiments.)
Regarding Claim 10, the combination of Babu and Khare teaches: wherein the managed cluster comprises at least one of an on-prem managed cluster operating in an enterprise computing environment or a cloud-based cluster operating in a cloud computing environment. (Babu Paragraph [0144] In certain examples, the functionalities described in this disclosure may be offered as services via a cloud environment. FIG. 17 is a simplified block diagram of a cloud-based system environment in which various services may be offered as cloud services in accordance with certain examples. In the example depicted in FIG. 17, cloud infrastructure system 1702 may provide one or more cloud services that may be requested by users using one or more client computing devices 1704, 1706, and 1708. Cloud infrastructure system 1702 may comprise one or more computers and/or servers that may include those described above for server 1612. The computers in cloud infrastructure system 1702 may be organized as general purpose computers, specialized server computers, server farms, server clusters, or any other appropriate arrangement and/or combination.)
Regarding Claims 11-12 and 17-20, they are system claims that corresponds to the method claims 1-2 and 7-10 above. Therefore, they are rejected for the same reason as method claim claims 1-2 and 7-10.
Claims 3-6 and 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over Babu et al (US 20190102700 A1, hereinafter Babu) in view of Khare et al (US 11249827 B2, hereinafter Khare) as applied in claims 2 and 12 above, and further in view of Cella et al. (US 12585231 B2, hereinafter Cella).
Regarding Claim 3, the combination of Babu and Khare teaches the invention as claimed in claim 2 above. However, the combination of Babu and Khare does not teach: wherein, prior to the converting, the API wrapper calls a model validation API for validating the machine learning model obtained from the machine learning model package, and wherein the API wrapper calls the model conversion API responsive to the machine learning model being valid.
In the same field of endeavor, Cella teaches: wherein, prior to the converting, the API wrapper calls a model validation API for validating the machine learning model obtained from the machine learning model package, and wherein the API wrapper calls the model conversion API responsive to the machine learning model being valid. (Col. 341-342 Lines 61-9 In embodiments, the validation circuit 9426 may validate the generated model, for example by testing it against test data provided by the governance library 9452. In some cases, the selected governance standards may require certain validations (e.g., validation that the model complies with safety requirements when processing data), and thus the governance library may contain test data and/or target output(s) for validating that the model successfully complies with the corresponding governance requirement(s). Col. 345 Lines 40-53 The classification module 9510 may receive input data, isolate/extract the input data, analyze the data, and classify the data. Col. 382 Lines 22-39 Some example applications provided by the platform 10110 for value chain management include payment processors, digital format conversion, production restrictions, export restriction filtering, and so on. Col. 402 Lines 4-46 In embodiments, a distributed manufacturing marketplace as described herein, may be integrated with or within another exchange... This may include integration by APIs, connectors, ports, brokers, and other interfaces, as well as integration by extraction, transformation and loading (ETL) technologies, smart contracts, wrappers, containers, or other capabilities.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the concept of incorporating an API wrapper to perform machine learning model validation, as suggested by Cella into the combination of references Babu and Khare because both of these systems are addressing the need to deploy machine learning models. Doing so would improve Babu and Khare by allowing users to validate data efficiently so that they would not be overwhelmed (Cella Column 2, Lines 33-41).
Regarding Claim 4, the combination of Babu and Khare in further view of Cella teaches: wherein validation of the machine learning model comprises at least one of: determining whether the machine learning model is free of malware; (Khare Col 6 Lines 51-55 At 513, a verification is caused to be performed when so requested. In some embodiments, the publishing/listing agent 125 performs this verification. In other embodiments, publishing/listing agent 125 calls another service to perform verification.)
Regarding Claim 5, the combination of Babu and Khare in further view of Cella teaches: further comprising: responsive to the machine learning model being invalid, deleting the machine learning model from a temporary location in a file system of the MLMM system. (Babu Paragraph [0075] In some embodiments, the machine learning platform may provide API(s) for publishing a version of a model for scoring. In some embodiments, the API(s) may specify whether a given version is the default version for scoring. In some embodiments, the machine learning platform may provide API(s) for importing and/or exporting a model. In some embodiments, the machine learning platform may provide API(s) for deleting a model. In some embodiments, the machine learning platform may support periodically publishing a model at a given frequency to implement continuous learning of a model using new data in new time windows. In some embodiments, the machine learning platform may provide API(s) for suggesting a model for a given dataset. Some examples of APIs are described in the Apendix in U.S. Provisional Patent Application No. 62/568,052, filed on Oct. 4, 2017, entitled “Machine Learning Platform”.).
Regarding Claim 6, the combination of Babu and Khare in further view of Cella teaches: further comprising: responsive to the machine learning model being invalid, generating an error code or message indicating that the machine learning model is invalid. (Khare Col 6 Lines 56-62 A determination of successful verification is made at 515 (when verification was performed). When the verification was not successful, an error is generated at 509. When the verification was successful, the package is published in the source control service 107 and 517. Published packages are available to the publishing/listing agent 125 to be served as a potential result to a code or data query.)
Regarding Claims 13-16, they are system claims that corresponding to the system claims 3-6. Therefore, they are rejected for the same reason as system claims 3-6 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. (US 11301762 B1) discloses Machine learning models trained in different frameworks translated into a common format.
Hiremath et al. (US 11853272 B2) discloses receiving input data used to train, test, and validate machine learning models.
Khare et al. (US 20200192733 A1) is an earlier publication of Khare, disclosing a machine learning repository service.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN A CARDOSO whose telephone number is (571)272-8512. The examiner can normally be reached M-F 7:30 - 5:00, alternate Friday's off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUSTIN CARDOSO/
Patent Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights