Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
Providing supporting paragraph(s) for each limitation of amended/new claim(s) in Remarks is strongly requested for clear and definite claim interpretations by Examiner (e.g., to avoid rejections under 35 U.S.C § 112(a) “Lack of written description”)
Applicant can schedule interviews (via Automated Interview Request (AIR)) at any stage of the prosecution (e.g., Non-Final, Final, and After-Final) to discuss any issues related to, for example, rejections under 35 U.S.C § 101 and § 102/103, for moving toward allowance.
If a limitation has bold brackets (i.e. [·]) around claim languages, the bracketed claim languages indicate that they have not been taught yet by the current prior art reference but they will be taught by another prior art reference afterwards.
If a limitation has one or more bold underlines, the one or more bold underlined claim languages indicate that they are taught by the current prior art reference, while the one or more non-underlined claim languages indicate that they have been taught already by one or more previous art references.
Priority
Acknowledgment is made of applicant's claim for the present application filed on 09/06/2023.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1
Step 1: “Is the claim to a process, machine, manufacture, or composition of matter?”
The claim is directed to a system. Therefore, yes.
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
generate, based on the input data, a second one or more outputs …; (i.e., mental process)
modify the data to obtain modified data associated with the application, (i.e., mental process)
perform one or more actions based on the performance level. (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
… for machine learning model evaluation, (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
the system comprising: one or more memories; and one or more processors, communicatively coupled to the one or more memories, configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
obtain, via application infrastructure associated with an application, data associated with the application, (insignificant extra-solution activity of receiving data, see MPEP 2106.05(g)) the data including a first one or more outputs of a deployed machine learning model associated with the application, (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
wherein the first one or more outputs are based on input data associated with the application; (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
… of a test machine learning model (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
wherein the modified data includes the second one or more outputs in place of the first one or more outputs; (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
provide, via the application infrastructure, the modified data to a processing component of the application; (insignificant extra-solution activity of transmitting data, see MPEP 2106.05(g))
obtain, based on providing the modified data, evaluation information indicating a performance level of the test machine learning model; and (insignificant extra-solution activity of receiving data, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Specifically, the claimed inventions simply append well-understood, routine and conventional activities previously known to the industry, both when viewed independently and as an ordered combination, specified at a high level of generality, to the judicial exception, (e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry). Therefore, no.
Regarding claim 2
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 1. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the application infrastructure include one or more modularized components, (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
wherein the one or more modularized components include the processing component and a scoring component, and (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
wherein the one or more processors, to obtain the data, are configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
obtain the data via the scoring component. (insignificant extra-solution activity of receiving data, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 3
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
configure the second one or more outputs to have the data format (i.e., mental process)
The claim is directed to an abstract idea. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the first one or more outputs are associated with a data format, (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h)) and wherein the one or more processors, to generate the second one or more outputs, are configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 4
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 1. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the first one or more outputs and the second one or more outputs are both associated with one or more target variables (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 5
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 1. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the first one or more outputs are associated with a first one or more target variables and the second one or more outputs are associated with a second one or more target variables. (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 6
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
configure the second one or more outputs to have a format of the one or more data fields, and (i.e., mental process)
replace the one or more data fields with the second one or more outputs to obtain the modified data. (i.e., mental process)
Therefore, no.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the data includes one or more data fields indicating the first one or more outputs (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h)), wherein the one or more processors, to generate the second one or more outputs, are configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
wherein the one or more processors, to modify the data, are configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 7
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 1. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the performance level satisfies a performance threshold (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h)), and wherein the one or more processors, to perform the one or more actions, are configured to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
cause, based on the performance level satisfying the performance threshold, the test machine learning model to be deployed via the application infrastructure (insignificant extra-solution activity of transmitting data, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 8
The claim is rejected for the reasons set forth in the rejection of Claim 1 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 9
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 8. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the performance level is based on one or more processing operations (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h)) performed via the processing component using the modified data (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 10
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 8. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
receiving a request to evaluate the second machine learning model; and (insignificant extra-solution activity of receiving data, see MPEP 2106.05(g))
obtaining, in response to the request, the second machine learning model, (insignificant extra-solution activity of receiving data, see MPEP 2106.05(g))
wherein generating the data associated with the application is in response to the request. (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 11
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 8. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
transmitting, via an application programming interface (API) call, the modified data to the processing component based on replacing the first one or more outputs with the second one or more outputs in the data (insignificant extra-solution activity of transmitting data, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 12
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”
preprocessing, …, input data to obtain preprocessed data; and (i.e., mental process)
wherein generating the second one or more outputs comprises: (i.e., mental process)
Therefore, no.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
via the application infrastructure (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
providing, via the application infrastructure, the preprocessed data to the first machine learning model to obtain the first one or more outputs, and (insignificant extra-solution activity of mere data gathering, see MPEP 2106.05(g))
providing, to the second machine learning model, the preprocessed data; and (adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, see MPEP 2106.05(f))
obtaining, based on providing the preprocessed data, the second one or more outputs (insignificant extra-solution activity of mere data outputting, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 13
The claim is rejected for the reasons set forth in the rejection of Claim 1 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 14
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 8. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the application infrastructure include one or more modularized components (a particular type or source of model/data, Field of Use and Technological Environment, see MPEP 2106.05(h)), wherein the one or more modularized components include the processing component and a scoring component configured to execute the first machine learning model. (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 15
The claim is rejected for the reasons set forth in the rejection of Claim 3 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 16
The claim is rejected for the reasons set forth in the rejection of Claim 1 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 17
Step 2A Prong 1: “Does the claim recite an abstract idea, law of nature, or natural phenomenon?” The claim recites the abstract idea identified above regarding claim 16. Therefore, yes.
Step 2A Prong 2: “Does the claim recite additional elements that integrate the judicial exception into a practical application?” The following elements are directed to additional elements:
wherein the one or more instructions, when executed by the one or more processors, further cause the device to: (well-understood, routine, and conventional generic computer and/or model, see MPEP 2106.05(f)) provide, for display, an indication of the performance level (insignificant extra-solution activity of transmitting data, see MPEP 2106.05(g))
Therefore, no.
Step 2B: “Does the claim recite additional elements that amount to significantly more than the judicial exception?”
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Therefore, no.
Regarding claim 18
The claim is rejected for the reasons set forth in the rejection of Claim 2 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 19
The claim is rejected for the reasons set forth in the rejection of Claim 4 under 35 U.S.C. 101, mutatis mutandis.
Regarding claim 20
The claim is rejected for the reasons set forth in the rejection of Claim 6 under 35 U.S.C. 101, mutatis mutandis.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by KREUZBERGER et al. (Machine Learning Operations (MLOps): Overview, Definition, and Architecture)
Regarding claim 1
KREUZBERGER teaches
A system for machine learning model evaluation, the system comprising:
one or more memories; and
(KREUZBERGER [sec(s) IV] “The model training infrastructure provides the foundational computation resources, e.g., CPUs, RAM, and GPUs. The provided infrastructure can be either distributed or non-distributed. In general, a scalable and distributed infrastructure is recommended [7], [23], [27], [28], [32], [33], [37], [45], [46], [δ, ζ, η, θ]. Examples include local machines (not scalable) or cloud computation [33] [η, θ], as well as non-distributed or dis tributed computation (several worker nodes) [7], [37].”;)
one or more processors, communicatively coupled to the one or more memories, configured to:
(KREUZBERGER [sec(s) IV] “The model training infrastructure provides the foundational computation resources, e.g., CPUs, RAM, and GPUs. The provided infrastructure can be either distributed or non-distributed. In general, a scalable and distributed infrastructure is recommended [7], [23], [27], [28], [32], [33], [37], [45], [46], [δ, ζ, η, θ]. Examples include local machines (not scalable) or cloud computation [33] [η, θ], as well as non-distributed or dis tributed computation (several worker nodes) [7], [37].”;)
obtain, via application infrastructure associated with an application, data associated with the application, the data including a first one or more outputs of a deployed machine learning model associated with the application,
(KREUZBERGER [fig(s) 4] [sec(s) IV] “C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
wherein the first one or more outputs are based on input data associated with the application;
(KREUZBERGER [fig(s) 4] [sec(s) IV] “C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
generate, based on the input data, a second one or more outputs of a test machine learning model;
(KREUZBERGER [fig(s) 4] [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
modify the data to obtain modified data associated with the application,
(KREUZBERGER [fig(s) 4] [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
wherein the modified data includes the second one or more outputs in place of the first one or more outputs;
(KREUZBERGER [fig(s) 4] [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
provide, via the application infrastructure, the modified data to a processing component of the application;
(KREUZBERGER [fig(s) 4] [sec(s) IV] “Real-time inference can be achieved by hosting the model in a RESTful web-service, batch inference can be an idempotent MapReduce workflow, and serverless inference is used when cost-efficient and scalable serving is required” [sec(s) V] “Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
obtain, based on providing the modified data, evaluation information indicating a performance level of the test machine learning model; and
(KREUZBERGER [fig(s) 4] [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61]. As a foundational requirement, the ML engineer (R7) manages the model-serving computation infrastructure. The (26) monitoring component (C9) observes continuously the model-serving performance and infrastructure in real-time.” [sec(s) IV] “P8 Continuous monitoring. Continuous monitoring implies the periodic assessment of data, model, code, infrastructure resources, and model serving performance (e.g., prediction accuracy) to detect potential errors or changes that influence the product quality [7], [23], [28], [32], [33], [α, β, γ, δ, ε, ζ, η].”;)
perform one or more actions based on the performance level.
(KREUZBERGER [fig(s) 4] [sec(s) V] “Once the model-monitoring component (C9) detects a drift in the data [60], the information is forwarded to the scheduler, which then triggers the automated ML workflow pipeline for retraining (continuous training).” [sec(s) IV] “As a foundational requirement, the ML engineer (R7) manages the model-serving computation infrastructure. The (26) monitoring component (C9) observes continuously the model-serving performance and infrastructure in real-time. Once a certain threshold is reached, such as detection of low prediction accuracy, the information is forwarded via the feedback loop.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 2
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER teaches
wherein the application infrastructure include one or more modularized components,
(KREUZBERGER [sec(s) IV] “B. TECHNICAL COMPONENTS After identifying the principles that need to be incorporated into MLOps, we now elaborate on the precise components and implement them in the ML systems design. In the following, the components are listed and described in a generic way with their essential functionalities. The references in brackets refer to the respective principles that the technical components are implementing. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API. As a foundational infrastructure layer, a scalable and distributed model serving infrastructure is recommended [23], [27], [33], [37], [39], [46], [α, β, δ, ζ, η, θ].”;)
wherein the one or more modularized components include the processing component and a scoring component, and
(KREUZBERGER [sec(s) IV] “B. TECHNICAL COMPONENTS After identifying the principles that need to be incorporated into MLOps, we now elaborate on the precise components and implement them in the ML systems design. In the following, the components are listed and described in a generic way with their essential functionalities. The references in brackets refer to the respective principles that the technical components are implementing. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API. As a foundational infrastructure layer, a scalable and distributed model serving infrastructure is recommended [23], [27], [33], [37], [39], [46], [α, β, δ, ζ, η, θ].”;)
wherein the one or more processors, to obtain the data, are configured to:
obtain the data via the scoring component.
(KREUZBERGER [sec(s) IV] “B. TECHNICAL COMPONENTS After identifying the principles that need to be incorporated into MLOps, we now elaborate on the precise components and implement them in the ML systems design. In the following, the components are listed and described in a generic way with their essential functionalities. The references in brackets refer to the respective principles that the technical components are implementing. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API. As a foundational infrastructure layer, a scalable and distributed model serving infrastructure is recommended [23], [27], [33], [37], [39], [46], [α, β, δ, ζ, η, θ].”;)
Regarding claim 3
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the first one or more outputs are associated with a data format,
(KREUZBERGER [sec(s) IV] “Real-time inference can be achieved by hosting the model in a RESTful web-service, batch inference can be an idempotent MapReduce workflow, and serverless inference is used when cost-efficient and scalable serving is required.” [sec(s) V] “Model-serving applications are often configured within a container and prediction requests are handled via a REST API.”;)
and wherein the one or more processors, to generate the second one or more outputs, are configured to: configure the second one or more outputs to have the data format.
(KREUZBERGER [sec(s) V] “Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 4
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the first one or more outputs and the second one or more outputs are both associated with one or more target variables.
(KREUZBERGER [fig(s) 4] [sec(s) IV] “C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 5
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the first one or more outputs are associated with a first one or more target variables and the second one or more outputs are associated with a second one or more target variables.
(KREUZBERGER [fig(s) 4] [sec(s) IV] “The model registry stores centrally the trained ML models together with their metadata. C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API.” [sec(s) V] “This includes the source and version of the feature data and model training code used to train the model. Also, the model version and status (e.g., staging or production-ready) is recorded. … The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 6
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the data includes one or more data fields indicating the first one or more outputs, wherein the one or more processors, to generate the second one or more outputs, are configured to:
configure the second one or more outputs to have a format of the one or more data fields, and
(KREUZBERGER [fig(s) 4] [sec(s) IV] “Real-time inference can be achieved by hosting the model in a RESTful web-service, batch inference can be an idempotent MapReduce workflow, and server less inference is used when cost-efficient and scalable serving is required.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
wherein the one or more processors, to modify the data, are configured to:
replace the one or more data fields with the second one or more outputs to obtain the modified data.
(KREUZBERGER [fig(s) 4] [sec(s) IV] “Real-time inference can be achieved by hosting the model in a RESTful web-service, batch inference can be an idempotent MapReduce workflow, and server less inference is used when cost-efficient and scalable serving is required.” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 7
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the performance level satisfies a performance threshold,
(KREUZBERGER [fig(s) 4] [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61]. … Once a certain threshold is reached, such as detection of low prediction accuracy, the information is forwarded via the feedback loop”;)
and wherein the one or more processors, to perform the one or more actions, are configured to:
cause, based on the performance level satisfying the performance threshold, the test machine learning model to be deployed via the application infrastructure.
(KREUZBERGER [sec(s) IV] “The CI/CD component ensures continuous integration, continuous delivery, and continuous deployment. It takes care of the build, test, delivery, and deploy steps.” [sec(s) V] “Once the status of a well-performing model is switched from staging to production, it is automatically handed over to the DevOps engineer or ML engineer for model deployment. From there, the (24) CI/CD component (C1) triggers the continuous deployment pipeline. The production-ready ML model and the model serving code are pulled (initially prepared by the software engineer (R5)). The continuous deployment pipeline carries out the build and test step of the ML model and serving code and deploys the model for production serving.”;)
Regarding claim 8
The claim is a method claim corresponding to the system claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 9
The combination of KREUZBERGER teaches claim 1.
KREUZBERGER further teaches
wherein the performance level is based on one or more processing operations performed via the processing component using the modified data.
(KREUZBERGER [fig(s) 4] [sec(s) IV] “C9 Monitoring Component (P8, P9). The monitoring component takes care of the continuous monitoring of the model serving performance (e.g., prediction accuracy). Additionally, monitoring of the ML infrastructure, CI/CD, and orchestration are required [7], [23], [24], [28], [32], [33], [50], [α, ζ, η, θ]. Examples include Prometheus with Grafana [η, ζ], ELK stack (Elasticsearch, Logstash, and Kibana) [α, η, ζ], and simply TensorBoard [θ]. Examples with built-in monitoring capabilities are Kubeflow [θ], MLflow [η], and AWS SageMaker model monitor or cloud watch [ζ].” [sec(s) V] “The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency). Model-serving applications are often configured within a container and prediction requests are handled via a REST API. When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61]. … Once a certain threshold is reached, such as detection of low prediction accuracy, the information is forwarded via the feedback loop.”;)
Regarding claim 10
The combination of KREUZBERGER teaches claim 8.
KREUZBERGER further teaches
receiving a request to evaluate the second machine learning model; and
obtaining, in response to the request, the second machine learning model,
(KREUZBERGER [sec(s) IV] “C1 CI/CD Component (P1, P6, P9). The CI/CD component ensures continuous integration, continuous delivery, and continuous deployment. It takes care of the build, test, delivery, and deploy steps. … C6 Model Registry (P3, P4). The model registry stores centrally the trained ML models together with their metadata. It has two main functionalities: storing the ML artifact and storing the ML metadata (see C7) [7], [24], [25], [34], [35], [α, β, γ, ε, ζ, η, θ]. Advanced storage examples include MLflow [α, η, ζ], AWS SageMaker Model Registry [ζ], Microsoft Azure ML ModelRegistry[ζ], and Neptune.ai[α]. Simple storage examples include Microsoft Azure Storage, Google Cloud Storage, and Amazon AWS S3 [23].” [sec(s) V] “Once the model-monitoring component (C9) detects a drift in the data [60], the information is forwarded to the scheduler, which then triggers the automated ML workflow pipeline for retraining (continuous training). As explained, a change in adequacy of the deployed model can be detected using distribution comparisons to identify drift. Retraining is not only triggered automatically when a statistical threshold is reached; it can also be triggered when new feature data is available, or it can be scheduled periodically.”;)
wherein generating the data associated with the application is in response to the request.
(KREUZBERGER [sec(s) IV] “C1 CI/CD Component (P1, P6, P9). The CI/CD component ensures continuous integration, continuous delivery, and continuous deployment. It takes care of the build, test, delivery, and deploy steps. … C6 Model Registry (P3, P4). The model registry stores centrally the trained ML models together with their metadata. It has two main functionalities: storing the ML artifact and storing the ML metadata (see C7) [7], [24], [25], [34], [35], [α, β, γ, ε, ζ, η, θ]. Advanced storage examples include MLflow [α, η, ζ], AWS SageMaker Model Registry [ζ], Microsoft Azure ML ModelRegistry[ζ], and Neptune.ai[α]. Simple storage examples include Microsoft Azure Storage, Google Cloud Storage, and Amazon AWS S3 [23].” [sec(s) V] “Once the model-monitoring component (C9) detects a drift in the data [60], the information is forwarded to the scheduler, which then triggers the automated ML workflow pipeline for retraining (continuous training). As explained, a change in adequacy of the deployed model can be detected using distribution comparisons to identify drift. Retraining is not only triggered automatically when a statistical threshold is reached; it can also be triggered when new feature data is available, or it can be scheduled periodically.”;)
Regarding claim 11
The combination of KREUZBERGER teaches claim 8.
KREUZBERGER further teaches
transmitting, via an application programming interface (API) call, the modified data to the processing component based on replacing the first one or more outputs with the second one or more outputs in the data.
(KREUZBERGER [sec(s) IV] “P8 Continuous monitoring. Continuous monitoring implies the periodic assessment of data, model, code, infrastructure resources, and model serving performance (e.g., pre diction accuracy) to detect potential errors or changes that influence the product quality [7], [23], [28], [32], [33], [α, β, γ, δ, ε, ζ, η]. P9 Feedback loops. Multiple feedback loops are required to integrate insights from the quality assessment step into the development or engineering process (e.g., a feedback loop from the experimental model engineering stage to the previous feature engineering stage). Another feedback loop is required from the monitoring component (e.g., observing the model serving performance) to the scheduler to enable the retraining [7], [23], [24], [34], [35], [α, β, δ, ζ, η, θ]. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API.” [sec(s) V] “Depending on the use case, features are extracted from either the offline or online database (or any kind of data store). … When deploying an ML/AI application, it’s a good practice to use A/B testing to determine in a real-world scenario which model performs better compared to another model, for example, deploying a ‘‘challenger model’’ in addition to an existing ‘‘champion model’’ to find out which one performs better by collecting feedback, for example, when predicting hotel booking cancellations [61].”;)
Regarding claim 12
The combination of KREUZBERGER teaches claim 8.
KREUZBERGER further teaches
preprocessing, via the application infrastructure, input data to obtain preprocessed data; and
(KREUZBERGER [fig(s) 4] “Feature Engineering Pipeline” [sec(s) V] “Feature engineering pipeline. The initially defined requirements for the feature engineering pipeline are taken by the data engineer (R4) and software engineer (R5) as a starting point to build up the prototype of the feature engineering pipeline. The initially defined requirements and rules are updated according to the iterative feedback coming either from the experimental model engineering stage or from the monitoring component observing the model’s performance in production. As a foundational requirement, the data engineer (R4) defines the code required for the CI/CD (C1) and orchestration component (C3) to ensure the task orchestration of the feature engineering pipeline. This role also defines the underlying infrastructure resource configuration. (8) First, the feature engineering pipeline connects to the raw data, which can be (for instance) streaming data, static batch data, or data from any cloud storage. (9) The data will be extracted from the data sources. (10) The data preprocessing begins with data transformation and cleaning tasks. The transformation rule artifact defined in the requirement gathering stage serves as input for this task, and the main aim of this task is to bring the data into a usable format. These transformation rules are continuously improved based on the feedback. (11) The feature engineering task calculates new and more advanced features based on other features. The predefined feature engineering rules serve as input for this task. These feature engineering rules are continuously improved based on the feedback. (12) Lastly, a data ingestion job loads batch or streaming data into the feature store system (C4). The target can either be the offline or online database (or any kind of data store).”;)
providing, via the application infrastructure, the preprocessed data to the first machine learning model to obtain the first one or more outputs, and
(KREUZBERGER [sec(s) IV] “C4Feature Store System(P3,P4). A feature store system ensures central storage of commonly used features. It has two databases configured: One database as an offline feature store to serve features with normal latency for experimentation, and one database as an online store to serve features with low latency for predictions in production [25], [28], [α, β, ζ, ε, θ].” [sec(s) V] “They check the distribution, and quality of the data, as well as performing validation checks. Furthermore, they ensure that the incoming data from the data sources is labeled, meaning that a target attribute is known, as this is a mandatory requirement for supervised ML. In this example, the data sources already had labeled data available as the labeling step was covered during an upstream process. … The continuous deployment pipeline carries out the build and test step of the ML model and serving code and deploys the model for production serving. The (25) model serving component (C8) makes predictions on new, unseen data coming from the feature store system (C4). This component can be designed by the software engineer (R5) as online inference for real time predictions or as batch inference for predictions concerning large volumes of input data. For real-time predictions, features must come from the online database (low latency), whereas for batch predictions, features can be served from the offline database (normal latency).”;)
wherein generating the second one or more outputs comprises:
providing, to the second machine learning model, the preprocessed data; and
obtaining, based on providing the preprocessed data, the second one or more outputs.
(KREUZBERGER [sec(s) IV] “P3 Reproducibility. Reproducibility is the ability to reproduce an ML experiment and obtain the exact same results [23], [27] [α, β, δ, ε, η]. … P6 Continuous ML training & evaluation. Continuous training (CT) means periodic retraining of the ML model based on new feature data. Continuous training is enabled through the support of a monitoring component, a feedback loop, and an automated ML workflow pipeline. Continuous training always includes an evaluation run to assess the change in model quality [23], [24], [28], [29], [β, δ, η, θ]. In general, to manage costs of retraining, it should be carefully considered, which update frequency is necessary for the use case (e.g., daily vs. weekly). A powerful tool to decrease cost of retraining is the use of online learning in large scale web applications, benefiting from iterative training steps compared to a full training data set. This way the model can also reflect recent impactful events like catastrophes. There is a magnitude of online learning optimization algorithms available [30].” [sec(s) V] “For all training job iterations, the ML metadata store (C7) records metadata such as parameters to train the model and the resulting performance metrics. This also includes the tracking and logging of the training job ID, training date and time, duration, and sources of artifacts. Additionally, the model specific metadata called ‘‘model lineage’’ combining the lineage of data and code is tracked for each newly registered model. This includes the source and version of the feature data and model training code used to train the model. Also, the model version and status (e.g., staging or production-ready) is recorded.”;)
Regarding claim 13
The claim is a method claim corresponding to the system claim 1 (the last limitation), and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 14
The combination of KREUZBERGER teaches claim 8.
KREUZBERGER teaches
wherein the application infrastructure include one or more modularized components,
(KREUZBERGER [sec(s) IV] “B. TECHNICAL COMPONENTS After identifying the principles that need to be incorporated into MLOps, we now elaborate on the precise components and implement them in the ML systems design. In the following, the components are listed and described in a generic way with their essential functionalities. The references in brackets refer to the respective principles that the technical components are implementing. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API. As a foundational infrastructure layer, a scalable and distributed model serving infrastructure is recommended [23], [27], [33], [37], [39], [46], [α, β, δ, ζ, η, θ].”;)
wherein the one or more modularized components include the processing component and a scoring component configured to execute the first machine learning model.
(KREUZBERGER [sec(s) IV] “B. TECHNICAL COMPONENTS After identifying the principles that need to be incorporated into MLOps, we now elaborate on the precise components and implement them in the ML systems design. In the following, the components are listed and described in a generic way with their essential functionalities. The references in brackets refer to the respective principles that the technical components are implementing. … C8 Model Serving Component (P1). The model serving component can be configured for different purposes. Examples are online inference for real-time predictions or batch inference for predictions using large volumes of input data. The serving can be provided, e.g., via a REST API. As a foundational infrastructure layer, a scalable and distributed model serving infrastructure is recommended [23], [27], [33], [37], [39], [46], [α, β, δ, ζ, η, θ].”;)
Regarding claim 15
The claim is a method claim corresponding to the system claim 3, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 16
The claim is a computer-readable medium claim corresponding to the system claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 17
The combination of KREUZBERGER teaches claim 16.
KREUZBERGER teaches
wherein the one or more instructions, when executed by the one or more processors, further cause the device to:
provide, for display, an indication of the performance level.
(KREUZBERGER [fig(s) 4] [sec(s) IV] “C9 Monitoring Component (P8, P9). The monitoring component takes care of the continuous monitoring of the model serving performance (e.g., prediction accuracy). Additionally, monitoring of the ML infrastructure, CI/CD, and orchestration are required [7], [23], [24], [28], [32], [33], [50], [α, ζ, η, θ]. Examples include Prometheus with Grafana [η, ζ], ELK stack (Elasticsearch, Logstash, and Kibana) [α, η, ζ], and simply TensorBoard [θ]. Examples with built-in monitoring capabilities are Kubeflow [θ], MLflow [η], and AWS SageMaker model monitor or cloud watch [ζ].” [sec(s) V] “As a foundational requirement, the ML engineer (R7) manages the model-serving computation infrastructure. The (26) monitoring component (C9) observes continuously the model-serving performance and infrastructure in real-time. Once a certain threshold is reached, such as detection of low prediction accuracy, the information is forwarded via the feedback loop.”;)
Regarding claim 18
The claim is a computer-readable medium claim corresponding to the system claim 2, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 19
The claim is a computer-readable medium claim corresponding to the system claim 4, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Regarding claim 20
The claim is a computer-readable medium claim corresponding to the system claim 6, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the system claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEHWAN KIM whose telephone number is (571)270-7409. The examiner can normally be reached Mon - Fri 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEHWAN KIM/Examiner, Art Unit 2129
Prosecution Insights