Detailed Action
Applicant amended claims 1, 8 and 15, canceled claim 3, 10 and 17 and presented claims 1-2, 4-9, 11-16, and 18-20 for reconsideration on 07/08/2026.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 1-2, 4-9, 11-16, and 18-20 of instant application are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims of 17/514,997 (12,353,445) and 18/162,625 (12,353,446). Although the conflicting claims are not identical, they are not patentably distinct from each other because the claims in 17/514,997 (12,353,445) and 18/162,625 (12,353,446) taught all features in the claim of the instant application as shown below for an independent claim. Dependent claims not included in claims of 17/514,997 (12,353,445) or 18/162,625 (12,353,446) would be rejected using references shown below.
19228554
17/514,997 (12,353,445)
18/162,625 (12,353,446)
1…storing a feature and metadata associated with the feature in a feature store, the metadata including a mapping from the feature to a machine learning model trained based on the feature;
1.store the feature in a feature store, wherein the feature store stores a first set of features and a second set of features; store, in the feature store, metadata in association with the feature, the metadata including a mapping from the feature to upstream lineage data indicating the one or more datasets used to determine the feature;…
1…storing the feature in a feature store; storing, in the feature store, metadata in association with the feature, the metadata including: (i) a mapping from the feature to upstream lineage data indicating the source data used to generate the feature, and (ii) a mapping from the feature to downstream lineage data
indicating the set of machine learning models trained based on the feature;
determining that a modification was made to the feature;
receive an update indication that the feature is to be updated;
determining, based on the metadata in the feature store including the mapping from the feature to the upstream lineage data, that a modification has been made to the
feature;
in response to determining the modification was made, identifying, based on the metadata in the feature store, a model serving endpoint that deploys the machine learning model trained based on the feature, wherein the model serving endpoint is a web service; and
determine, responsive to receipt of the update indication and based on the metadata including the mapping from the feature to the downstream lineage
data, the model serving endpoint that deploys the first model trained on the feature;
in response to determining that the modification has been made to the feature, identifying, based on metadata in the feature store including the mapping from the feature
to the downstream lineage data: (i) a first machine
learning model from the set of machine learning models trained based on the feature and (ii) a model serving endpoint that deploys the first machine learning model;
and
transmitting a notification to the model serving endpoint that deploys the machine learning model, the notification indicating that the feature has been modified.
transmit a notification to the model serving endpoint, the notification indicating that the feature is updated…
transmitting a notification to the model serving endpoint that deploys the first machine learning model, the notification indicating that the feature has been modified,
wherein the model serving endpoint is a web service.
2, 4-9, 11-16, 18-20
2-20
2-20
Examiner's Notes
The Examiner cites particular sections in the references as applied to the claims below for the convenience of the applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant(s) fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-9, 11-16, and 18-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Dirac et al., US 2015/0379072 A1 (hereinafter Dirac), in view of Pramod Singh, “Deploy Machine Learning Models to Production” (hereinafter Singh).
Claim 1. Dirac teaches:
A method comprising:
storing a feature and metadata associated with the feature in a feature store, the metadata including a mapping from the feature to a machine learning model trained based on the feature; (Dirac, a generated data source is a feature store storing retrieved data to be used for training a model: ¶ 74, “A client request to create a data source artifact 602 may include, for example, an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… the client request to create a data source may also include a request to re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”, ¶ 79, “a client may create an artifact such as a model, and may want that same model to be re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time. In some cases the points in time may be specified explicitly (e.g., by the client requesting the equivalent of “re-run model M1 on the currently available data set at data source DS1 at 11:00, 15:00 and 19:00 every day”). In other cases the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1). A respective job may be placed in the MLS job queue for each recurring training or execution iteration. The MLS may implement a set of programmatic interface enabling such scheduled recurring operations in some embodiments. Using such an interface, a client may specify a set of model/alias/recipe artifacts (or respective versions of the same underling artifact) to be used for each of the iterations, and/or the resource configurations to be used. Such programmatic interfaces may be referred to as "pipelining APIs” in some embodiments. In addition to the artifact types shown in FIG.6, pipeline artifacts may be stored in the MLS artifact repository in some embodiments, with each instance of a pipeline artifact representing a named set of recurring operations requested via such APIs. In one embodiment, a separately managed data pipelining service implemented at the provider network may be used in conjunction with the MLS for supporting such recurrent operations”)
determining that a modification was made to the feature; (Dirac, a modification is determined: ¶ 79, “a client may create an artifact such as a model, and may want that same model to be re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time…”)
in response to determining the modification was made, identifying, based on the metadata in the feature store, a model serving endpoint that deploys the machine learning model trained based on the feature; and; and (Dirac, in response to a modification, a model servicing endpoint based on the instruction provided by the user is identified for retraining the model: ¶ 79, “a client may create an artifact such as a model, and may want that same model to be re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time…”)
transmitting a notification to the model serving endpoint that deploys the machine learning model, the notification indicating that the feature has been modified. (Dirac, ¶ 79, wherein “the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1”). A respective job may be placed in the MLS job queue for each recurring training or execution iteration” suggest that a model servicing endpoint based on the instruction provided by the user is notified for retraining the model)
Dirac did not specifically disclose, but Singh discloses the model serving endpoint is a web service. (Singh, chapter 3, “we mainly use Jupyter notebooks to develop and test the models locally, but when we want to connect our ML model to an app or a web service, we essentially need to deploy it using a web server. This web server can be hosted locally or in the cloud. There are many different ways in which a machine learning model can be deployed, but in this chapter, we are going to explore two methods: Flask and Streamlit”)
It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing the model serving endpoint is a web service because doing so would provide for an explicit disclosure of a model serving endpoint as a web service as an alternative for achieving the same predictable result of updating a model based on the underlined data as disclosed by Dirac.
Claims 8 and 15 are rejected under the same rationale as above.
Dirac as modified teach:
Claim 2. The method of claim 1, further comprising:
sending, by the model serving endpoint: (i) a call to the feature store to access the modified feature, and (ii) a call to the machine learning model to generate an outcome based on the modified feature. (Dirac, APIs are used for retrieving new records and retraining the models: ¶ 44, “The MLS programmatic interfaces may enable users to submit respective requests for several related tasks of a given machine learning workflow, such as tasks for extracting records from data sources, generating statistics on the records, feature processing, model training, prediction, and so on. A given invocation of a programmatic interface (such as an API) may correspond to a request for one or more operations or tasks on one or more instances of a supported type of entity. Some tasks (and the corresponding APIs) may involve multiple different entity types—e.g., an API requesting a creation of a data source may result in the generation of a data source entity instance as well as a statistics entity instance. Some of the tasks of a given workflow may be dependent on the results of other tasks…an asynchronous approach may be taken to scheduling the tasks, in which MLS clients can submit additional tasks that depend on the output of earlier-submitted tasks without waiting for the earlier-submitted tasks to complete. For example, a client may submit respective requests for tasks T2 and T3 before an earlier-submitted task T1 completes, even though the execution of T2 depends at least partly on the results of T1, and the execution of T3 depends at least partly on the results of T2. In such embodiments, the MLS may take care of ensuring that a given task is scheduled for execution only when its dependencies (if any dependencies exist) have been met”, ¶ 69, “As indicated by the request handler's timeline TL2, the jobs corresponding to each API call may be created and queued shortly after the API is invoked”; ¶ 142, “The ORs may be provided programmatically to the requesting client (e.g., as an array or collection returned in response to the API call representing the OR extraction request), and/or to a specified destination such as a model or a feature processing recipe (element 2525)”)
Claims 9 and 16 are rejected under the same rationale as above.
Claim 4. The method of claim 1, further comprising:
accessing source data stored in a data store, the source data comprising raw data; and generating the feature based on the source data. (Dirac, data source/feature is generated from raw input data: ¶ 74, “A client request to create a data source artifact 602 may include, for example, an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… the client request to create a data source may also include a request to re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”)
Claims 11 and 18 are rejected under the same rationale as above.
Claim 5. The method of claim 4, wherein the metadata stored in the feature store includes:
(i) a mapping from the feature to upstream lineage data indicating the source data used to generate the feature, and (ii) a mapping from the feature to downstream lineage data indicating a set of machine learning models including the machine learning model trained based on the feature. (Dirac, raw data, manipulated data/feature and models using the feature are specified: ¶ 74, “A client request to create a data source artifact 602 may include, for example, an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… the client request to create a data source may also include a request to re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”; ¶ 79, “the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1”). A respective job may be placed in the MLS job queue for each recurring training or execution iteration”)
Claims 12 and 19 are rejected under the same rationale as above.
Claim 6. The method of claim 5, further comprising:
updating the mapping from the feature to the upstream lineage data based on the modification to the feature, wherein the mapping is updated to reflect modified upstream lineage data indicating revised source data. (Dirac, modified raw data, manipulated data/feature and models using the feature are specified: ¶ 74, “A client request to create a data source artifact 602 may include, for example, an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… the client request to create a data source may also include a request to re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”; ¶ 79, “the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1”). A respective job may be placed in the MLS job queue for each recurring training or execution iteration”)
Claims 13 and 20 are rejected under the same rationale as above.
Claim 7. The method of claim 5, further comprising:
determining that the feature is used to train a new machine learning model, where the new machine learning model is not in the set of machine learning models; and updating the mapping from the feature to the downstream lineage data to indicate the new machine learning model. (Dirac, modified raw data, manipulated data/feature and models including a new model using the feature are specified: ¶ 42, “For MLS clients that have high availability and data durability requirements, machine learning data (e.g., raw input data, transformed/manipulated input data, intermediate results, or final results) and/or models may be replicated across different geographical locations”, ¶ 47, “The model developers may continue to experiment with various algorithms, parameters and/or input data sets to obtain improved versions of the underlying model, and may be able to change the pointer to point to an enhanced version to improve the quality of predictions obtained by the business analysts”, ¶ 74, “A client request to create a data source artifact 602 may include, for example, an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… the client request to create a data source may also include a request to re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”; ¶ 79, “the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1”). A respective job may be placed in the MLS job queue for each recurring training or execution iteration”)
Claims 14 is rejected under the same rationale as above.
Claims 3, 10 and 17. (Canceled)
Response to Amendment and Arguments
Double patenting rejections are maintained. “Applicant will evaluate filing a terminal disclaimer” ‘if a claim is allowed”. Remarks, 9.
Applicant’s arguments with respect to rejected claims have been considered but are not persuasive for at least the following reasons.
Applicant argues, “The Office action on page 5 argues that Dirac teaches the mapping from the feature to a machine learning model trained based on the feature at paragraphs 74 and 79. However…the cited paragraphs do not describe a feature store storing metadata of a feature, the metadata including a mapping from the feature to a model trained based on the feature, as recited in claim 1…the Office action on page 6-7 argues that Dirac teaches the "identifying" and the "transmitting" steps of claim 1 at paragraph 79. However, paragraph 79 of Dirac merely describes a client requesting that a model be re-trained or re-executed on different input data at specified points in time or upon specified conditions (e.g., "re-run model Ml whenever the next set of 1000000 new records becomes available from data source DS1"), with a respective job "placed in the MLS job queue for each recurring training or execution iteration." Paragraph 79 of Dirac thus describes client-instruction-driven scheduling of a training or execution job…neither Paragraph 79 of Dirac nor Singh disclose or suggest identifying, based on the metadata in the feature store, a model serving endpoint that deploys the machine learning model trained based on the feature, wherein the model serving endpoint is a web service, and transmitting to such a model serving endpoint a notification indicating that the feature has been modified, as recited in amended claim 1”. Remarks, 8-9.
In response, The combination of Dirac and Singh teach “identifying, based on the metadata in the feature store, a model serving endpoint that deploys the machine learning model trained based on the feature, wherein the model serving endpoint is a web service, and transmitting to such a model serving endpoint a notification indicating that the feature has been modified” as shown above. It is not clear how “client-instruction-driven scheduling of a training or execution job”, as noted by applicant, does not teach identifying and transmitting steps. Dirac stores a feature and the feature’s metadata as in ¶¶ 74, 79. The metadata according to the claim, includes “a mapping from the feature to a machine learning model trained based on the feature”. The stored feature is mapped to a model by “an indication of an address or location from which data records can be read, and some indication of the format or schema of the data records… …re-arrange the raw input data, e.g., by sampling or splitting the data records using an I/O library of the MLS”.
The step of “identifying” is in response to determining that a modification was made to the feature. The modification is determined “whenever the next set of 1000000 new records becomes available from data source DS1”. The model associated with the “1000000 new records” is identified because “a client may create an artifact such as a model, and may want that same model to be re-trained and/or re-executed for different input data sets (e.g., using the same configuration of resources for each of the training or prediction iterations) at specified points in time…”.
A notification is transmitted to a model serving point because “the client may indicate the conditions under which the iterations are to be scheduled (e.g., by the client requesting the equivalent of “re-run model M1 whenever the next set of 1000000 new records becomes available from data source DS1”). A respective job may be placed in the MLS job queue for each recurring training or execution iteration. The MLS may implement a set of programmatic interface enabling such scheduled recurring operations in some embodiments. Using such an interface, a client may specify a set of model/alias/recipe artifacts (or respective versions of the same underling artifact) to be used for each of the iterations, and/or the resource configurations to be used. Such programmatic interfaces may be referred to as "pipelining APIs” in some embodiments. In addition to the artifact types shown in FIG.6, pipeline artifacts may be stored in the MLS artifact repository in some embodiments, with each instance of a pipeline artifact representing a named set of recurring operations requested via such APIs…a separately managed data pipelining service implemented at the provider network may be used in conjunction with the MLS for supporting such recurrent operations” discloses that a model servicing endpoint based on the instruction provided by the user is notified for retraining the model.
As noted in rejection of claim 1 above, Dirac did not specifically disclose, but Singh discloses in chapter 3 that the model serving endpoint is a web service. It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing the model serving endpoint is a web service because doing so would provide for an explicit disclosure of a model serving endpoint as a web service as an alternative for achieving the same predictable result of updating a model based on the underlined data as disclosed by Dirac.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. It is suggested that the Applicant review these documents before submitting any amendments.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mohsen Almani whose telephone number is (571)270-7722. The examiner can normally be reached on M-F, 9 AM-5 PM, ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached on 571-272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHSEN ALMANI/Primary Examiner, Art Unit 2159
