Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/6/2026 has been entered.
Remarks
This Office Action is responsive to Applicants' Amendment filed on March 6, 2026, in which claims 1 and 18 are currently amended. Claims 1-19 are currently pending.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on April 7, 2026 and March 6, 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments with respect to rejection of claims 1-19 under 35 U.S.C. 103 based on amendment have been considered and are persuasive. The argument is moot in view of a new ground of rejection set forth below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 18, "a relationship between the input node of the second model and the output node of the first model is previously unknown," is indefinite. it's unclear who or what the relationship was unknown to and to what degree it was previously unknown. Previous is a relative term and there is no relative basis for comparison in the claim. As the bounds of "previously unknown" cannot be determined, similarly, the scope of the claim is ambiguous. In the interest of further Examination the claim is interpreted as "generating a relationship between the input node of the second model and the output node of the first model".
The remaining claims are rejected with respect to their dependence on the rejected claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6, 9-14, 18, and 19 are rejected under U.S.C. §103 as being unpatentable over the combination of Agarwal (US20200033163A1) and Feng (US20190370420A1).
PNG
media_image1.png
562
552
media_image1.png
Greyscale
FIG. 3A of Agarwal
PNG
media_image2.png
438
706
media_image2.png
Greyscale
FIG. 4 of Agarwal
Regarding claim 1, Agarwal teaches A computer-implemented method for model chaining, ([¶0124] "the back end server system comprises at least one server that comprises a processor and a memory for storing instructions that, when executed by the processor, cause the server to: (i) receive the featurized data from the sensor assembly; (ii) determine an occurrence of one or more events via the featurized data; (iii) train, via machine learning, one or more first order virtual sensor implemented by the server to detect the one or more events based on the featurized data; and (iv) monitor, via the virtual sensor, for subsequent occurrences of the one or more events based on featurized data from the sensor assembly." [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on." Agarwal discloses a system of training early stage models and feeding their outputs to later-stage models (interpreted as model chaining))
the method comprising: obtaining a parameter output node of a first model based on simulating one or more first objects using at least the first model and first error detection data that indicates one or more errors that occurred in historical operation of the first model, ([¶0052] "a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment." [¶0120] "The sensing system 100 including virtual sensors 118, as described herein, can be utilized to provide a uniform sensing substrate for all things related to smart building management, including [...] fault detection, etc." [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on." first model interpreted as first virtual sensor. Generating explicit faulty label outputs from a learned virtual sensor that represents the behavior of a physical sensor/system is interpreted as synonymous with simulating one or more first objects (sensors) using error detection data that indicates one or more errors that occurred in historical operation of the first model. See also FIG. 4)
wherein the first model is to be included in a chain of two or more models; ([¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on." See also FIG. 4 first order virtual sensor 1)
and wherein the first error detection data indicates a physical sensor error in a first system associated with the first model;([¶0052] "a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment." [¶0120] "The sensing system 100 including virtual sensors 118, as described herein, can be utilized to provide a uniform sensing substrate for all things related to smart building management, including [...] fault detection, etc.")
obtaining a parameter input node of a second model based on simulating one or more second objects using at least the second model ([¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on." See also FIG. 4 where directional arrows indicate input/output relationships between models as well as input/output relationships between the virtual sensor models and the physical sensors they are simulating)
and second error detection data that indicates one or more errors that occurred in historical operation of the second model, ([¶0052] "a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment." [¶0120] "The sensing system 100 including virtual sensors 118, as described herein, can be utilized to provide a uniform sensing substrate for all things related to smart building management, including [...] fault detection, etc.")
to obtain a parameter output node of the first model and a parameter input node of the second model; wherein the second model is to be included in the chain of two or more models([¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on." See also FIG. 4 where directional arrows indicate input/output relationships between models as well as input/output relationships between the virtual sensor models and the physical sensors they are simulating)
and wherein the second error detection data indicates a physical sensor error in a second system associated with the second model;([¶0052] "a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment." [¶0120] "The sensing system 100 including virtual sensors 118, as described herein, can be utilized to provide a uniform sensing substrate for all things related to smart building management, including [...] fault detection, etc.")
generating a connection prediction, using a model simulator and the parameter output node of the first model and the parameter input node of the second model, that the parameter output node of the first model is related to the parameter input node of the second model;([¶0065] "FIG. 3A depicts the computer system 104 including a second machine learning module 122 that receives the outputs (data) from the first order virtual sensors 120 to generate a second order virtual sensors 124." See FIG. 3B where Second Machine Learning Module 122 is clearly different from first and second order virtual sensors that the second machine learning module connects)
wherein a relationship between the input node of the second model and the output node of the first model is previously unknown, wherein the model simulator is different from the first model, wherein the model simulator is different from the second model, ([¶0065] "FIG. 3A depicts the computer system 104 including a second machine learning module 122 that receives the outputs (data) from the first order virtual sensors 120 to generate a second order virtual sensors 124." See FIG. 3B where Second Machine Learning Module 122 is clearly different from first and second order virtual sensors that the second machine learning module connects)
and wherein the model simulator is trained to at least generate predictions indicating relationships between parameter input nodes and parameter output nodes of other models that lack a nodal relationship to the model simulator([¶0008] "the server can have a library of previously trained machine learning models and/or may train machine learning models from prior data collection steps, crowd sourcing, or the like, for different activities and events, and the sensing system can directly send sensor data and have the server determine what events have occurred" [¶0065] "first order virtual sensors 120, include classifiers trained by a machine learning module on the output from one or more lower order sensors to identify the occurrence of a trained-for event or condition. The higher order virtual sensors can be trained on the outputs of at least one immediately lower order of virtual sensor in the hierarchical structure" [¶0045] "the machine learning module 116 can generate a machine learning model to detect correlations between the data and events that have occurred. In one aspect, the machine learning module 116 generates a classifier, which is an algorithm that is trained via a machine learning model to assign an input to one or more categories based upon the training that the classifier received. In this aspect, the classifier can be trained to identify the occurrence of a given event based upon the grouped, featurized data that is provided to the machine learning module 116 as training data. In training the classifier to identify an event, the machine learning module 116 can assess the informational power of different sensor channels and may select appropriate thresholds for optimal accuracy in characterizing the training data. The training by the machine learning module 116 causes the classifier to learn what sensor data streams are associated with an event type and, further, what characteristics of those data streams identify the event type with particularity")
and determining based at least in part on the connection prediction of the model simulator, to at least chain the first and second models and chaining the first and second models, wherein chaining the first and second models includes linking the parameter output node of the first model directly with the parameter input node of the second model, wherein the model simulator is separate from the chained first and second models([¶0065] "FIG. 3A depicts the computer system 104 including a second machine learning module 122 that receives the outputs (data) from the first order virtual sensors 120 to generate a second order virtual sensors 124." See FIG. 3B where Second Machine Learning Module 122 is clearly different from first and second order virtual sensors that the second machine learning module connects)
wherein the method is performed using one or more processors([¶0127] "the sensor assembly 102 includes a microcontroller 121 that includes a processor 123 coupled to a memory 125.").
While Agarwal discloses training a machine learning model to generate the coupling between first and second virtual sensors, Agarwal does not explicitly teach wherein the model simulator simulates individual models of the chain of two or more models separately from execution of the chain of two or more models.
Feng, in the same field of endeavor, teaches wherein the model simulator simulates individual models of the chain of two or more models separately from execution of the chain of two or more models ([¶0125] "there may be multiple, alternative model components available for simulating a given element of the high-level system design 128, and each component may be a model of that element" [¶0137] "the execution engine utilized by the co-simulation component discovery engine 118 is different than the execution engine used to execute a model component in a model." Feng discloses a separate co-simulation discovery layer that works with individual model components before and apart from execution of the final realized co-simulation and says the discovery engine may access only the inputs and outputs of those components ([¶0137]).).
Agarwal as well as Feng are directed towards stacked generalization for digital twins. Therefore, Agarwal as well as Feng are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Agarwal with the teachings of Feng by using a model simulator separate from the chain of execution of the two or more models. Feng provides as additional motivation for combination ([¶0261] “The master execution engine 912 may dynamically update a step size of a communication rate along a connection between co-simulation components. The master execution engine 912 may update a step size of the communication rate of data along the connection in the model based on the sensitivity analysis and/or the error. In some implementations, as the sensitivity of the connection increases over time, the step size of the communication rate may be reduced to reduce the possibility of error or inaccuracy in the simulation.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 2, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1 further comprising: optimizing the chained first and second models by recurrently linking related parameter output nodes with related parameter input nodes(Agarwal [¶0060] "creation of the virtual sensor 118 is an iterative process and the training of the classifier that forms the virtual sensor 118 may continue to improve and/or modify the virtual sensor 118 (e.g., using a feedback loop)." In Agarwal the chaining is not limited to one static first-to-second link, but rather creation is iterative and the higher-order architecture allows a second-order virtual sensor to subscribe to one or more first-order virtual sensors with the same principles extending to third and higher order virtual sensors. Functionally, that is a repeated or recurrent extension of the chain by adding further related upstream/downstream connections.).
Regarding claim 3, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 2, wherein recurrently linking related parameter output nodes with related parameter input nodes comprises: using the model simulator using current nodal relationships to predict new nodal relationships between parameter output nodes and parameter input nodes; (Agarwal [¶0061] "Each virtual sensor 118 can subscribe to (i.e., receive data from) the data stream of one or multiple sensors 110, in any combination" [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on" [¶0066] "FIG. 4 depicts the higher order virtual sensors receiving data from different levels or orders of sensors" In Agarwal the chaining is not limited to one static first-to-second link, but rather creation is iterative and the higher-order architecture allows a second-order virtual sensor to subscribe to one or more first-order virtual sensors with the same principles extending to third and higher order virtual sensors. Functionally, that is a repeated or recurrent extension of the chain by adding further related upstream/downstream connections.)
and chaining related parameter input nodes and parameter output nodes by re-linking the parameter output nodes with the parameter input nodes based on the current nodal relationships and predicted new nodal relationships(Agarwal [¶0061] "Each virtual sensor 118 can subscribe to (i.e., receive data from) the data stream of one or multiple sensors 110, in any combination" [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on" [¶0066] "FIG. 4 depicts the higher order virtual sensors receiving data from different levels or orders of sensors" Agarwal's deployed second-order and higher virtual sensors are reconfigured by subscription to the upstream sensor/virtual-sensor streams that the training system found relevant. This is interpreted as synonymous with re-linking the parameter output nodes with the parameter input nodes based on the current and new nodal relationships.).
Regarding claim 4, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1 further comprising: optimizing the chained first and second models by iteratively optimizing a converging series of the chained first and second models(Agarwal [¶0060] "creation of the virtual sensor 118 is an iterative process and the training of the classifier that forms the virtual sensor 118 may continue to improve and/or modify the virtual sensor 118 (e.g., using a feedback loop).").
Regarding claim 6, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1, wherein the model simulator executes a recurrent neural network(Agarwal [¶0057] "the virtual sensors 118 could all use the same machine learning technique or they could use different machine learning techniques. For example, some virtual sensors 118 could use […] neural networks" [¶0054] "There are many variants of neural networks with deep architecture depending on the probability specification and network architecture, including, but not limited to [...] recurrent neural network (RNN)-enhanced models").
Regarding claim 9, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1, wherein the first model is one of a known or black box system(Agarwal [¶0120] "The sensing system 100 including virtual sensors 118, as described herein, can be utilized to provide a uniform sensing substrate for all things related to smart building management, including […] fault detection, etc").
Regarding claim 10, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1, wherein the one or more first objects are one of physical or virtual devices(Agarwal [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on.").
Regarding claim 11, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 10, wherein the one or more first objects at least one of detect, measure, position, signal, gauge, or sense external stimuli(Agarwal [¶0070] "Each of the sensor assemblies 102 includes a plurality of sensors 110 for detecting various physical or natural phenomena in the environment in which the sensor assembly 102 is located and a control circuit for executing the various functions of the sensor assembly 102").
Regarding claim 12, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 11, wherein the one or more first objects are at least one of user configurable, editable, or removable(Agarwal [¶0052] " the graphical user interface 500 could be utilized to provide a “knocking” annotation 502 at the time on the sensor data stream 504 corresponding to when there was knocking on a door. The annotation 502 thus provides a label for the sensor data for the machine learning module 116 to train a virtual sensor 118 to detect the corresponding event. As another example, a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment.").
Regarding claim 13, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1, wherein the first and second models are simulated for a time range or a point in time(Agarwal [¶0083] "FIG. 7 depicts a timeline 200 illustrating various illustrative data sampling timescales for a variety of sensors and the events detectable by various sensors 110 at those sampling rates. In this example illustration, the vibration sensor 138, acoustic sensor 144, and EMI sensor 146 can sample on timescales on the order of milliseconds to minutes" [¶0093] "a virtual sensor 116 could be trained to detect rain, as depicted in FIG. 10, according to the temperature data 252 and/or the humidity data 254 due to the fact that rain is correlated with a drop in the temperature and an increase in the humidity. As another example, a virtual sensor 116 could be trained to detect night time " [¶0096] "a virtual sensor 116 could be trained to detect when the automobile is approaching a highway according to the acceleration data 292, which indicates that the automobile has been gradually accelerating for an extended period of time").
Regarding claim 14, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1 further comprising: causing display of the chained first and second models in a graphical user interface that depicts at least one of interconnections between the first and second models, the parameter input node, the parameter output node, or the one or more first objects(Agarwal [¶0052] "The graphical user interface 500 could be displayed on a client 106 (FIG. 5) connected to the sensing system 100, the computer system 104, or another computer system or device that is in communication with the sensing system 100. The graphical user interface 500 can allow users to visualize the sensor data streams (which can be either the raw sensor data or the featurized sensor data) and then indicate when various event types occurred, such as by annotating the sensor data streams with events types and the times that the event types occurred. By indicating when various event types occurred, the machine learning module 116 can then train a machine learning model, such as a classifier, to correlate various characteristics of the featurized sensor data streams with the occurrences of the particular events types. For example, the graphical user interface 500 could be utilized to provide a “knocking” annotation 502 at the time on the sensor data stream 504 corresponding to when there was knocking on a door. The annotation 502 thus provides a label for the sensor data for the machine learning module 116 to train a virtual sensor 118 to detect the corresponding event. As another example, a user could annotate a “faulty” label to a vibration sensor reading from a machine in a factory that is vibrating due to mechanical misalignment.").
Regarding claim 18, claim 18 is directed towards a system for performing the method of claim 1. Therefore, the rejection applied to claim 1 also applies to claim 18. Claim 18 also recites additional elements one or more non-transitory computer readable storage mediums storing program instructions; and one or more processors configured to execute the program instructions, wherein the program instructions, when executed, cause the system to (Agarwal [¶0124] "the back end server system comprises at least one server that comprises a processor and a memory for storing instructions that, when executed by the processor, cause the server to: (i) receive the featurized data from the sensor assembly; (ii) determine an occurrence of one or more events via the featurized data; (iii) train, via machine learning, one or more first order virtual sensor implemented by the server to detect the one or more events based on the featurized data; and (iv) monitor, via the virtual sensor, for subsequent occurrences of the one or more events based on featurized data from the sensor assembly." [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on.").
Regarding claim 19, the combination of Agarwal and Feng teaches A computer program product comprising one or more computer-readable storage mediums having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method Claim 1. (Agarwal [¶0124] "the back end server system comprises at least one server that comprises a processor and a memory for storing instructions that, when executed by the processor, cause the server to: (i) receive the featurized data from the sensor assembly; (ii) determine an occurrence of one or more events via the featurized data; (iii) train, via machine learning, one or more first order virtual sensor implemented by the server to detect the one or more events based on the featurized data; and (iv) monitor, via the virtual sensor, for subsequent occurrences of the one or more events based on featurized data from the sensor assembly." [¶0065] "the second order virtual sensor 124 receives data streams from the first order virtual sensors 120 that the second machine learning module 122 determined correlated to the event that the particular second order virtual sensor 124 was being trained on.").
Claim 5 is rejected under U.S.C. §103 as being unpatentable over the combination of Agarwal and Feng and Drori (“Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar”, 2019).
Regarding claim 5, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 4.
However, the combination of Agarwal and Feng doesn't explicitly teach wherein the converging series of the chained first and second models converges towards an optimum, wherein a gradient on the series of the chained first and second models converges towards the optimum.
Drori, in the same field of endeavor, teaches The computer-implemented method of Claim 4, wherein the converging series of the chained first and second models converges towards an optimum, wherein a gradient on the series of the chained first and second models converges towards the optimum([p. 3 §2.1] "The parameters are updated by stochastic gradient descent on the following loss function: [...] maximizing cross entropy between policy vector p and search probabilities , minimizing mean squared error between predicted performance v and actual pipeline evaluation e").
The combination of Agarwal and Feng as well as Drori are directed towards stacked generalization. Therefore, the combination of Agarwal and Feng as well as Drori are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Agarwal and Feng with the teachings of Drori by using the stacked generalization architecture and training in Drori to implement the digital twin models in Agarwal and Feng. Drori provides as additional motivation for combination ([Abstract] “In this work we extend AlphaD3M by using a pipeline grammar and a pre-trained model which generalizes from many different datasets and similar tasks. Our results demonstrate improved performance compared with our earlier work and existing methods on AutoML benchmark datasets for classification and regression tasks.”).
Claims 7 and 8 are rejected under U.S.C. §103 as being unpatentable over the combination of Agarwal and Feng and in further view of Gulsun (US9767557B1).
Regarding claim 7, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 6.
However, the combination of Agarwal and Feng doesn't explicitly teach further comprising: training the recurrent neural network, wherein training the recurrent neural network comprises unrolling the recurrent neural network.
Gulsun, in the same field of endeavor, teaches training the recurrent neural network, wherein training the recurrent neural network comprises unrolling the recurrent neural network ([Col. 6 l. 12-30] "After unrolling, an RNN can be trained based on ground truth training samples with back-propagation, similar to a conventional feed-forward neural network").
The combination of Agarwal and Feng as well as Gulsun are directed towards neural networks systems for medical imaging. Therefore, the combination of Agarwal and Feng as well as Gulsun are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Agarwal and Feng with the teachings of Gulsun by substituting the neural network in Zhang with a recurrent neural network and unrolling the RNN for training. Gulsun provides as additional motivation for combination ([Col. 6 l. 12-25] "After unrolling, an RNN can be trained based on ground truth training samples with back-propagation, similar to a conventional feed-forward neural network"). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 8, the combination of Agarwal, Feng, and Gulsun teaches The computer-implemented method of Claim 7, wherein training the recurrent neural network further comprises applying a backpropagation to the unrolled recurrent neural network to calculate and accumulate one or more gradients (Gulsun [Col. 6 l. 12-30] "After unrolling, an RNN can be trained based on ground truth training samples with back-propagation, similar to a conventional feed-forward neural network").
Claims 15, 16, and 17 are rejected under U.S.C. §103 as being unpatentable over the combination of Agarwal and Feng and in further view of Santos (“Visus: An Interactive System for Automatic Machine Learning Model Building and Curation”, 2019).
Regarding claim 15, the combination of Agarwal and Feng teaches The computer-implemented method of Claim 1.
However, the combination of Agarwal and Feng doesn't explicitly teach the first model is associated with a health value that indicates a health of the first model.
Santos, in the same field of endeavor, teaches the first model is associated with a health value that indicates a health of the first model ([p. 4] "To allow exploration of the pipelines generated by the Model Search, Visus displays them in a solution table as shown in Figure 3(E2). In this table, users can see the different solutions and their associated performance metrics. Users can also sort solutions by metric, allowing them to quickly identify the best solutions according to each metric. Additionally, Visus provides a histogram of the scores associated with each performance metric, which allows the users to visualize the distribution of scores of all generated solutions (see Figure 3(E1))" See FIG. 3 which shows scores indicating health of first model).
The combination of Agarwal and Feng as well as Santos are directed towards machine learning pipeline synthesis. Therefore, the combination of Agarwal and Feng as well as Santos are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Agarwal and Feng with the teachings of Santos by analyzing health of multiple models in a pipeline synthesis system having four or more primitives. Santos provides as additional motivation for combination ([p. 1 §1] “it also enables experts that possess domain knowledge to assess, validate, and potentially improve model outcomes”).
Regarding claim 16, the combination of Agarwal, Feng, and Santos teaches The computer-implemented method of Claim 15 further comprising: generating a model hierarchy based on the health of the first model and a health of the second model(Santos [p. 4] "To allow exploration of the pipelines generated by the Model Search, Visus displays them in a solution table as shown in Figure 3(E2). In this table, users can see the different solutions and their associated performance metrics. Users can also sort solutions by metric, allowing them to quickly identify the best solutions according to each metric. Additionally, Visus provides a histogram of the scores associated with each performance metric, which allows the users to visualize the distribution of scores of all generated solutions (see Figure 3(E1))" See FIG. 3 which shows hierarchy based on scores indicating health of first and second model).
Regarding claim 17, the combination of Agarwal, Feng, and Santos teaches The computer-implemented method of Claim 16 further comprising: grouping a third model with the first model and a fourth model with the second model that share related parameter nodes(Santos See FIG. 3 which explicitly shows generated pipeline having more than four primitives).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Wolpert (“STACKED GENERALIZATION”, 1992) is directed towards generalized black-box machine learning model ensembles.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIDNEY VINCENT BOSTWICK/Examiner, Art Unit 2124
Prosecution Insights