DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Status of Claims
Claims 1-20 are now pending.
Effective Filing Date
Examiner acknowledges that the instant application claims domestic benefit to provisionally filed application 63087841, filed 10/05/2020. This provisionally filed application has been reviewed and DOES NOT provide sufficient support for the instant application’s claims.
Examiner acknowledges that the instant application is a continuation of application number 17471520 filed 09/10/2021.
As such, the effective filing date of the application is 09/10/2021.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, it recites:
“A computer-implemented method for training a machine learning model to control robotic assembly tasks, the method comprising:
receiving transitions generated by one or more actors;
grouping the transitions into a plurality of sequences;
assigning a priority to each sequence included in the plurality of sequences;
sampling sequences from the plurality of sequences based on the assigned priorities; and performing a plurality of training operations on the machine learning model based on the plurality of sequences to generate a trained machine learning model.”
The limitations as drafted, under their broadest reasonable interpretation, covers performance
of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the step of “receiving transitions generated by one or more actors;” in the context of this claim, given the broadest reasonable interpretation encompasses the act of a user watching reviewing records of gathered data. Similarly, the steps of grouping, assigning and sampling could be performed mentally. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claim 1 only recites that the method is “computer-implemented” and “to control robotic assembly tasks.” All of the components are recited at a high-level of generality (i.e., as a generic computer and generic robot) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional generic computer components to perform the claimed method amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claim is not patent eligible.
Independent claims 11 and 20 are found to not be patent eligible for similar reasons to those of claim 1. The steps claimed are similarly directed to an abstract idea and are not integrated into a practical application. Claim 11 only recites that the instructions as claimed are stored on a generic “non-transitory computer-readable media” which are executed on one or more generic processors to control a generic robot. Claim 20 similarly claims a generic “system” which includes one or more “memories” storing instructions and one or more generic “processors.” The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception for reasons similar to those outlined above.
Dependent claims 2-10 and 12-19 are similarly rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The dependent claims have been given the full two-part analysis including analyzing the additional limitations both individually and in combination. The dependent claims, when analyzed individually, and in combination, are also held to be patent ineligible under 35 U.S.C. 101. The additional recited limitations of the dependent claims fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the dependent claims merely further narrow the abstract idea. These claims only recite the generic computing components. All of the components are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, the claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more
than the judicial exception. As discussed above with respect to integration of the abstract idea into a
practical application, the additional elements of a computing device to perform the claimed steps amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
As such, the claims are not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 6, 9-12, 14, 15 and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Pietquin et al. (US 20200151562 A1), hereinafter Pietquin.
Regarding claim 1, Pietquin discloses:
A computer-implemented method for training a machine learning model to control robotic assembly tasks, the method comprising:
receiving transitions generated by one or more actors (see at least [0052]: “The reinforcement learning system 100 includes a replay buffer 150, that is memory which stores reinforcement learning transitions as operation transition data, generated as a consequence of the agent 102a interacting with the environment 104 during operation of the RL system. The operation transition data may comprise experience tuples, each comprising state data, action data, new state data, and reward data, for example (s, a, s′, r). The stored experience tuples are used for training the reinforcement learning system.”);
grouping the transitions into a plurality of sequences (see at least [0030]: “The operation transition data may comprise tuples, that is data groups, comprising the state data, the action data, the reward data, and new state data representing the new state. A second input, which may be the same input as the first input, receives training data defining demonstration transition data for demonstration transitions. The demonstration transition data may comprise a set of the tuples from a demonstration of the task, such as a control task, within the environment. Thus reinforcement learning transitions stored in the replay buffer include the demonstration transition data. The neural network system may be configured to train the at least one actor neural network and the at least one critic neural network off-policy. The off-policy training may use the error signal and stored tuples from the replay buffer comprising tuples from both the operation transition data and the demonstration transition data.”)
assigning a priority to each sequence included in the plurality of sequences; sampling sequences from the plurality of sequences based on the assigned priorities (see at least [0061]: “A sample selection system 160 selects experience tuples from replay buffer 150 for use in off-policy training of the RL system 100. The sample selection system samples more important experience tuples more frequently. The probability of sampling a particular experience tuple may depend on its priority, which may be partly determined by the last TD error calculated for the transition represented by the experience tuple. The priority may include a positive constant for demonstration experience tuples to increase the probability that these are sampled. The priority may also include a small positive constant for all experience tuples to ensure that all transitions are sampled with some probability.”)
and performing a plurality of training operations on the machine learning model based on the plurality of sequences to generate a trained machine learning model (see the training operations in Figures 3a and 3b.)
Regarding claim 2, Pietquin discloses:
The computer-implemented method of claim 1, wherein each transition comprises at least one of an observation, an action, or a reward (see at least [0016]: “Each transition may comprise state data characterizing a state of the environment, action data defining an action performed, reward data representing a reward from the action, and new state data representing a new state to which the environment transitions in response to the action.”)
Regarding claim 3, Pietquin discloses:
The computer-implemented method of claim 1, further comprising storing the plurality of sequences in a replay buffer (see at least [0030]: “A replay buffer stores reinforcement learning transitions comprising operation transition data from operation of the system.”)
Regarding claim 4, Pietquin discloses:
The computer-implemented method of claim 1, wherein the machine learning model comprises a policy network and a value network (see at least “actor” and “critic” networks as shown in Fig. 1.)
Regarding claim 6, Pietquin discloses:
The computer-implemented method of claim 1, wherein at least one sequence included in the plurality of sequences comprises transitions generated within an episode (see at least [0021]: “In implementations the method may perform multiple training steps of the actor-critic system, that is multiple weight update steps, for each environment transition (state-action-new state update). This helps facilitate efficient use of the data gathered from operation of the system, which tends to be more costly than processing time. In implementations the method may separate operation in the environment, and training, into different time intervals or episodes, facilitating real-time operation of the system.”)
Regarding claim 9, Pietquin discloses:
The computer-implemented method of claim 1, wherein the priority assigned to each sequence included in the plurality of sequences is based at least in part on a temporal difference error (see at least [0059]: “During training the output of the critic neural network is used to train the actor neural network. Parameters of the critic neural network are also adjusted to learn an action-value function or Q-function, Q(s,a), representing the action-value or Q-value. The critic neural network may be trained using a temporal-difference (TD) error, as described further later. The TD error is dependent on the reward and this ultimately drives training of both the critic and actor neural networks.”)
Regarding claim 10, Pietquin discloses:
The computer-implemented method of claim 1, further comprising periodically synchronizing updated parameters of the machine learning model with the one or more actors (See at least [0014]: “The system may comprise learning and target actor neural networks and learning and target critic neural networks. The neural network system may be configured to update the learning critic neural network off-policy using the error signal. The error signal may be derived from the target critic target neural network, the target actor neural network, and the stored tuples of both the operation transition data and the demonstration transition data. The system may be configured to update the learning actor neural network using a deterministic policy gradient comprising a product of a gradient of the output of the learning critic neural network and a gradient of the output of the learning actor neural network evaluated using the stored tuples of both the operation transition data and the demonstration transition data. The system may be configured to, at intervals, update weights of the target actor neural network using the learning actor neural network and to update weights of the target critic neural network using the learning critic neural network. The at intervals updating may involve the weights of the target actor and critic neural networks slowly tracking those of the learning actor and critic neural networks.”)
Regarding claim 11, Pietquin discloses:
One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to train a machine learning model to control robotic assembly tasks, by performing the operations of:
receiving transitions generated by one or more actors (see at least [0052]: “The reinforcement learning system 100 includes a replay buffer 150, that is memory which stores reinforcement learning transitions as operation transition data, generated as a consequence of the agent 102a interacting with the environment 104 during operation of the RL system. The operation transition data may comprise experience tuples, each comprising state data, action data, new state data, and reward data, for example (s, a, s′, r). The stored experience tuples are used for training the reinforcement learning system.”);
grouping the transitions into a plurality of sequences (see at least [0030]: “The operation transition data may comprise tuples, that is data groups, comprising the state data, the action data, the reward data, and new state data representing the new state. A second input, which may be the same input as the first input, receives training data defining demonstration transition data for demonstration transitions. The demonstration transition data may comprise a set of the tuples from a demonstration of the task, such as a control task, within the environment. Thus reinforcement learning transitions stored in the replay buffer include the demonstration transition data. The neural network system may be configured to train the at least one actor neural network and the at least one critic neural network off-policy. The off-policy training may use the error signal and stored tuples from the replay buffer comprising tuples from both the operation transition data and the demonstration transition data.”)
assigning a priority to each sequence included in the plurality of sequences; sampling sequences from the plurality of sequences based on the assigned priorities (see at least [0061]: “A sample selection system 160 selects experience tuples from replay buffer 150 for use in off-policy training of the RL system 100. The sample selection system samples more important experience tuples more frequently. The probability of sampling a particular experience tuple may depend on its priority, which may be partly determined by the last TD error calculated for the transition represented by the experience tuple. The priority may include a positive constant for demonstration experience tuples to increase the probability that these are sampled. The priority may also include a small positive constant for all experience tuples to ensure that all transitions are sampled with some probability.”)
and performing a plurality of training operations on the machine learning model based on the plurality of sequences to generate a trained machine learning model (see the training operations in Figures 3a and 3b.)
Regarding claim 12, Pietquin discloses:
The one or more non-transitory computer-readable media of claim 11, wherein the operations further include maintaining, for each sequence in the plurality of sequences, a record of one or more gradient magnitudes generated during training (See at least [0026]: “The learning actor network may be updated using a deterministic policy gradient, integrating (averaging) over just the state space rather than both the state and action space, because the action policy is deterministic based upon the state. This policy gradient defines the performance of a policy function mapping the state data to the action data. The policy gradient may be approximated from a product of the gradient of the output of the (learning) critic neural network with respect to actions and the gradient of the output of the (learning) actor neural network with respect to its weights. These gradients may again evaluated on a sample or minibatch taken from the replay buffer.”)
Regarding claim 14, Pietquin discloses:
The one or more non-transitory computer-readable media of claim 11, wherein the operations further include generating a sampling probability for each sequence included in the plurality of sequences based on a non-uniform distribution (See at least [0012]: “The reinforcement learning system may include a sample selection system or sampling module to sample the reinforcement learning transitions according to a sampling probability. In implementations the sampling probability prioritizes sampling of the demonstration transition data tuples.”)
Regarding claim 15, Pietquin discloses:
The one or more non-transitory computer-readable media of claim 11, wherein the operations further include grouping transitions generated within an episode into one or more sequences included in the plurality of sequences (see at least [0021]: “In implementations the method may perform multiple training steps of the actor-critic system, that is multiple weight update steps, for each environment transition (state-action-new state update). This helps facilitate efficient use of the data gathered from operation of the system, which tends to be more costly than processing time. In implementations the method may separate operation in the environment, and training, into different time intervals or episodes, facilitating real-time operation of the system.”)
Regarding claim 18, Pietquin discloses:
The one or more non-transitory computer-readable media of claim 11, wherein the operations further include generating a temporal difference error for at least one sequence included the plurality of sequences and assign the priority based on the temporal difference error (see at least [0018]: “Optionally the replay buffer may also store an actor-critic error signal, such as a Temporal-Difference (TD) error signal, with the operation transitions to allow further prioritized replay by TD error, or “surprise”. Transitions may be discarded from the replay buffer as new transitions become available during operation of the system, for example discarding the oldest first. In implementations, however, some or all of the demonstration transitions are retained in the replay buffer throughout the training. The task demonstration may be performed before RL training of the system. The provision of a large replay buffer facilitates off-policy training using uncorrelated sample transitions.”)
Regarding claim 19, Pietquin discloses:
The one or more non-transitory computer-readable media of claim 11, wherein the operations further include periodically synchronizing updated parameters of the machine learning model with the one or more actors (see at least [0014]: “The system may comprise learning and target actor neural networks and learning and target critic neural networks. The neural network system may be configured to update the learning critic neural network off-policy using the error signal. The error signal may be derived from the target critic target neural network, the target actor neural network, and the stored tuples of both the operation transition data and the demonstration transition data. The system may be configured to update the learning actor neural network using a deterministic policy gradient comprising a product of a gradient of the output of the learning critic neural network and a gradient of the output of the learning actor neural network evaluated using the stored tuples of both the operation transition data and the demonstration transition data. The system may be configured to, at intervals, update weights of the target actor neural network using the learning actor neural network and to update weights of the target critic neural network using the learning critic neural network. The at intervals updating may involve the weights of the target actor and critic neural networks slowly tracking those of the learning actor and critic neural networks.”)
Regarding claim 20, Pietquin discloses:
A system, comprising:
one or more memories that include instructions (see at least [0078]: “The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.”)
and one or more processors that are coupled to the one or more memories (see at least [0078]: “Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.”)
and, when executing the instructions, are configured to train a machine learning model to control robotic assembly tasks, by performing the operations of:
receiving transitions generated by one or more actors (see at least [0052]: “The reinforcement learning system 100 includes a replay buffer 150, that is memory which stores reinforcement learning transitions as operation transition data, generated as a consequence of the agent 102a interacting with the environment 104 during operation of the RL system. The operation transition data may comprise experience tuples, each comprising state data, action data, new state data, and reward data, for example (s, a, s′, r). The stored experience tuples are used for training the reinforcement learning system.”);
grouping the transitions into a plurality of sequences (see at least [0030]: “The operation transition data may comprise tuples, that is data groups, comprising the state data, the action data, the reward data, and new state data representing the new state. A second input, which may be the same input as the first input, receives training data defining demonstration transition data for demonstration transitions. The demonstration transition data may comprise a set of the tuples from a demonstration of the task, such as a control task, within the environment. Thus reinforcement learning transitions stored in the replay buffer include the demonstration transition data. The neural network system may be configured to train the at least one actor neural network and the at least one critic neural network off-policy. The off-policy training may use the error signal and stored tuples from the replay buffer comprising tuples from both the operation transition data and the demonstration transition data.”)
assigning a priority to each sequence included in the plurality of sequences; sampling sequences from the plurality of sequences based on the assigned priorities (see at least [0061]: “A sample selection system 160 selects experience tuples from replay buffer 150 for use in off-policy training of the RL system 100. The sample selection system samples more important experience tuples more frequently. The probability of sampling a particular experience tuple may depend on its priority, which may be partly determined by the last TD error calculated for the transition represented by the experience tuple. The priority may include a positive constant for demonstration experience tuples to increase the probability that these are sampled. The priority may also include a small positive constant for all experience tuples to ensure that all transitions are sampled with some probability.”)
and performing a plurality of training operations on the machine learning model based on the plurality of sequences to generate a trained machine learning model (see the training operations in Figures 3a and 3b.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Pietquin in view of Hafner et al. (US 20210201156 A1), hereinafter Hafner.
Regarding claim 5, Pietquin discloses the computer-implemented method of claim 4.
Pietquin does not explicitly disclose, but Hafner, in the analogous field of machine learning teaches wherein at least one of the policy network or the value network comprises a long short-term memory (LSTM) layer (see at least [0040]: “The policy neural network 110 can have any of a variety of neural network architectures. For example, the architecture of the policy neural network 110 may include a sequence of one or more convolutional layers, followed by a recurrent layer (e.g., a long short-term memory (LSTM) layer) and an output layer that generates the action selection policy. ”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the LSTM layer as taught as Hafner because as described in [0040] of Hafner’s disclosure: “The policy neural network 110 can have any of a variety of neural network architectures.” The LSTM layer is listed as one example of a layer which a policy neural network can have.
Claims 7, 8, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Pietquin in view of Kompella et al. (US 20220101064 A1), hereinafter Kompella.
Regarding claim 7, Pietquin discloses the computer-implemented method of claim 6.
Pietquin does not explicitly disclose, but Kompella, in the analogous field of machine learning teaches wherein the episode includes multiple sequences in the plurality of sequences that include overlapping transitions (see at least [0073]: “In the continual case, however, the agent continues executing actions without a reset. This case can be treated as a special case of episodic, where the agent's initial state is the last observed state. An episode in the continual loop is considered to be a moving window of T−1 overlapping transitions.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the methods as taught by Kompella because as stated in Kompella [0006]-[0007]: “Prior ER (Experience Replay) algorithms were mostly applied to single RL task learning settings. A few that were applied in multi-task settings require either rewards to be binary or abundant. Some of these also require a parameterized specification of goals. In view of the foregoing, there is a need for a system and methods that improve upon off-polity reinforcement learning.”
Regarding claim 8, Pietquin discloses the computer-implemented method of claim 7.
Pietquin does not explicitly disclose, but Kompella teaches wherein an amount of overlap between the sequences included in the plurality of sequences is based on a number of transitions in the episode (see at least [0073]: “In the continual case, however, the agent continues executing actions without a reset. This case can be treated as a special case of episodic, where the agent's initial state is the last observed state. An episode in the continual loop is considered to be a moving window of T−1 overlapping transitions.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the methods as taught by Kompella because as stated in Kompella [0006]-[0007]: “Prior ER (Experience Replay) algorithms were mostly applied to single RL task learning settings. A few that were applied in multi-task settings require either rewards to be binary or abundant. Some of these also require a parameterized specification of goals. In view of the foregoing, there is a need for a system and methods that improve upon off-polity reinforcement learning.”
Regarding claim 16, Pietquin discloses the one or more non-transitory computer-readable media of claim 15.
Pietquin does not explicitly disclose, but Kompella teaches wherein the operations further include overlapping transitions between sequences included in the plurality of sequences (see at least [0073]: “The agent's control loop 18 can be categorized broadly as (a) episodic or (b) continual. In the episodic case, at the end of each episode, the agent's state is reset to an initial state (that can be different from its last observed state). Episodes in this case have non-overlapping transitions. In the continual case, however, the agent continues executing actions without a reset. This case can be treated as a special case of episodic, where the agent's initial state is the last observed state. An episode in the continual loop is considered to be a moving window of T−1 overlapping transitions.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the methods as taught by Kompella because as stated in Kompella [0006]-[0007]: “Prior ER (Experience Replay) algorithms were mostly applied to single RL task learning settings. A few that were applied in multi-task settings require either rewards to be binary or abundant. Some of these also require a parameterized specification of goals. In view of the foregoing, there is a need for a system and methods that improve upon off-polity reinforcement learning.”
Regarding claim 17, Pietquin discloses the one or more non-transitory computer-readable media of claim 15.
Pietquin does not explicitly disclose, but Kompella teaches wherein the operations further include determining an amount of overlap between sequences included in the plurality of sequences based on a number of transitions in the episode (see at least [0073]: “The agent's control loop 18 can be categorized broadly as (a) episodic or (b) continual. In the episodic case, at the end of each episode, the agent's state is reset to an initial state (that can be different from its last observed state). Episodes in this case have non-overlapping transitions. In the continual case, however, the agent continues executing actions without a reset. This case can be treated as a special case of episodic, where the agent's initial state is the last observed state. An episode in the continual loop is considered to be a moving window of T−1 overlapping transitions.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the methods as taught by Kompella because as stated in Kompella [0006]-[0007]: “Prior ER (Experience Replay) algorithms were mostly applied to single RL task learning settings. A few that were applied in multi-task settings require either rewards to be binary or abundant. Some of these also require a parameterized specification of goals. In view of the foregoing, there is a need for a system and methods that improve upon off-polity reinforcement learning.”
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Pietquin in view of Park et al. (US 20190361412 A1), hereinafter Park.
Regarding claim 13, Pietquin discloses the one or more non-transitory computer-readable media of claim 11.
Pietquin does not explicitly disclose, but Park, in the analogous field of data processing teaches wherein the operations further include tracking, for each sequence included in the plurality of sequences, a timestamp corresponding to when the sequence was last sampled (see at least [0156]: “In some embodiments, each data sample is received with a timestamp indicating a time at which the corresponding data value was measured or calculated. In other embodiments, data collector 612 adds timestamps to the data samples based on the times at which the data samples are received. Data collector 612 can generate raw timeseries data for each of the data points for which data samples are received. Each timeseries can include a series of data values for the same data point and a timestamp for each of the data values.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Pietquin with the timestamps as taught by Park in order to better track data during use.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Morlot et al. (US 20230108874 A1), which discloses:
“Generating a digital twin of a complex system including receiving at least one training dataset in which each sample includes information on a state and on associated action, including related time information, training a generative model over states, actions and time information to learn a topological space representing attainable system states, in an unsupervised fashion over those states, actions and time information, wherein the generative model learns the mapping to realistic samples includes the space and transitions associated with those samples subject to the actions, and outputting a digital twin including the topological space and transitions between the attainable states subject to the actions, for simulating behaviors of the system by the digital twin to properly achieve one or more tasks pertaining to the system. Applications to reinforcement learning, notably for biological cells.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658
/THOMAS E WORDEN/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights