CTNF 18/041,824 CTNF 84695 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 112 07-30-01 AIA The following is a quotation of the first paragraph of 35 U.S.C. 112(a): (a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112: The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention. Claims 9-10, 18-19 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claims 9, 18, and the dependent claims thereof recite “ designing the task being performed by the controllable physical device based on active learning, such that the task includes task elements configured to yield most observations from the physical environment that are correlated to one or more sensitive parameters among the one or more physics parameters.” Emphasis added. The specification, at [0036], describes using sensitivity analysis to determine one or more sensitive parameters associated with a task. However, the specification provides no guidance as to how to “design” a task once the one or more sensitive parameters have been identified. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Step 2A Prong 1: Claim 1 recites, outside of the brackets: A [computer-implemented] method comprising: [(a) operating a controllable physical device to perform a task in a physical environment], (b) running forward simulations of the task by a [physics engine] based on one or more physics parameters, wherein the [physics engine] communicates with a [parameter data layer] in which each of the one or more physics parameters is represented as a probabilistic variable with an associated probability distribution, wherein for each forward simulation run, a tuple of parameter values is sampled from the probability distribution of the one or more physics parameters and fed to the [physics engine], (c) obtaining an observation pertaining to the task from the physical environment and a corresponding forward simulation outcome associated with each sampled tuple of parameter values, and (d) updating the probability distribution of the one or more physics parameters in the [parameter data layer] based on the observation from the physical environment and the corresponding forward simulation outcomes. The above limitations, but for the additional elements as described below, covers performance of the limitations in the human mind or on pen and paper, including observation, evaluation, judgment, opinion. Therefore, the limitations fall into the “mental processes” grouping of abstract ideas. Alternatively, the “running forward simulations” and “updating the probability distribution” limitations fall into the “mathematical concepts” grouping of abstract ideas. Claim 12 recites similar limitations. Step 2A Prong 2: This judicial exception is not integrated into a practical application. Claim 1 recites, “computer-implemented,” “operating a controllable physical device to perform a task in a physical environment,” and “physics engine,” and “parameter data layer.” Claim 12 recites similar limitations, along with “a processor,” and “a memory storing instructions.” The “operating” limitation represents the environment in which the abstract idea is used, in that the operating provides context for how the other claimed steps fit into a model updating process. This element is thus a mere indication of the field of use or technological environment in which the judicial exception is performed, like the step of administering a drug providing 6-thioguanine to patients with an immune mediated gastrointestinal disorder in Mayo, which the Supreme Court treated as merely indicating the field of use in which the recited correlations were identified. Alternatively, this element is also insignificant extra-solution activity because it merely gathers data for use in calculating the extent of curing completion. Further alternatively, this limitation amounts to “apply it” as the claim only recites machinery operated in its ordinary capacity. The remaining limitations (i.e. “computer-implemented,” “physics engine,” “parameter data layer,” “a processor,” and “a memory storing instructions”) amount to mere instructions to apply the exception via generic computer components. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application. The claim is directed to an abstract idea. Step 2B: The claims does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because. As discussed with respect to Step 2A Prong 2, the generically claimed “computer-implemented,” “physics engine,” “parameter data layer,” “a processor,” and “a memory storing instructions” amount to mere instructions to apply the exception via generic computer components. The “operating” limitation amounts to a mere indication of the field of use or technological environment, or alternatively “apply it.” For the consideration under extra-solution activity in Step 2A Prong 2, this has been re-evaluated in Step 2B and determined to be well-understood, routine, conventional activity in the field. The Chebotar et al. reference, in § IV(D) describes “operating a controllable physical device” in a manner (using a commercially available robot) that indicates that the additional elements are sufficiently well-known that it does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. 112(a). Accordingly, even in combination, these additional elements are not significantly more than the abstract idea. For these reasons, there is no inventive concept. The claim is not patent eligible. Dependent claims 2-6, 8-10, 13-15, 17-19 merely narrow the abstract idea, above, and may be considered a mental process and/or mathematical concept. Accordingly, even when viewed in combination, the claim does not amount to a practical application or significantly more than the abstract idea. Dependent claims 7 and 16 further narrows the abstract idea, above, via a type of co-simulation. The claims also recite a “digital twin” which amounts to generic computer implementation of the abstract idea. Accordingly, even when viewed in combination, the claim does not amount to a practical application or significantly more than the abstract idea. Dependent claims 11 recites a “non-transitory computer readable medium.” Similar to independent claim 1, the “non-transitory computer readable medium” amounts to mere instructions to apply the exception via a generic computer component. Accordingly, even when viewed in combination, the claim does not amount to a practical application or significantly more than the abstract idea. Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15-aia AIA Claim(s) 1, 2, 7-9, 11-13, 16-18 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Chebotar et al., “Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience”; 2019 INTENRATIONAL CONFERENCE ON ROBOTICS AND AUTOMOTATION, May 20-24, 2019 (Applicant’s IDS dated February 16, 2023, NPL cite 5) . Claim 1: Chebotar discloses: A computer-implemented method (Chebotar abstract) comprising: operating a controllable physical device to perform a task in a physical environment (Chebotar Fig. 3, “Reality”; § IV(A) “Tasks” disclosing controllable tasks in a physical environment; § IV(D) “Real robot experiments” disclosing operation of those tasks), running forward simulations of the task by a physics engine based on one or more physics parameters (Chebotar, in Fig. 3 showing RL/Simulation Training; § IV(B) “Simulation engine” disclosing multiple forward simulations of a task based on rigid body dynamics. § IV(C) showing other physics parameters), wherein the physics engine communicates with a parameter data layer in which each of the one or more physics parameters is represented as a probabilistic variable with an associated probability distribution (Chebotar § III(A) showing a distribution of simulation parameters; § III(C) showing the simulation parameters via a Gaussian distribution; ), wherein for each forward simulation run, a tuple of parameter values is sampled from the probability distribution of the one or more physics parameters and fed to the physics engine (Chebotar Algorithm I “SimOpt framework” showing providing a tuple of parameter values for simulation, § IV(D) showing 3 updates with 9600 simulations per update), (c) obtaining an observation pertaining to the task from the physical environment and a corresponding forward simulation outcome associated with each sampled tuple of parameter values (Chebotar Fig. 3 showing receipt of both simulation and reality data; Algorithm I “SimOpt framework” disclosing receiving real world observations and simulation outcomes; § III(B) disclosing use of both real world and simulation outcomes), and (d) updating the probability distribution of the one or more physics parameters in the parameter data layer based on the observation from the physical environment and the corresponding forward simulation outcomes (Chebotar Algorithm I “SimOpt framework” showing updating probability distribution based on the observations/simulations; § III(B) showing optimization of parameters based on observations/simulations; § IV(D) updates to parameters to improve performance following each iteration of the SimOpt procedure). Claim 2: Chebotar discloses: wherein the one or more physics parameters comprises a plurality of correlated physics parameters, the plurality of physics parameters being modeled as a probabilistic graphical model in the parameter data layer (Chebotar Fig. 4, § IV(B) disclosing graphical modeling of parameters which, as per § III(C) are probabilistically modeled). Claim 7: Chebotar discloses: performing a co-simulation of the physical task on a digital twin of the physical environment based on a mean value of each of the one or more physics parameters, which is obtained from the updated probability distribution of the one or more physics parameters in the parameter data layer (Chebotar Fig. 3 showing the simulation as a digital twin of the physical task; Fig. 6, § IV(C-D) showing iterative performance of simulations and updating mean values of various parameters). Claim 8: Chebotar discloses: predicting an uncertainty associated with the co-simulation of the physical task based on a variance in a current probability distribution of the one or more physics parameters in the parameter data layer (Chebotar Fig. 3 showing the simulation as a digital twin of the physical task; Fig. 6, § IV(C-D) showing probability distributions including standard deviations of each iteration). Claim 9: Chebotar discloses: designing the task being performed by the controllable physical device based on active learning, such that the task includes task elements configured to yield most observations from the physical environment that are correlated to one or more sensitive parameters among the one or more physics parameters (Chebotar § IV(D) showing two tasks including task elements to yield observations about various sensitive parameters of the one or more physics parameters). Claim 11: See above relevant rejection of claim 1. In addition, Chebotar, in Fig. 3, § IV(B-D) shows the system is computerized, which necessitates a non-transitory computer readable medium. Claim 12, 13, 16-18: See above relevant rejection of claims 1, 2, 7, 8, 9. In addition, Chebotar, in Fig. 3, § IV(B-D) shows the system is computerized, which necessitates a processor and memory . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim (s) 3, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chebotar et al in view of Ramos et al. (US 2020/0368906 A1) . Claim 3, 14: Chebotar, while generally disclosing various parameters, does not explicitly disclose the following limitation. However, Ramos discloses: wherein the one or more physics parameters comprise at least one parameter that is not directly observable from the physical environment, the at least one parameter being correlated to a physical effect directly observable from the physical environment (Ramos [0023], [0075], [0078] showing friction as a not directly observable parameter (see also instant specification [0025] describing friction as a parameter not directly observable)). It would have been obvious to one of ordinary skill in the art before the effective filing date to combine, with a reasonable expectation of success, the robotic simulation of Chebotar with the robotic simulation of Ramos that includes particular parameters because “Unfortunately, in some cases, lack of knowledge about the correct simulation parameters, oversimplified simulation models, or insufficient numerical precision for differential equation solvers may prevent the results of a simulation from being seamlessly transferable to the real-world systems.” 07-21-aia AIA Claim (s) 4, 5, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chebotar et al in view of Mahmood et al., “Setting up a Reinforcement Learning Task with a Real-World Robot,” International Conference on Intelligent Robots and Systems, October 1-5, 2018 in view of Rajeswaren et al., “EPOpt: Learning Robust Neural Network Policies Using Model Ensembles”; 5 October 2016 (Applicant’s IDS dated 2/16/2023 cite 3) . Claim 4 and 15: With respect to the limitation: wherein the steps (b), (c) and (d) are recursively performed over a series of discrete time steps during the performing of the step (a) based on a Bayesian filter, such that the updated probability distributions of the one or more physics parameters for a particular time step are utilized for sampling parameter values that are fed to the physics engine for running forward simulations for the next time step. Chebotar, in e.g. Fig. 3 discloses an RL iterative simulation framework in which forward running simulations are run for the task, but does not specifically disclose discretizing the task into time steps. However, Mahmood, in the § I similar discloses an RL framework for robotic tasks, of which, at § II, § III, § IV(A), and Fig. 2, teaches that tasks may be decomposed into discrete time steps including robotic and modeling at each step. One of ordinary skill in the art would have recognized that applying the known technique of Mahmood to Chebotar would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Mahmood to the teaching of Chebotar would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such time steps, as § II describes Markov Decision Processes as including discrete time steps. Further, applying discrete time steps to Chebotar that already utilizes a Markov Decision Process (§ III(A)), would have been recognized by one of ordinary skill in the art as resulting in an improved system that would allow more granular task controls. Chebotar/Mahmood, while generally discloses performing modeling over time periods, and Chebotar, in Fig. 3, Algorithm I discloses updating probability distributions, does not disclose doing so using a Bayesian filter. However, Rajeswaran, in Abstract, § 1, § 3.2 teaches that the RL robotic simulation using Bayesian updates for its parameters. One of ordinary skill in the art would have recognized that applying the known technique of Rajeswaran to Chebotar/Mahmood would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Rajeswaran to the teaching of Chebotar/Mahmood would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such a Bayesian filter. Further, applying a Bayesian to Chebotar that already utilizes distributions, would have been recognized by one of ordinary skill in the art as resulting in an improved system that would allow the particular modeling benefits known to Bayesian filter methodologies. Claim 5: With respect to the limitation: wherein the Bayesian filter comprises a particle filter wherein each sampled tuple of parameter values is represented as a particle, wherein the particle filter reinforces those particles whose forward simulation outcomes are closest to the observation from the physical environment in preference to those particles whose forward simulation outcomes are farther from the observation from the physical environment. Chebotar, as described above particularly in Fig. 3, Algorithm I, uses reality to update simulated parameter distributions, but does not explicitly disclose using a Bayesian filter with a particular filter that reinforces particles that are closest to real world observation. However, Rajeswaran, in § 3.2 describes using a particle filter with weighted samples. One of ordinary skill in the art would have recognized that applying the known technique of Rajeswaran to Chebotar/Mahmood would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Rajeswaran to the teaching of Chebotar/Mahmood would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such a particle filter. Further, applying a particle filter to Chebotar that already utilizes distributions, would have been recognized by one of ordinary skill in the art as resulting in an improved system that would allow the particular modeling benefits known to particle filter methodologies . 07-21-aia AIA Claim (s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chebotar et al in view of Cunha et al. (US 2018/0107767 A1) Claim 6: With respect to the limitation: modeling an initial probability distribution for each of the one or more physics parameters, wherein the initial probability distribution is a Gaussian distribution modeled around a user-specified mean value of a respective physics parameter Chebotar Algorithm I, § III(C) discloses a Gaussian distribution as an initial probability distribution of a parameter, and in Fig. 6, § IV(D) discloses the distribution includes a mean. Chebotar does not explicit disclose that the mean is “user-specified.” However, Cunha, in [0031] discloses that initial parameter values may be user-specified. One of ordinary skill in the art would have recognized that applying the known technique of Cunha to Chebotar would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Cunha to the teaching of Chebotar would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such user specified parameters. Further, applying user specified parameters to Chebotar that already include parameters, would have been recognized by one of ordinary skill in the art as resulting in an improved system that would allow more efficient and focused simulations based on user knowledge . 07-21-aia AIA Claim (s) 10, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chebotar et al. in view of Tahmasbi et al., “Investigation, sensitivity analysis, and multi-objective optimization of effective parameters on temperature and force in robotic drilling cortical bone”, Proceedings of the Institution of Mechanical Engineers, November 2017 . Claim 10, 19: Chebotar discloses: determining the one or more sensitive parameters (Chebotar Fig. 3) by: running forward simulations of the task using sampled parameter values from the probability distribution of the one or more physics parameters in the parameter data layer (Chebotar Fig. 3 showing the simulation as a digital twin of the physical task; Fig. 6, § IV(C-D) showing probability distributions including standard deviations of each iteration), estimating a simulation uncertainty associated with the task based on outcomes of the forward simulations (Chebotar Fig. 6, § IV(C-D) showing probability distributions including standard deviations of each iteration), and With respect to the limitation: identifying which of the one or more physics parameters contribute maximally to the simulation uncertainty. Chebotar, as shown above, generally discloses determining uncertainties associated with parameters, but does not specifically disclose determining specific sensitive parameters based on uncertainty. However, Tahmasbi, in Abstract, teaches using sensitivity analysis to determine the most effective input parameters of a model, which, as per the Introduction, last two paragraphs, sensitivity analysis can use uncertainties to determine the most effective and ineffective parameters. It would have been obvious to one or ordinary skill in the art before the effective filing date of the invention to combine, with a reasonable expectation of success, the simulations of Chebotar that include uncertainty measurements with the technique of Tahmasbi which utilize uncertainty measurements to identify particular effective parameters in order to “ascertain the effect” of particular input parameters. Tahmasbi, abstract. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN H FLYNN whose telephone number is (571)270-3108. The examiner can normally be reached Monday-Friday, 8:00 am - 5:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Beth Boswell can be reached at 571-272-6737. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /KEVIN H FLYNN/Primary Examiner, Art Unit 3600 Application/Control Number: 18/041,824 Page 2 Art Unit: 3600 Application/Control Number: 18/041,824 Page 3 Art Unit: 3600 Application/Control Number: 18/041,824 Page 4 Art Unit: 3600 Application/Control Number: 18/041,824 Page 5 Art Unit: 3600 Application/Control Number: 18/041,824 Page 6 Art Unit: 3600 Application/Control Number: 18/041,824 Page 7 Art Unit: 3600 Application/Control Number: 18/041,824 Page 8 Art Unit: 3600 Application/Control Number: 18/041,824 Page 9 Art Unit: 3600 Application/Control Number: 18/041,824 Page 10 Art Unit: 3600 Application/Control Number: 18/041,824 Page 11 Art Unit: 3600 Application/Control Number: 18/041,824 Page 12 Art Unit: 3600 Application/Control Number: 18/041,824 Page 13 Art Unit: 3600 Application/Control Number: 18/041,824 Page 14 Art Unit: 3600 Application/Control Number: 18/041,824 Page 15 Art Unit: 3600 Application/Control Number: 18/041,824 Page 16 Art Unit: 3600 Application/Control Number: 18/041,824 Page 17 Art Unit: 3600 Application/Control Number: 18/041,824 Page 18 Art Unit: 3600
Prosecution Insights