DETAILED ACTION
This is the first office action in response to U.S. application 19/090,796. All claims are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 7, 9-14, 16-18 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Vogelsong (US 10926408).
Regarding claim 1, Vogelsong teaches a system (Fig. 2A), comprising:
a communication interface (Col. 8 lines 17-33 discuss the system including operating system 212 interacting with the computing system 200 as well as the physics simulation engine 214, feedback engine 216, and machine learning system 218 discussing that the systems can be in direct communication with one another or distributed among computing devices within a networked computing system where the systems being in communication with each other is interpreted as a communication interface further supported by Fig. 3A showing the robot control policy in communication with the physics simulation engine which is in communication with the feedback engine similar to the instant application’s depiction of the communication interface 414 in Fig. 4B); and
a processor coupled to the communication interface (Col 8 lines 1-33 discuss the system utilizing processors to perform the interaction between the systems) and configured to:
receive from a robotic control system that includes an internal state machine, via the communication interface, a communication indicating an action to be performed by a robotic element, wherein the internal state machine is used to maintain a view of a state of the robotic element (Fig. 3A shows the robotic control policy communicating information to the physics simulation engine with Col. 3 lines 44-57 discussing the robot control policy outputs a specification of actions that the robot should take and with Col. 9 lines 24-29 stating “The recorded observations data repository 232 can store data representing recorded task performance trials. Recorded observations can include audio signals, still images, video images sequences, electromagnetic tracking data, and textual information, depending upon the nature of a particular task.” which is interpreted as an internal state machine);
simulate performance of the action by the robotic element using at least one mock agent and a state tracker that monitors a simulated state of the at least one mock agent independently of the internal state machine (Col. 12 lines 37-47 “The virtual phase 300A of the training technique involves physics simulation engine 214 running simulations of the robotic system performing the task…the robot 110 stepping through multiple actions for the task 101 in the simulated environment 30” where the robot in the simulated environment is interpreted as the mock agent and with Col. 12 lines 48-63 discussing the system monitoring the simulated environment that are supplying inputs from the simulated environment where this is interpreted as a state tracker);
update a state tracking data to reflect a virtual change to one or more state variables as a result of simulated performance of the action (Col. 13 lines 29-34 “The physics simulation engine 214 can be additionally or alternatively configured to record events occurring within the simulated environment 305 and send this data to the feedback engine 214, where it may be programmatically evaluated by a machine learning classifier as described herein.”); and
report to the robotic control system, via the communication interface, successful is completion of the action by the robotic element (Col. 13 lines 35-38 “Recordings of these simulated trials are provided to the feedback engine 214, which generates success/reward scores or outputs comparison preferences indicating which of a number of performances was more successful…The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task” where Fig. 3A shows the feedback engine in communication with the robotic control policy via the machine learning system).
Regarding claim 2, Vogelsong teaches a memory coupled to the processor and configured to store the state tracking data (Col. 13 lines 35-38 discuss the recorded (stored) simulated trials (state tracking data) to the feedback engine where Col. 8 lines 17-33 discuss this being executed with a memory coupled to one or more processors).
Regarding claim 3, Vogelsong teaches wherein the action indicated in the communication is determined by the robotic control system according to a plan generated by the robotic control system to fulfill a set of requirements using a set of robotic elements that includes the robotic element (Fig. 3A shows the robotic control policy communicating information to the physics simulation engine with Col. 3 lines 34-36 stating “A control policy for this task 100 would specify the actions the robot 110 can take to complete the steps needed for completing the task 101.”).
Regarding claim 4, Vogelsong teaches wherein the robotic control system is configured to use a simulation result generated based at least in part on the simulated performance of the action by the robotic element to validate or refine the plan (Col. 13 lines 35-52 state “Recordings of these simulated trials are provided to the feedback engine 214, which generates success/reward scores or outputs comparison preferences indicating which of a number of performances was more successful…The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task”).
Regarding claim 7, Vogelsong teaches wherein the processor is configured to simulate performance of the action by the robotic element at least in part by using a model comprising data that represents a physical configuration and dynamic behavior characteristics of the robotic element (Col. 12 lines 48-63 “The simulated environment 305 approximates the real-world environment in which the robotic system 110 will perform the task 101. The simulated environment 305 represents a comprehensive, high-fidelity simulation of the scenarios and tasks involved, including any objects being manipulated, the robot performing the manipulation…robot kinematics may be known with a high degree of precision from its manufacturing specifications and/or calibration test data”).
Regarding claim 9, Vogelsong teaches wherein the robotic element comprises a robotic arm having an end effector (Col. 7 lines 22-25 “The robotic system 110 can be a robot having a number of linkages coupled by a number of joints (motorized or passive) and one or more end effectors configured to interact with the robot's environment”).
Regarding claim 10, Vogelsong teaches wherein the action includes using the robotic arm and end effector to pick an item from a source location and place the item in a destination location (Col. 3 lines 31-43 “an example robot 110 performing an example task 101 of tossing an object 105 into a box 120…a robotic system picks up ordered items from storage structures and transfers them to bins or boxes for shipment to customers.”).
Regarding claim 11, Vogelsong teaches wherein the source location comprises a source tray or other source receptacle and the destination location comprises a destination tray or other receptacle (Col. 3 lines 31-43 “an example robot 110 performing an example task 101 of tossing an object 105 into a box 120…a robotic system picks up ordered items from storage structures and transfers them to bins or boxes for shipment to customers.”).
Regarding claim 12, Vogelsong teaches wherein the processor is configured to detect an error condition associated with simulated performance of the action by the robotic element (Col. 11 lines 6-17 “the feedback can be a performance score, a binary “succeeded/failed” indication, or other appropriate metrics for indicating the level of success of a particular task performance.”).
Regarding claim 13, Vogelsong teaches wherein the processor is configured to provide to the robotic control system a feedback data representing the error condition (Col. 11 lines 6-17 “the feedback can be a performance score, a binary “succeeded/failed” indication, or other appropriate metrics for indicating the level of success of a particular task performance.”).
Regarding claim 14, Vogelsong teaches wherein the robotic control system is configured to refine a plan based at least in part on the feedback data (Col. 13 lines 39-42 “The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task”).
Regarding claim 16, Vogelsong teaches a method (Fig. 3A), comprising:
receiving from a robotic control system that includes an internal state machine, via a communication interface, a communication indicating an action to be performed by a robotic element, wherein the internal state machine is used to machine a view of a state of the robotic element (Fig. 3A shows the robotic control policy communicating information to the physics simulation engine with Col. 3 lines 44-57 discussing the robot control policy outputs a specification of actions that the robot should take and with Col. 9 lines 24-29 stating “The recorded observations data repository 232 can store data representing recorded task performance trials. Recorded observations can include audio signals, still images, video images sequences, electromagnetic tracking data, and textual information, depending upon the nature of a particular task.” which is interpreted as an internal state machine);
simulating performance of the action by the robotic element using at least one mock agent and a state tracker that monitors a simulated state of the at least one mock agent independently of the internal state machine (Col. 12 lines 37-47 “The virtual phase 300A of the training technique involves physics simulation engine 214 running simulations of the robotic system performing the task…the robot 110 stepping through multiple actions for the task 101 in the simulated environment 30” where the robot in the simulated environment is interpreted as the mock agent and with Col. 12 lines 48-63 discussing the system monitoring the simulated environment that are supplying inputs from the simulated environment where this is interpreted as a state tracker);
updating a state tracking data to reflect a virtual change to one or more state variables as a result of simulated performance of the action (Col. 13 lines 29-34 “The physics simulation engine 214 can be additionally or alternatively configured to record events occurring within the simulated environment 305 and send this data to the feedback engine 214, where it may be programmatically evaluated by a machine learning classifier as described herein.”); and
reporting to the robotic control system, via the communication interface, successful completion of the action by the robotic element (Col. 13 lines 35-38 “Recordings of these simulated trials are provided to the feedback engine 214, which generates success/reward scores or outputs comparison preferences indicating which of a number of performances was more successful…The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task” where Fig. 3A shows the feedback engine in communication with the robotic control policy via the machine learning system).
Regarding claim 17, Vogelsong teaches wherein the action indicated in the communication is determined by the robotic control system according to a plan generated by the robotic control system to fulfill a set of requirements using a set of robotic elements that includes the robotic element (Fig. 3A shows the robotic control policy communicating information to the physics simulation engine with Col. 3 lines 34-36 stating “A control policy for this task 100 would specify the actions the robot 110 can take to complete the steps needed for completing the task 101.”).
Regarding claim 18, Vogelsong teaches wherein the robotic control system is configured to use a io simulation result generated based at least in part on the simulated performance of the action by the robotic element to validate or refine the plan (Col. 13 lines 35-52 state “Recordings of these simulated trials are provided to the feedback engine 214, which generates success/reward scores or outputs comparison preferences indicating which of a number of performances was more successful…The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task”).
Regarding claim 20, Vogelsong teaches a computer program product embodied in a non-transitory computer readable medium and comprising computer instructions (Col. 21 line 60 – Col. 22 line 9 discuss the inventive system being implemented with a non-transitory computer-readable storage medium) for:
receiving from a robotic control system that includes an internal state machine, via a communication interface, a communication indicating an action to be performed by a robotic element, wherein the internal state machine is used to maintain a view of a state of the robotic element (Fig. 3A shows the robotic control policy communicating information to the physics simulation engine with Col. 3 lines 44-57 discussing the robot control policy outputs a specification of actions that the robot should take and with Col. 9 lines 24-29 stating “The recorded observations data repository 232 can store data representing recorded task performance trials. Recorded observations can include audio signals, still images, video images sequences, electromagnetic tracking data, and textual information, depending upon the nature of a particular task.” which is interpreted as an internal state machine);
simulating performance of the action by the robotic element using at least one mock agent and a state tracker that monitors a simulated state of the at least one mock agent independently of the internal state machine (Col. 12 lines 37-47 “The virtual phase 300A of the training technique involves physics simulation engine 214 running simulations of the robotic system performing the task…the robot 110 stepping through multiple actions for the task 101 in the simulated environment 30” where the robot in the simulated environment is interpreted as the mock agent and with Col. 12 lines 48-63 discussing the system monitoring the simulated environment that are supplying inputs from the simulated environment where this is interpreted as a state tracker);
updating a state tracking data to reflect a virtual change to one or more state variables as a result of simulated performance of the action (Col. 13 lines 29-34 “The physics simulation engine 214 can be additionally or alternatively configured to record events occurring within the simulated environment 305 and send this data to the feedback engine 214, where it may be programmatically evaluated by a machine learning classifier as described herein.”); and
reporting to the robotic control system, via the communication interface, successful completion of the action by the robotic element (Col. 13 lines 35-38 “Recordings of these simulated trials are provided to the feedback engine 214, which generates success/reward scores or outputs comparison preferences indicating which of a number of performances was more successful…The evaluation from the feedback engine 214 guides the machine learning system 218 to generate and refine a robotic control policy for the task” where Fig. 3A shows the feedback engine in communication with the robotic control policy via the machine learning system).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-6 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Vogelsong in view of Graca (US 20140148949).
Regarding claim 5, Vogelsong teaches the robotic control system providing the robotic control policy developed using the simulated tasks to the controller 208 of the robotic system to control the system while performing trails of the task in the real world environment (Col. 13 lines 57-63) but does not explicitly teach wherein the communication is received from the robotic control system and the report is sent to the robotic control system via a same interface used by the robotic control system to communicate with a non-simulated agent configured to control operation of the robotic element to perform the action in the real, physical world.
Graca teaches wherein the communication is received from the robotic control system and the report is sent to the robotic control system via a same interface used by the robotic control system to communicate with a non-simulated agent configured to control operation of the robotic element to perform the action in the real, physical world ([0022] “The robot control system 20 includes a data storage device 22 for obtaining data and storing data communicated from the controllers 17, 18, 19. The data storage device 22 can also be included with any of the controllers 17, 18, 19. The data storage device 22 is capable of being connected to the real robot work cell 10 and to a robot simulation device 30 positioned external to the real robot work cell 10”).
Vogelsong teaches providing the robotic control policy developed using the simulated tasks to the controller 208 of the robotic system to control the system while performing trails of the task in the real world environment. Graca teaches using a data storage device in communication with the controllers and the robot simulation device. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to take the system of Vogelsong and modify it with the storage device communication of Graca as this allows for direct communication with the robot controllers which minimizes the amount of required equipment making the system more cost efficient.
Regarding claim 6, Vogelsong teaches using a generated robotic control policy during a simulation of a task but does not explicitly teach wherein the processor is further configured to emulate a robotic control agent associated with the robotic element.
Graca teaches wherein the processor is further configured to emulate a robotic control agent associated with the robotic element ([0022] “The robot simulation device 30 is a device used to simulate operation of the robots 11, 12, 13, 14, 15, 16 and facilitate obtaining optimum placement and movement of the robots 11, 12, 13, 14, 15, 16. The robot simulation device 30 has a processor 31 and a memory 32.”).
Vogelsong teaches simulating a robotic system to develop a control policy. Graca teaches emulating the operation of the robots using a processor. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to take the system of Vogelsong and modify it with the simulated operation of the robots of Graca so that the simulated kinematic motions of the robot were motions that could be performed by the controller in a real world environment.
Regarding claim 19, Vogelsong teaches the robotic control system providing the robotic control policy developed using the simulated tasks to the controller 208 of the robotic system to control the system while performing trails of the task in the real world environment (Col. 13 lines 57-63) but does not explicitly teach wherein the communication is received from the robotic control system and the report is sent to the robotic control system via a same interface used by the robotic control system to communicate with a non-simulated agent configured to control operation of the robotic element to perform the action in the real, physical world.
Graca teaches wherein the communication is received from the robotic control system and the report is sent to the robotic control system via a same interface used by the robotic control system to communicate with a non-simulated agent configured to control operation of the robotic element to perform the action in the real, physical world ([0022] “The robot control system 20 includes a data storage device 22 for obtaining data and storing data communicated from the controllers 17, 18, 19. The data storage device 22 can also be included with any of the controllers 17, 18, 19. The data storage device 22 is capable of being connected to the real robot work cell 10 and to a robot simulation device 30 positioned external to the real robot work cell 10”).
Vogelsong teaches providing the robotic control policy developed using the simulated tasks to the controller 208 of the robotic system to control the system while performing trails of the task in the real world environment. Graca teaches using a data storage device in communication with the controllers and the robot simulation device. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to take the system of Vogelsong and modify it with the storage device communication of Graca as this allows for direct communication with the robot controllers which minimizes the amount of required equipment making the system more cost efficient.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Vogelsong in view of Kamrani (US 20070106421).
Regarding claim 8, Vogelsong teaches a simulated environment which approximates the real-world environment of the robotic system but does not explicitly teach wherein the model includes data indicating an amount of time associated with performance of the action by the robotic element ([0124] “After finding the boundaries in each direction, a simulation of path will be performed to calculate the cycle-times for each of the tests in the experiment. The task is simulated with a virtual controller, that is a simulated robot control system that gives accurate cycle-time simulations, and the time is measured during the simulation.”).
Kamrani teaches wherein the model includes data indicating an amount of time associated with performance of the action by the robotic element ([0124] “After finding the boundaries in each direction, a simulation of path will be performed to calculate the cycle-times for each of the tests in the experiment. The task is simulated with a virtual controller, that is a simulated robot control system that gives accurate cycle-time simulations, and the time is measured during the simulation.”).
Vogelsong teaches a robotic system using simulated tasks. Kamrani teaches including the cycle times within the simulated tasks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to take the simulation of Vogelsong and modify it with the time cycle of Kamrani as Kamrani teaches that this allows to optimize the position and orientation of the task creating a more efficient system.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Vogelsong in view of Takeda (US 20180178379).
Regarding claim 15, Vogelsong teaches estimating timing within the robotic simulation (Col. 4 lines 6-11) but does not explicitly teach wherein the at least one mock agent reads the state tracking data to estimate an amount of time the at least one agent would take to perform an assigned task or operation.
Takeda teaches wherein the at least one mock agent reads the state tracking data to estimate an amount of time the at least one agent would take to perform an assigned task or operation ([0041] discusses virtually performing a task with a robot to measure a time required to perform an operation).
Vogelsong teaches simulating robotic performance of tasks. Takeda teaches measuring a time required for a robot to perform a task in a simulation. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the system of Vogelsong with the time measurement of Takeda as Takeda teaches that this allows for optimization of the task by minimized task time [0043] making the system more efficient.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Tunyasuvunakool (US 20190126472) teaches controlling an agent within a simulated environment; and Diankov (US 20230016733) teaches simulating a robotic control policy.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIELLE M JACKSON whose telephone number is (303)297-4364. The examiner can normally be reached Monday-Friday 7:00-4:30 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached at (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.M.J./Examiner, Art Unit 3657 /ABBY LIN/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights