DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 21-40 as originally filed are pending and have been considered as follows.
Priority
1. Acknowledgement is made that this application is a continuation of U.S. Patent Application No. 17/652,607 filed on 02/25/2022, which claims priority to U.S. Provisional Application No. 63/153,811 filed on 02/25/2021.
Information Disclosure Statement
2. The information disclosure statements (IDS) filed on 11/25/2024, 03/17/2025, 06/09/2025, 08/05/2025, 09/10/2025, and 12/25/2025 are being considered by the examiner.
Double Patenting
3. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 21-25, 28-32, and 35-39 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-6, 8-13, and 15-20 of U.S. Patent No. 12153414. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims cover substantially the same scope.
Refer to the table below to see claim mapping for double patenting:
Instant Application -18/959,158
U.S. Patent No. 12153414
21. (New) A method for training a control system, comprising:
1. A method for training a control system, comprising:
receiving, by a computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system; and
receiving, by a computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system;
based on the initial teacher policy, training, by the computing system, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
based on the initial teacher policy, generating, by the computing system, an initial student policy by training the control system to perform the task exemplified in the trajectory example, wherein the control system exists is a second action space that is lower dimension from the first action space, wherein movements of the control system in the second action space are limited compared to movements of the human operator in the first action space, wherein training the control system comprises:
causing the control system to mimic the movements of the human operator while performing the task,
causing the control system to mimic the movements of the human operator while performing the task, and
monitoring the movements of the control system using sensors,
monitoring the movements of the control system using sensors, and
generating an output trajectory of the control system performing the task based on the monitored movements,
generating an output trajectory of the control system performing the task based on the monitored movements;
determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
providing, by the computing system, the output trajectory of the control system to the human operator for determining a reproducibility of the trajectory example based on the output trajectory generated by the control system;
receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
receiving, by the computing system, an updated teacher policy based on an updated trajectory example generated by the human operator responsive to the determined reproducibility of the trajectory example; and
causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example,
based on the updated teacher policy, generating, by the computing system, an updated student policy by training the control system to perform the task exemplified in the updated trajectory example.
monitoring additional movements of the control system using the sensors,
generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
2. The method of claim 1, wherein generating, by the computing system, the updated student policy by training the control system to perform the task exemplified in the updated trajectory example comprises:
outputting an updated output trajectory of the control system performing the task.
determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.
22. (New) The method of claim 21, further comprising:
3. The method of claim 2, further comprising:
receiving, by the computing system, a further updated teacher policy comprising a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
receiving, by the computing system, a further updated teacher policy comprising a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
based on the further updated teacher policy, generating, by the computing system, a further updated output trajectory of the control system performing the task.
based on the further updated teacher policy, generating, by the computing system, a further updated student policy by training the control system to perform the task exemplified in the further updated trajectory example.
23. (New) The method of claim 21, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
4. The method of claim 1, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
24. (New) The method of claim 23, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
5. The method of claim 4, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
25. (New) The method of claim 21, further comprising:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
6. The method of claim 1, further comprising:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
7. The method of claim 1, wherein the task is a manufacturing task.
26. (New) The method of claim 21, wherein the control system learns to perform the task performed by the human operator in the first action space.
27. (New) The method of claim 21, wherein the updated teacher policy adapts a motion profile of the human operator to more closely match identified limitations of the control system performing the task.
28. (New) A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:
15. A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:
receiving, by the computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system; and
receiving, by a computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system;
based on the initial teacher policy, training, by the computing system, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
based on the initial teacher policy, generating, by the computing system, an initial student policy by training the control system to perform the task exemplified in the trajectory example, wherein the control system exists is a second action space that is lower dimension from the first action space, wherein movements of the control system in the second action space are limited compared to movements of the human operator in the first action space, wherein training the control system comprises:
causing the control system to mimic the movements of the human operator while performing the task,
causing the control system to mimic the movements of the human operator while performing the task, and
monitoring the movements of the control system using sensors,
monitoring the movements of the control system using sensors, and
generating an output trajectory of the control system performing the task based on the monitored movements,
generating an output trajectory of the control system performing the task based on the monitored movements;
determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
providing, by the computing system, the output trajectory of the control system to the human operator for determining a reproducibility of the trajectory example based on the output trajectory generated by the control system;
receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
receiving, by the computing system, an updated teacher policy based on an updated trajectory example generated by the human operator responsive to the determined reproducibility of the trajectory example; and
causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example,
based on the updated teacher policy, generating, by the computing system, an updated student policy by training the control system to perform the task exemplified in the updated trajectory example.
monitoring additional movements of the control system using the sensors,
generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
16. The non-transitory computer readable medium of claim 15, wherein generating, by the computing system, the updated student policy by training the control system to perform the task exemplified in the updated trajectory example comprises:
outputting an updated output trajectory of the control system performing the task.
determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.
29. (New) The non-transitory computer readable medium of claim 28, further comprising:
17. The non-transitory computer readable medium of claim 16, further comprising:
receiving, by the computing system, a further updated teacher policy comprising a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
receiving, by the computing system, a further updated teacher policy comprising a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
based on the further updated teacher policy, generating, by the computing system, a further updated output trajectory of the control system performing the task.
based on the further updated trajectory example, generating, by the computing system, a further updated student policy by training the control system to perform the task exemplified in the further updated trajectory example.
30. (New) The non-transitory computer readable medium of claim 28, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
18. The non-transitory computer readable medium of claim 15, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
31. (New) The non-transitory computer readable medium of claim 30, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
19. The non-transitory computer readable medium of claim 18, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
32. (New) The non-transitory computer readable medium of claim 28, further comprising:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
20. The non-transitory computer readable medium of claim 15, further comprising:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
33. (New) The non-transitory computer readable medium of claim 28, wherein the control system learns to perform the task performed by the human operator in the first action space.
34. (New) The non-transitory computer readable medium of claim 28, wherein the updated teacher policy adapts a motion profile of the human operator to more closely match identified limitations of the control system performing the task.
35. (New) A system comprising:
8. A system comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations, comprising:
receiving an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system; and
receiving an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the system; based on the initial teacher policy,
based on the initial teacher policy, training the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
based on the initial teacher policy, generating an initial student policy by training the control system to perform the task exemplified in the trajectory example, wherein the control system exists is a second action space that is lower dimension from the first action space, wherein movements of the control system in the second action space are limited compared to movements of the human operator in the first action space, wherein training the control system comprises:
causing the control system to mimic the movements of the human operator while performing the task,
causing the control system to mimic the movements of the human operator while performing the task, and
monitoring the movements of the control system using sensors, generating an output trajectory of the control system performing the task based on the monitored movements,
monitoring the movements of the control system using sensors, and generating an output trajectory of the control system performing the task based on the monitored movements;
determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
providing the output trajectory of the control system to the human operator for determining a reproducibility of the trajectory example based on the output trajectory generated by the control system;
receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
receiving an updated teacher policy based on an updated trajectory example generated by the human operator responsive to the determined reproducibility of the trajectory example; and
causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example,
based on the updated teacher policy, generating an updated student policy by training the control system to perform the task exemplified in the updated trajectory example.
monitoring additional movements of the control system using the sensors,
generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
9. The system of claim 8, wherein generating the updated student policy by training the control system to perform the task exemplified in the updated trajectory example comprises: outputting an updated output trajectory of the robot control system performing the task.
determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.
36. (New) The system of claim 35, wherein the operations further comprise:
10. The system of claim 9, wherein the operations further comprise:
receiving a further updated teacher policy comprising a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
receiving a further updated teacher policy based on a further updated trajectory example generated by the human operator based on the trajectory example, the output trajectory, the updated trajectory example, and the updated output trajectory of the control system performing the task; and
based on the further updated teacher policy, generating a further updated output trajectory of the control system performing the task.
based on the further updated teacher policy, generating a further updated student policy by training the control system to perform the task exemplified in the further updated trajectory example.
37. (New) The system of claim 35, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
11. The system of claim 8, wherein the trajectory example is projected from a first environment in which the human operator performs the task into a second environment in which the control system performs the task, wherein the first environment is a higher dimensional environment than the second environment.
38. (New) The system of claim 37, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
12. The system of claim 11, wherein the updated trajectory example is projected from the first environment in which the human operator performs the task into the second environment in which the control system performs the task.
39. (New) The system of claim 35, wherein the operations further comprise:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
13. The system of claim 8, wherein the operations further comprise:
minimizing a distance between the task as performed by the human operator and the task as performed by the control system.
14. The system of claim 8, wherein the task is a manufacturing task.
40. (New) The system of claim 35, wherein the updated teacher policy adapts a motion profile of the human operator to more closely match identified limitations of the control system performing the task.
Allowable Subject Matter
4. Claims 20-41 would be allowable upon filing of a terminal disclaimer to overcome the nonstatutory double patenting rejection set forth in this Office action.
5. The following is a statement of reasons for the indication of allowable subject matter:
The available prior art fails to teach or suggest based on the initial teacher policy, training, by the computing system, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising: determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount, receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example, causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example, monitoring additional movements of the control system using the sensors, generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount, in combination with the further limitations of the independent claims.
Takahashi (US 20200409379) teaches a method and apparatus for training a mobile robot that comprises receiving teacher policy based on a trajectory example generated by a user and training the robot to perform autonomous movements through learning steps.
Gonzalez Aguirre et al. (US 20190135300) teaches a method and apparatus for detecting multimodal anomaly in an autonomous vehicle that includes training a neural network where information from a higher dimensional space is compressed into a lower dimensional space. Oleynik (US 20190291277) teaches a system and method for training a robotic system to replicate actions and tasks from a human operator.
The combination of Takahashi, Gonzalez Aguirre, and Oleynik fails to teach training, by the computing system and based on the initial teacher policy, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising: determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount, receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example, causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example, monitoring additional movements of the control system using the sensors, generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.
Therefore, the combination of features is considered to be allowable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIEN MINH LE whose telephone number is (571)272-3903. The examiner can normally be reached Monday to Friday (8:30am-5:30pm eastern time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached on (571)272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M.L./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights