Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This is a non-final Office Action on the merits. Claims 1-28 are currently pending and are addressed below.
Information Disclosure Statement
The information disclosure statements (IDS) are being considered by the examiner.
Election/Restrictions
Claims 14-28 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 06/01/2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-5 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claims 4-5, the claims recite multiple recitations of “at least one additional sensor”, rendering the scope unclear, since it is unclear which “additional sensor” is being referenced in the various claim limitations.
In the art rejections below the claims have been treated as best understood by the examiner. Any claim not explicitly rejected under this heading is rejected as being dependent on an indefinite claim.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 2 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. In the instant case, the definition of “one or more physical actions” including “any other physical action” does not further limit claim 1 since it does not further limit the “one or more physical actions” beyond the scope of claim 1. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 7-10, and 13 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kolluri et al. (US 2021/0362328).
Regarding claim 1:
Kolluri teaches a robotic training system comprising:
one or more leader robotic devices configured for human manipulation (demonstration device 122, see at least Fig. 12, ¶0217-0231, alternately local robot 170 manipulated by user in demonstration mode, see at least ¶0051-0053, ¶0072); and
one or more follower robotic devices configured to replicate at least one movement of the one or more leader robotic devices (robots 170a-n, execution mode, see at least ¶0057); and
one or more force sensors, torque sensors, or force-torque sensors on at least one of: the one or more leader robotic devices, one or more of the one or more follower robotic devices, or any combination thereof (see at least ¶0046, ¶0080, ¶0219-0223);
where the robotic training system is configured to:
process force data, torque data, or force-torque data measured by the one or more force sensors, torque sensors, or force-torque sensors among the one or more leader robotic devices and the one or more follower robotic devices, where the force data, torque data, or force-torque data corresponds to one or more physical actions performed by the robotic training system (see at least ¶0046, ¶0223); and
record the force data, torque data, or force-torque data as demonstration data, where the demonstration data is adapted for training of at least one artificial intelligence (AI) model, where the AI model is designed to train at least one robot to perform the one or more physical actions performed by the robotic training system (see at least ¶0044-0065).
Regarding claim 2:
Kolluri further teaches where the one or more physical actions comprise at least one of: lifting, twisting, pouring, translating, carrying, moving, tipping, any other physical action, or any combination thereof (see at least ¶0218-0221).
Regarding claim 3:
Kolluri further teaches at least one user control interface physically mounted on the one or more leader robotic devices, the at least one user control interface configured to control at least one of: one or more demonstration recording operations, synchronization between the one or more leader and follower robotic devices, or any combination thereof (see at least Fig. 12C, handle 1220, trigger 1230, ¶0221, broadly interpreted, actuating the trigger controls demonstration recording operations).
Regarding claim 7:
Kolluri teaches a robotic training system comprising:
one or more leader robotic devices configured for human manipulation (demonstration device 122, see at least Fig. 12, ¶0217-0231, alternately local robot 170 manipulated by user in demonstration mode, see at least ¶0051-0053, ¶0072); and
one or more follower robotic devices configured to replicate at least one movement of the one or more leader robotic devices (robots 170a-n, execution mode, see at least ¶0057);
one or more interchangeable electromechanical end effector tools mounted on at least one of: the one or more leader robotic devices, the one or more of follower robotic devices, or any combination thereof (tool, see at least ¶0011, ¶0256, the Examiner notes that any tool is, in principle “interchangeable” since it is at least possible to change the tool with at least a tool of the same type.);
where the robotic training system is configured to:
process information corresponding to one or more physical actions performed using the one or more interchangeable electromechanical end effector tools among the one or more leader robotic devices and the one or more follower robotic devices (see at least ¶0046, ¶0223); and
record the information corresponding to the one or more physical actions performed using the one or more interchangeable electromechanical end effector tools as demonstration data, where the demonstration data is adapted for training of at least one artificial intelligence (AI) model, where the AI model is designed to train at least one robot to perform the one or more physical actions performed by the robotic training system (see at least ¶0044-0065).
Regarding claim 8:
Kolluri teaches A handheld device for robotic skill demonstration comprising:
a body portion configured to be manipulated by a user (demonstration device 122, see at least Fig. 12, ¶0217-0231); and
one or more force sensors, torque sensors, or force-torque sensors configured to measure force data, torque data, or force-torque data (see at least ¶0219-0223);
where the handheld device is configured to record the force data, the torque data, or the force-torque data as demonstration data, where the force data, torque data, or force-torque data corresponds to one or more physical actions performed by the user with the handheld device (see at least ¶0223);,
and where the demonstration data is adapted for training of at least one artificial intelligence (AI) model, where the AI model is designed to train, using the force data, torque data, or force-torque data, at least one robot to perform the one or more physical actions demonstrated by the user (see at least ¶0044-0065).
Regarding claim 9:
Kolluri further teaches further comprising one or more interchangeable electromechanical end effector tools (tool, see at least ¶0011, ¶0256, the Examiner notes that any tool is, in principle “interchangeable” since it is at least possible to change the tool with at least a tool of the same type.).
Regarding claim 10:
Kolluri further teaches wherein the force data, the torque data, or the force-torque data measured by the one or more force sensors, torque sensors, or force-torque sensors comprise at least one of: force associated with gripping of the one or more interchangeable electromechanical end effector tools, torque associated with weighted rotation of the one or more interchangeable electromechanical end effector tools, or any combination thereof (see at least ¶0223).
Regarding claim 13:
Kolluri further teaches one or more cameras present on a mobile device, where the handheld device is configured to house the mobile device, the one or more cameras allowing a vantage point of the mobile device to be synchronized with the vantage point of visual data captured by the one or more cameras (camera wristband, see at least Fig. 3B-D, ¶0114-0121).
Claim Rejections - 35 USC § 103
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Kolluri as applied to claim 1 above, and further in view of Taitler (US 2009/0132088).
Regarding claim 4-5:
Kolluri teaches the limitations as in claim 1 above.
Kolluri further teaches additional sensors, including at least depth sensors, non-visible spectrum data, kinematic data, thermal data, and sound data (see at least ¶0046).
Kolluri does not explicitly teach at least one sensor positioned independently from and provides a view of the one or more follower robotic devices.
Taitler teaches a system and method of robotic demonstration learning, including at least one leader robotic device, and at least one follower robotic device and further comprising a sensor system comprising at least one additional sensor, wherein the sensor system is configured to provide multiple viewpoints of a workspace (see at least ¶0018, ¶0058-0066).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the robotic demonstration learning system and method as taught by Kolluri with the well-known technique of providing an external camera to monitor the robot and surrounding area as taught by Taitler in order to provide additional context for robot training, including an additional viewpoint to capture data not represented by onboard sensors.
Claim Rejections - 35 USC § 103
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Kolluri as applied to claim 1 above, and further in view of Niemeyer et al. (US 2007/0052496).
Regarding claim 6:
Kolluri teaches the limitations as in claim 1 above.
Kolluri additionally at least suggests a haptic feedback interface (see at least ¶0015, ¶0283).
Kolluri does not clearly and unambiguously teach the haptic feedback being based on state of the follower robotic device.
However, it would have been obvious to one of ordinary skill in the art before the time of filing of the invention, based on Kolluri’s teaching of tactile feedback, to provide the tactile (haptic) feedback based on the follower robot’s state.
Alternately or in addition, Niemeyer teaches a leader/follower (master/slave) robotic system and method, including a motion modification module configured to perform at least one of: modifying a perceived weight of the one or more leader robotic devices, providing haptic feedback based on states of the one or more follower robotic devices, adjusting movement scaling between leader and follower robotic devices, or any combination thereof (see at least ¶0003).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the leader/follower robotic system and method as taught by Kolluri with the motion scaling and/or haptic feedback technique as taught by Niemeyer in order to provide more complete information to the user and can improve their ability to perform complex tasks and to achieve stronger, larger, or smaller interfaces.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11-12 is rejected under 35 U.S.C. 103 as being unpatentable over Kolluri as in claim 8 above, in view of Kozuka (US 2021/0016439).
Regarding claim 11:
Kolluri teaches the limitations as in claim 8 above. Kolluri further teaches cameras for capturing additional training data.
Kolluri does not explicitly teach the cameras configured to capture force/torque data via deformation.
Kozuka teaches a system and method of training a robot including one or more cameras, where the one or more cameras are configured to capture additional force data, torque data, or force-torque data by capturing physical evidence of deformation (see at least abstract, ¶0092-0094, ¶0104).
It would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the robot training system and method as taught by Kolluri with the technique of using image data depicting deformation as taught by Kozuka in order to train a robot grasping operation with an appropriate grasp force so as to securely hold a particular object without damaging the object or the end effector.
Regarding claim 12:
Kolluri further teaches one or more cameras, where the one or more cameras comprise at least one of: a time-of-flight “ToF” camera; a depth camera; or any combination thereof (see at least ¶0046, ¶0078, ¶0098, ¶0243).
Kolluri doesn’t explicitly teach the depth camera being a stereo depth camera.
Kolluri does further teach the robot tool comprising multiple cameras, including plural camera types, including depth cameras (see ¶0078).
It would have been obvious to one of ordinary skill in the art before the time of filing of the invention to modify the plurality of cameras as taught by Kolluri with stereo depth cameras and/or time of flight cameras in order to achieve better vision capability.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sinnet et al. (US 2020/0139554) discloses a system and method of robotic demonstration learning, including a plurality of interchangeable tools.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN J RINK whose telephone number is (571)272-4863. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anna Momper can be reached on (571) 270-5788. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Rink/ Primary Examiner, Art Unit 3619
Prosecution Insights