DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments and Amendments
Applicant's arguments and amendments, filed May 11th, 2026, have overcome each and every 35 U.S.C. 101 rejection previously set forth in the Non-Final Office Action sent on February 9th, 2026.
Applicant’s arguments and amendments, regarding the rejections of claims 1-7 and 9-10 under 35 U.S.C. 103 have been fully considered and they are persuasive. However, because the additional limitations in independent claims 1 and 9-10 are merely the same limitations extracted from dependent claims 2-3 and 7, the combination of prior art continues to provide support for every element of the claimed invention. Examiner notes that while the additional limitation of the robot performing the control command does overcome the previous 101 rejections, it does not overcome the prior art of record as explained in the 103 rejections below. Lastly, the amended limitations resulted in additional claim objections laid out below.
Claim Objections
Claims 1-2 are objected to because of the following informalities:
Claim 1 Line 15: “and they wrist posture” should be revised to “and a wrist posture” to avoid lacking antecedent basis.
Claim 2 Line 2: “wherein the a robot” should be revised to “wherein the robot”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7 and 9-10 are rejected under 35 U.S.C. 103 as being obvious over Zhang et al. (US Pub No: 2023/0046044 A1, hereinafter Zhang) in view of Sachs et al. (US Pub No: 2022/0151718 A1, hereinafter Sachs), Chang et al. (US Pub No: 2024/0375283 A1, hereinafter Chang), Ohnishi et al. (US Pub No: 2021/0197391 A1, hereinafter Ohnishi), and Hecht et al. (US Pub No: 2022/0242433 A1, hereinafter Hecht).
Regarding Claim 1:
Zhang discloses:
A remote operation control device, in a robot remote operation of operating a robot by recognizing movement of an operator and transferring the movement of the operator to the robot, the device comprising. Paragraph [0025] describes a user console 110 that remotely manipulates the robotic arms. Paragraph [0064] describes a movement of an end effector 222 that mimics that of a user input device.
and a control command generation unit generating a control command on the basis of an estimated motion of the operator and information acquired by the relationship acquisition unit. Paragraph [0071] describes a robot performing actions in the same degree of freedom.
a robot body joint angle estimation unit estimating a joint angle of the whole body of the robot on the basis of a finger joint angle and a wrist posture of the operator included in the second sensor value. Paragraph [0032] describes a system translating the surgeon’s hand, wrist and finger movements through the master UIDs 116 into precise real-time movements of the surgical tools. Paragraph [0045] describes defining consistent cording frames for the joint angles.
Zhang does not disclose estimating a motion of the operator based on a first sensor value obtained by an environmental sensor acquiring information of the surrounding environment and a second sensor value indicating movement of the operator obtained by the operator sensor.
Sachs, in an analogous field of endeavors, teaches:
an intention estimation unit estimating a motion of the operator on the basis of a first sensor value obtained by an environmental sensor acquiring information of the robot or a surrounding environment of the robot, and a second sensor value indicating movement of the operator obtained by an operator sensor. Paragraph [0009] describes a sensor system that determines the orientation in space of the head mounted display relative to a reference point. Paragraph [0102] describes sensors to track the position of the surgeon’s arms and body. Paragraph [0080] describes mimicking a human arm.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Sachs to show estimating a motion of the operator based on a first sensor value obtained by an environmental sensor acquiring information of the surrounding environment and a second sensor value indicating movement of the operator obtained by the operator sensor. One would have been motivated to do so in order to determine the operator’s position in space relative to a orientation in space ([0009] of Sachs).
Zhang does not disclose a relationship acquiring unit that acquires a relationship between a first operation target object and a second target object.
Chang, in an analogous field of endeavor, teaches:
a relationship acquisition unit acquiring a relationship between a first operation target object and a second target object. Paragraph [0022] describes 3 calibration objects to determine a transformation relationship a first and second state.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Chang to show a relationship acquiring unit that acquires a relationship between a first operation target object and a second target object. One would have been motivated to do so that the robot can adjust and adapt to changes if the position and orientation of the work object changes ([0003] of Chang).
Zhang does not disclose the robot performs the control command and replicates the estimated motion of the operator and an object posture estimation unit estimating a posture of each of the first operation target object and the second target object on the basis of the first sensor value.
Ohnishi, in an analogous field of endeavor, teaches:
the robot performs the control command and replicates the estimated motion of the operator and an object posture estimation unit estimating a posture of each of the first operation target object and the second target object on the basis of the first sensor value. Paragraph [0026] describes a target position and target posture of the robot to pick up a first object and a second target position and target posture to pick up a second object.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Ohnishi to show estimating a posture and motion of each of the first target object and a second target object to perform the control command. One would have been motivated to do so in order to shorten the time the robot takes to perform subsequent actions ([0006] of Ohnishi).
Zhang does not disclose the relationship acquisition unit extracts an appearance feature amount and a geometric feature amount for each of the operation target objects using an image included in the first sensor value and acquires a relationship between the first operation target object and the second target object using the appearance feature amount and the geometric feature amount extracted for each of the operation target objects.
Hecht, in an analogous field of endeavor, teaches:
the relationship acquisition unit extracts an appearance feature amount and a geometric feature amount for each of the operation target objects using an image included in the first sensor value and acquires a relationship between the first operation target object and the second target object using the appearance feature amount and the geometric feature amount extracted for each of the operation target objects. Paragraph [0079] describes highlight a bicyclist 92 using a bounding box 112 that is overlaid on the image. In order for a bicyclist to be identified, the features of the bicycle must be identified.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Hecht to show extracting an appearance and geometric feature of each of the target objects. One would have been motivated to do so to draw attention of the user to the region without occluding the region (Abstract of Hecht).
Claim 9 is substantially similar to claim 1 and is rejected on the same grounds.
Regarding Claim 10:
Zhang discloses:
A computer-readable non-transitory storage medium storing a program for causing a computer of a remote operation control device. Paragraph [0064] describes a programmed processor. Paragraph [0075] describes a memory.
The rest of claim 10 is substantially similar to claim 1 and is rejected on the same grounds.
Regarding Claim 2:
Zhang discloses:
The remote operation control device according to claim 1, wherein the robot body joint angle estimation unit estimates joint angles of the whole body of the robot, and wherein the control command generation unit generates a control command also using an estimated joint angle of the whole body of the robot. Paragraph [0032] describes a system translating the surgeon’s hand, wrist and finger movements through the master UIDs 116 into precise real-time movements of the surgical tools. Paragraph [0045] describes defining consistent cording frames for the joint angles.
Regarding Claim 3:
Ohnishi teaches:
The remote operation control device according to claim 1, wherein the control command generation unit generates a control command also using an estimated posture of each of the first operation target object and the second target object. Paragraph [0026] describes a target position and target posture of the robot to pick up a first object and a second target position and target posture to pick up a second object.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Ohnishi to show estimating a posture of each of the first target object and a second target object to generate the control command. One would have been motivated to do so in order to shorten the time the robot takes to perform subsequent actions ([0006] of Ohnishi).
Regarding Claim 4:
Zhang discloses:
The remote operation control device according to claim 1, wherein the control command generation unit generates a control command with which a degree of freedom in motion of the operator is reduced by generating a control command appropriate for a degree of freedom in part of the motion of the operator. Paragraph [0071] describes an effector that can only move in one degree of freedom.
Regarding Claim 5:
Chang teaches:
The remote operation control device according to claim 1, wherein the second target object is an object having a relationship with the first operation target object in operation. Paragraph [0022] describes 3 calibration objects to determine a transformation relationship a first and second state.
The reason to combine Chang with Zhang is for the same reasons as in claim 1.
Regarding Claim 7:
Hecht teaches:
The remote operation control device according to claim 1, wherein the relationship acquisition unit extracts the appearance feature amount and the geometric feature amount for each of the operation target objects using identification information related to an object. Paragraph [0079] describes highlight a bicyclist 92 using a bounding box 112 that is overlaid on the image. In order for a bicyclist to be identified, the features of the bicycle must be identified.
a position and a size of a region of interest. Paragraph [0077] and figure 4 describes a plurality of regions and therefore their size.
and a luminance within the region of interest detected using the image included in the first sensor value, and outputs the relationship in text. Paragraph [0077] describes increasing the brightness of image regions by modifying the original image. Paragraph [0068] describes that this is based on the user’s attentiveness score. Figure 4 shows that there could be multiple target objects and therefore multiple different geometric features. Paragraph [0044] describes a user interface 34 that is configured to interact with the user via text or graphical displays.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Hecht to show extracting an appearance feature of each of the target objects, a position and size of the region interest and a luminance within the region of interest. One would have been motivated to do so to draw attention of the user to the region without occluding the region (Abstract of Hecht).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Sachs, Chang, Ohnishi, and Hecht, and further in view of Okazaki (US Pub No: 2009/0105880 A1, hereinafter Okazaki).
Regarding Claim 6:
Zhang, Sachs, Chang, Ohnishi, and Hecht teach the above limitations in claim 1, but do not teach a database that describes a relationship between the target objects and a surrounding environment.
Okazaki, in an analogous field of endeavor, teaches:
The remote operation control device according to claim 1, wherein when there are a plurality of the operation target objects, the relationship acquisition unit acquires relationship information from a database in which a relationship between the operation target objects or between the operation target objects and a surrounding environment is described in advance using information identifying the target objects from the first sensor value. Paragraph [0074] describes a interference judging that develops a relationship between the robot arm 5 and the surrounding environment that is stored in the map database 52.
Therefore, it would have been prima facie obvious to one of the ordinary skill in the art before the effective filing date, with a reasonable expectation for success, to have modified Zhang to incorporate the teachings of Okazaki to show a database that describes a relationship between the target objects and a surrounding environment. One would have been motivated to do so to realize safe robot control without damaging humans or objects ([0006] of Okazaki).
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 8 had previously overcome any prior art rejections and was rejected under 35 U.S.C. 101. Since all 101 rejections have been resolved in the Applicant’s amendments, Claim 8 now contains allowable subject matter. Specifically, the following limitations were found to be novel improvements in the art of transferring motion for remote robotic operation(s):
and the control command generation unit generates a first feature vector by encoding a set of an estimated motion of the operator, an estimated joint angle of the whole body of the robot, and an estimated posture of each of the first operation target object and the second target object,
generates a second feature vector by encoding text output by the relationship acquisition unit,
and generates a control command that is a joint angle trajectory sequence of the robot by associating the first feature vector and the second feature vector with each other and encoding associated data.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bryant Tang whose telephone number is (571)270-0145. The examiner can normally be reached M-F 8-5 PDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571)272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYANT TANG/Examiner, Art Unit 3658
/THOMAS E WORDEN/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights