MASTER/SLAVE REGISTRATION AND CONTROL FOR TELEOPERATION

Application No. 19/005,692

• Filed: Dec 30, 2024
• Examiner: HUANG, FRANK F
• Art Unit: 2485
• Tech Center: 2400 — Computer Networks
• Assignee: Intuitive Surgical Operations, Inc.
• OA Round: 1 (Non-Final)

Office Action

Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .

Double Patenting

The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim 37-56 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,213,757, 11,534,252, 11,857,280. Although the claims at issue are not identical, they are not patentably distinct from each other because they recites substantial similar subject matters.
Instant Application
U.S. Patent No. 12,213,757
37. A teleoperated system comprising:
an input device;
a display configured to display images; a manipulator arm comprising a plurality of links coupled by a plurality of joints; a motor system coupled to move the manipulator arm; and a control system comprising one or more processors and a memory, the memory comprising programmed instructions adapted to cause the one or more processors to perform operations comprising:
determining an orientation of an end-effector reference frame relative to a field- of-view reference frame,
the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,

determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with the input device,and
the display reference frame being associated with the display,
establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to- display alignment relationship,
wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and
wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent ofa position relationship between the input-device reference frame and the display reference frame, and
commanding, in response to a change in a pose of the input device and based on the first alignment relationship,
the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.

38. The teleoperated system of claim 37,
wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship.

39. The teleoperated system of claim 38, wherein:the operations further comprise: establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship; andcommanding the motor system to move the manipulator arm is further based on the second alignment relationship.

40. The teleoperated system of claim 39, wherein the orientation of the input-device reference frame relative to the display reference frame is a first relative orientation and the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein:commanding the motor system to move the manipulator arm is further based on the difference, such that achieving the change in the pose of the end effector reduces the difference;or the operations further comprise: updating the second alignment relationship to reduce the difference.

41. The teleoperated system of claim 37, wherein the first alignment relationship comprises the input-device-to-display alignment relationship.

42. The teleoperated system of claim 37, wherein commanding the motor system to move the manipulator arm comprises:commanding the motor system to move the plurality of joints such that a change in an orientation of the end effector relative to the field-of-view reference frame corresponds to a change in an orientation of the input device relative to the display reference frame.

43. The teleoperated system of The teleoperated system of wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:determining a complete orientation of the field-of-view reference frame, and determining a complete orientation of the end-effector reference frame; andwherein determining an orientation of the input-device reference frame relative to the display reference frame comprises:determining a complete orientation of the display reference frame, and determining a complete orientation of the input-device reference frame.

44. The teleoperated system of claim 37, wherein:the teleoperated system is a medical system;the end effector is part of a tool supported by the manipulator arm.

45. The teleoperated system of claim 37, wherein establishing the first alignment relationship comprises:establishing the first alignment relationship in response to an indication to begin teleoperation.

46. The teleoperated system of claim 37, wherein the operations further comprise: updating the first alignment relationship by:updating the first alignment relationship while commanding the motor system to move the manipulator arm; or updating the first alignment relationship at a predetermined time interval.

47. The teleoperated system of claim 37, wherein the operations further comprise: determining the display reference frame based on a reference frame of a link of a structure supporting the display.

48. The teleoperated system of claim 37, further comprising:a common control support structure supporting the display and the input device.

49. The teleoperated system of claim 37, wherein:the display is a first display;the operations further comprise: causing an image to be presented simultaneously by the first display and a second display physically distinct from the first display; andthe display reference frame is not associated with the second display.

50. The teleoperated system of claim 37, wherein the operations further comprise:determining the display reference frame in relation to an image displayed by the display.

51. The teleoperated system of claim 50, further comprising:a display support structure supporting the display; anda device support structure supporting the input device, the device support structure physically separate from the display support structure.

52. A method for operating a teleoperated medical system comprising an input device, a display configured to display images, a manipulator arm comprising a plurality of links coupled by a plurality of joints, and a motor system coupled to move the manipulator arm, the method comprising:determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with an input device, and the display reference frame being associated with the display,establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship,wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, andwherein the input-device-to- display alignment relationship is between the input- device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame,and commanding, in response to a change in a pose of the input device and using the motor system to drive the manipulator arm based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.

53. The method of The method of wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:determining a complete orientation of the field-of-view reference frame, and determining a complete orientation of the end-effector reference frame; andwherein determining an orientation of the input-device reference frame relative to the display reference frame comprises: determining a complete orientation of the display reference frame, and determining a complete orientation of the input-device reference frame.

54. The method of claim 52, further comprising:determining the display reference frame based on a reference frame of a link of a structure supporting the display.

55. The method of claim 52, further comprising:determining the display reference frame in relation to an image displayed by the display.

56. A non-transitory machine-readable medium comprising a plurality of machine- readable instructions which, when executed by one or more processors associated with a teleoperated robotic system comprising an input device, a display configured to display images, a manipulator arm comprising a plurality of links coupled by a plurality of joints, and a motor system coupled to move the manipulator arm, are adapted to cause the one or more processors to perform a method comprising:determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with an input device, and the display reference frame being associated with the display,establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship,wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of aposition relationship between the end-effector reference frame and the field-of-view reference frame, andwherein the input-device-to- display alignment relationship is between the input- device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame,and commanding, in response to a change in a pose of the input device and using the motor system to drive the manipulator arm based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.
1. A teleoperated system comprising:

a first means for determining:

an orientation of an end-effector reference frame relative to a field-of-view reference frame,

the end-effector reference frame moveable relative to the field-of-view reference frame, the end-effector reference frame being defined for an end effector of a tool, and the field-of-view reference frame being defined for a field of view of an imaging device, and
an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being defined for a master input device, and
the display reference frame being defined for a displayed image;
a second means for establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship,
wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame; and

a third means for commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device.

2. The teleoperated system of claim 1,
wherein: the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship;
the second means is configured for establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship; and
the third means is configured for commanding the change in the pose of the end effector further based on the second alignment relationship.
3. The teleoperated system of claim 2, wherein an orientation of the input-device reference frame relative to the display reference frame is a first relative orientation, wherein the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein the teleoperated system further comprises:
a fourth means for updating the second alignment relationship multiple times to gradually reduce the difference.

4. The teleoperated system of claim 1,
wherein the pose of the end effector is relative to the field-of-view reference frame;
wherein the pose of the master input device is relative to the display reference frame; and
wherein the change in the pose of the end effector includes: a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master input device relative to the display reference frame.

5. The teleoperated system of claim 1, further comprising:
a fourth means for establishing a teleoperated master-slave control relationship based on the first alignment relationship.

6. The teleoperated system of claim 1, wherein the first means is configured for:
determining a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the input-device reference frame; or
determining a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining a complete position of the input-device reference frame relative to the display reference frame.
7. The teleoperated system of claim 1, wherein the first means is configured for:
determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining less than a complete position of the input-device reference frame relative to the display reference frame.
8. A teleoperated system comprising:
a first means for determining:
a complete orientation of a field-of-view reference frame defined for a field of view of an imaging device,
a complete orientation of an end-effector reference frame defined for an end effector of a tool;
a complete orientation of a display reference frame defined for a display;
a complete orientation of a master-device reference frame defined for a master device;
a second means for establishing a teleoperated master/slave control relationship between a master device and the end effector by establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or a master-device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and is based on less than complete position information relating the end-effector reference frame and the field-of-view reference frame, wherein the master-device-to-display alignment relationship is between the master-device reference frame and the display reference frame, and wherein the first alignment relationship between the master-device reference frame and the display reference frame being is based on less than complete position information relating the master-device reference frame and the display reference frame; and
a third means for executing the master/slave control relationship between the master device and the end effector by changing a pose of the end effector to correspond to a change in a pose of the master device.
9. The teleoperated system of claim 8,
wherein the pose of the end effector is relative to the field-of-view reference frame;
wherein the pose of the master device is relative to the display reference frame; and
wherein the change in the pose of the end effector includes: a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master device relative to the display reference frame.
10. The teleoperated system of claim 8, wherein the first means is configured for:
determining partial position information of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the master-device reference frame.
11. The teleoperated system of claim 8, wherein changing the pose of the end effector to correspond to the change in the pose of the master device comprises:
changing a direction of movement of the end effector to correspond to a change in direction of movement of the master device.
12. A teleoperated system comprising:
a first means for determining
an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector of a tool, and the field-of-view reference frame being associated with a field of view of an imaging device, and
an orientation of an input-device reference frame relative to an image-presentation reference frame, the input-device reference frame being associated with an input device, and the image-presentation reference frame being associated with an image presented to a user;
a second means for establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-image-presentation alignment relationship;
wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and
wherein the input-device-to-image-presentation alignment relationship is between the input-device reference frame and the image-presentation reference frame and independent of a position relationship between the input-device reference frame and the image-presentation reference frame; and
a third means for commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the input device.
13. The teleoperated system of claim 12, wherein:
the second means is configured for establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-image-presentation alignment relationship; and
the third means is configured for commanding the change in the pose of the end effector is further based on the second alignment relationship.
14. The teleoperated system of claim 13, wherein the orientation of the input-device reference frame relative to the image-presentation reference frame is a first relative orientation, wherein the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, and wherein the first relative orientation differs from the second relative orientation by a difference, the teleoperated system further comprising:
a fourth means for updating the second alignment relationship multiple times to gradually reduce the difference.
15. The teleoperated system of claim 12, wherein the first alignment relationship comprises the input-device-to-image-presentation alignment relationship.
16. The teleoperated system of claim 12, wherein commanding the change in the pose of the end effector comprises:
commanding a change in an orientation of the end effector relative to the field-of-view reference frame to correspond to a change in an orientation of the input device relative to the image-presentation reference frame.
17. The teleoperated system of claim 12, further comprising:
a fourth means for establishing a teleoperated master-slave control relationship based on the first alignment relationship.
18. The teleoperated system of claim 12,
wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:
determining a complete orientation of the field-of-view reference frame, and
determining a complete orientation of the end-effector reference frame; and
wherein determining the orientation of the input-device reference frame relative to the image-presentation reference frame comprises:
determining a complete orientation of the image-presentation reference frame, and
determining a complete orientation of the input-device reference frame.
19. The teleoperated system of claim 12, wherein the first means is configured for:
determining a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the image-presentation reference frame, and the input-device reference frame; or
determining a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining a complete position of the input-device reference frame relative to the image-presentation reference frame.
20. The teleoperated system of claim 12, wherein the first means is configured for:
determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining less than a complete position of the input-device reference frame relative to the image-presentation reference frame.
U.S. Patent No. 12,213,757
U.S. Patent No. 11,857,280
37. A teleoperated system comprising:a first means for determining: an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame moveable relative to the field-of-view reference frame, the end-effector reference frame being defined for an end effector of a tool, and the field-of-view reference frame being defined for a field of view of an imaging device, and an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being defined for a master input device, and the display reference frame being defined for a displayed image; a second means for establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input- device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame; and a third means for commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device.38. The teleoperated system of claim 37, wherein: the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship; the second means is configured for establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship; and the third means is configured for commanding the change in the pose of the end effector further based on the second alignment relationship.

39. The teleoperated system of claim 38, wherein an orientation of the input- device reference frame relative to the display reference frame is a first relative orientation, wherein the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein the teleoperated system further comprises: a fourth means for updating the second alignment relationship multiple times to gradually reduce the difference.

40. The teleoperated system of The teleoperated system of wherein the pose of the end effector is relative to the field-of-view reference frame; wherein the pose of the master input device is relative to the display reference frame; and wherein the change in the pose of the end effector includes: a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master input device relative to the display reference frame.

41. The teleoperated system of claim 37, further comprising: a fourth means for establishing a teleoperated master-slave control relationship based on the first alignment relationship.

42. The teleoperated system of claim 37, wherein the first means is configured for: determining a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the input-device reference frame; or determining a complete position of the end-effector reference frame relative to the field- of-view reference frame; or determining a complete position of the input-device reference frame relative to the display reference frame.

43. The teleoperated system of claim 37, wherein the first means is configured for: determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or determining less than a complete position of the input-device reference frame relative to the display reference frame. 44. A teleoperated system comprising:a first means for determining: a complete orientation of a field-of-view reference frame defined for a field of view of an imaging device, a complete orientation of an end-effector reference frame defined for an end effector of a tool; a complete orientation of a display reference frame defined for a display; a complete orientation of a master-device reference frame defined for a master device; a second means for establishing a teleoperated master/slave control relationship between a master device and the end effector by establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or a master-device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and is based on less than complete position information relating the end-effector reference frame and the field-of-view reference frame, wherein the master-device-to-display alignment relationship is between the master-device reference frame and the display reference frame, and wherein the first alignment relationship between the master-device reference frame and the display reference frame being is based on less than complete position information relating the master-device reference frame and the display reference frame; and a third means for executing the master/slave control relationship between the master device and the end effector by changing a pose of the end effector to correspond to a change in a pose of the master device.

45. The teleoperated system of The teleoperated system of wherein the pose of the end effector is relative to the field-of-view reference frame; wherein the pose of the master device is relative to the display reference frame; and wherein the change in the pose of the end effector includes: a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master device relative to the display reference frame.

46. The teleoperated system of claim 44, wherein the first means is configured for:determining partial position information of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the master-device reference frame.

47. The teleoperated system of claim 44, wherein changing the pose of the end effector to correspond to the change in the pose of the master device comprises:changing a direction of movement of the end effector to correspond to a change in direction of movement of the master device.

48. A teleoperated system comprising:a first means for determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector of a tool, and the field-of-view reference frame being associated with a field of view of an imaging device, and an orientation of an input-device reference frame relative to an image-presentation reference frame, the input-device reference frame being associated with an input device, and the image-presentation reference frame being associated with an image presented to a user; a second means for establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input- device-to-image-presentation alignment relationship;wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-image-presentation alignment relationship is between the input-device reference frame and the image-presentation reference frame and independent of a position relationship between the input-device reference frame and the image-presentation reference frame; and a third means for commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the input device.

49. The teleoperated system of claim 48, wherein:the second means is configured for establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-image-presentation alignment relationship; and the third means is configured for commanding the change in the pose of the end effector is further based on the second alignment relationship.

50. The teleoperated system of claim 49, wherein the orientation of the input-device reference frame relative to the image-presentation reference frame is a first relative orientation, wherein the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, and wherein the first relative orientation differs from the second relative orientation by a difference, the teleoperated system further comprising: a fourth means for updating the second alignment relationship multiple times to gradually reduce the difference.

51. The teleoperated system of claim 48, wherein the first alignment relationship comprises the input-device-to-image-presentation alignment relationship.

52. The teleoperated system of claim 48, wherein commanding the change in the pose of the end effector comprises:commanding a change in an orientation of the end effector relative to the field-of-view reference frame to correspond to a change in an orientation of the input device relative to the image-presentation reference frame.

53. The teleoperated system of claim 48, further comprising:a fourth means for establishing a teleoperated master-slave control relationship based on the first alignment relationship.

54. The teleoperated system of The teleoperated system of wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises: determining a complete orientation of the field-of-view reference frame, and determining a complete orientation of the end-effector reference frame; and wherein determining the orientation of the input-device reference frame relative to the image-presentation reference frame comprises: determining a complete orientation of the image-presentation reference frame, and determining a complete orientation of the input-device reference frame.

55. The teleoperated system of claim 48, wherein the first means is configured for:determining a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the image- presentation reference frame, and the input-device reference frame; or determining a complete position of the end-effector reference frame relative to the field- of-view reference frame; or determining a complete position of the input-device reference frame relative to the image-presentation reference frame.

56. The teleoperated system of claim 48, wherein the first means is configured for:determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or determining less than a complete position of the input-device reference frame relative to the image-presentation reference frame.
1. A teleoperated system comprising:
an input device; and
a control system comprising one or more processors and a memory, the memory comprising programmed instructions adapted to cause the one or more processors to perform operations comprising:
determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector of a tool, and the field-of-view reference frame being associated with a field of view of an imaging device,
determining an orientation of an input-device reference frame relative to an image-presentation reference frame, the input-device reference frame being associated with the input device, and the image-presentation reference frame being associated with an image presented to a user,
establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-image-presentation alignment relationship,
wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and
wherein the input-device-to-image-presentation alignment relationship is between the input-device reference frame and the image-presentation reference frame and independent of a position relationship between the input-device reference frame and the image-presentation reference frame, and
commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the input device.
2. The teleoperated system of claim 1, wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship.
3. The teleoperated system of claim 2, wherein:
the operations further comprise: establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-image-presentation alignment relationship; and
commanding the change in the pose of the end effector is further based on the second alignment relationship.
4. The teleoperated system of claim 3, wherein the orientation of the input-device reference frame relative to the image-presentation reference frame is a first relative orientation and the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein the operations further comprise:
updating the second alignment relationship multiple times to gradually reduce the difference.
5. The teleoperated system of claim 1, wherein the first alignment relationship comprises the input-device-to-image-presentation alignment relationship.
6. The teleoperated system of claim 1, wherein commanding the change in the pose of the end effector comprises:
commanding a change in an orientation of the end effector relative to the field-of-view reference frame to correspond to a change in an orientation of the input device relative to the image-presentation reference frame.
7. The teleoperated system of claim 1, wherein the operations further comprise:
establishing a teleoperated master-slave control relationship based on the first alignment relationship.
8. The teleoperated system of claim 1,
wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:
determining a complete orientation of the field-of-view reference frame, and
determining a complete orientation of the end-effector reference frame; and
wherein determining an orientation of the input-device reference frame relative to the image-presentation reference frame comprises:
determining a complete orientation of the image-presentation reference frame, and
determining a complete orientation of the input-device reference frame.
9. The teleoperated system of claim 1, wherein the programmed instructions are not adapted to cause the one or more processors to:
determine a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the image-presentation reference frame, and the input-device reference frame; or
determine a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determine a complete position of the input-device reference frame relative to the image-presentation reference frame.
10. The teleoperated system of claim 1, wherein the operations further comprise:
determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining less than a complete position of the input-device reference frame relative to the image-presentation reference frame.
11. The teleoperated system of claim 1, wherein:
the teleoperated system includes a teleoperated medical system;
the tool includes a medical tool;
the teleoperated system further comprises a manipulator arm configured to removably support the tool, the manipulator arm comprising a plurality of joints and a plurality of links; and
commanding the change in the pose of the end effector comprises commanding the manipulator arm to change the pose of the end effector.
12. The teleoperated system of claim 1, wherein establishing the first alignment relationship comprises:
establishing the first alignment relationship in response to an indication to begin teleoperation.
13. The teleoperated system of claim 1, wherein the operations further comprise updating the first alignment relationship by:
updating the first alignment relationship while commanding a change in a pose of the end effector in response to a change in a pose of the input device; or
updating the first alignment relationship at a predetermined time interval.
14. A method for operating a medical system comprising:
determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector of a tool, and the field-of-view reference frame being associated with a field of view of an imaging device;
determining an orientation of an input-device reference frame relative to an image-presentation reference frame, the input-device reference frame being associated with an input device of the medical system, and the image-presentation reference frame being associated with an image presented to a user;
establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-image-presentation alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-image-presentation alignment relationship is between the input-device reference frame and the image-presentation reference frame and independent of a position relationship between the input-device reference frame and the image-presentation reference frame; and
commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the input device.
15. The method of claim 14, wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship, the method further comprising:
establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-image-presentation alignment relationship, wherein commanding the change in the pose of the end effector is further based on the second alignment relationship.
16. The method of claim 14,
wherein the operations further comprise: establishing a teleoperated master-slave control relationship based on the first alignment relationship;
wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:
determining a complete orientation of the field-of-view reference frame, and
determining a complete orientation of the end-effector reference frame; and
wherein determining the orientation of an input-device reference frame relative to an image-presentation reference frame comprises:
determining a complete orientation of the image-presentation reference frame, and
determining a complete orientation of the input-device reference frame.
17. The method of claim 14, further comprising:
determining a less than complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the image-presentation reference frame, and the input-device reference frame.
18. A teleoperated system comprising:
a master device, a master-device reference frame being associated with the master device; and
a control system comprising a memory storing instructions that, when executed by the control system, cause the control system to perform operations comprising:
determining a complete orientation of a field-of-view reference frame associated with a field of view of an imaging device;
determining a complete orientation of an end-effector reference frame associated with an end effector of a tool;
determining a complete orientation of an image-presentation reference frame associated with an image presented to a user;
determining a complete orientation of the master-device reference frame;
establishing a teleoperated master-slave control relationship between the master device and the end effector by establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or a master-device-to-image-presentation alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and is based on less than complete position information relating the end-effector reference frame and the field-of-view reference frame, and wherein the master-device-to-image-presentation alignment relationship is between the master-device reference frame and the image-presentation reference frame, wherein the first alignment relationship between the master-device reference frame and the image-presentation reference frame being is based on less than complete position information relating the master-device reference frame and the image-presentation reference frame; and
executing the teleoperated master-slave control relationship between the master device and the end effector by changing a pose of the end effector corresponding to a change in a pose of the master device.
19. The teleoperated system of claim 18, wherein the teleoperated system is a tele-surgical system comprising:
the imaging device, wherein the imaging device comprises an endoscopic camera;
the tool, wherein the tool comprises a surgical tool comprising the end effector; and
a manipulator arm configured to removably support the tool, the manipulator arm comprising a plurality of joints and a plurality of links, wherein changing the pose of the end effector comprises using the manipulator arm to change the pose of the end effector.
20. The teleoperated system of claim 18, wherein the operations further comprise
determining partial position information of at least one reference frame, the reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the image-presentation reference frame, and the master-device reference frame.
Instant Application
U.S. Patent No. 11,534,252
37. (New) A teleoperated system comprising:an input device;a display configured to display images;a manipulator arm comprising a plurality of links coupled by a plurality of joints;a motor system coupled to move the manipulator arm; anda control system comprising one or more processors and a memory, the memory comprising programmed instructions adapted to cause the one or more processors to perform operations comprising:determining an orientation of an end-effector reference frame relative to a field- of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with the input device,and the display reference frame being associated with the display,establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to- display alignment relationship,wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, andwherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent ofa position relationship between the input-device reference frame and the display reference frame, and commanding, in response to a change in a pose of the input device and based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.

38. (New) The teleoperated system of claim 37, wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship.

39. (New) The teleoperated system of claim 38, wherein:the operations further comprise: establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship; andcommanding the motor system to move the manipulator arm is further based on the second alignment relationship.

40. (New) The teleoperated system of claim 39, wherein the orientation of the input-device reference frame relative to the display reference frame is a first relative orientation and the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein:commanding the motor system to move the manipulator arm is further based on the difference, such that achieving the change in the pose of the end effector reduces the difference;or the operations further comprise: updating the second alignment relationship to reduce the difference.

41. (New) The teleoperated system of claim 37, wherein the first alignment relationship comprises the input-device-to-display alignment relationship.

42. (New) The teleoperated system of claim 37, wherein commanding the motor system to move the manipulator arm comprises:commanding the motor system to move the plurality of joints such that a change in an orientation of the end effector relative to the field-of-view reference frame corresponds to a change in an orientation of the input device relative to the display reference frame.

43. (New) The teleoperated system of The teleoperated system of wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:determining a complete orientation of the field-of-view reference frame, and determining a complete orientation of the end-effector reference frame; andwherein determining an orientation of the input-device reference frame relative to the display reference frame comprises:determining a complete orientation of the display reference frame, and determining a complete orientation of the input-device reference frame.

44. (New) The teleoperated system of claim 37, wherein:the teleoperated system is a medical system;the end effector is part of a tool supported by the manipulator arm.

45. (New) The teleoperated system of claim 37, wherein establishing the first alignment relationship comprises:establishing the first alignment relationship in response to an indication to begin teleoperation.

46. (New) The teleoperated system of claim 37, wherein the operations further comprise: updating the first alignment relationship by:updating the first alignment relationship while commanding the motor system to move the manipulator arm; or updating the first alignment relationship at a predetermined time interval.

47. (New) The teleoperated system of claim 37, wherein the operations further comprise: determining the display reference frame based on a reference frame of a link of a structure supporting the display.

48. (New) The teleoperated system of claim 37, further comprising:a common control support structure supporting the display and the input device.

49. (New) The teleoperated system of claim 37, wherein:the display is a first display;the operations further comprise: causing an image to be presented simultaneously by the first display and a second display physically distinct from the first display; andthe display reference frame is not associated with the second display.

50. (New) The teleoperated system of claim 37, wherein the operations further comprise:determining the display reference frame in relation to an image displayed by the display.

51. (New) The teleoperated system of claim 50, further comprising:a display support structure supporting the display; anda device support structure supporting the input device, the device support structure physically separate from the display support structure.

52. (New) A method for operating a teleoperated medical system comprising an input device, a display configured to display images, a manipulator arm comprising a plurality of links coupled by a plurality of joints, and a motor system coupled to move the manipulator arm, the method comprising:determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with an input device, and the display reference frame being associated with the display,establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship,wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, andwherein the input-device-to- display alignment relationship is between the input- device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame,and commanding, in response to a change in a pose of the input device and using the motor system to drive the manipulator arm based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.

53. (New) The method of The method of wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:determining a complete orientation of the field-of-view reference frame, and determining a complete orientation of the end-effector reference frame; andwherein determining an orientation of the input-device reference frame relative to the display reference frame comprises: determining a complete orientation of the display reference frame, and determining a complete orientation of the input-device reference frame.

54. (New) The method of claim 52, further comprising:determining the display reference frame based on a reference frame of a link of a structure supporting the display.

55. (New) The method of claim 52, further comprising:determining the display reference frame in relation to an image displayed by the display.

56. (New) A non-transitory machine-readable medium comprising a plurality of machine- readable instructions which, when executed by one or more processors associated with a teleoperated robotic system comprising an input device, a display configured to display images, a manipulator arm comprising a plurality of links coupled by a plurality of joints, and a motor system coupled to move the manipulator arm, are adapted to cause the one or more processors to perform a method comprising:determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device,determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with an input device, and the display reference frame being associated with the display,establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship,wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of aposition relationship between the end-effector reference frame and the field-of-view reference frame, andwherein the input-device-to- display alignment relationship is between the input- device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame,and commanding, in response to a change in a pose of the input device and using the motor system to drive the manipulator arm based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.
1. A teleoperated system comprising:
a display;
a master input device; and
a control system comprising one or more processors and a memory, the memory comprising programmed instructions adapted to cause the one or more processors to perform operations comprising:
determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame moveable relative to the field-of-view reference frame, the end-effector reference frame being defined for an end effector of a tool, and the field-of-view reference frame being defined for a field of view of an imaging device,
determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being defined for the master input device, and the display reference frame being defined for an image displayed by the display,
establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame, and
commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device.
2. The teleoperated system of claim 1, wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship.
3. The teleoperated system of claim 2, wherein:
the operations further comprise: establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship; and
commanding the change in the pose of the end effector is further based on the second alignment relationship.
4. The teleoperated system of claim 3, wherein the orientation of the input-device reference frame relative to the display reference frame is a first relative orientation and the orientation of the end-effector reference frame relative to the field-of-view reference frame is a second relative orientation, wherein the first relative orientation differs from the second relative orientation by a difference, and wherein the operations further comprise:
updating the second alignment relationship multiple times to gradually reduce the difference.
5. The teleoperated system of claim 1, wherein the first alignment relationship comprises the input-device-to-display alignment relationship.
6. The teleoperated system of claim 1,
wherein the pose of the end effector is relative to the field-of-view reference frame;
wherein the pose of the master input device is relative to the display reference frame; and
wherein the change in the pose of the end effector includes:
a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master input device relative to the display reference frame.
7. The teleoperated system of claim 1, wherein the operations further comprise:
establishing a teleoperated master-slave control relationship based on the first alignment relationship.
8. The teleoperated system of claim 1,
wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:
determining a complete orientation of the field-of-view reference frame, and
determining a complete orientation of the end-effector reference frame; and
wherein determining an orientation of the input-device reference frame relative to the display reference frame comprises:
determining a complete orientation of the display reference frame, and
determining a complete orientation of the input-device reference frame.
9. The teleoperated system of claim 1, wherein the programmed instructions are not adapted to cause the one or more processors to:
determining a complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the input-device reference frame; or
determining a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining a complete position of the input-device reference frame relative to the display reference frame.
10. The teleoperated system of claim 1, wherein the operations further comprise:
determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining less than a complete position of the input-device reference frame relative to the display reference frame.
11. The teleoperated system of claim 1, wherein:
the teleoperated system is a teleoperated medical system;
the tool is a medical tool;
the teleoperated system further comprises a manipulator arm configured to removably support the tool, the manipulator arm comprising a plurality of joints and a plurality of links; and
commanding the change in the pose of the end effector comprises commanding the manipulator arm to change the pose of the end effector.
12. The teleoperated system of claim 1, wherein establishing the first alignment relationship comprises:
establishing the first alignment relationship in response to an indication to begin teleoperation.
13. The teleoperated system of claim 1, wherein the operations further comprise updating the first alignment relationship by:
updating the first alignment relationship while commanding a change in a pose of the end effector in response to a change in a pose of the master input device; or
updating the first alignment relationship at a predetermined time interval.
14. A method for operating a medical system comprising:
determining an orientation of an end-effector reference frame relative to a field-of-view reference frame, the end-effector reference frame moveable relative to the field-of-view reference frame, the end-effector reference frame being defined for an end effector of a tool, and the field-of-view reference frame being defined for a field of view of an imaging device;
determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being defined for a master input device of the medical system, and the display reference frame being defined for a display of the medical system;
establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame; and
commanding, based on the first alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device.
15. The method of claim 14, wherein the first alignment relationship comprises the end-effector-to-field-of-view alignment relationship, the method further comprising:
establishing a second alignment relationship, the second alignment relationship comprising the input-device-to-display alignment relationship, wherein commanding the change in the pose of the end effector is further based on the second alignment relationship.
16. The method of claim 14,
wherein determining the orientation of the end-effector reference frame relative to the field-of-view reference frame comprises:
determining a complete orientation of the field-of-view reference frame, and
determining a complete orientation of the end-effector reference frame; and
wherein determining the orientation of an input-device reference frame relative to a display reference frame comprises:
determining a complete orientation of the display reference frame, and
determining a complete orientation of the input-device reference frame.
17. The method of claim 14, further comprising:
determining a less than complete position of at least one reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the input-device reference frame; or
determining less than a complete position of the end-effector reference frame relative to the field-of-view reference frame; or
determining less than a complete position of the input-device reference frame relative to the display reference frame.
18. A teleoperated system comprising:
a display, a display reference frame being defined for the display;
a master device, a master-device reference frame being defined for the master device; and
a control system comprising a memory storing instructions that, when executed by the control system, cause the control system to perform operations comprising:
determining a complete orientation of a field-of-view reference frame defined for a field of view of an imaging device;
determining a complete orientation of an end-effector reference frame defined for an end effector of a tool;
determining a complete orientation of the display reference frame;
determining a complete orientation of the master-device reference frame;
establishing a teleoperated master/slave control relationship between the master device and the end effector by establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or a master-device-to-display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and is based on less than complete position information relating the end-effector reference frame and the field-of-view reference frame, and wherein the master-device-to-display alignment relationship is between the master-device reference frame and the display reference frame, wherein the first alignment relationship between the master-device reference frame and the display reference frame being is based on less than complete position information relating the master-device reference frame and the display reference frame; and
executing the master/slave control relationship between the master device and the end effector by changing a pose of the end effector corresponding to a change in a pose of the master device.
19. The teleoperated system of claim 18,
wherein the pose of the end effector is relative to the field-of-view reference frame;
wherein the pose of the master device is relative to the display reference frame; and
wherein the change in the pose of the end effector includes:
a change in an orientation of the end effector relative to the field-of-view reference frame corresponding to a change in an orientation of the master device relative to the display reference frame.
20. The teleoperated system of claim 18, wherein the teleoperated system is a telesurgical system comprising:
the imaging device, wherein the imaging device comprises an endoscopic camera;
the tool, wherein the tool comprises a surgical tool comprising the end effector
a manipulator arm configured to removably support the tool, the manipulator arm comprising a plurality of joints and a plurality of links, wherein changing the pose of the end effector comprises using the manipulator arm to change the pose of the end effector.
21. The teleoperated system of claim 18, wherein the operations further comprise
determining partial position information of at least one reference frame, the reference frame selected from the group consisting of: the field-of-view reference frame, the end-effector reference frame, the display reference frame, and the master-device reference frame.
22. The teleoperated system of claim 18, wherein changing a pose of the end effector corresponding to a change in a pose of the master device comprises:
changing a direction of movement of the end effector corresponding to a change in direction of movement of the master device.
23. The method of claim 14, wherein the first alignment relationship comprises the input-device-to-display alignment relationship.

Allowable Subject Matter
The following is an examiner’s statement of reasons for allowance:
US 20140171964 A1 discloses Apparatus for Minimal Invasive Surgery (MIS) comprising a master device, a slave device, a detector for detecting a parameter of or associated with the slave device, and a shape locking system for locking the shape of the master device in response to a parameter detected by the detector. [0005] The continuing evolution of the technology, including force feedback and virtual immobilization through real-time motion adaptation, will permit more complex procedures such as beating heart surgery to be carried out under a static frame-of-reference. Current systems for human robot interaction are generally based on mechanical systems either using standard input devices or bespoke designed master-slave manipulators.
US 20150173846 A1 discloses a system captures and displays video of surgeries. The system may include at least one digital image sensor optically coupled to one or more lenses and configured to capture a video sequence of a scene in a surgery; at least one interface configured to receive at least one region on interest (ROI) of the captured video sequence; an electronic display, selected so that at least one of the digital image sensors has a pixel resolution which is substantially greater than the pixel resolution of the electronic display; and a computer processor configured to: receive the at least one captured video sequence and the at least one received ROI and display over the at least one electronic display a portion of the captured video sequence based on the at least one selected ROI.
US 8535336 B2 discloses the present disclosure provides for systems and methods for Nested Cannula configuration. Nested Cannula systems include a plurality of telescoping, pre-shaped tubes configured and dimensioned to reach target locations within a particular anatomical region. A three dimensional image is read for the particular anatomical region and structure in question. A series of arcs are generated between a point of the anatomical region and a target location, ensuring collision-free motion for each of the tubes at each specific diameter. The target location is determined based upon the medical procedure being performed and the location and orientation in six degrees of freedom of the anatomical structure in question. The series of arcs are used to configure and dimension the plurality of tubes. The Nested Cannula system is adapted to reach relatively small and complex target locations, such as to deliver photodynamic therapy, balloon angioplasty or Broncho Alveolar Lavage.
None of the cited prior arts discloses instant application with limitations: ” determining an orientation of an end-effector reference frame relative to a field- of-view reference frame, the end-effector reference frame being associated with an end effector supported by the manipulator arm, and the field-of-view reference frame being associated with a field of view of an imaging device, determining an orientation of an input-device reference frame relative to a display reference frame, the input-device reference frame being associated with the input device, and the display reference frame being associated with the display, establishing a first alignment relationship, the first alignment relationship comprising an end-effector-to-field-of-view alignment relationship or an input-device-to- display alignment relationship, wherein the end-effector-to-field-of-view alignment relationship is between the end-effector reference frame and the field-of-view reference frame and independent of a position relationship between the end-effector reference frame and the field-of-view reference frame, and wherein the input-device-to-display alignment relationship is between the input-device reference frame and the display reference frame and independent of a position relationship between the input-device reference frame and the display reference frame, and commanding, in response to a change in a pose of the input device and based on the first alignment relationship, the motor system to move the manipulator arm such that a change in a pose of the end effector corresponds to a change in a pose of the input device.”.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 10883708 B2 LED bulb has multiple features
US 10646156 B1 Adaptive image processing in assisted reproductive imaging modalities
US 20200138518 A1 OPTICAL GUIDANCE FOR SURGICAL, MEDICAL, AND DENTAL PROCEDURES
US 20150173846 A1 MICROSURGERY SYSTEM FOR DISPLAYING IN REAL TIME MAGNIFIED DIGITAL IMAGE SEQUENCES OF AN OPERATED AREA
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK F HUANG whose telephone number is [phone]. The examiner can normally be reached Monday-Friday, 8:30 am - 6:00 pm (Eastern Time), Federal Alternative First Friday Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached on [phone].. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.

/FRANK F HUANG/Primary Examiner, Art Unit 2485

Prosecution Timeline

• Dec 30, 2024: Application Filed
• Jan 08, 2026: Non-Final Rejection — §DP (current)