DETAILED ACTION
Claims 1-20 are pending in the present application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/02/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. PGPubs 2023/0367289 to Edwards et al..
PNG
media_image1.png
228
378
media_image1.png
Greyscale
Regarding claim 1, Edwards et al. teach a system for assisting a user (abstract), the system comprising:
a moveable end effector in a world space (Fig 5A, par 0092, “a physical-world controller 5008 instructs a robot arm 5001 to rotate at axes 5002 and 5003 to move the end effector 5005 forward 5004. This particular embodiment depicts a robot arm with only two axes, but the principles can be applied to other robot arm/end effector configurations with three or more axes (not shown) and the associated kinematics and movement profiles of such configurations”, par 0120, “An opponent robot 6002 wields a sword 6004 which is held by an end effector 6003. The robot may include multiple axes (not shown) and is attached to the PW controller which is attached to the computer, similar to FIG. 5A.”);
a display configured to display a virtual space to the user (par 0091, “the sensor 4021 is a discrete device, but, in other implementations, could be integrated into the VR device 4008, the robot base 4002, or the opponent 4001. The computer 4020 hosts the virtual world and sends corresponding image data over a wireless connection to the VR headset 4008, which displays corresponding imagery to the user 4007”); and
at least one processor configured to (par 0168-0171):
obtain a pose of the end effector in the world space; generate a visual representation of an avatar based at least in part on the pose of the end effector; superimpose the visual representation of the avatar with a location of the end effector in the virtual space (Figs 6A-6B, par 0120-0122, “FIG. 6B depicts the virtual-world components that correspond to the embodiment of FIG. 6A. The virtual world includes a virtual player 6105 and a virtual camera 6108 that is moved in the virtual world to match the position and orientation of the VR headset 6008 of FIG. 6A, and this camera's view is shown to the user 6007 via the VR headset 6008. The virtual world also includes a virtual sword 6106 that is moved in the virtual world to match the position and orientation of user's physical sword 6005 as tracked by the tracker 6006. The virtual world includes a virtual opponent 6102 that is holding a virtual sword 6104 in his hand 6103. The virtual world also includes a virtual robot 6101 which matches the movements and positions of the physical robot 6002, but the virtual robot 6101 is not rendered as a visual object in the virtual world and thus is invisible to the virtual camera 6108 and is therefore not displayed on the VR headset 6008 to the user 6007. The virtual robot 6101 is the virtual twin of the physical robot 6002 of FIG. 6A. The VW controller uses the actions of the virtual player 6105 to instruct the virtual opponent 6102 to execute swordfight animations that are appropriate responses to the virtual player's movements. …. During these motions, the virtual robot 6101 matches the virtual end effector (not shown) to the position and rotation of the opponent's virtual hand 6103. The VW controller (not shown) sends the axis angles of the virtual robot 6101 to the PW controller, which instructs the physical robot 6002 to match those angles. This causes the physical robot 6002 to move its end effector 6003 in the same manner as the virtual hand 6103 of the virtual opponent 6102, and thus also move the physical sword 6004 to match the virtual sword 6104. The VW controller can attach a virtual physics collider to the virtual swords 6104 and 6106, and use virtual physics simulation to ensure the swords can clash and deflect but not pass through each other. The VW controller can augment the virtual opponent 6102 animations, by using blend trees, ragdoll physics, or other tools, to better match the virtual opponent 6102 movements to the collision limits of the virtual sword 6104—so the limb movements will follow the limitations applied to the virtual sword 6104, rather than continue the animation and dislocate the virtual opponent's 6102 hand 6103 from the virtual sword 6104 ”).
Regarding claim 2, Edwards et al. teach all the limitation of claim 1, and further teach wherein the avatar includes at least a portion of a human avatar (Fig 6B, par 0120-0122, “a virtual opponent 6102 that is holding a virtual sword 6104 in his hand 6103”).
Regarding claim 3, Edwards et al. teach all the limitation of claim 1, and further teach further comprising one or more sensors configured to sense the pose of the end effector in the world space (par 0039, par 0092-0093, par 0105, “the virtual camera 5109 does not need to be in such a position, or it may be excluded entirely and the virtual person 5108 can be instantiated at a location determined by mathematical calculations and transformations that may use reference points—which may include the (X=0, Y=0, Z=0) origin point, the installation location of the PW robot arm 5001, the location of the PW end effector 5005, the location of the PW camera 5013, and/or a location calculated based on placement of multiple cameras, sensors, detection markers, or other knowledge of either the physical or virtual world”).
Regarding claim 4, Edwards et al. teach all the limitation of claim 1, and further teach wherein the pose is a commanded pose (par 0102, par 0108, “This command instructs the robot controller 5008 to utilize the Move protocol with interpolation option #1 to move the PW end effector 5005 to the (X,Y,Z) coordinates (400 mm, 300 mm, 200 mm) (where “mm” denotes millimeters, and the position is relative to the center of the base of the robot arm 5001) and rotate the PW end effector 5005 to the (Rx, Ry, Rz) orientation (180, 0, 180)” ….control movement of end effector through command).
Regarding claim 5, Edwards et al. teach all the limitation of claim 1, and further teach wherein the at least one processor is further configured to transform a reference frame of the world space to a reference frame of the virtual space to identify a location of the end effector in the virtual space (Figs 6A-6B, par 0120-0122, generate virtual world based on real would and determine virtual end effector in virtual space through match real world).
Regarding claim 6, Edwards et al. teach all the limitation of claim 1, and further teach further comprising a robotic arm configured to move the end effector (Fig 6A, par 0120, “An opponent robot 6002 wields a sword 6004 which is held by an end effector 6003. The robot may include multiple axes (not shown) and is attached to the PW controller which is attached to the computer, similar to FIG. 5A.”).
Regarding claim 7, Edwards et al. teach all the limitation of claim 6, and further teach further comprising one or more sensors configured to sense joint configurations of the robotic arm (par 0039, par 0100, par 0151, “In the instance depicted in FIG. 9A, the user 9008 is moving the surgical tool 9006 downwards 9007. Similar to FIG. 5A, the robot 9005 tracks the position and orientation of the tool 9006 and end effector 9009 by monitoring the angles of its joint axes (not shown) and transmits that information via a wired connection 9004 to the PW controller 9003, which transmits that information to the computer 9001. The VW controller tracks changes in this position and rotation information and determines (by a method that will be described in the subsequent paragraph) an amount of resistance force that should be applied to the surgical tool 9006. The computer 9001 sends information via connection 9002 instructing the PW controller 9003 to apply this resistance force, and the PW controller 9003 sends power and commands via connection 9004 to the robot 9005 to apply that resistance force via the robot's joint actuators (not shown). This thus creates an amount of resistance or force feedback to the movement of the surgical tool 9006 in the hand 9010 of the user 9008”).
Regarding claim 8, Edwards et al. teach all the limitation of claim 1, and further teach wherein the display is configured to be worn by the user (Fig 3A, par 0078, Fig 6A, par 0120, “FIG. 6A depicts the physical components of another embodiment that utilizes the second operating mode where a user 6007 wears a VR headset 6008 on his head and holds a sword 6005 that has an attached tracker 6006”).
Regarding claim 9, Edwards et al. teach all the limitation of claim 1, and further teach wherein the end effector is configured to interact with a body of the user (Figs 6A-6B, par 0120-0122, “if the virtual player 6105 raises his virtual sword 6106 and prepares to swing it downwards, then the VW controller will trigger an animation in which the virtual opponent 6102 raises his virtual hand 6103 and his virtual sword 6104 in a direction perpendicular to the virtual sword 6106 of the virtual player 6105, thus forming a block for an impending slash or strike. If the virtual player 6105 holds his virtual sword 6106 in a blocking position that is too far to the virtual player's left, the VW controller will trigger an animation in which the virtual opponent 6102 performs a jab with his virtual sword 6104 towards the player's undefended right side. During these motions, the virtual robot 6101 matches the virtual end effector (not shown) to the position and rotation of the opponent's virtual hand 6103. The VW controller (not shown) sends the axis angles of the virtual robot 6101 to the PW controller, which instructs the physical robot 6002 to match those angles. This causes the physical robot 6002 to move its end effector 6003 in the same manner as the virtual hand 6103 of the virtual opponent 6102, and thus also move the physical sword 6004 to match the virtual sword 6104. The VW controller can attach a virtual physics collider to the virtual swords 6104 and 6106, and use virtual physics simulation to ensure the swords can clash and deflect but not pass through each other. The VW controller can augment the virtual opponent 6102 animations, by using blend trees, ragdoll physics, or other tools, to better match the virtual opponent 6102 movements to the collision limits of the virtual sword 6104—so the limb movements will follow the limitations applied to the virtual sword 6104, rather than continue the animation and dislocate the virtual opponent's 6102 hand 6103 from the virtual sword 6104”).
Regarding claim 10, Edwards et al. teach all the limitation of claim 9, and further teach wherein the end effector is configured to manipulate and/or grasp a portion of the body of the user (Fig 8A-8B, par 0139-0140, “FIG. 8A depicts the physical-world components of an embodiment that uses the fourth operating mode for a robotic surgery. In this particular embodiment, the VW controller hosted on a computer 8001 generates a virtual twin (8101 of FIG. 8B) of the PW robot 8005 and uses an inverse kinematics function to generate the axis (not shown) movements necessary to move the end effector sensor 8009 of the robot 8005 downwards with a specified amount of force 8007. These instructions are sent via a wired connection 8002 to a PW controller 8003, which then instructs the robot 8005 via a wired connection 8004 to move a scalpel 8006 downwards along the patient 8008”).
Regarding claims 11-19, the method claims 11-19 are similar in scope to claims 1-2, 5, 4, 3, 6-7, and 9-10 and are rejected under the same rational.
Regarding claim 20, Edwards et al. teach a non-transitory computer readable memory including processor executable instructions that when executed by at least one processor perform the method of claim 11 (par 0198-0199). The remaining limitations of the claim are similar in scope to claim 11 and rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jin Ge whose telephone number is (571)272-5556. The examiner can normally be reached 8:00 to 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIN . GE
Examiner
Art Unit 2619
/JIN GE/Primary Examiner, Art Unit 2619
Prosecution Insights