DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on May 14, 2026 has been entered.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on May 14, 2026 was filed after the mailing date of the notice of allowance on May 06, 2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claim(s) 1-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elias et al. (US 2019/0101981 A1, hereinafter referred as “Elias”) in view of Arora et al. (US 2023/0326153 A1, hereinafter referred as “Arora”).
Regarding claim 1, Elias discloses a data collection system (¶0005 discloses VR glove capable of measuring the movement of individual finger and thumb bones) comprising:
an augmented reality (AR) headset configured to be worn by a user (¶0003 and ¶0022 discloses VR headset and the VR gloves can be attached to a user); and
a wearable data collection device (VR glove) comprising:
a hand element configured to receive a hand of the user (Fig. 2 and abstract discloses the VR glove can include a plurality of inertial measurement units (IMUs) to track the movement of one or more finger and/or hand sections);
a plurality of finger elements extending from the hand element (Fig. 2 and abstract discloses the VR glove can include a plurality of inertial measurement units (IMUs) to track the movement of one or more finger and/or hand sections);
a plurality of joints, wherein each joint of the plurality of joints couples a finger element of the plurality of finger elements to the hand element (Fig. 2, ¶0029 and ¶0034 discloses the VR glove 230 can be capable of fine-level motion capture. That is, the VR glove 230 can be capable of discerning between the movement of an entire finger (e.g., the user waiving his or her index finger) and the movement of a finger joint (e.g., the user bending the index finger));
a controller mount on the wearable data collection device (VR glove) configured to secure a controller (Fig. 2 and ¶0057 discloses the VR glove 330 can be a knitted or woven glove where one or more (e.g., all) electronics components can be integrated into the fabric of the glove) associated with the AR headset (Fig. 3 and ¶0053 discloses hand controller 332 can one or more components including, but not limited to, a memory 325, a connector 327, and a transceiver 338… The connector 327 can be used to connect the VR glove 330 to one or more components (e.g., VR headset) for wired communications), wherein the AR headset is configured to track a position and orientation of the controller (Fig. 4A and ¶0069 discloses controller located on a VR headset can …receive information associated with the motions of steps 414-426 and can update the simulated environment; Figs. 2-3 and ¶0065 discloses controller (e.g., controller 323 illustrated in FIG. 3) can receive one or more signals, which can be indicative of the local frame 492, from the respective IMU (step 418 of process 400); and abstract discloses the IMUs can include one or more motion sensors, such as a gyroscope and an accelerometer, for measuring the orientation, position, and velocity of objects (e.g., finger bones) that the IMU can be attached);
a plurality of sensors mounted on the wearable data collection device (VR glove) configured to capture sensor data (Fig. 2 and ¶0027 discloses the IMUs 202 can be configured to measure the acceleration and the rotational rate of the user's bone in order to capture the motion of the user's hand and/or fingers) during a recording session (¶0061 discloses examples of the disclosure can include determining (e.g., including recording) the range of motion for the user's hand); and
a processing circuit operatively coupled to the plurality of sensors configured to collect (¶0032 discloses one or more (e.g., each) IMUs 302 can be coupled to a microcontroller unit (MCU) 304. The IMU 302 can measure inertial motion (e.g., acceleration and rotational rate) of the corresponding finger or thumb bone and can communicate the information to the MCU 304. The MCU 304 can process the information and/or communicate the information to a controller 323) and transmit the sensor data to the AR headset (Fig. 3 and ¶0053 discloses hand controller 332 can one or more components including, but not limited to, a memory 325, a connector 327, and a transceiver 338… The connector 327 can be used to connect the VR glove 330 to one or more components (e.g., VR headset)).
Elias doesn’t disclose capture head position and orientation data during a recording session; and wherein the data collection system is configured to associate the head position and orientation data with the sensor data and the position and orientation of the controller.
However, in the same field of endeavor, Arora discloses capture head position and orientation data during a recording session (¶0054 and ¶0081 discloses a mixed reality headset can track the position and orientation of a recording user’s head); and wherein the data collection system is configured to associate the head position and orientation data (¶0054 and ¶0081 discloses a mixed reality headset can track the position and orientation of a recording user’s head) with the sensor data (¶0050 discloses handheld controller 300 includes tracking components (e.g., an IMU or other suitable sensors) for detecting position or orientation, such as position or orientation relative to wearable head device 2102) and the position and orientation of the controller (¶0050 discloses such input may correspond to a position, orientation, and/or movement of the handheld controller (and, by extension, to a position, orientation, and/or movement of a hand of a user holding the controller)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that the recordings can be utilized to reconstruct the task accurately which makes the demonstration more useful for imitation learning.
Regarding claim 2, Elias discloses the data collection system of claim 1, wherein the AR headset includes one or more cameras configured to track the controller to determine the position and orientation of the wearable data collection device (VR glove) (¶0078 discloses the camera (e.g., included in a VR headset) could take images of the user's fingers and can check whether the orientation of the fingers in the images differ from the system's simulation. Additionally or alternatively, the camera can be used to track the position(s) of the user's hand(s); abstract and ¶0062 discloses taking measurements of the orientation and the position of objects that the IMU can be attached to and can include using one or more cameras (not shown); and Fig. 2 and ¶0057 discloses the VR glove 330 can be a knitted or woven glove where one or more (e.g., all) electronics components including controller 223 can be integrated into the fabric of the glove).
Regarding claim 3, Elias discloses the data collection system of claim 1, wherein the controller mount secures the controller to the wearable data collection device (VR glove) such that the controller moves in coordination with the wearable data collection device (VR glove) (Fig. 2 and ¶0057 discloses the VR glove 330 can be a knitted or woven glove where one or more (e.g., all) electronics components can be integrated into the fabric of the glove).
Regarding claim 4, Elias discloses the data collection system of claim 1, wherein the AR headset further comprises a camera mounted on the AR headset configured to capture visual data of an environment in a field of view of the user (¶0019 discloses the one or more cameras can be used to capture the user's real environment in AR technology).
Regarding claim 5, Elias discloses the data collection system of claim 1, wherein the plurality of sensors comprises: at least one pressure sensor positioned on each of the plurality of finger elements (Figs. 2-3 and ¶0042 discloses the VR glove can also include one or more force sensors 306. The force sensors 306 can be located at the fingertips of the VR glove 330); at least one position sensor at each of the plurality of joints configured to capture angle data (¶0029 and ¶0034 discloses the controller 323 can process the signals from the respective bus 322 individually to track the motion of a specific finger bone and/or can process two or more signals collectively to track the motion of the finger joint(s)); and at least one camera mounted on the wearable data collection device (VR glove) (¶0026 discloses VR glove 230 can include a plurality of electronic components; and ¶0069 discloses one or more other components (e.g., cameras, optical sensors, the reset electrodes 214 illustrated in FIG. 2) can be provide the same and/or additional information related to one or more the user's hand movement, location, and position).
Regarding claim 6, Elias discloses the data collection system of claim 1, further comprising a connection interface configured to transmit the sensor data from the processing circuit to the AR headset, wherein the connection interface comprises at least one of a wired connection (Fig. 3 and ¶0053 discloses hand controller 332 can one or more components including, but not limited to, a memory 325, a connector 327, and a transceiver 338… The connector 327 can be used to connect the VR glove 330 to one or more components (e.g., VR headset) for wired communications) and a wireless connection (Fig. 2 and ¶0027 discloses the transceiver 238 can be configured to communicate with an external device (e.g., the VR headset and/or the host device)).
6. Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elias in view of Arora, in further view of Jarvis et al. (US 2024/0408757 A1, hereinafter referred as “Jarvis8757”), and still in further view of Ranjbar et al. (US 12,365,093 B1, hereinafter referred as “Ranjbar”).
Regarding claim 7, Elias doesn’t disclose the data collection system of claim 1, wherein the data collection system is configured to: record head position and orientation data from the AR headset along with the sensor data from the wearable data collection device; and use the sensor data and the head position and orientation data to train a neural network that controls a robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device.
However, in the same field of endeavor, Jarvis8757 discloses wherein the data collection system is configured to: record head position and orientation data from the AR headset (¶0048 and ¶0073 discloses connect to the MR devices 340 to get the position/pose information of the human data collector) along with the sensor data from the wearable data collection device (¶0064 discloses the human data collector 412 may wear an intelligent glove having various sensors embedded in the glove… The collected data may then be provided to the wearable computation subsystem for recording); and use the sensor data and the head position and orientation data to train a neural network (¶0014 discloses the data collection device may comprise a human-machine operation interface worn by the human data collector and used to perform the various human-operated robot tasks related to testing and/or training the machine learning model or other software; and ¶0017 discloses the computation device includes processing capabilities that allow it to execute the machine learning model and the other software so that the machine learning model and other software can be trained and/or tested) that controls a robotic counterpart device (¶0009 discloses a robot teaching and testing system that performs human-operated robot tasks according to instructions generated from generative AI models)...
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that inconsistencies or noise in human hand movement can be eliminated.
Elias as modified doesn’t disclose robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device.
However, in the same field of endeavor, Ranjbar discloses robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device (col. 6, lines 11-24 discloses the sensor configuration of the sensor glove 104 may generally match a sensor configuration of a target robot hand... For example, if a robot hand of interest has tactile sensors, the sensor glove 104 may include tactile sensors).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Elias so that so that the synthetic source data can contain the same scope of sensor data for the subject hand that true robot data would have for the target robot hand (col. 6, lines 13-16).
7. Claim(s) 8-10 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elias in view of Arora and in further view of Jarvis et al. (US 2023/0072317 A1, hereinafter referred as “Jarvis2317”).
Regarding claim 8, Elias discloses a method of collecting training data using a data collection system (¶0005 discloses VR glove capable of measuring the movement of individual finger and thumb bones) comprising an augmented reality (AR) headset and a wearable data collection device (VR glove) (¶0003 and ¶0022 discloses VR headset and the VR gloves can be attached to a user), the method comprising:
…tracking, by the AR headset, a position and orientation of a controller (Fig. 4A and ¶0069 discloses controller located on a VR headset can …receive information associated with the motions of steps 414-426 and can update the simulated environment; Figs. 2-3 and ¶0065 discloses controller (e.g., controller 323 illustrated in FIG. 3) can receive one or more signals, which can be indicative of the local frame 492, from the respective IMU (step 418 of process 400); and abstract discloses the IMUs can include one or more motion sensors, such as a gyroscope and an accelerometer, for measuring the orientation, position, and velocity of objects (e.g., finger bones) that the IMU can be attached) secured to the wearable data collection device (VR glove) (Fig. 2 and ¶0057 discloses the VR glove 330 can be a knitted or woven glove where one or more (e.g., all) electronics components can be integrated into the fabric of the glove);
capturing sensor data via a plurality of sensors mounted on the wearable data collection device (VR glove) (Fig. 2 and ¶0027 discloses the IMUs 202 can be configured to measure the acceleration and the rotational rate of the user's bone in order to capture the motion of the user's hand and/or fingers) during the recording session (¶0061 discloses examples of the disclosure can include determining (e.g., including recording) the range of motion for the user's hand);
transmitting the sensor data from the wearable data collection device (VR glove) to the AR headset (Fig. 3 and ¶0053 discloses hand controller 332 can one or more components including, but not limited to, a memory 325, a connector 327, and a transceiver 338… The connector 327 can be used to connect the VR glove 330 to one or more components (e.g., VR headset)) via a processing circuit (¶0032 discloses one or more (e.g., each) IMUs 302 can be coupled to a microcontroller unit (MCU) 304. The IMU 302 can measure inertial motion (e.g., acceleration and rotational rate) of the corresponding finger or thumb bone and can communicate the information to the MCU 304. The MCU 304 can process the information and/or communicate the information to a controller 323).
Elias doesn’t disclose capturing head position and orientation data via the AR headset during the recording session; associating the head position and orientation data with the sensor data and the position and orientation of the wearable data collection device; and initiating a recording session in response to receiving a user input and terminating the recording session in response to receiving a second user input.
However, in the same field of endeavor, Arora discloses capturing head position and orientation data via the AR headset during the recording session (¶0054 and ¶0081 discloses a mixed reality headset can track the position and orientation of a recording user’s head); and associating the head position and orientation data (¶0054 and ¶0081 discloses a mixed reality headset can track the position and orientation of a recording user’s head) with the sensor data (¶0050 discloses handheld controller 300 includes tracking components (e.g., an IMU or other suitable sensors) for detecting position or orientation, such as position or orientation relative to wearable head device 2102) and the position and orientation (¶0050 discloses such input may correspond to a position, orientation, and/or movement of the handheld controller (and, by extension, to a position, orientation, and/or movement of a hand of a user holding the controller)) of the wearable data collection device (¶0043 discloses mixed reality system 112 may also include a handheld controller 300, and/or an auxiliary unit 320, which may be a wearable beltpack).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that the recordings can be utilized to reconstruct the task accurately which makes the demonstration more useful for imitation learning.
Elias as modified doesn’t disclose initiating a recording session in response to receiving a user input and terminating the recording session in response to receiving a second user input.
However, in the same field of endeavor, Jarvis2317 discloses initiating a recording session in response to receiving a user input and terminating the recording session in response to receiving a second user input (Fig. 1B and ¶0035 discloses the data collector 105 may use the voice user interface 122 to provide audio commands such as ‘begin recording’ at the start of a data collection process or at the start of an instructed action or ‘stop recording’ at the end of the data collection process or the end of an instructed action).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that bounded, intentional demonstrations are used for inputting data for machine learning.
Regarding claim 9, Elias discloses the method of claim 8, further comprising capturing visual data of an environment in a field of view of a user using a camera on the AR headset (¶0019 discloses the one or more cameras can be used to capture the user's real environment in AR technology).
Elias doesn’t disclose …during a recording session.
However, in the same field of endeavor, Jarvis2317 discloses …during the recording session (Fig. 1B and ¶0035 discloses the data collector 105 may use the voice user interface 122 to provide audio commands such as ‘begin recording’ at the start of a data collection process or at the start of an instructed action or ‘stop recording’ at the end of the data collection process or the end of an instructed action).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that bounded, intentional demonstrations are used for inputting data for machine learning.
Regarding claim 10, Elias discloses the method of claim 8, wherein the plurality of sensors comprises: at least one pressure sensor positioned on each of a plurality of finger elements of the wearable data collection device (VR glove) (Figs. 2-3 and ¶0042 discloses the VR glove can also include one or more force sensors 306. The force sensors 306 can be located at the fingertips of the VR glove 330); at least one position sensor at each of a plurality of joints that couple the plurality of finger elements to a hand element of the wearable data collection device (VR glove) (¶0029 and ¶0034 discloses the controller 323 can process the signals from the respective bus 322 individually to track the motion of a specific finger bone and/or can process two or more signals collectively to track the motion of the finger joint(s)); and at least one camera mounted on the wearable data collection device (VR glove) (¶0026 discloses VR glove 230 can include a plurality of electronic components; and ¶0069 discloses one or more other components (e.g., cameras, optical sensors, the reset electrodes 214 illustrated in FIG. 2) can be provide the same and/or additional information related to one or more the user's hand movement, location, and position).
Regarding claim 12, Elias discloses the method of claim 8, wherein transmitting the sensor data to the AR headset comprises transmitting the sensor data via at least one of a wired connection (Fig. 3 and ¶0053 discloses hand controller 332 can one or more components including, but not limited to, a memory 325, a connector 327, and a transceiver 338… The connector 327 can be used to connect the VR glove 330 to one or more components (e.g., VR headset) for wired communications) and a wireless connection (Fig. 2 and ¶0027 discloses the transceiver 238 can be configured to communicate with an external device (e.g., the VR headset and/or the host device)).
8. Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elias in view of Arora, in further view of Jarvis2317, and still in further view of Harris et al. (US 2015/0253847 A1, hereinafter referred as “Harris”).
Regarding claim 13, Elias as modified doesn’t disclose the method of claim 8, wherein the controller is mechanically mounted to the wearable data collection device such that the controller moves in coordination with the wearable data collection device.
However, in the same field of endeavor, Harris discloses wherein the controller is mechanically mounted to the wearable data collection device such that the controller moves in coordination with the wearable data collection device (Fig. 1 and ¶0030 discloses the control module 100 is attachable to the glove 110, but the control module 100 is not enclosed by the glove. For example, the control module 100 may comprise a clasp that is configured to attach to a slit in the fabric of the glove 110).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Elias so that the control module 100 individually can comprise water proof or water resistant construction (¶0030).
9. Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Elias in view of Arora, in further view of Jarvis2317, in further view of Jarvis8757 and still in further view of Ranjbar.
Regarding claim 14, Elias as modified doesn’t disclose the method of claim 8, further comprising training, using the sensor data and the position and orientation of the controller, a neural network that controls a robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device.
However, in the same field of endeavor, Jarvis8757 discloses training, using the sensor data (¶0064 discloses the human data collector 412 may wear an intelligent glove having various sensors embedded in the glove… The collected data may then be provided to the wearable computation subsystem for recording) and the position and orientation of the controller (¶0048 and ¶0073 discloses connect to the MR devices 340 to get the position/pose information of the human data collector), a neural network (¶0014 discloses the data collection device may comprise a human-machine operation interface worn by the human data collector and used to perform the various human-operated robot tasks related to testing and/or training the machine learning model or other software; and ¶0017 discloses the computation device includes processing capabilities that allow it to execute the machine learning model and the other software so that the machine learning model and other software can be trained and/or tested) that controls a robotic counterpart device (¶0009 discloses a robot teaching and testing system that performs human-operated robot tasks according to instructions generated from generative AI models)...
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Elias so that inconsistencies or noise in human hand movement can be eliminated.
Elias as modified doesn’t disclose …robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device.
However, in the same field of endeavor, Ranjbar discloses robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device (col. 6, lines 11-24 discloses the sensor configuration of the sensor glove 104 may generally match a sensor configuration of a target robot hand... For example, if a robot hand of interest has tactile sensors, the sensor glove 104 may include tactile sensors).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Elias so that the synthetic source data can contain the same scope of sensor data for the subject hand that true robot data would have for the target robot hand (col. 6, lines 13-16).
Allowable Subject Matter
10. Claims 15-20 are allowed.
The closest prior art of Jarvis8757 in ¶0050 discloses the collected data may be provided to a wearable computation subsystem for recording. ¶0064 discloses the human data collector 412 may wear an intelligent glove having various sensors embedded in the glove… The collected data may then be provided to the wearable computation subsystem for recording. ¶0048 and ¶0073 discloses connect to the MR devices 340 to get the position/pose information of the human data collector. ¶0014 discloses the data collection device may comprise a human-machine operation interface worn by the human data collector and used to perform the various human-operated robot tasks related to testing and/or training the machine learning model or other software; and ¶0017 discloses the computation device includes processing capabilities that allow it to execute the machine learning model and the other software so that the machine learning model and other software can be trained and/or tested. ¶0009 discloses a robot teaching and testing system that performs human-operated robot tasks according to instructions generated from generative AI models.
However, the prior art, alone or in combination, doesn’t disclose “receiving additional sensor data and additional position and orientation data from multiple recording sessions, wherein the multiple recording sessions comprise recordings of different tasks performed with the wearable data collection device; analyzing the additional sensor data to identify one or more patterns; and refining the trained neural network model based on the one or more patterns to improve performance of the robotic counterpart device,” as recited in claim 15.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PRIYANK J SHAH whose telephone number is (571)270-3732. The examiner can normally be reached on 10:00 - 6:00 M-F.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ghebretinsae, Temesghen can be reached on (571) 272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PRIYANK J SHAH/Primary Examiner, Art Unit 2626
Prosecution Insights