DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Amendment
Applicant’s preliminary amendment filed on June 10, 2024 has been entered. Claims1, 3-6, 8-9, 12-14 and 16-19 have been amended. Claim 20 has been canceled. Claims 1-19 and 21 are pending in this application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 14-17, 19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954 cited in IDS dated 7/19/2024) in view of Reitmayr et al. (US 2014/0126769).
Regarding claim 1, Parsley discloses a tracking system for simulating body motion into a computing environment (Fig. 1; [0001], [0025]), the system comprising
one or more processors ([0025], [0034], e.g., a control unit 10 comprises a processor 11);
an optical sensor ([0032], e.g., camera 24) configured to signal that movement of the optical sensor has occurred through the one or more processors detecting that captured successive frames are different, the one or more processors measuring the movement by referencing automatically determined set points across the successive frames (Fig. 4; [0009], [0034]-[0035], [0040], e.g., detects and tracks the movement of the camera by comparing the locations of representations of the indicia 30 in images captured at different times);
a plurality of inertial measurement units ([0003], [0027]-[0028], e.g., a plurality of inertial measurement units 20), each controlled by the one or more processors to measure rotational data ([0028], e.g., measuring rotational data by gyroscopic); and
a hub in communication with the inertial measurement units and the optical sensor ([0028], [0032], e.g., the control unit 10 communicates with the inertial measurement units 20 and the camera 24),
wherein the hub receives the rotational data from the plurality of inertial measurement units over one or more wireless communication channels ([0028]], e.g., the control unit 10 receives the rotational data from the initial measurement units 20 over wireless communication channel),
the hub controlled by the one or more processors ([0034], e.g., the processor 11) to combine the rotational data obtained while tracking the body motion with data of the measured movement obtained while tracking the body motion ([0040], e.g., the processor synthesises the estimated position as determined by means of the camera 24 and the estimated motions of the subject's body as determined by the accelerometers 20), to output a data stream that enables simulation of the body motion in the computing environment ([0025], e.g., generate outputs such as a stream of position estimates, a video stream (optionally in conjunction with another processor 15) or motion demands to a robot or another item of mobile equipment),
wherein a body part movement in the computing environment is deduced from measured rotational data of the body part and the measured rotational data of other connected body parts and a position of the body in the computing environment is deduced from the data of the measured movement ([0013], [0027]-[0028], [0040]-[0041], e.g., a position and a motion of a robot or an image in the virtual reality environment is determined by the measured rotational data of the body parts from the inertial measurement units 20 and the position of the body determined by the camera).
Parsley does not specifically disclose wherein the one or more processors measuring the movement by referencing undefined set points across the successive frames.
However, Reitmayr discloses a tracking system (Figs 1-2; [0025], e.g., an electronic device 100) comprising:
one or more processors ([0022], e.g., one or more processors 161, 166 and 168); and
a camera ([0025], e.g., camera 114) configured to signal that movement of the camera has occurred through the one or more processors detecting that captured successive frames are different ([0025]-[0026], [0040], [0072], e.g., a first reference image and subsequent captured images are at a different depth or location),
the one or more processors measuring the movement by referencing automatically determined and undefined set points across the successive frames ([0026], [0040], [0072], e.g., measuring a camera position and orientation (pose) by referencing automatically determined and undefined a set of 3D interest points across the successive frames).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Reitmayr in the invention of Parsley for measuring a position and orientation of a camera by referencing automatically determined and undefined a set of 3D interest points in order to measure a position and orientation of the camera without the use of markers or indicia in a scene (see [0027] of Reitmayr).
Regarding claim 5, Parsley in view of Reitmayr further discloses the tracking system of claim 1, wherein measurement of movement commences from the optical sensor capturing a first frame, the first frame providing a starting point for tracking the body motion (Reitmayr, [0029], e.g., a first reference image provides a starting point for tracking the body motion/camera).
Regarding claim 14, Parsley further discloses the tracking system of claim 1, wherein a base pose for adoption before body motion capture can commence is predetermined in the computing environment ([0040]-[0041], e.g., a predetermined pose is adopted).
Regarding claim 15, Parsley further discloses the tracking system of claim 14, wherein the measured rotational data is used to derive an offset from the base pose, the offset being usable to construct a current pose ([0040], e.g., the current pose relative to the reference pose is determined by acceleration data from the accelerometers).
Regarding claim 16, Parsley further discloses the tracking system of claim 1, wherein the hub is configured to allow extraction of the measured rotational data from one or more of the plurality of inertial measurement units and/or the data of the measured movement from the optical sensor, obtained while tracking the body motion, for recording as a macro ([0028], e.g., the control unit 10 collects the data gathered by the accelerometers and estimates the motion of each of the subject's limbs. The control unit can integrate the measured accelerations received from each accelerometer to estimate the position of the respective body part to which that accelerometer is attached with respect to an initial reference position).
Regarding claim 17, Parsley in view of Reitmayr further discloses the tracking system of claim 1, wherein a position of the body obtained from the data of the measured movement is based on visual simultaneous localisation and mapping (Reitmayr, [0009]).
Regarding claim 19, Parsley further discloses the tracking system of claim 1, wherein at least one of the one or more processors is hosted in a computer platform (Fig. 2; [0041], e.g., a processor 15 is hosted in a virtual reality headset 17), wherein the deduction of the body part movement in the computing environment and the deduction of the position of the body in the computing environment is performed in the computer platform to generate the data stream in the computer platform ([0013], [0025], [0041], e.g., generate the video stream in the virtual reality headset).
Regarding claim 21, Parsley discloses a method of simulating body motion into a computing environment (Fig. 1; [0001], [0013], [0025]), the method comprising
measuring movement of an optical sensor ([0032], e.g., camera 24) obtained while tracking the body motion by referencing automatically determined set points across successive frames captured by the optical sensor that are different (Fig. 4; [0009], [0034]-[0035], [0040], e.g., detects and tracks the movement of the camera by comparing the locations of representations of the indicia 30 in images captured at different times);
combining, in a hub ([0028], [0032], e.g., the control unit 10), measured rotational data obtained while tracking the body motion with data of the measured movement ([0040], e.g., the processor synthesises the estimated position as determined by means of the camera 24 and the estimated motions of the subject's body as determined by the accelerometers 20), the measured rotational data being received in the hub from a plurality of inertial measurement units over one or more wireless communication channels ([0028]], e.g., the control unit 10 receives the rotational data from the initial measurement units 20 over wireless communication channel); and
output a data stream that enables simulation of the body motion in the computing environment ([0025], e.g., generate outputs such as a stream of position estimates, a video stream (optionally in conjunction with another processor 15) or motion demands to a robot or another item of mobile equipment),
wherein a body part movement in the computing environment is deduced from measured rotational data of the body part and the measured rotational data of other connected body parts and a position of the body in the computing environment is deduced from the data of the measured movement ([0013], [0027]-[0028], [0040]-[0041], e.g., a position and a motion of a robot or an image in the virtual reality environment is determined by the measured rotational data of the body parts from the inertial measurement units 20 and the position of the body determined by the camera).
Parsley does not specifically disclose wherein the one or more processors measuring the movement by referencing automatically determined and undefined set points across the successive frames.
However, Reitmayr discloses a tracking system (Figs 1-2; [0025], e.g., an electronic device 100) comprising:
one or more processors ([0022], e.g., one or more processors 161, 166 and 168); and
a camera ([0025], e.g., camera 114) configured to signal that movement of the camera has occurred through the one or more processors detecting that captured successive frames are different ([0025]-[0026], [0040], [0072], e.g., a first reference image and subsequent captured images are at a different depth or location),
the one or more processors measuring the movement by referencing automatically determined and undefined set points across the successive frames ([0026], [0040], [0072], e.g., measuring a camera position and orientation (pose) by referencing automatically determined and undefined a set of 3D interest points across the successive frames).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Reitmayr in the invention of Parsley for measuring a position and orientation of a camera by referencing automatically determined and undefined a set of 3D interest points in order to measure a position and orientation of the camera without the use of markers or indicia in a scene (see [0027] of Reitmayr).
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769), and further in view of Lee et al. (US 2015/0279053).
Regarding claim 2, Parsley in view of Reitmayr does not specifically disclose the tracking system of claim 1, wherein a quaternion representation of the rotational data is used to deduce the body part movement in the computing environment.
However, Lee discloses a tracking system (Figs 1-2; [0035]-[0036], e.g., the system for motion estimation) comprising a plurality of inertial sensors configured to measure rotational data ([0037], [0048], e.g., each of the motion sensors 200 can include an accelerometer 201, a gyroscope 202 and a magnetometer 203), wherein a quaternion representation of the rotational data is used to deduce body part movement in a computing environment ([0026], [0037], [0048], [0060], e.g., the motion estimation device 400 can use quaternion to deduce body part movement in virtual reality environment).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Lee in the invention of Parsley in view of Reitmayr for deducing body movement in a computing environment by using motion information such as quaternion because the quaternion can be used to compute rotation data and position data.
Claim(s) 3 is rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769), and further in view of Kaifosh et al. (US 2018/0020978).
Regarding claim 3, Parsley in view of Reitmayr does not specifically disclose the tracking system of claim 1, wherein the deduction of the body part movement in the computing environment is based on one or more forward kinematic algorithms.
However, Kaifosh discloses a tracking system (Fig. 1; [0048]) wherein deduction of a body part movement in a computing environment is based on one or more forward kinematic algorithms ([0080], [0126]-[0127], e.g., forward kinematics is used to determine the position of the wrist in a virtual reality environment).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Kaifosh in the invention of Parsley in view of Reitmayr for using forward kinematics to determine a position of a wrist or a non-anchored segment in order to determine a body part movement in a computing environment based on forward kinematic algorithms.
Claim(s) 4 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769), and further in view of Chamdani et al. (US 2017/0084070).
Regarding claim 4, Parsley further discloses the tracking system of claim 1, wherein the optical sensor is integrated with the hub and movement of the optical sensor occurs from motion of the body on which the hub is worn ([0032], [0042], e.g., the camera 24 can be integrated with the hub (the communication device 22 and the control unit 10 are integrated together)).
Parsley in view of Reitmayr does not disclose wherein the hub wherein the hub receives the rotational data from the plurality of inertial measurement units over one or more wireless communication channels.
However, Chamdani discloses a tracking system (Figs 1A-1B and 2, [0067]-[0071], e.g., a track system) wherein a hub ([0076], e.g., the hub 204) receives rotational data from a plurality of inertial measurement units over one or more wireless communication channels ([0018], [0069], [0076], e.g., the hub module collects rotational data from each of the inertial sensors over wireless communication channels).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Chamdani in the invention of Parsley in view of Reitmayr for including a wireless transceiver in each of a plurality of inertial sensors for intercommunication with a hub in order to collect sensing data in a wireless signal.
Regarding claim 6, Parsley in view of Reitmayr does not specifically disclose the tracking system of claim 1, wherein the hub is configured to pair with the plurality of inertial measurement units through proximity detection of emitted wireless signal from the plurality of inertial measurement units.
However, Chamdani discloses a tracking system (Figs 1A-1B and 2, [0067]-[0071], e.g., a track system) wherein a hub ([0076], e.g., the hub 204) is configured to pair with a plurality of inertial measurement units through proximity detection of emitted wireless signal from the plurality of inertial measurement units ([0018], [0067]-[0076], [0090], e.g., establishing an intercommunication session between the hub module and each of the inertial sensors through proximity detection of emitted wireless signal from the plurality of inertial sensors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Chamdani in the invention of Parsley in view of Reitmayr for pairing the hub with a plurality of inertial measurement units through proximity detection of emitted wireless signal from the plurality of inertial measurement units in order to collect sensor data in a wireless signal.
Claim(s) 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769) and Chamdani et al. (US 2017/0084070), and further in view of Huang et al. (US 2021/0157396).
Regarding claim 7, Parsley in view of Reitmayr and Chamdani discloses the tracking system of claim 6. Chamdani further discloses the tracking system is further configured to determine assignment of a body part on which each of the plurality of inertial measurement units is worn through analysis of output rotational data (Fig. 6J; [0070], [0180], e.g., since the inertial sensors are respectively attached to different parts of a body, the sensing data from each of the inertial sensors can be used to analyze attributes of the motion made at each of the body parts).
Parsley in view of Reitmayr and Chamdani does not specifically disclose wherein the hub is configured to determine assignment of the body part on which each of the plurality of inertial measurement units is worn through strength of emitted wireless signal relative to the hub.
However, Huang discloses a tracking system (Figs 1-3; [0016], a motion tracking system 10) wherein a hub ([0030], e.g., the processor 250) is configured to determine assignment of a body part on which each of the plurality of inertial measurement units is worn through strength of emitted wireless signal relative to the hub ([0031], [0033], e.g., the signal strengths would be recorded for a time period to generate a sequence of the second sensing data and then a displacement of a corresponding human body portion can be estimated through double integral on the detected acceleration (i.e., the second sensing data) of the human body portion in three axes).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Huang in the invention of Parsley in view of Reitmayr and Chamdani for determining assignment of a body part on which each of a plurality of inertial measurement units is worn through strength of emitted wireless signal relative to a hub in order to estimate a displacement of a corresponding body part based on the strength of emitted wireless signal.
Regarding claim 8, Parsley in view of Reitmayr, Chamdani and Huang further discloses the tracking system of claim 7, wherein the hub is further configured during pairing to save a unique identifier of each of the plurality of inertial measurement units against the assigned body part (Chamdani, [0019], [0096]-[0097], e.g., each the plurality of inertial sensors has a unique identifier).
Claim(s) 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769) and Chamdani et al. (US 2017/0084070), and further in view of LU HILL et al. (US 2019/0103033).
Regarding claim 9, Parsley in view of Reitmayr and Chamdani does not specifically disclose the tracking system of claim 6, wherein following pairing the one or more processors is configured to perform calibration using data on dimensions of body parts on which the plurality of inertial measurement units is worn before tracking of the body motion commences.
However, LU HILL discloses a tracking system (Fig. 3; [0035], e.g., a sensor system 300) configured to perform calibration using data on dimensions of body parts on which a plurality of inertial measurement units is worn before tracking of body motion commences (Fig. 6; [0033], [0048], e.g., performing an initial calibration using the size of each portion of the user’s body on which a plurality of inertial sensors 606-616 is positioned). 204
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of LU HILL in the invention of Parsley in view of Reitmayr and Chamdani for performing a calibration by using the size of each portion of the user’s body on which a plurality of inertial sensors is worn in order to ensure that motion of users of different sizes is correctly identified by a system.
Regarding claim 10, Parsley, Reitmayr, Chamdani and LU HILL further discloses the tracking system of claim 9, wherein the one or more processors analyses images containing the body parts to derive dimensions of the body parts (LU, HILL, [0033], e.g., image processing techniques may be used to estimate the size of each portion, the users, height, plumb-line, etc.).
Regarding claim 11, Parsley, Reitmayr, Chamdani and LU HILL further discloses the tracking system of claim 10, wherein the dimensions are derived using one or more of a machine learning algorithm and skeletal structure models (LU, HILL, [0033], [0048], e.g., using a machine learning image analysis system and a shadow object 600 corresponding to the sensor-tracked object).
Regarding claim 12, Parsley, Reitmayr, Chamdani and LU HILL further discloses the tracking system of claim 10, wherein the images are taken by the optical sensor (LU, HILL, Fig. 2; [0028], e.g., the image capture device 204).
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769), Chamdani et al. (US 2017/0084070) and LU HILL et al. (US 2019/0103033), and further in view of Huang et al. (US 2021/0157396).
Regarding claim 13, LU HILL further discloses the tracking system, wherein sensor data from the plurality of the inertial measurement units being worn on the respective body parts can be used to determine dimensions of the body parts ([0033]).
Parsley, Reitmayr, Chamdani and LU HILL does not specifically disclose wherein the derivation of the dimensions is done in conjunction with the plurality of the inertial measurement units to cross reference strength of their emitted wireless signals against measurement data based on corresponding body part images.
However, Huang discloses a tracking system (Figs 1-3; [0016], a motion tracking system 10) wherein position information is determined in conjunction with a plurality of inertial measurement units to cross reference strength of their emitted wireless signals against measurement data based on corresponding body part images (Figs 1-3; [0030]-[0032], [0034], [0037]-[0038], [0041], e.g., the processor is configured to determine position information according to a combination of position information obtained based on second sensing data and third sensing data, wherein the second sensing data is obtained based on strength of wireless signals emitted by a plurality of inertial sensors 130 and the third sensing data is obtained based on images captured from image sensors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings od Huang in the invention of Parsley, Reitmayr, Chamdani and LU HILL for deriving dimensions of body parts by cross referencing strength of emitted wireless signals against measurement data based on corresponding body part images in order to improve accuracy of the dimensions of body parts.
Claim(s) 18 is rejected under 35 U.S.C. 103 as being unpatentable over Parsley et al. (US 2020/0209954) in view of Reitmayr et al. (US 2014/0126769), and further in view of Kochi et al. (US 2007/0263924).
Regarding claim 18, Parsley in view of Reitmayr does not specifically disclose the tracking system of claim 1, wherein the optical sensor is any one or more of a stereoscopic camera, LIDAR and optical sonar sensors.
However, Kochi discloses a system (Figs 1-2; [0067]-[0068], e.g., an image processing device 100) comprising:
one or more processors ([0068], e.g., one or more processors 5); and
an optical sensor ([0067], [0072], e.g., the photographing camera) configured to signal that movement of the optical sensor has occurred through the one or more processors detecting that captured successive frames are different (Fig. 3; [0072], e.g., a series of photographed images that change over time),
wherein the optical sensor is any one or more of a stereoscopic camera, LIDAR and optical sonar sensors ([0076], e.g., the photographing camera is a stereoscopic camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the teachings of Kochi in the invention of Parsley in view of Reitmayr for capturing a series of images in order to precisely measure a position or posture of the photographing camera using moving images (see [0007] of Kochi).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Latella, JR. (US 2019/0038187) a tracking system configured to pair a plurality of inertial sensors to a computing device (Fig. 1; [0027], [0101]).
Sullivan et al. (US 2010/0164862) discloses a tracking system (Figs 1-2, e.g., [0027]-[0030], e.g., a visual motion sensing system 104 and a physical motion sensing system 106) wherein a hub (e.g., the computer system 204 and the physical data collector 208) is configured to pair with a plurality of inertial measurement units through proximity detection of emitted wireless signal from the plurality of inertial measurement units ([0022], [0030], e.g., pair with a plurality of inertial sensors 108 through proximity detection of wireless signals 209).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HONG ZHOU whose telephone number is (571)270-5372. The examiner can normally be reached 9:00-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BENJAMIN C LEE can be reached at 571-272-2963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HONG ZHOU/Primary Examiner, Art Unit 2629
Prosecution Insights