DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 9 are rejected under 35 U.S.C. 103 as being unpatentable over Servati (US 20260041354 A1) in view of Deshpande (US 20160296345 A1) in further view of Hemken (US 20170225329 A1).
Regarding claim 1, Servati teaches A wearable data collection device comprising: ([0001] This invention relates to stretchable textile smart sensor apparel and other wearable devices for tracking one or more body metrics.)
a hand element configured to receive a hand of a user; ([0036] This embodiment enables tracking of hands, fingers or body parts even during interaction with different objects [0038] FIG. 1(a), which is a smart textile glove 100 worn on the hand of a user 101 who is holding an object 102)
a plurality of sensors mounted on the wearable data collection device and configured to capture sensor data during a recording session; and ([0034] Embodiments described herein relate to a wearable device comprising one or more stretchable textile sensors (“smart textile wearable device”) for tracking one or more body metrics such as movement, muscle strength, force, temperature, sweat, heart rate, blood pressure, electrocardiogramals, electromyography (EMG) signals, electroencephalography (EEG) signals, and electrodermal activity (EDA) signals [0038] The smart textile glove 100 comprises stretchable inner and outer textile layers 116, 117 sandwiching and embedded with yarn sensors with different sensing modalities, including: stretch sensing yarn sensors 110, force sensing yarn sensors 111 and bio-signal sensing yarn sensors 112 for sensing temperature, sweat and other biological metrics. The yarn sensors 110, 111 112 are located in the glove 110 where the desired body metrics can be measured when the glove is worn, and for example can be located in the proximity of finger, wrist joints, palm of the hand, tips of the fingers or around the wrist when the glove is worn. The yarn sensors 110, 111, 112 are communicatively connected by vine-like stretchy interconnects 113 that electrically connect to each yarn sensor individually and are embedded in the textile layers 116, 117 without making limitations in stretchability, comfort, breathability, washability and fit for the user. The stretchy interconnects 113 communicatively connect the yarn sensors 110, 111, 112 to an embedded integrated circuit 120 in the smart textile glove 100 that serves in processing and/or wirelessly communicating with the data gateway and feedback device 103.)
a processing circuit operatively coupled to the plurality of sensors and configured to collect and transmit the sensor data. ([0062] Alternatively, one or both of the embedded integrated circuit 120 or removable integrated circuit 121 in the smart sensing wearable device 100 includes a processor and a non-transitory computer readable medium having stored thereon the machine learning (ML) program 200 executable by the processor, in which case the multimodal sensor data can be processed directly on the smart textile wearable device 100 and the health parameter outputs can be transmitted to the data gateway and feedback device 103 for display or notification through sound or haptic feedback on the wearable device 100 or other devices. [0069] Finally, both software applications transmit the collected data to a database 590)
Servati does not expressly disclose but Deshpande discloses a plurality of finger elements extending from the hand element; ([0010] the present disclosure may include a hand exoskeleton comprising a plurality of finger exoskeletons)
a plurality of joints, wherein each joint of the plurality of joints couples a finger element of the plurality of finger elements to the hand element, and wherein: ([0010] the present disclosure may include a hand exoskeleton comprising a plurality of finger exoskeletons, each finger exoskeleton comprising a plurality of joints and a plurality of sensors configured to measure rotation of at least some of the plurality of joints of the finger exoskeleton.)
the plurality of joints enable movement of the plurality of finger elements relative to the hand element within a constrained range of motion; and ([0026] Joints 120 may be any juncture of finger exoskeleton at which movement may occur. The movement may be rotation, lateral motion, or any other motion. Joints 120 may be hinge joints configured to provide rotation about a pivot point. [0029] FIG. 1, while the MCP and DIP joints may utilize sliding joints 130 to protect against non-normal forces, the PIP joint may not utilize a sliding joint. This may allow for greater range of motion without requiring an additional motor. For example, by using joints 120 c and 120 d to replicate the motion of the PIP joint, the PIP joint has much greater range of motion without interference from the exoskeleton.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Servati does not expressly disclose but Hemken discloses the constrained range of motion corresponds to movement capabilities of a robotic counterpart device; ([0005] Using various embodiments, methods, apparatuses, systems and techniques, data for motions and movements, changes in posture, changes in position, vocalizations, gestures, grasping, gait, and other bodily dynamics of living subjects are captured using sensors. Using the resulting sensor data, a supervised learning algorithm or program (AI program) implemented in a machine-learning computer program can be programmed to control an autonomous robot in a substantially similar manner as the living subject.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Hemken with a reasonable expectation of success by collecting data to be used for the purpose of teaching autonomous robots tasks that can be performed by humans as taught by Hemken ([0002]).
Regarding claim 2, Servati does not expressly disclose but Deshpande discloses The wearable data collection device of claim 1, wherein the plurality of finger elements comprises at least three finger elements including a thumb element, an index finger element, and a pinky finger element. ([0059] As shown in FIG. 11, each of the digits from the index finger to the pinky finger may utilize a finger exoskeleton similar to finger exoskeleton 100 [0048] FIG. 9 illustrates an example embodiment of a robotic thumb exoskeleton)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Regarding claim 3, Servati does not expressly disclose but Deshpande discloses The wearable data collection device of claim 2, wherein the thumb element is fixed relative to the hand element, and wherein the index finger element and the pinky finger element are movable relative to the hand element. ([0059] As shown in FIG. 11, each of the digits from the index finger to the pinky finger may utilize a finger exoskeleton similar to finger exoskeleton 100 [0057] some embodiments, medical tape, velcro, or any other suitable material may be used to secure attachment 956 to the metacarpal bone and may prevent movement between thumb exoskeleton 900 and the thumb of the user.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Regarding claim 4, Servati teaches The wearable data collection device of claim 1, wherein the plurality of sensors comprises:
at least one pressure sensor positioned on each of the plurality of finger elements; ([0081] pressure sensing yarn sensor 433)
at least one position sensor at each of the plurality of joints configured to capture angle data; and ([0068] Other data can be used as ground truth or multimodal monitoring such as tagging by an expert (e.g. doctor, clinician, coach or subjective input from the user or patient) for a particular event, processed video or movement from a model to match, text input from the user, force plate sensors, weight of the loads that the user is using during an exercise, or simulation using inverse kinematics of movements for muscle forces, EMG, ECG, or other modality, global position system or other data source.)
at least one camera mounted on the wearable data collection device and configured to capture visual data. ([0037] the angle tracking can be accurate and in different orientations extracted from IMUs or complemented with computer vision using camera as well as stretch and force sensors and can be for normal flexion/extension or abnormal conditions such as hyper extension, and valus/vargus rotation angles)
Regarding claim 5, Servati does not expressly disclose but Deshpande discloses The wearable data collection device of claim 1, further comprising a mount configured to hold a device that tracks position and orientation of the wearable data collection device in space during the recording session. ([0025] Mounting components 110 may be attached to a user of finger exoskeleton 100 using attachment straps 112. [0032] For example, the orientation and location of the framework coupled to mounting component 110 a may be adjusted based on the user of finger exoskeleton 100.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Regarding claim 6, Servati teaches The wearable data collection device of claim 1, further comprising a plurality of contact surfaces positioned on the plurality of finger elements, wherein the contact surfaces are configured to contact objects being manipulated by the wearable data collection device. ([0029] FIG. 6(e) are photographs and a table of an example implementation showing different grasped objects (photographs) and its confusion matrix (table) based on the training of the output ML model.)
Regarding claim 9, Servati teaches The wearable data collection device of claim 1, wherein the sensor data captured during the recording session is used to train a neural network ([0065] However, each layer of the core ML program 200 can include different neural networks (NN), long short-term memory (LSTM), convolutional neural network (CNN), 2-layer stacked bi-directional LSTM (Bi-LSTM), fully connected network (FC), hidden Markov model, generative neural network (GAN), or any other machine learning algorithms known to experts in the field. The core ML program 200 output is determined by real-time data as well as a sliding time window (for example 2 sec) of data to accommodate for the known trends in the time-dependent signals, which is implemented as a pre-processing in the input layer 203.)
Servati does not expressly disclose but Hemken discloses that controls the robotic counterpart device, and wherein the robotic counterpart device has a joint and sensor configuration that matches the wearable data collection device. ([0047] As block 203, the data processing unit of the sensor apparatus receives data (objective measurements) transmitted by the sensor. At block 205, the data processing unit, saves the data to a storage medium using a format that can be processed by another computing device using which an autonomous robot can perform substantially in a similar manner to the living subject when the data was generated.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Hemken with a reasonable expectation of success by collecting data to be used for the purpose of teaching autonomous robots tasks that can be performed by humans as taught by Hemken ([0002]).
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Servati (US 20260041354 A1) in view of Deshpande (US 20160296345 A1) in further view of Hemken (US 20170225329 A1) in further view of Yuan (US 20250123685 A1)
Regarding claim 7, Servati does not expressly disclose but Yuan discloses The wearable data collection device of claim 6, wherein at least one of the plurality of contact surfaces comprises a rubber material configured to deform when contacting an object. ([0176] The embodiments of the present disclosure may measure the deformation of a plurality of degrees of freedom (e.g., three dimensions including the bending around the X-axis, bending around the Z-axis, and stretching along the Y-axis) by a single sensor to realize the detection of posture data of the finger in a plurality of dimensions, which is conducive to the simplification of the preparation process and the miniaturization of the design while improving the detection accuracy. [0179] In some embodiments, the polymeric material includes but is not limited to, silicone, rubber, resin, etc)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Yuan with a reasonable expectation of success by determining a correspondence relationship between sensor data and finger joint angles as taught by Yuan ([0002]).
Regarding claim 8, Servati does not expressly disclose but Yuan discloses The wearable data collection device of claim 1, further comprising an activation mechanism configured to:
initiate the recording session in response to a first user input; and ([0248] In some embodiments, the sample sensor data may be data collected by a sample strain sensor while the sample user is performing a hand movement (e.g., the sample user is mimicking a preset gesture). The target moment is the moment when a sample user makes/completes a specific gesture, which may be in the form of a timestamp. Different gestures completed by the sample user correspond to different target moments. The processing device 410B may record and/or store the sample sensor data collected by the sample strain sensors at each target moment to obtain a set of sample sensor data corresponding to each target moment.)
terminate the recording session in response to a second user input. ([0220] Parameters of the initial pre-training model may be iteratively updated based on the value of the loss function until training ending conditions are satisfied (e.g., the loss function converges, a specific count of iterations has been performed, etc.) to obtain an updated initial pre-training model. The processing device 410A may use the updated initial pre-training model as the corrected first mapping relationship, thereby realizing the correction of the first mapping relationship using the migration learning algorithm)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Yuan with a reasonable expectation of success by determining a correspondence relationship between sensor data and finger joint angles as taught by Yuan ([0002]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Servati (US 20260041354 A1) in view of Yuan (US 20250123685 A1)
Regarding claim 10, Servati teaches A method of collecting training data using a wearable data collection device, the method comprising: ([0001] This invention relates to stretchable textile smart sensor apparel and other wearable devices for tracking one or more body metrics.)
initiating a recording session in response to a first user input received via an activation mechanism on the wearable data collection device; ([0017] The first training dataset can be produced by simultaneously collecting body metric data tracked by an external device and by the at least one body metric sensor on the wearable device, while the wearable device is performing specified movements with known health parameter outputs. The external device can comprise one or more motion capture cameras and the wearable device comprises markers; during training of the core ML program, the one or more motion camera cameras track the markers when the wearable device is performing the specified movements.)
capturing sensor data via a plurality of sensors mounted on the wearable data collection device during the recording session; and ([0034] Embodiments described herein relate to a wearable device comprising one or more stretchable textile sensors (“smart textile wearable device”) for tracking one or more body metrics such as movement, muscle strength, force, temperature, sweat, heart rate, blood pressure, electrocardiogramals, electromyography (EMG) signals, electroencephalography (EEG) signals, and electrodermal activity (EDA) signals [0038] The smart textile glove 100 comprises stretchable inner and outer textile layers 116, 117 sandwiching and embedded with yarn sensors with different sensing modalities, including: stretch sensing yarn sensors 110, force sensing yarn sensors 111 and bio-signal sensing yarn sensors 112 for sensing temperature, sweat and other biological metrics. The yarn sensors 110, 111 112 are located in the glove 110 where the desired body metrics can be measured when the glove is worn, and for example can be located in the proximity of finger, wrist joints, palm of the hand, tips of the fingers or around the wrist when the glove is worn. The yarn sensors 110, 111, 112 are communicatively connected by vine-like stretchy interconnects 113 that electrically connect to each yarn sensor individually and are embedded in the textile layers 116, 117 without making limitations in stretchability, comfort, breathability, washability and fit for the user. The stretchy interconnects 113 communicatively connect the yarn sensors 110, 111, 112 to an embedded integrated circuit 120 in the smart textile glove 100 that serves in processing and/or wirelessly communicating with the data gateway and feedback device 103.)
processing the sensor data via a processing circuit operatively coupled to the plurality of sensors; ([0017] a computing device comprising a processor and a non-transitory computer readable medium having stored thereon a trained core machine learning (ML) program executable by the processor to receive raw body metric data from the at least one body metric sensor, correlate the raw body metric data with a corresponding health parameter output stored in a first training dataset, and display the corresponding health parameter output.)
transmitting the processed sensor data to an external device; and([0062] Alternatively, one or both of the embedded integrated circuit 120 or removable integrated circuit 121 in the smart sensing wearable device 100 includes a processor and a non-transitory computer readable medium having stored thereon the machine learning (ML) program 200 executable by the processor, in which case the multimodal sensor data can be processed directly on the smart textile wearable device 100 and the health parameter outputs can be transmitted to the data gateway and feedback device 103 for display or notification through sound or haptic feedback on the wearable device 100 or other devices. [0069] Finally, both software applications transmit the collected data to a database 590)
Servati does not expressly disclose but Yuan discloses terminating the recording session in response to a second user input received via the activation mechanism on the wearable data collection device. ([0220] Parameters of the initial pre-training model may be iteratively updated based on the value of the loss function until training ending conditions are satisfied (e.g., the loss function converges, a specific count of iterations has been performed, etc.) to obtain an updated initial pre-training model. The processing device 410A may use the updated initial pre-training model as the corrected first mapping relationship, thereby realizing the correction of the first mapping relationship using the migration learning algorithm)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Yuan with a reasonable expectation of success by determining a correspondence relationship between sensor data and finger joint angles as taught by Yuan ([0002]).
Claims 11-15 are rejected under 35 U.S.C. 103 as being unpatentable over Servati (US 20260041354 A1) in view of Yuan (US 20250123685 A1) in further view of Deshpande (US 20160296345 A1) in further view of Hemken (US 20170225329 A1)
Regarding claim 11, Servati teaches The method of claim 10, wherein the wearable data collection device comprises:
a hand element configured to receive a hand of a user; and([0036] This embodiment enables tracking of hands, fingers or body parts even during interaction with different objects [0038] FIG. 1(a), which is a smart textile glove 100 worn on the hand of a user 101 who is holding an object 102)
Servati does not expressly disclose but Deshpande discloses a plurality of finger elements extending from the hand element, wherein the plurality of finger elements are coupled to the hand element by a plurality of joints, and ([0010] the present disclosure may include a hand exoskeleton comprising a plurality of finger exoskeletons, each finger exoskeleton comprising a plurality of joints and a plurality of sensors configured to measure rotation of at least some of the plurality of joints of the finger exoskeleton.)
wherein movement of the plurality of finger elements is constrained within a range of motion. ([0026] Joints 120 may be any juncture of finger exoskeleton at which movement may occur. The movement may be rotation, lateral motion, or any other motion. Joints 120 may be hinge joints configured to provide rotation about a pivot point. [0029] FIG. 1, while the MCP and DIP joints may utilize sliding joints 130 to protect against non-normal forces, the PIP joint may not utilize a sliding joint. This may allow for greater range of motion without requiring an additional motor. For example, by using joints 120 c and 120 d to replicate the motion of the PIP joint, the PIP joint has much greater range of motion without interference from the exoskeleton.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Servati does not expressly disclose but Hemken discloses that corresponds to movement capabilities of a robotic counterpart device ([0005] Using various embodiments, methods, apparatuses, systems and techniques, data for motions and movements, changes in posture, changes in position, vocalizations, gestures, grasping, gait, and other bodily dynamics of living subjects are captured using sensors. Using the resulting sensor data, a supervised learning algorithm or program (AI program) implemented in a machine-learning computer program can be programmed to control an autonomous robot in a substantially similar manner as the living subject.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Hemken with a reasonable expectation of success by collecting data to be used for the purpose of teaching autonomous robots tasks that can be performed by humans as taught by Hemken ([0002]).
Regarding claim 12, Servati does not expressly disclose but Deshpande discloses The method of claim 11, wherein the plurality of finger elements comprises at least three finger elements including a thumb element, an index finger element, and a pinky finger element ([0059] As shown in FIG. 11, each of the digits from the index finger to the pinky finger may utilize a finger exoskeleton similar to finger exoskeleton 100 [0048] FIG. 9 illustrates an example embodiment of a robotic thumb exoskeleton), and wherein the method further comprises maintaining the thumb element in a fixed position relative to the hand element while enabling movement of the index finger element and the pinky finger element relative to the hand element. ([0059] As shown in FIG. 11, each of the digits from the index finger to the pinky finger may utilize a finger exoskeleton similar to finger exoskeleton 100 [0057] some embodiments, medical tape, velcro, or any other suitable material may be used to secure attachment 956 to the metacarpal bone and may prevent movement between thumb exoskeleton 900 and the thumb of the user.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Regarding claim 13, Servati teaches The method of claim 11, wherein capturing the sensor data comprises:
detecting pressure applied to objects using at least one pressure sensor positioned on each of the plurality of finger elements; ([0081] pressure sensing yarn sensor 433 [0029] FIG. 6(e) are photographs and a table of an example implementation showing different grasped objects (photographs) and its confusion matrix (table) based on the training of the output ML model.)
measuring angle data using at least one position sensor at each of the plurality of joints; and ([0068] Other data can be used as ground truth or multimodal monitoring such as tagging by an expert (e.g. doctor, clinician, coach or subjective input from the user or patient) for a particular event, processed video or movement from a model to match, text input from the user, force plate sensors, weight of the loads that the user is using during an exercise, or simulation using inverse kinematics of movements for muscle forces, EMG, ECG, or other modality, global position system or other data source.)
recording visual data using at least one camera mounted on the wearable data collection device. ([0037] the angle tracking can be accurate and in different orientations extracted from IMUs or complemented with computer vision using camera as well as stretch and force sensors and can be for normal flexion/extension or abnormal conditions such as hyper extension, and valus/vargus rotation angles)
Regarding claim 14, Servati teaches The method of claim 11, further comprising:
manipulating objects with the wearable data collection device such that the objects contact a plurality of contact surfaces positioned on the plurality of finger elements; and([0029] FIG. 6(e) are photographs and a table of an example implementation showing different grasped objects (photographs) and its confusion matrix (table) based on the training of the output ML model.)
Servati does not expressly disclose but Yuan discloses deforming at least one of the plurality of contact surfaces comprising a rubber material when contacting an object. ([0176] The embodiments of the present disclosure may measure the deformation of a plurality of degrees of freedom (e.g., three dimensions including the bending around the X-axis, bending around the Z-axis, and stretching along the Y-axis) by a single sensor to realize the detection of posture data of the finger in a plurality of dimensions, which is conducive to the simplification of the preparation process and the miniaturization of the design while improving the detection accuracy. [0179] In some embodiments, the polymeric material includes but is not limited to, silicone, rubber, resin, etc)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Yuan with a reasonable expectation of success by determining a correspondence relationship between sensor data and finger joint angles as taught by Yuan ([0002]).
Regarding claim 15, Servati does not expressly disclose but Deshpande discloses The method of claim 10, further comprising tracking position and orientation of the wearable data collection device in space during the recording session using a secondary device mounted on the wearable data collection device. ([0025] Mounting components 110 may be attached to a user of finger exoskeleton 100 using attachment straps 112. [0032] For example, the orientation and location of the framework coupled to mounting component 110 a may be adjusted based on the user of finger exoskeleton 100.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Servati with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Claims 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Hemken (US 20170225329 A1) in view of Servati (US 20260041354 A1) in further view of Deshpande (US 20160296345 A1)
Regarding claim 16, Hemken teaches A method of training a robotic control model, the method comprising: ([0002] data collection to be used for the purpose of teaching autonomous robots tasks that can be performed by humans.)
wherein the trained neural network model is configured to control a robotic counterpart device having a joint and sensor configuration that matches the wearable data collection device. ([0005] Using various embodiments, methods, apparatuses, systems and techniques, data for motions and movements, changes in posture, changes in position, vocalizations, gestures, grasping, gait, and other bodily dynamics of living subjects are captured using sensors. Using the resulting sensor data, a supervised learning algorithm or program (AI program) implemented in a machine-learning computer program can be programmed to control an autonomous robot in a substantially similar manner as the living subject.)
Hemken does not expressly disclose but Servati discloses receiving sensor data captured during a recording session by a plurality of sensors mounted on a wearable data collection device ([0034] Embodiments described herein relate to a wearable device comprising one or more stretchable textile sensors (“smart textile wearable device”) for tracking one or more body metrics such as movement, muscle strength, force, temperature, sweat, heart rate, blood pressure, electrocardiogramals, electromyography (EMG) signals, electroencephalography (EEG) signals, and electrodermal activity (EDA) signals [0038] The smart textile glove 100 comprises stretchable inner and outer textile layers 116, 117 sandwiching and embedded with yarn sensors with different sensing modalities, including: stretch sensing yarn sensors 110, force sensing yarn sensors 111 and bio-signal sensing yarn sensors 112 for sensing temperature, sweat and other biological metrics. The yarn sensors 110, 111 112 are located in the glove 110 where the desired body metrics can be measured when the glove is worn, and for example can be located in the proximity of finger, wrist joints, palm of the hand, tips of the fingers or around the wrist when the glove is worn. The yarn sensors 110, 111, 112 are communicatively connected by vine-like stretchy interconnects 113 that electrically connect to each yarn sensor individually and are embedded in the textile layers 116, 117 without making limitations in stretchability, comfort, breathability, washability and fit for the user. The stretchy interconnects 113 communicatively connect the yarn sensors 110, 111, 112 to an embedded integrated circuit 120 in the smart textile glove 100 that serves in processing and/or wirelessly communicating with the data gateway and feedback device 103.), wherein the wearable data collection device comprises a hand element configured to receive a hand of a user ([0036] This embodiment enables tracking of hands, fingers or body parts even during interaction with different objects [0038] FIG. 1(a), which is a smart textile glove 100 worn on the hand of a user 101 who is holding an object 102)
processing the sensor data to generate training data for a neural network; and([0065] However, each layer of the core ML program 200 can include different neural networks (NN), long short-term memory (LSTM), convolutional neural network (CNN), 2-layer stacked bi-directional LSTM (Bi-LSTM), fully connected network (FC), hidden Markov model, generative neural network (GAN), or any other machine learning algorithms known to experts in the field. The core ML program 200 output is determined by real-time data as well as a sliding time window (for example 2 sec) of data to accommodate for the known trends in the time-dependent signals, which is implemented as a pre-processing in the input layer 203.)
training the neural network using the training data to generate a trained neural network model ([0065] However, each layer of the core ML program 200 can include different neural networks (NN), long short-term memory (LSTM), convolutional neural network (CNN), 2-layer stacked bi-directional LSTM (Bi-LSTM), fully connected network (FC), hidden Markov model, generative neural network (GAN), or any other machine learning algorithms known to experts in the field. The core ML program 200 output is determined by real-time data as well as a sliding time window (for example 2 sec) of data to accommodate for the known trends in the time-dependent signals, which is implemented as a pre-processing in the input layer 203.),
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Servati with a reasonable expectation of success by tracking one or more body metrics using a wearable device as taught by Servati ([0001]).
Hemken does not expressly disclose but Deshpande discloses and a plurality of finger elements extending from the hand element; ([0010] the present disclosure may include a hand exoskeleton comprising a plurality of finger exoskeletons)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Regarding claim 17, Hemken teaches The method of claim 16, further comprising controlling the robotic counterpart device with the trained neural network model, wherein controlling the robotic counterpart device with the trained neural network model comprises:
receiving real-time sensor data from multiple sensors on the robotic counterpart device; ([0057] receives as input a full set of sensor apparatus data 603)
processing the real-time sensor data using the trained neural network model to determine control signals; and ([0057] In FIG. 6B, the robotic device's machine-learning program 604)
transmitting the control signals to the robotic counterpart device to control movement of the robotic counterpart device. ([0057] It outputs the set of a signals 605 needed to actuate all of the number a of the robot's motion generating signals and communications outputs during the next single time interval, time interval n+1, t(n+1).)
Regarding claim 18, Hemken does not expressly disclose but Servati discloses The method of claim 16, wherein the sensor data comprises:
pressure data from at least one pressure sensor positioned on each of the plurality of finger elements; ([0081] pressure sensing yarn sensor 433)
angle data from at least one position sensor at each of a plurality of joints that couple the plurality of finger elements to the hand element; and([0068] Other data can be used as ground truth or multimodal monitoring such as tagging by an expert (e.g. doctor, clinician, coach or subjective input from the user or patient) for a particular event, processed video or movement from a model to match, text input from the user, force plate sensors, weight of the loads that the user is using during an exercise, or simulation using inverse kinematics of movements for muscle forces, EMG, ECG, or other modality, global position system or other data source.)
visual data from at least one camera mounted on the wearable data collection device. ([0037] the angle tracking can be accurate and in different orientations extracted from IMUs or complemented with computer vision using camera as well as stretch and force sensors and can be for normal flexion/extension or abnormal conditions such as hyper extension, and valus/vargus rotation angles)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Servati with a reasonable expectation of success by tracking one or more body metrics using a wearable device as taught by Servati ([0001]).
Regarding claim 19, Hemken does not expressly disclose but Deshpande discloses The method of claim 16, further comprising:
receiving position and orientation data of the wearable data collection device captured during the recording session by a positioning device mounted on the wearable data collection device; and([0025] Mounting components 110 may be attached to a user of finger exoskeleton 100 using attachment straps 112. [0032] For example, the orientation and location of the framework coupled to mounting component 110 a may be adjusted based on the user of finger exoskeleton 100.)
incorporating the position and orientation data into the training data for the neural network. ([0040] Computing device 720 may include any instrumentality or aggregate of instrumentalities operable to compute, calculate, determine, classify, process, transmit, receive, retrieve, originate, switch, store, display, communicate, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, computing device 720 may be a personal computer (e.g., desktop or laptop), tablet computer, mobile device (e.g., personal digital assistant (PDA) or smart phone), server (e.g., blade server or rack server), a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. Computing device 720 may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of computing device 720 may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, touchscreen, and/or a video display. Computing device 720 may also include one or more buses operable to transmit communication between the various hardware components.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Deshpande with a reasonable expectation of success by including robotic finger exoskeleton with a plurality of joints and a plurality of sensors to measure rotational position of the joints as taught by Deshpande ([0022]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Hemken (US 20170225329 A1) in view of Servati (US 20260041354 A1) in further view of Deshpande (US 20160296345 A1) in further view of Buckley (US 5673367 A)
Regarding claim 20, Hemken does not expressly disclose but Servati discloses The method of claim 16, further comprising:
receiving additional sensor data from multiple recording sessions from the wearable data collection device, wherein the multiple recording sessions comprise recordings of different tasks performed with the wearable data collection device; ([0066] Using the motion capture camera system to track the markers on the smart textile glove 100 making random movements, performing controlled specific tasks or gestures, or interacting with objects, a training dataset is produced of certain pose parameter outputs corresponding to the tracked hand movements, such as wrist, finger and forearm joint angles and positions.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Servati with a reasonable expectation of success by tracking one or more body metrics using a wearable device as taught by Servati ([0001]).
Hemken does not expressly disclose but Buckley discloses analyzing the additional sensor data to identify one or more patterns; and (Col 8 Line 3-7 Data at each time interval, Δt, constitutes a specific pattern. The resultant set of input (proximity and force in this case) and output (prediction of next angle of movement) patterns over all the time increments, Δt, constitutes a training set.)
refining the trained neural network model based on the one or more patterns to improve performance of the robotic counterpart device. (Col 8 Line 7-12 This process can be designed to include other objects in the training set. Using this training set, the network is then taught to map this cause and effect relationship between force and proximity feedback and the next desired change in joint angle for each finger. Hence, an animatable sequence is created and motion is learned.)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention to modify Hemken with the teachings of Buckley with a reasonable expectation of success by controlling the grasping function of a robotic hand for neural network control of robotic motion as taught by Buckley (Col 1 Line 19-22).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SARAH TRAN whose telephone number is (313)446-6642. The examiner can normally be reached 8am-5pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached at (571) 272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.A.T./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights