DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Claim rejection under 35 USC 101 is withdrawn in view of amendments. Applicant's arguments filed on 06/22/2026 have been fully considered but they are not persuasive or moot in view of new ground of rejection provided below which was necessitated based on Applicant’s amendments to the claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8, 11, 12, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle).
Regarding Claim 1, Chao teaches a system for human-cobot (collaborative robot) ergonomic interaction, comprising:
a communication interface operable to receive sensor data related to human motion (See at
least Fig. 3A shows communication interface operable to receive sensor data related to human motion, Para [0042] “FIG. 3A illustrates an example system 300 that can be used to model realistic motion of a human hand for a human-robot interaction that can be used in accordance with various embodiments. It should be understood that such interactions can be performed with both hands or other portions of a human body, and are also not limited to (inter)actions such as object handovers which are presented as a primary example herein. In this example system 300, one or more cameras 302 or sensors can be positioned with respect to a physical environment 304 that includes, or will include, a human hand 306 and at least one physical object 308 with which that hand is to interact, such as to grasp and move the object. The camera(s) 302 can capture image data 310 (e.g., a sequence of images or video) representative of the state of the physical environment, including position, orientation, and pose information for the hand and object in the physical environment. A camera 302 used for such purposes can include a camera for capturing two-dimensional images, a camera for capturing stereoscopic images, or a camera for capturing images that include color and depth information (e.g., RGB-D images). Other sensors or devices may be used to capture a representation of the environment as well, as may include depth sensors, ultrasonic sensors, LIDAR scanners, and the like.”);
…
human intent prediction processor circuitry operable to interpret the sensor data to predict an
object the human intends to grasp (See at least Fig 4 item 406, 408, Para [0048] “…A determined number of instances, such as around 1,000, may be performed and captured, as may include at least a minimum number of different types of objects and different handover or target locations, poses, and orientations. At least a subset of this captured image data can be analyzed 406 to identify at least a position and pose of the hand and the object in individual images or video frames. This can include identifying a location or coordinates, such as a bounding box, or determining a segmentation of these images that identify segments of the image corresponding to the hand and object. In other embodiments, image features may be extracted and identified for the hand and object and then encoded into a latent space for modeling and motion/behavior prediction, among other such options…”), and to proactively select a destination container for the predicted object (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”); and
cobot motion processor circuitry operable to determine a position or orientation for the cobot
to place the selected destination container based on the predicted object and the strain scores (See at least Fig 5 item 504, 512, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”),
However, Chao does not explicitly spell out …
ergonomic assessment processor circuitry operable to continuously evaluate the sensor data in real-time during active human-cobot interaction to generate individual strain scores for plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint based on an integration of motion of the respective human joint over a period of time;…
wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints.
Baek teaches …
to generate individual strain scores for plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint based on an integration of motion of the respective human joint over a period of time (See at least Para [0017] “According to one embodiment, a method of evaluating workplace worker injury risks includes videotaping a worker who is not wearing any motion sensors, and who is engaged in routine repetitive movements, to provide recorded videos as input data. The recorded videos are analyzed to resolve multiple joints of the worker. The recorded videos are analyzed for measurable kinematic variables related to each joint. The measurable kinematic variables are analyzed to provide job risk assessment reports as output. The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”);…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Baek and include the feature of generating individual strain scores for plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint based on an integration of motion of the respective human joint over a period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Buerkle teaches … ergonomic assessment processor circuitry operable to continuously evaluate the sensor data in real-time during active human-cobot interaction (See at least Page 521 Col 2 Para 2 “…In the case of human sensor data, the main idea is to stream raw data into the classifier…”, Page 523 Col 1 Para 3 “Similar to a guitar, where a cord consists of different notes, EMGsensors provide muscle activity data in a complex signal within a time domain. The signal consists of different frequen cies and associated intensities. Consequently, it is necessary to perform a wave transform to retrieve relevant features (under lying frequencies and their intensity), which can be mapped to payloads and muscular fatigue.”, Page 522 Col 2 Para 1 “…it allows for distinguishing different levels of payload from EMG signals over time…”)…
wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints (See at least Page 523 Col 1 Para 3 “Similar to a guitar, where a cord consists of different notes, EMGsensors provide muscle activity data in a complex signal within a time domain. The signal consists of different frequen cies and associated intensities. Consequently, it is necessary to perform a wave transform to retrieve relevant features (under lying frequencies and their intensity), which can be mapped to payloads and muscular fatigue.”, Page 526 Col 1 Para 2 “…In the following, the Mondrian Forest will be applied and optimized to predict the different payloads and the fatigue threshold from the collected EMGdata.”, Page 524 Col 1 Para 5 “…Moreover, instead of participants pressing the assistance request manually, the Mondrian Forest is envisioned to identify the fatigue threshold and trigger a robotic assistance request automatically afterward.”, Page 524 Col 2 Para 2 “The gripper is holding a thin metal plate, which was fitted with a Styrofoam edging to avoid scratching participants. As soon as the assistance button is pressed, the UR10 would slowly move upward in a straight line, until it touched the engine cover. Afterward, it would move an additional 7 cm upward at the 10kg payload setting to assist the human operator in lifting the weight. During this assistance operation, the force/torque readings of the gripper would allow for quantifying the amount of weight that the UR10 is holding.”, discloses robot changing movement to assist human to lift the weight which is construed as cobot adjusting its position to assist human and prevent exacerbation of the strain level).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chao with the teachings of Buerkle and include the feature of continuously evaluate the sensor data in real-time during active human-cobot interaction and upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints, thereby improving collaborative human robot performance (See at least Page 522 Col 2 Para 4 “Thus, this would allow for the use of an incremental learner to continuously adapt to a human operator while improving its performance over time…”, Page 526 Col 1 Para 2 “…Thus, providing a collaborative robot with cognitive skills regarding its human partner will improve teamwork by compensating each other’s weaknesses…”).
Regarding Claim 8, modified Chao teaches all the elements of claim 1.
However, Chao does not explicitly spell out the system of claim 1, wherein the ergonomic
assessment processor circuitry is further operable to update the strain score after the cobot has placed the selected destination container.
Baek teaches the system of claim 1, wherein the ergonomic assessment processor circuitry is
further operable to update the strain score after the cobot has placed the selected destination container (See at least Para [0099 “Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the system of Chao with the teachings of Baek
and include the feature of the wherein the ergonomic assessment processor circuitry being operable to update the strain score after the cobot has placed the selected destination container thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 11, modified Chao teaches all the elements of claim 1. Chao further teaches the system of claim 1, wherein the cobot motion processor circuitry is further operable to use a graph-based algorithm to determine the position or orientation for the cobot to place the selected destination container (See at least Para [0056] “… In at least one embodiment, training logic 715 may include, or be coupled to code and/or data storage 701 to store graph code or other software to control timing and/or order, in which weight and/or other parameter information is to be loaded to configure, logic, including integer and/or floating point units (collectively, arithmetic logic units (ALUs)…”).
Regarding Claim 12, modified Chao teaches all the elements of claim 1. Chao further teaches the system of claim 11, wherein the cobot motion processor circuitry is further operable to determine the position or orientation for the cobot to place the selected destination container while taking into account an operating range of the cobot, a speed of the cobot, or a potential environmental obstacle (See at least Para [0047] “… In many instances this will result in the end effector being moved toward the target grasp position and orientation, but due to factors such as motion of the hand or object may involve moving away from the object to avoid a collision or obstruction relative to the hand or object…”).
Regarding Claim 13, Chao teaches a component of a system for human-cobot (collaborative
robot) ergonomic interaction, comprising:
processor circuitry (See at least Para [0057] “In at least one embodiment, any portion of code and/or data storage 701 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and/or code and/or data storage 701 may be cache memory, dynamic randomly addressable memory (“DRAM”), static randomly addressable memory (“SRAM”), non-volatile memory (e.g., Flash memory), or other storage. In at least one embodiment, choice of whether code and/or code and/or data storage 701 is internal or external to a processor, for example, or comprised of DRAM, SRAM, Flash or some other storage type may depend on available storage on-chip versus off-chip, latency requirements of training and/or inferencing functions being performed, batch size of data used in inferencing and/or training of a neural network, or some combination of these factors.”); and
a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry (See at least Para [0221] “…In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein…”), cause the processor circuitry to:
receive sensor data related to human motion (See at least Fig. 3A shows communication
interface operable to receive sensor data related to human motion, Para [0042] “FIG. 3A illustrates an example system 300 that can be used to model realistic motion of a human hand for a human-robot interaction that can be used in accordance with various embodiments. It should be understood that such interactions can be performed with both hands or other portions of a human body, and are also not limited to (inter)actions such as object handovers which are presented as a primary example herein. In this example system 300, one or more cameras 302 or sensors can be positioned with respect to a physical environment 304 that includes, or will include, a human hand 306 and at least one physical object 308 with which that hand is to interact, such as to grasp and move the object. The camera(s) 302 can capture image data 310 (e.g., a sequence of images or video) representative of the state of the physical environment, including position, orientation, and pose information for the hand and object in the physical environment. A camera 302 used for such purposes can include a camera for capturing two-dimensional images, a camera for capturing stereoscopic images, or a camera for capturing images that include color and depth information (e.g., RGB-D images). Other sensors or devices may be used to capture a representation of the environment as well, as may include depth sensors, ultrasonic sensors, LIDAR scanners, and the like.”);
…
interpret the sensor data to predict an object the human intends to grasp (See at least
Fig 4 item 406, 408, Para [0048] “…A determined number of instances, such as around 1,000, may be performed and captured, as may include at least a minimum number of different types of objects and different handover or target locations, poses, and orientations. At least a subset of this captured image data can be analyzed 406 to identify at least a position and pose of the hand and the object in individual images or video frames. This can include identifying a location or coordinates, such as a bounding box, or determining a segmentation of these images that identify segments of the image corresponding to the hand and object. In other embodiments, image features may be extracted and identified for the hand and object and then encoded into a latent space for modeling and motion/behavior prediction, among other such options…”), and to proactively select a destination container for the predicted object (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”); and
determine a position or orientation for the cobot to place the selected destination
container based on the predicted object and the strain scores (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”),
However, Chao does not explicitly spell out …
Continuously evaluate the sensor data in real-time during active human-cobot
interaction to generate individual strain scores for plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint based on an integration of motion of the respective human joint over a period of time; …
wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints.
Baek teaches …
to generate individual strain scores for plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint based on an integration of motion of the respective human joint over a period of time (See at least Para [0017] “According to one embodiment, a method of evaluating workplace worker injury risks includes videotaping a worker who is not wearing any motion sensors, and who is engaged in routine repetitive movements, to provide recorded videos as input data. The recorded videos are analyzed to resolve multiple joints of the worker. The recorded videos are analyzed for measurable kinematic variables related to each joint. The measurable kinematic variables are analyzed to provide job risk assessment reports as output. The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the system of Chao with the teachings of Baek and include the feature of evaluating the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Buerkle teaches … Continuously evaluate the sensor data in real-time during active
human-cobot interaction (See at least Page 521 Col 2 Para 2 “…In the case of human sensor data, the main idea is to stream raw data into the classifier…”, Page 523 Col 1 Para 3 “Similar to a guitar, where a cord consists of different notes, EMGsensors provide muscle activity data in a complex signal within a time domain. The signal consists of different frequen cies and associated intensities. Consequently, it is necessary to perform a wave transform to retrieve relevant features (under lying frequencies and their intensity), which can be mapped to payloads and muscular fatigue.”, Page 522 Col 2 Para 1 “…it allows for distinguishing different levels of payload from EMG signals over time…”)…
wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints (See at least Page 523 Col 1 Para 3 “Similar to a guitar, where a cord consists of different notes, EMGsensors provide muscle activity data in a complex signal within a time domain. The signal consists of different frequen cies and associated intensities. Consequently, it is necessary to perform a wave transform to retrieve relevant features (under lying frequencies and their intensity), which can be mapped to payloads and muscular fatigue.”, Page 526 Col 1 Para 2 “…In the following, the Mondrian Forest will be applied and optimized to predict the different payloads and the fatigue threshold from the collected EMGdata.”, Page 524 Col 1 Para 5 “…Moreover, instead of participants pressing the assistance request manually, the Mondrian Forest is envisioned to identify the fatigue threshold and trigger a robotic assistance request automatically afterward.”, Page 524 Col 2 Para 2 “The gripper is holding a thin metal plate, which was fitted with a Styrofoam edging to avoid scratching participants. As soon as the assistance button is pressed, the UR10 would slowly move upward in a straight line, until it touched the engine cover. Afterward, it would move an additional 7 cm upward at the 10kg payload setting to assist the human operator in lifting the weight. During this assistance operation, the force/torque readings of the gripper would allow for quantifying the amount of weight that the UR10 is holding.”, discloses robot changing movement to assist human to lift the weight which is construed as cobot adjusting its position to assist human and prevent exacerbation of the strain level).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chao with the teachings of Buerkle and include the feature of continuously evaluate the sensor data in real-time during active human-cobot interaction and upon determining that any of the individual strain scores exceeds a predetermined threshold, the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container to prevent exacerbation of the strain level of any of the plurality of human joints, thereby improving collaborative human robot performance (See at least Page 522 Col 2 Para 4 “Thus, this would allow for the use of an incremental learner to continuously adapt to a human operator while improving its performance over time…”, Page 526 Col 1 Para 2 “…Thus, providing a collaborative robot with cognitive skills regarding its human partner will improve teamwork by compensating each other’s weaknesses…”).
Regarding Claim 20, modified Chao teaches all the elements of claim 13.
However, Chao does not explicitly spell out the component of claim 13, wherein the instructions further cause the processor circuitry to:
update the strain score after the cobot has placed the selected destination container.
Baek teaches the the component of claim 13, wherein the instructions further cause the processor circuitry to:
update the strain score after the cobot has placed the selected destination container (See at least Para [0099 “Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the system of Chao with the teachings of Baek
and include the feature of the ergonomic assessment processor circuitry being operable to update the strain score after the cobot has placed the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Claim(s) 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle), and further in view of Batzianoulis et al. (Batzianoulis I, Krausz NE, Simon AM, Hargrove L, Billard A. Decoding the grasping intention from electromyography during reaching motions. J Neuroeng Rehabil. 2018 Jun 26;15(1):57. doi: 10.1186/s12984-018-0396-5. PMID: 29940991; PMCID: PMC6020187). (Hereinafter Batzianoulis).
Regarding Claim 5, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time and an average deviation of the motion of the respective human joint from its natural rest position.
Baek teaches wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time (See at least Para [0011] “…C examples include use of the Lumbar Motion Monitor to measure kinematics of the lumbar spine, electrogoniometers to measure angular displacement of certain joints (e.g., most commonly the wrist, but also the knee, shoulder, and elbow)…”, Para [0017] “…The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0073] “We combine these three matrices using a weighted sum”, Para [0074] “The Hungarian algorithm uses the weighted sum to determine the assignment between Kalman filter tracker bounding boxes and CNN-detected bounding boxes. After the assignment is completed, we select only admissible assignments, by thresholding each of the similarity measures...”, Para [0099] “…Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record…”, Para [0069] “After processing the entire video frames with the Kalman filter trackers, we compare the similarity among trackers using the appearance model. The dissimilarity between the trackers is defined as the weighted sum of the Euclidean distance between PCA means and the cosine distance between the principal components.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Baek and include the feature of the strain score being generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Batzianoulis teaches … and an average deviation of the motion of the respective human joint from its natural rest position (See at least Page 7 “Fig 4 : The confusion matrices present the average classification accuracies and their standard deviations for the five grasp types…” ).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Batzianoulis and include the feature of and an average deviation of the motion of the respective human joint from its natural rest position, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 17, modified Chao teaches all the elements of claim 13.
Chao does not explicitly spell out wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time and an average deviation of the motion of the respective human joint from its natural rest position.
Baek teaches wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time and an average deviation of the motion of the respective human joint from its natural rest position (See at least Para [0011] “…C examples include use of the Lumbar Motion Monitor to measure kinematics of the lumbar spine, electrogoniometers to measure angular displacement of certain joints (e.g., most commonly the wrist, but also the knee, shoulder, and elbow)…”, Para [0017] “…The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0073] “We combine these three matrices using a weighted sum”, Para [0074] “The Hungarian algorithm uses the weighted sum to determine the assignment between Kalman filter tracker bounding boxes and CNN-detected bounding boxes. After the assignment is completed, we select only admissible assignments, by thresholding each of the similarity measures...”, Para [0099] “…Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record…”, Para [0069] “After processing the entire video frames with the Kalman filter trackers, we compare the similarity among trackers using the appearance model. The dissimilarity between the trackers is defined as the weighted sum of the Euclidean distance between PCA means and the cosine distance between the principal components.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Baek and include the feature of the strain score being generated based on a weighted sum of a cumulative angular displacement during the motion of a respective one of the plurality of human joints over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Batzianoulis teaches … and an average deviation of the motion of the respective human joint from its natural rest position (See at least Page 7 “Fig 4 : The confusion matrices present the average classification accuracies and their standard deviations for the five grasp types…” ).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Batzianoulis and include the feature of an average deviation of the motion of the respective human joint from its natural rest position, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim(s) 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle), and further in view of Schmid et al. (WO2024132714A1) (Hereinafter Schmid).
Regarding Claim 6, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the strain score is further based on an exponential decay of the strain score.
Schmid teaches the system of claim 1, wherein the strain score is further based on an exponential decay of the strain score (See at least Page 4 Para 11 “In an embodiment, the weight of a given heat strain score, when calculating the acute heat load, is subject to exponential decay with a time constant of 2 - 10 days.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Schmid and include the feature of the strain score being based on an exponential decay of the strain score, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 18, modified Chao teaches all the elements of claim 13.
Chao does not explicitly spell out the component of claim 13, wherein the strain score is further based on an exponential decay of the strain score.
Schmid teaches the component of claim 13, wherein the strain score is further based on an exponential decay of the strain score (See at least Page 4 Para 11 “In an embodiment, the weight of a given heat strain score, when calculating the acute heat load, is subject to exponential decay with a time constant of 2 - 10 days.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Schmid and include the feature of the strain score being based on an exponential decay of the strain score, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim(s) 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle), and further in view of Gomez Gutierrez et al (US 20240217103 A1) (Hereinafter Gomez Gutierrez).
Regarding Claim 7, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the cobot motion processor
circuitry is further operable to simulate inverse kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container.
Baek teaches teaches the system of claim 1, wherein the cobot motion processor circuitry is further operable to simulate … kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container (See at least Fig 1 item 10 – Kinematic Activities, Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the cobot motion processor circuitry being operable to simulate kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
However, neither Chao nor Baek explicitly spell out kinematics modeling being inverse.
Gomez Gutierrez teaches inverse kinematic modeling (See at least Para [0066] “…For 6D poses, inverse kinematics techniques may be used to compute the corresponding joint configurations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Gomez Gutierrez and include the feature of the inverse kinematic modeling, thereby provide simplified automatic calculation of necessary joint movements for precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 19, modified Chao teaches all the elements of claim 7.
Chao does not explicitly spell out the component of claim 13, wherein the instructions further
cause the processor circuitry to:
simulate inverse kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container.
Baek teaches the component of claim 13, wherein the instructions further cause
the processor circuitry to:
simulate … kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container (See at least Fig 1 item 10 – Kinematic Activities, Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the cobot motion processor circuitry being operable to simulate kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
However, neither Chao nor Baek explicitly spell out kinematics modeling being inverse.
Gomez Gutierrez teaches inverse kinematic modeling (See at least Para [0066] “…For 6D poses, inverse kinematics techniques may be used to compute the corresponding joint configurations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Gomez Gutierrez and include the feature of the inverse kinematic modeling, thereby provide simplified automatic calculation of necessary joint movements for precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle), and further in view of (Wagner et al.) (US 20170136632 A1) (Hereinafter Wagner).
Regarding Claim 9, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the human intent prediction processor circuitry is further operable to select the destination container for the predicted object from a plurality of candidate destination containers.
Wagner teaches the system of claim 1, wherein the human intent prediction processor circuitry is further operable to select the destination container for the predicted object from a plurality of candidate destination containers (See at least Para [0005] “… The sortation system includes a programmable motion device including an end effector, a perception system for recognizing any of the identity, location, or orientation of an object presented in a plurality of objects, a grasp selection system for selecting a grasp location on the object, the grasp location being chosen to provide a secure grasp of the object by the end effector to permit the object to be moved from the plurality of objects to one of a plurality of destination locations, and a motion planning system for providing a motion path for the transport of the object when grasped by the end effector from the plurality of objects to the one of the plurality of destination locations, wherein the motion path is chosen to provide a path from the plurality of objects to the one of the plurality of destination locations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Wagner and include the feature of the human intent prediction processor circuitry being operable to select the destination container for the predicted object from a plurality of candidate destination containers, thereby provide efficiency, robustness and safety while moving items from one place to another (See at least Para [0066] “Another advantage of the varied set is the ability to address several customer metrics without having to re-plan motions. The database is sorted and indexed by customer metrics like time, robustness, safety, distance to obstacles etc. and given a new customer metric, all the database needs to do is to reevaluate the metric on the existing trajectories, thereby resorting the list of trajectories, and automatically producing the best trajectory that satisfies the new customer metric without having to re-plan motions.”).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), Buerkle et al. (A. Buerkle et al., "An Incremental Learning Approach to Detect Muscular Fatigue in Human– Robot Collaboration," in IEEE Transactions on Human-Machine Systems, vol. 53, no. 3, pp. 520-528, June 2023) (Hereinafter Buerkle), and further in view of Matijevich et al. (US 20210236020 A1) (Hereinafter Matijevich).
Regarding Claim 10, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out wherein the cobot motion processor circuitry is further operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of any of the plurality of human joints.
Matijevich teaches wherein the cobot motion processor circuitry is further operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of any of the plurality of human joints (See at least Para [0023] “In one embodiment, the processing unit is further configured to alert the user when the musculoskeletal loading and/or microdamage accumulation is greater than a threshold that has been predefined or a threshold that has been calibrated for the specific user.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the system of Chao with the teachings of Matijevich and include the feature of the cobot motion processor circuitry being operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of the plurality of human joints, thereby provide notification in case of an emergency (See at least Para [0265] “…The investigators also have considerable experience in developing custom embedded systems for wearable exoskeletons that provide compact, energy efficient, self-contained operation.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHEDA HOQUE whose telephone number is (571)270-5310. The examiner can normally be reached Monday-Friday 8:00 am- 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at 571-270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHEDA HOQUE/Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights