DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed on have been fully considered but they are not persuasive.
The Applicant argues on page 7 of the Applicant’s remarks that “The Examiner incorrectly characterized sensor data reception as "insignificant extra- solution activity" and "mere data gathering." This is in error for several reasons. First, the sensor data reception is integral to the solution, not peripheral to it. The system continuously processes sensor data to generate strain scores that directly control robot motion. Data gathering is only insignificant if it is a well-understood, routine, conventional activity previously known to the industry. The Examiner has provided no evidence that sensor fusion for cumulative ergonomic strain assessment in robot control is conventional in the field of collaborative robotics. Second, this is not generic data gathering but specialized sensing for a specific technological purpose; the claims require "sensor data related to human motion" that is processed to generate strain scores representing cumulative joint loading, which then directly influences cobot positioning. Third, the Examiner's cited cases involved generic data collection over networks for routine purposes such as email virus scanning, phone image transmission, and price aggregation. Those cases addressed situations where conventional computers performed conventional data collection and transmission functions. Here, the system performs specialized processing (temporal integration of joint motion to calculate cumulative strain) and uses the results for safety-critical robot control, a fundamentally different application.”. Also, the Examiner argues on page 8 of the Applicant’s remarks that “The Examiner's finding that processor circuitry and computer-readable storage medium are well-understood, routine, and conventional fails to account for the specific processing operations and specialized configuration required by the claims. The claimed processing performs temporal integration of joint motion over time to generate cumulative strain scores. This is not conventional processing in collaborative robotics, where standard systems use instantaneous sensor readings and threshold-based reactive responses rather than temporal integration of cumulative physiological strain.”. The Examiner respectfully disagrees. The use of sensor fusion in the field of collaborative robotics for cumulative ergonomic strain assessment is indeed conventional. It involves the integration of various sensor technologies to provide a comprehensive view of an operator’s posture and movements, which is crucial for assessing ergonomic strain. This approach is widely accepted and utilized in the industry to ensure the safety and well-being of operators during collaborative tasks with robots. As the Examiner pointed out before, by incorporating active control step in the claim language may overcome the 35 USC § 101 rejection since it will indicate integrating the abstract idea into a practical application.
Also, the Applicant argues on page 11 of the Applicant’s remarks that “The Office Action cites Baek as teaching the limitation of "generating a strain score based on an integration of motion of the at least one human joint over a period of time." However, Baek does not teach or suggest this requirement. Baek merely detects kinematic variables and applies existing ergonomic assessment tools, explaining that "the computing device 16... make[s] assessments of ergonomic metrics... to create a risk assessment... [which] may be a score, a risk level, or similar report." (Baek 1[0042]).”. The Examiner respectfully disagrees. Baek teaches detecting kinematic variables which includes joint position and angles (See at least Para [0015] “… Using this system which does not rely upon wearable sensors (including passive sensors such as visual markers or reflectors), occupational safety engineers in manufacturing plants are able to assess in real time the kinematic aspects of the workers (joint positions and angles for multiple joints simultaneously), and the impact of various physical activities (posture, repetition, force, pull, reach . . . ), to determine the risks of injuries from repetitive motion to shoulder, elbow, wrist, and hand, and to reduce and possibly prevent work-related injuries from happening…”, Para [0017] “According to one embodiment, a method of evaluating workplace worker injury risks includes videotaping a worker who is not wearing any motion sensors, and who is engaged in routine repetitive movements, to provide recorded videos as input data. The recorded videos are analyzed to resolve multiple joints of the worker. The recorded videos are analyzed for measurable kinematic variables related to each joint. The measurable kinematic variables are analyzed to provide job risk assessment reports as output. The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns. Additional workers may be monitored simultaneously but reported on separately. The method may include an ergonomic assessment as well as an assessment of kinematic variables. The method may output data assessment reports with health and risk recommendations.”). Also, as applicant already mentioned that Baek discloses creating risk assessment which could be a score of workers (See at least Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0111] “In summary the system and method are designed to obtain data associated with the repetitive work, the information comprising previous health and medical history of the workers, health-related habits of the workers, type and role of the workers at work, times of events of work-related musculoskeletal injuries, environmental conditions such as temperature of the workplace, motion capture of the workers at work. It generates a statistical model for the data associated with the repetitive work; evaluating individuals using quantitative scores; and reporting the injury risk scores to the employer.”). Also, Baek teaches generating a strain score based on an integration of motion of the at least one human joint over a period of time. Baek teaches about time series posture data to generate the risk score (See at least Para [0013] The method described herein represents a substantial improvement to current biomechanical exposure assessment methods. Specifically, the method offers the same benefits of measurement (i.e., accurate and precise time series posture data) without the need to attach sensors to workers and while retaining visual information about the context of the work being assessed that is valued by practitioners for interpretation….”, Para [0015] “It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive. A system and method are provided for a Prevention and Safety Management (PSM) system and method for automated analysis of ergonomics for workers in the manufacturing industry using computer vision and deep machine learning. Using this system which does not rely upon wearable sensors (including passive sensors such as visual markers or reflectors), occupational safety engineers in manufacturing plants are able to assess in real time the kinematic aspects of the workers (joint positions and angles for multiple joints simultaneously), and the impact of various physical activities (posture, repetition, force, pull, reach . . . ), to determine the risks of injuries from repetitive motion to shoulder, elbow, wrist, and hand, and to reduce and possibly prevent work-related injuries from happening…”, Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”). Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
The Applicant also argues on page 12 of the Applicant’s remarks that “In contrast, the claimed subject matter requires determining the cobot's position or orientation based on both (i) a predicted object and (ii) a strain score, where the strain score reflects ergonomic load derived from an integration of human joint motion. Chao contains no disclosure of strain scores, ergonomic assessment, musculoskeletal risk, physiological metrics, or any use of cumulative human strain to influence robot motion planning. Chao's system is concerned exclusively with interaction efficiency and handover feasibility, not ergonomic safety. Nothing in Chao suggests using ergonomic strain, let alone a calculated strain score, to modify, weight, or determine robot positioning. Accordingly, Chao cannot satisfy the claim's requirement of determining cobot positioning "based on the predicted object and the strain score."”. The Examiner respectfully disagrees. Chao depends on Baek in the rejection for the teaching of strain scores, ergonomic assessment, musculoskeletal risk, physiological metrics, or any use of cumulative human strain (See at least Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”). And, the combination of Chao and Baek teaches the strain scores, ergonomic assessment, musculoskeletal risk, physiological metrics, or any use of cumulative human strain.
Moreover, the Applicant argues on Page 12 and 13 of the Applicant’s remarks that “Even if the Examiner were correct in asserting that Baek teaches a general ergonomic "risk assessment," and even if Baek were combined with Chao's robot-positioning system, the combination still would not teach or suggest determining a cobot position or orientation "based on the predicted object and the strain score" as explicitly required by the claim. The Office Action12 does not explain, and the cited references do not disclose, how Chao's simulation-based, predicted- behavior handover optimization could be modified to incorporate Baek's ergonomic metrics so as to generate a real-time strain score representing cumulative joint loading over time, let alone use that strain score to determine cobot positioning. This required integration of object prediction and time-integrated strain scoring represents a novel technical coupling that neither reference teaches individually, and nothing in the art suggests combining them in this manner.”. The Examiner respectfully disagrees. Chao already teaches object prediction (See at least Para [0043] “… Motion models for the hand and the object can be updated for individual motions determined for individual sequences to generate at least one motion model for a human hand, and may also generate at least one model for an object held in a human hand, to provide for predictions of realistic motion and behavior of a hand and object during an interaction, such as an object handover.”). Chao depends on Baek in the rejection for the teachings of time-integrated strain scoring (See at least Para [0013] “The method described herein represents a substantial improvement to current biomechanical exposure assessment methods. Specifically, the method offers the same benefits of measurement (i.e., accurate and precise time series posture data) without the need to attach sensors to workers and while retaining visual information about the context of the work being assessed that is valued by practitioners for interpretation.”, Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”. Therefore, combining Chao with the teachings of Baek should be obvious to one of the ordinary skill in the art.
Furthermore, the Applicant argues on page 13 of the Applicant’s remarks that “Furthermore, Baek and Chao address fundamentally different technical problems in incompatible domains. Chao concerns simulating human-robot handovers to train robot control models in a virtual environment, explaining that "motion capture can be the Applicant teaches on Page 13 of the Applicant’s remarks that performed... without any need for an actual robot to be present or an actual handover to be performed." (Chao 1[0030]). Chao's system is explicitly designed for offline simulation and training, not live human-robot collaboration, and contains no ergonomic or physiological considerations. In contrast, Baek is directed to post-hoc workplace safety documentation, describing "a Prevention and Safety Management (PSM) system ... for automated analysis of ergonomics for workers in the manufacturing industry" (Baek [0015]). Baek performs passive monitoring after the fact and does not control robots or operate in real time. The Examiner respectfully disagrees. Chao also performs in a physical environment (See at least [0050] “FIG. 5 illustrates an example process 500 for controlling a robot to perform a handover action based, at least in part, upon predicted motion or behavior of a human hand as modeled and accounted for in training of a control algorithm, such as is described in the process 400 of FIG. 4 . In this example process, a handover action to be performed by a robot, or other automated or semi-automated assembly, can be determined 502, where this handover action can involve the robot taking an object currently, or to be, held by a human hand. A current state of the physical environment, including the robot, hand, and object can be determined based at least on image or sensor data captured of the physical environment, as well as orientation or motion data provided by the robot. Based at least in part upon this current state, as well as modeled human motion or behavior determined from simulations and accounted for in a robot control model, as discussed with respect to FIG. 4 , an optimal location and orientation for the robot to perform the handover action can be determined 504 . For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover. At least a first motion, or subset of motions, of this sequence can be performed 508, and a determination made 510 as to whether an end effector, or other portion of this robot, has arrived at this target location and orientation to perform the handover. If not, this process can continue (unless another end criterion is satisfied as discussed herein, such as human-robot contact or the object being dropped or damaged. If the end effector of the robot is determined to have successfully arrived at the target location and orientation then the robot can be caused to perform the handover action at that location and orientation, such as may involve holding the object for a minimum period of time to allow the human hand to move away, or detecting such movement of the human hand, and then performing a subsequent task based on successful completion of the handover.”. Chao teaches the information about object prediction and depends on Baek for the information of analysis of ergonomics for workers in the manufacturing industry (See at least Para [0040]). Therefore, combining Chao with the teachings of Baek should be obvious to one of the skill in the art.
Again furthermore, the Applicant argues on Page 13 of the Applicant’s remarks that “Neither Baek nor Chao address real-time ergonomic feedback control, nor do they suggest incorporating cumulative joint strain as a control input for robot motion. Thus, even in combination, the cited references fail to teach or suggest the claimed invention.”. The Examiner respectfully disagrees. Chao depends on Baek in the rejection for the teachings of real-time ergonomic feedback (See at least Para [0015] “It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive. A system and method are provided for a Prevention and Safety Management (PSM) system and method for automated analysis of ergonomics for workers in the manufacturing industry using computer vision and deep machine learning. Using this system which does not rely upon wearable sensors (including passive sensors such as visual markers or reflectors), occupational safety engineers in manufacturing plants are able to assess in real time the kinematic aspects of the workers (joint positions and angles for multiple joints simultaneously), and the impact of various physical activities (posture, repetition, force, pull, reach . . . ), to determine the risks of injuries from repetitive motion to shoulder, elbow, wrist, and hand, and to reduce and possibly prevent work-related injuries from happening…”). Thus, in combination, the cited references teach or suggest the claimed invention.
Even furthermore, the Applicant argues on page 13 and page 14 of the Applicant’s remarks that “Newly added dependent claim 21 further recites: "wherein the ergonomic assessment processor circuitry is operable to continuously evaluate the sensor data in real-time during active human-cobot interaction to generate updated strain scores." These real-time feedback and dynamic adjustment features are inherent to the claimed system's operation but are entirely absent from Baek's post-hoc ergonomic analysis approach. Baek analyzes recorded workplace videos using "deep machine learning algorithms to resolve the image data into kinematic activities" for the purpose of generating workplace safety documentation, not for controlling a robot during active collaboration. Nothing in Baek suggests continuous, frame-to-frame updating of strain values during an ongoing task, nor any real-time coupling of ergonomic metrics to robot motion.”. The Examiner respectfully disagrees. Baek teaches continuous, frame-to-frame updating of strain values during an ongoing task, and real-time coupling of ergonomic metrics to robot motion (See at least Para [0015] “It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive. A system and method are provided for a Prevention and Safety Management (PSM) system and method for automated analysis of ergonomics for workers in the manufacturing industry using computer vision and deep machine learning. Using this system which does not rely upon wearable sensors (including passive sensors such as visual markers or reflectors), occupational safety engineers in manufacturing plants are able to assess in real time the kinematic aspects of the workers (joint positions and angles for multiple joints simultaneously), and the impact of various physical activities (posture, repetition, force, pull, reach . . . ), to determine the risks of injuries from repetitive motion to shoulder, elbow, wrist, and hand, and to reduce and possibly prevent work-related injuries from happening…”, Para [0042] “… These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0070] “Furthermore, each of the Kalman filter trackers is associated with an adaptive appearance model. The adaptive appearance model records the history of visual appearances. The CNN-feature map at the tracked bounding box region is cropped for each frame and warped into a 32×32 map (i.e. ROI pooling). Here, to avoid storing 32×32 feature maps for a large number of video frames per tracker naively, we use the adaptive principal component analysis (PCA) model. The adaptive PCA model compresses a large set of feature maps by using the mean image and principal components. Instead of finding principal components each time a new cropped image is added, which involves the computationally-heavy eigendecomposition calculation repeatedly, adaptive PCA allows updating the mean and the principal components using an update formula. The adaptive appearance model tracked in this manner is used to merge two Kalman filter trackers and to solve the assignment problem”).
And moreover, the Applicant argues on Page 14 of the Applicant’s remarks that “Even if one attempted to adapt Baek for real-time use, which Baek does not contemplate, there is still no disclosure or suggestion of continuously updating integrated strain scores during human-cobot interaction, no teaching of dynamically modifying robot behavior based on those updated scores, and no mechanism for integrating ergonomic strain assessment directly into robot motion planning. The real-time, dynamic control loop required by dependent claim 21 is therefore neither taught nor suggested by Baek, either alone or in combination with the cited references.”. The Examiner respectfully disagrees. The real-time, dynamic control loop required by dependent claim 21 is taught or suggested by Baek (See at least Para [0015], [0042], [0070]).
Again furthermore, the Applicant argues on Page 14 of the Applicant’s remarks that “Baek contains no disclosure of any of these features. Baek does not generate strain scores at all, let alone individual strain scores for multiple joints derived from integration of joint-specific motion. Baek does not monitor how strain evolves or progresses over time and does not include any mechanism for threshold-based ergonomic intervention. Moreover, Baek performs no robot control and contains no teaching of adjusting a cobot's position or orientation in response to ergonomic metrics. Baek's general ergonomic "risk assessment" is a post-hoc, categorical evaluation that does not implement the real-time, joint-specific, feedback-driven control loop recited across claims 2-4.”. The Examiner respectfully disagrees. Baek discloses generating strain scores for multiple joints derived from integration of joint-specific motion (See at least Para [0042]) and monitor how strain evolves or progresses over time and does not include any mechanism for threshold-based ergonomic intervention (See at least Para [0015]). In the rejection, Chao teaches robot control associated with human interaction (See at least Para [0024]). Thus, in combination, the cited references teach or suggest the claimed invention.
The Applicant argues on Page 15 of the Applicant’s remarks that “None of the cited references teach or suggest this formula. Although Baek mentions "weighted sums" in the context of tracker matching (e.g., similarity calculations for assigning tracked skeletons to workers) (Baek 111[0073]-[0074]), those weighted sums relate solely to computer vision data association. They have no connection to ergonomic strain calculation, cumulative angular displacement, deviation from natural rest position, or any physiological metric. Thus, Baek's weighted-sum discussion is directed to a different problem and cannot reasonably be equated with the specific mathematical generation of strain scores required by claim 5.”. The Examiner respectfully disagrees. Applicant mentioned that Baek mentions "weighted sums”. In the rejection, Chao depends on both Baek and Batzianoulis for the teachings of claim 5. Baek teaches the strain score being generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time (See at least Para [0011], [0017], [0069], [0073], [0074], [0099]) whereas Batzianoulis teaches deviation from natural rest position (See at least Page 7 “Fig 4). Thus, in combination, the cited references teach or suggest the claimed invention.
The Applicant again argues on Page 15 of the Applicant’s remarks that “Claim 6 further requires that "the strain score is further based on an exponential decay of the strain score." None of the cited references discloses exponential decay of ergonomic strain, cumulative joint load, or any analogous ergonomic metric. There is no suggestion in Baek, Chao, or any other reference that strain should decay exponentially as time progresses. This feature is wholly absent from the prior art.”. The Examiner respectfully disagrees. Chao depends on Baek in the rejection for the teachings of Ergonomic metric such as ergonomic strain, cumulative joint load (See at least Para [0042]) and he depends on Schmid for the teachings of exponential decay as time progresses (See at least Page 4 Para 11).
The Applicant further argues on Page 15 of the Applicant’s remarks that “Claim 7 recites that "the cobot motion processor circuitry is further operable to simulate inverse kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container." The Office Action relies on Gomez Gutierrez to teach inverse kinematics, but that reference teaches inverse kinematics solely for computing robot joint configurations ("For 6D poses, inverse kinematics techniques may be used to compute the corresponding joint configurations...," Gomez1[0066]). Gomez does not disclose or suggest using inverse kinematics to simulate human motion, nor using such simulation to determine ergonomically optimal container placement for a cobot. The claimed use of inverse kinematics to model human biomechanics for ergonomic optimization is a novel application not taught or suggested by the cited references.”. The Examiner respectfully disagrees. Chao depends on Baek in the rejection for the teachings of cobot motion processor circuitry being operable to simulate kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container (See at least Fig 1 item 10 – Kinematic Activities, Para [0042]) and he depends on Gomez Gutierrez for the teachings of inverse kinematic modeling (See at least Para [0066]). Thus, in combination, the cited references teach or suggest the claimed invention.
The Applicant furthermore argues on Page 15 of the Applicant’s remarks that “The additional references cited by the Office Action do not cure these deficiencies. Accordingly, claims 5-7 include specific mathematical, physiological, and control-system limitations that are not taught or suggested by any of the cited references, either individually or in combination.”. The Examiner respectfully disagrees. The combination of the cited references teach or suggest the claimed invention as discussed above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
analysis – Step 1
Claim 1 is directed to a system of controlling a cobot (i.e., a process). Therefore, claim 1 is
within at least one of the four statutory categories.
analysis – Step 2A, Prong I
Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to
determine whether they recite subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes.
Independent claim 1 includes limitations that recite an abstract idea (emphasized below) and will be used as a representative claim for the remainder of the 101 rejection. Claim 1 recites:
A system for human-cobot (collaborative robot) ergonomic interaction, comprising:
a communication interface operable to receive sensor data related to human motion;
ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a
strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time;
human intent prediction processor circuitry operable to interpret the sensor data to predict an
object the human intends to grasp, and to select a destination container for the predicted object; and
cobot motion processor circuitry operable to determine a position or orientation for the cobot
to place the selected destination container based on the predicted object and the strain score.
The examiner submits that the foregoing bolded limitation(s) constitute mathematical concepts.
For example, “generate a strain score …” in the context of this claim encompasses performing mathematical calculation to obtain certain results to control the cobot.
101 Analysis – Step 2A, Prong II
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract idea into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
A system for human-cobot (collaborative robot) ergonomic interaction, comprising:
a communication interface operable to receive sensor data related to human motion;
ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a
strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time;
human intent prediction processor circuitry operable to interpret the sensor data to predict an
object the human intends to grasp, and to select a destination container for the predicted object; and
cobot motion processor circuitry operable to determine a position or orientation for the cobot
to place the selected destination container based on the predicted object and the strain score.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “a communication interface operable to receive sensor data related to human motion”, “ergonomic assessment processor circuitry operable to”, “human intent prediction processor circuitry operable to”, “cobot motion processor circuitry operable to”, “cobot motion processor circuitry operable to”, the examiner submits that these limitations are insignificant extra-solution activities which does not integrate the abstract idea into practical application. In general a communication interface operable to receive sensor data related to human motion is recited at a high level of generality and amounts to mere data gathering, which is a form of insignificant extra-solution activity. Lastly, the “for the cobot to place the selected destination container …” merely describes how to generally “apply” the otherwise mathematical calculation in a generic or general purpose robot control environment. The cobot is recited at a high level of generality.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
Regarding Step 2B of the 2019 PEG, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application.
Further, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The additional limitations of “human intent prediction processor circuitry operable to …”, and “cobot motion processor circuitry operable to …”are well-understood, routine, and conventional activities and the specification does not provide any indication that controlling the cobot is anything other than a conventional computer within a robot. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner.
Dependent claim(s) 2-7, and 9-12 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of dependent claims are directed toward additional aspects of the judicial exception and/or well-understood, routine and conventional additional elements that do not integrate the judicial exception into a practical application [provide concise explanation]. Therefore, dependent claims 2-7, and 9-12 are not patent eligible under the same rationale as provided for in the rejection of [independent claim].
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
analysis – Step 1
Claim 13 is directed to a component of a system of controlling a cobot (i.e., a process).
Therefore, claim 13 is within at least one of the four statutory categories.
analysis – Step 2A, Prong I
Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to
determine whether they recite subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes.
Independent claim 13 includes limitations that recite an abstract idea (emphasized below) and will be used as a representative claim for the remainder of the 101 rejection. Claim 13 recites:
A component of a system for human-cobot (collaborative robot) ergonomic interaction,
comprising:
processor circuitry; and
a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry, cause the processor circuitry to:
receive sensor data related to human motion;
evaluate the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time;
interpret the sensor data to predict an object the human intends to grasp, and to
select a destination container for the predicted object; and
determine a position or orientation for the cobot to place the selected destination
container based on the predicted object and the strain score.
The examiner submits that the foregoing bolded limitation(s) constitute mathematical concepts.
For example, “generate a strain score …” in the context of this claim encompasses performing mathematical calculation to obtain certain results to control the cobot.
101 Analysis – Step 2A, Prong II
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
A component of a system for human-cobot (collaborative robot) ergonomic interaction,
comprising:
processor circuitry; and
a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry, cause the processor circuitry to:
receive sensor data related to human motion;
evaluate the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time;
interpret the sensor data to predict an object the human intends to grasp, and to
select a destination container for the predicted object; and
determine a position or orientation for the cobot to place the selected destination
container based on the predicted object and the strain score.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “receive sensor data related to human motion”, “processor circuitry”, “a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry”, the examiner submits that these limitations are insignificant extra-solution activities which does not integrate the abstract idea into practical application. In general receive sensor data related to human motion is recited at a high level of generality and amounts to mere data gathering, which is a form of insignificant extra-solution activity. Lastly, the “for the cobot to place the selected destination container …” merely describes how to generally “apply” the otherwise mathematical calculation in a generic or general purpose robot control environment. The cobot is recited at a high level of generality.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
Regarding Step 2B of the 2019 PEG, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application.
Further, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The additional limitations of “processor circuitry”, and “a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry …” are well-understood, routine, and conventional activities and the specification does not provide any indication that controlling the cobot is anything other than a conventional computer within a robot. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner.
Dependent claim(s) 14-19 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of dependent claims are directed toward additional aspects of the judicial exception and/or well-understood, routine and conventional additional elements that do not integrate the judicial exception into a practical application. Therefore, dependent claims 14-19 are not patent eligible under the same rationale as provided for in the rejection of [independent claim].
Therefore, claim(s) 1-17 and 9-19 are ineligible under 35 USC §101. Incorporating active control step in the claim language may overcome the rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 8, 11, 12, 13, 14, 15, 16, 20 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek).
Regarding Claim 1, Chao teaches a system for human-cobot (collaborative robot) ergonomic interaction, comprising:
a communication interface operable to receive sensor data related to human motion (See at
least Fig. 3A shows communication interface operable to receive sensor data related to human motion, Para [0042] “FIG. 3A illustrates an example system 300 that can be used to model realistic motion of a human hand for a human-robot interaction that can be used in accordance with various embodiments. It should be understood that such interactions can be performed with both hands or other portions of a human body, and are also not limited to (inter)actions such as object handovers which are presented as a primary example herein. In this example system 300, one or more cameras 302 or sensors can be positioned with respect to a physical environment 304 that includes, or will include, a human hand 306 and at least one physical object 308 with which that hand is to interact, such as to grasp and move the object. The camera(s) 302 can capture image data 310 (e.g., a sequence of images or video) representative of the state of the physical environment, including position, orientation, and pose information for the hand and object in the physical environment. A camera 302 used for such purposes can include a camera for capturing two-dimensional images, a camera for capturing stereoscopic images, or a camera for capturing images that include color and depth information (e.g., RGB-D images). Other sensors or devices may be used to capture a representation of the environment as well, as may include depth sensors, ultrasonic sensors, LIDAR scanners, and the like.”);
…
human intent prediction processor circuitry operable to interpret the sensor data to predict an
object the human intends to grasp (See at least Fig 4 item 406, 408, Para [0048] “…A determined number of instances, such as around 1,000, may be performed and captured, as may include at least a minimum number of different types of objects and different handover or target locations, poses, and orientations. At least a subset of this captured image data can be analyzed 406 to identify at least a position and pose of the hand and the object in individual images or video frames. This can include identifying a location or coordinates, such as a bounding box, or determining a segmentation of these images that identify segments of the image corresponding to the hand and object. In other embodiments, image features may be extracted and identified for the hand and object and then encoded into a latent space for modeling and motion/behavior prediction, among other such options…”), and to select a destination container for the predicted object (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”); and
cobot motion processor circuitry operable to determine a position or orientation for the cobot
to place the selected destination container based on the predicted object and the strain score (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”).
However, Chao does not explicitly spell out …
ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a
strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time…
Baek teaches …
ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a
strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time (See at least Para [0017] “According to one embodiment, a method of evaluating workplace worker injury risks includes videotaping a worker who is not wearing any motion sensors, and who is engaged in routine repetitive movements, to provide recorded videos as input data. The recorded videos are analyzed to resolve multiple joints of the worker. The recorded videos are analyzed for measurable kinematic variables related to each joint. The measurable kinematic variables are analyzed to provide job risk assessment reports as output. The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of ergonomic assessment processor circuitry operable to evaluate the sensor data to generate a strain score for at least one human joint, wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
5. Regarding Claim 2, modified Chao teaches all the elements of claim 1.
However, Chao does not explicitly spell out the system of claim 1, wherein the ergonomic assessment processor circuitry is further operable to generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time.
Baek teaches the system of claim 1, wherein the ergonomic assessment processor circuitry is further operable to generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time (See at least Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the ergonomic assessment processor circuitry being operable to generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
6. Regarding Claim 3, modified Chao teaches all the elements of claim 2.
However, Chao does not explicitly spell out the system of claim 2, wherein the ergonomic assessment processor circuitry is further operable to monitor progressions of the individual strain scores.
Baek teaches the system of claim 2, wherein the ergonomic assessment processor circuitry is further operable to monitor progressions of the individual strain scores (See at least Para [0102] “The waveform data might also be useful for optimizing performance of workers. For examples the waveforms of high production or safer workers might be studied and used as examples for training new workers. Uses for the information gathered may include retraining workers and inboarding new employees. It can be used to interface with environmental and health data if desired. Integration with worker compensation financial data can occur. As well, there is the ability for workers to access their record and monitor their progress.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the ergonomic assessment processor circuitry being operable to monitor progressions of the individual strain scores, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 4, modified Chao teaches all the elements of claim 3. Chao further teaches … the cobot motion processor circuitry is further operable to adjust the position or orientation for the cobot to place the selected destination container (See at least Para [0027] “… A success score, or other such metric, can be generated or determined for each such simulation, and this feedback can be used to adjust the robot handover control instructions …”)…
However, Chao does not explicitly spell out the system of claim 3, wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, … to prevent exacerbation of the strain level of any of the plurality of human joints.
Baek teaches the system of claim 3, wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, … to prevent exacerbation of the strain level of any of the plurality of human joints (See at least Para [0063] “When the tracker ages beyond a threshold, we consider we lost the objects and terminate the tracker.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of preventing exacerbation of the strain level of any of the plurality of human joints
upon determining that any of the individual strain scores exceeds a predetermined threshold, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 8, modified Chao teaches all the elements of claim 1.
However, Chao does not explicitly spell out the system of claim 1, wherein the ergonomic
assessment processor circuitry is further operable to update the strain score after the cobot has placed the selected destination container.
Baek teaches the system of claim 1, wherein the ergonomic assessment processor circuitry is
further operable to update the strain score after the cobot has placed the selected destination container (See at least Para [0099 “Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek
and include the feature of the wherein the ergonomic assessment processor circuitry being operable to update the strain score after the cobot has placed the selected destination container thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 11, modified Chao teaches all the elements of claim 1. Chao further teaches the system of claim 1, wherein the cobot motion processor circuitry is further operable to use a graph-based algorithm to determine the position or orientation for the cobot to place the selected destination container (See at least Para [0056] “… In at least one embodiment, training logic 715 may include, or be coupled to code and/or data storage 701 to store graph code or other software to control timing and/or order, in which weight and/or other parameter information is to be loaded to configure, logic, including integer and/or floating point units (collectively, arithmetic logic units (ALUs)…”).
Regarding Claim 12, modified Chao teaches all the elements of claim 1. Chao further teaches the system of claim 11, wherein the cobot motion processor circuitry is further operable to determine the position or orientation for the cobot to place the selected destination container while taking into account an operating range of the cobot, a speed of the cobot, or a potential environmental obstacle (See at least Para [0047] “… In many instances this will result in the end effector being moved toward the target grasp position and orientation, but due to factors such as motion of the hand or object may involve moving away from the object to avoid a collision or obstruction relative to the hand or object…”).
Regarding Claim 13, Chao teaches A component of a system for human-cobot (collaborative
robot) ergonomic interaction, comprising:
processor circuitry (See at least Para [0057] “In at least one embodiment, any portion of code and/or data storage 701 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and/or code and/or data storage 701 may be cache memory, dynamic randomly addressable memory (“DRAM”), static randomly addressable memory (“SRAM”), non-volatile memory (e.g., Flash memory), or other storage. In at least one embodiment, choice of whether code and/or code and/or data storage 701 is internal or external to a processor, for example, or comprised of DRAM, SRAM, Flash or some other storage type may depend on available storage on-chip versus off-chip, latency requirements of training and/or inferencing functions being performed, batch size of data used in inferencing and/or training of a neural network, or some combination of these factors.”); and
a non-transitory computer-readable storage medium including instructions that, when executed by the processor circuitry (See at least Para [0221] “…In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein…”), cause the processor circuitry to:
receive sensor data related to human motion (See at least Fig. 3A shows communication
interface operable to receive sensor data related to human motion, Para [0042] “FIG. 3A illustrates an example system 300 that can be used to model realistic motion of a human hand for a human-robot interaction that can be used in accordance with various embodiments. It should be understood that such interactions can be performed with both hands or other portions of a human body, and are also not limited to (inter)actions such as object handovers which are presented as a primary example herein. In this example system 300, one or more cameras 302 or sensors can be positioned with respect to a physical environment 304 that includes, or will include, a human hand 306 and at least one physical object 308 with which that hand is to interact, such as to grasp and move the object. The camera(s) 302 can capture image data 310 (e.g., a sequence of images or video) representative of the state of the physical environment, including position, orientation, and pose information for the hand and object in the physical environment. A camera 302 used for such purposes can include a camera for capturing two-dimensional images, a camera for capturing stereoscopic images, or a camera for capturing images that include color and depth information (e.g., RGB-D images). Other sensors or devices may be used to capture a representation of the environment as well, as may include depth sensors, ultrasonic sensors, LIDAR scanners, and the like.”);
…
interpret the sensor data to predict an object the human intends to grasp (See at least
Fig 4 item 406, 408, Para [0048] “…A determined number of instances, such as around 1,000, may be performed and captured, as may include at least a minimum number of different types of objects and different handover or target locations, poses, and orientations. At least a subset of this captured image data can be analyzed 406 to identify at least a position and pose of the hand and the object in individual images or video frames. This can include identifying a location or coordinates, such as a bounding box, or determining a segmentation of these images that identify segments of the image corresponding to the hand and object. In other embodiments, image features may be extracted and identified for the hand and object and then encoded into a latent space for modeling and motion/behavior prediction, among other such options…”), and to select a destination container for the predicted object (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”); and
determine a position or orientation for the cobot to place the selected destination
container based on the predicted object and the strain score (See at least Fig 5 item 504, Para [0050] “…as discussed with respect to FIG. 4, an optimal location and orientation for the robot to perform the handover action can be determined 504. For this current state, and based at least on this predicted human behavior, and optimal sequence of motions can be predicted 506 to direct the robot to a currently-determined optimal location and orientation for the handover…”).
However, Chao does not explicitly spell out …
evaluate the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time;
Baek teaches …
evaluate the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time (See at least Para [0017] “According to one embodiment, a method of evaluating workplace worker injury risks includes videotaping a worker who is not wearing any motion sensors, and who is engaged in routine repetitive movements, to provide recorded videos as input data. The recorded videos are analyzed to resolve multiple joints of the worker. The recorded videos are analyzed for measurable kinematic variables related to each joint. The measurable kinematic variables are analyzed to provide job risk assessment reports as output. The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of evaluating the sensor data to generate a strain score for at least one human joint,
wherein the strain score represents a strain level of the at least one human joint based on an integration of motion of the at least one human joint over a period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 14, modified Chao teaches all the elements of claim 13.
Chao does not explicitly spell out the component of claim 13, wherein the instructions further cause the processor circuitry to:
generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time .
Baek teaches the component of claim 13, wherein the instructions further cause the processor circuitry to:
generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time (See at least Para [0042] “…The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0112] “Furthermore, the collection and analysis of a large volume of video data and over a prolonged period of time can, when paired with health outcome data collected at the worker level (e.g., work environment, demographics, symptom self-reports or clinical assessment) and/or at the organizational level (e.g., OSHA 300 or other records-based surveillance systems), lead to improved understanding of dose-response relationships necessary to optimize task and work design, decrease the number of injuries and decrease health care expenses for manufacturers.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of generate individual strain scores for a plurality of human joints, wherein each of the strain scores represents a strain level of its respective human joint, generated by integrating motion of the respective human joint over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 15, modified Chao teaches all the elements of claim 14.
However, Chao does not explicitly spell out the component of claim 14, wherein the instructions further cause the processor circuitry to:
monitor progressions of the individual strain scores.
Baek teaches the the component of claim 14, wherein the instructions further cause the processor circuitry to:
monitor progressions of the individual strain scores (See at least Para [0102] “The waveform data might also be useful for optimizing performance of workers. For examples the waveforms of high production or safer workers might be studied and used as examples for training new workers. Uses for the information gathered may include retraining workers and inboarding new employees. It can be used to interface with environmental and health data if desired. Integration with worker compensation financial data can occur. As well, there is the ability for workers to access their record and monitor their progress.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of monitoring progressions of the individual strain scores, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 16, modified Chao teaches all the elements of claim 15. Chao further teaches … the instructions further cause the processor circuitry to:adjust the position or orientation for the cobot to place the selected destination container (See at least Para [0027] “… A success score, or other such metric, can be generated or determined for each such simulation, and this feedback can be used to adjust the robot handover control instructions …”)…
However, Chao does not explicitly spell out the component of claim 15, wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, … to prevent exacerbation of the strain level of any of the plurality of human joints.
Baek teaches the component of claim 15, wherein upon determining that any of the individual strain scores exceeds a predetermined threshold, … to prevent exacerbation of the strain level of any of the plurality of human joints (See at least Para [0063] “When the tracker ages beyond a threshold, we consider we lost the objects and terminate the tracker.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of preventing exacerbation of the strain level of any of the plurality of human joints
upon determining that any of the individual strain scores exceeds a predetermined threshold, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 20, modified Chao teaches all the elements of claim 13.
However, Chao does not explicitly spell out the component of claim 13, wherein the instructions further cause the processor circuitry to:
update the strain score after the cobot has placed the selected destination container.
Baek teaches the the component of claim 13, wherein the instructions further cause the processor circuitry to:
update the strain score after the cobot has placed the selected destination container (See at least Para [0099 “Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek
and include the feature of the ergonomic assessment processor circuitry being operable to update the strain score after the cobot has placed the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Regarding Claim 21, modified Chao teaches all the elements of claim 1.
However, Chao does not explicitly spell out the system of claim 1, wherein the ergonomic
assessment processor circuitry is operable to continuously evaluate the sensor data in real-time during active human-cobot interaction to generate updated strain scores.
Baek teaches the system of claim 1, wherein the ergonomic assessment processor circuitry is
operable to continuously evaluate the sensor data in real-time during active human-cobot interaction to generate updated strain scores (See at least Para [0015] “It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive. A system and method are provided for a Prevention and Safety Management (PSM) system and method for automated analysis of ergonomics for workers in the manufacturing industry using computer vision and deep machine learning. Using this system which does not rely upon wearable sensors (including passive sensors such as visual markers or reflectors), occupational safety engineers in manufacturing plants are able to assess in real time the kinematic aspects of the workers (joint positions and angles for multiple joints simultaneously), and the impact of various physical activities (posture, repetition, force, pull, reach . . . ), to determine the risks of injuries from repetitive motion to shoulder, elbow, wrist, and hand, and to reduce and possibly prevent work-related injuries from happening…”, Para [0042] “… These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”, Para [0068] “Furthermore, each of the Kalman filter trackers is associated with an adaptive appearance model. The adaptive appearance model records the history of visual appearances. The CNN-feature map at the tracked bounding box region is cropped for each frame and warped into a 32×32 map (i.e. ROI pooling). Here, to avoid storing 32×32 feature maps for a large number of video frames per tracker naively, we use the adaptive principal component analysis (PCA) model. The adaptive PCA model compresses a large set of feature maps by using the mean image and principal components. Instead of finding principal components each time a new cropped image is added, which involves the computationally-heavy eigendecomposition calculation repeatedly, adaptive PCA allows updating the mean and the principal components using an update formula. The adaptive appearance model tracked in this manner is used to merge two Kalman filter trackers and to solve the assignment problem.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective
filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek
and include the feature of ergonomic assessment processor circuitry being operable to continuously evaluate the sensor data in real-time during active human-cobot interaction to generate updated strain scores, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Claim(s) 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of Batzianoulis et al. (Batzianoulis I, Krausz NE, Simon AM, Hargrove L, Billard A. Decoding the grasping intention from electromyography during reaching motions. J Neuroeng Rehabil. 2018 Jun 26;15(1):57. doi: 10.1186/s12984-018-0396-5. PMID: 29940991; PMCID: PMC6020187). (Hereinafter Batzianoulis).
Regarding Claim 5, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time and an average deviation of the motion of the at least one joint from its natural rest position.
Baek teaches the system of claim 1, wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time (See at least Para [0011] “…C examples include use of the Lumbar Motion Monitor to measure kinematics of the lumbar spine, electrogoniometers to measure angular displacement of certain joints (e.g., most commonly the wrist, but also the knee, shoulder, and elbow)…”, Para [0017] “…The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0073] “We combine these three matrices using a weighted sum”, Para [0074] “The Hungarian algorithm uses the weighted sum to determine the assignment between Kalman filter tracker bounding boxes and CNN-detected bounding boxes. After the assignment is completed, we select only admissible assignments, by thresholding each of the similarity measures...”, Para [0099] “…Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record…”, Para [0069] “After processing the entire video frames with the Kalman filter trackers, we compare the similarity among trackers using the appearance model. The dissimilarity between the trackers is defined as the weighted sum of the Euclidean distance between PCA means and the cosine distance between the principal components.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the strain score being generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Batzianoulis teaches … and an average deviation of the motion of the at least one joint from its natural rest position (See at least Page 7 “Fig 4 : The confusion matrices present the average classification accuracies and their standard deviations for the five grasp types…” ).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Batzianoulis and include the feature of an average deviation of the motion of the at least one joint from its natural rest position, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 17, modified Chao teaches all the elements of claim 13.
Chao does not explicitly spell out the system of claim 1, wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time and an average deviation of the motion of the at least one joint from its natural rest position.
Baek teaches the component of claim 13, wherein the strain score is generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time (See at least Para [0011] “…C examples include use of the Lumbar Motion Monitor to measure kinematics of the lumbar spine, electrogoniometers to measure angular displacement of certain joints (e.g., most commonly the wrist, but also the knee, shoulder, and elbow)…”, Para [0017] “…The kinematic variables may include at least some of joint positions, angles, range of motion, walking, posture, push, pull, reach, force, repetition, duration, musculoskeletal health, movement velocity, rest/recovery time and variations in movement patterns…”, Para [0073] “We combine these three matrices using a weighted sum”, Para [0074] “The Hungarian algorithm uses the weighted sum to determine the assignment between Kalman filter tracker bounding boxes and CNN-detected bounding boxes. After the assignment is completed, we select only admissible assignments, by thresholding each of the similarity measures...”, Para [0099] “…Based on the likely 3-D postures of each worker in each frame, the joint angles and changes in joint angles over time can be calculated and recorded in that worker's database record…”, Para [0069] “After processing the entire video frames with the Kalman filter trackers, we compare the similarity among trackers using the appearance model. The dissimilarity between the trackers is defined as the weighted sum of the Euclidean distance between PCA means and the cosine distance between the principal components.”)…
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the strain score being generated based on a weighted sum of a cumulative angular displacement during the motion of the at least one human joint over the period of time, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
Batzianoulis teaches … and an average deviation of the motion of the at least one joint from its natural rest position (See at least Page 7 “Fig 4 : The confusion matrices present the average classification accuracies and their standard deviations for the five grasp types…” ).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Batzianoulis and include the feature of an average deviation of the motion of the at least one joint from its natural rest position, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim(s) 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of Schmid et al. (WO2024132714A1) (Hereinafter Schmid).
Regarding Claim 6, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the strain score is further based on an exponential decay of the strain score.
Schmid teaches the system of claim 1, wherein the strain score is further based on an exponential decay of the strain score (See at least Page 4 Para 11 “In an embodiment, the weight of a given heat strain score, when calculating the acute heat load, is subject to exponential decay with a time constant of 2 - 10 days.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Schmid and include the feature of the strain score being based on an exponential decay of the strain score, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 18, modified Chao teaches all the elements of claim 13.
Chao does not explicitly spell out the component of claim 13, wherein the strain score is further based on an exponential decay of the strain score.
Schmid teaches the component of claim 13, wherein the strain score is further based on an exponential decay of the strain score (See at least Page 4 Para 11 “In an embodiment, the weight of a given heat strain score, when calculating the acute heat load, is subject to exponential decay with a time constant of 2 - 10 days.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Schmid and include the feature of the strain score being based on an exponential decay of the strain score, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim(s) 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of Gomez Gutierrez et al (US 20240217103 A1) (Hereinafter Gomez Gutierrez).
Regarding Claim 7, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the cobot motion processor
circuitry is further operable to simulate inverse kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container.
Baek teaches teaches the system of claim 1, wherein the cobot motion processor circuitry is further operable to simulate … kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container (See at least Fig 1 item 10 – Kinematic Activities, Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the cobot motion processor circuitry being operable to simulate kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
However, neither Chao nor Baek explicitly spell out kinematics modeling being inverse.
Gomez Gutierrez teaches inverse kinematic modeling (See at least Para [0066] “…For 6D poses, inverse kinematics techniques may be used to compute the corresponding joint configurations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Gomez Gutierrez and include the feature of the inverse kinematic modeling, thereby provide simplified automatic calculation of necessary joint movements for precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Regarding Claim 19, modified Chao teaches all the elements of claim 7.
Chao does not explicitly spell out the component of claim 13, wherein the instructions further
cause the processor circuitry to:
simulate inverse kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container.
Baek teaches the component of claim 13, wherein the instructions further cause
the processor circuitry to:
simulate … kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container (See at least Fig 1 item 10 – Kinematic Activities, Para [0042] “With further reference to FIG. 1, the image capturing device 12 transmits image data (e.g., AVI, Flash Video, MPEG, WebM, WMV, GIF, and other known video data formats) to a computing device, such as a computing cloud 16 . The computing device 16 uses deep machine learning algorithms to resolve the image data into kinematic activities. The computing device 16 is adapted to perform unique analyses of the resolved kinematic activities of multiple body joints simultaneously and make assessments of ergonomic metrics including joint positions and angles, walk/posture, lift, push, pull, reach, force, repetition, duration, and to distinguish and report on each one separately. These ergonomic metrics are analyzed by a computing device 16 adapted to act as a risk assessment tool by applying existing ergonomic models to the ergonomic metrics to create a risk assessment of the workers. The risk assessment may be a score, a risk level, or similar report.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Baek and include the feature of the cobot motion processor circuitry being operable to simulate kinematics modeling the human motion to determine the position or orientation for the cobot to place the selected destination container, thereby provide precise ergonomic assessment to enhance safety by decreasing the number of work injuries (See at least Para [0113] “This system, designed for automated analysis of ergonomics (body posture and positioning identification) for example for meat processing workers using the power of computer vision and deep machine learning, prevents and decreases drastically upper extremities musculoskeletal injuries associated with repetitive stress injuries and reduces the high costs associated with these injuries. In many ways this Prevention and Safety Management system (PSM) is a tremendous improvement, possibly even “an inflection point”, in the way manufacturers presently monitor, prevent, and mitigate risks of work-related injuries.”).
However, neither Chao nor Baek explicitly spell out kinematics modeling being inverse.
Gomez Gutierrez teaches inverse kinematic modeling (See at least Para [0066] “…For 6D poses, inverse kinematics techniques may be used to compute the corresponding joint configurations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Gomez Gutierrez and include the feature of the inverse kinematic modeling, thereby provide simplified automatic calculation of necessary joint movements for precise ergonomic assessment to enhance safety by decreasing the number of work injuries.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of (Wagner et al.) (US 20170136632 A1) (Hereinafter Wagner).
Regarding Claim 9, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the human intent prediction processor circuitry is further operable to select the destination container for the predicted object from a plurality of candidate destination containers.
Wagner teaches the system of claim 1, wherein the human intent prediction processor circuitry is further operable to select the destination container for the predicted object from a plurality of candidate destination containers (See at least Para [0005] “… The sortation system includes a programmable motion device including an end effector, a perception system for recognizing any of the identity, location, or orientation of an object presented in a plurality of objects, a grasp selection system for selecting a grasp location on the object, the grasp location being chosen to provide a secure grasp of the object by the end effector to permit the object to be moved from the plurality of objects to one of a plurality of destination locations, and a motion planning system for providing a motion path for the transport of the object when grasped by the end effector from the plurality of objects to the one of the plurality of destination locations, wherein the motion path is chosen to provide a path from the plurality of objects to the one of the plurality of destination locations…”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Wagner and include the feature of the human intent prediction processor circuitry being operable to select the destination container for the predicted object from a plurality of candidate destination containers, thereby provide efficiency, robustness and safety while moving items from one place to another (See at least Para [0066] “Another advantage of the varied set is the ability to address several customer metrics without having to re-plan motions. The database is sorted and indexed by customer metrics like time, robustness, safety, distance to obstacles etc. and given a new customer metric, all the database needs to do is to reevaluate the metric on the existing trajectories, thereby resorting the list of trajectories, and automatically producing the best trajectory that satisfies the new customer metric without having to re-plan motions.”).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Chao et a. (US 2023/0294276 A1) (Hereinafter Chao) in view of Baek et al. (US 20220237537 A1) (Hereinafter Baek), and further in view of Matijevich et al. (US 20210236020 A1) (Hereinafter Matijevich).
Regarding Claim 10, modified Chao teaches all the elements of claim 1.
Chao does not explicitly spell out the system of claim 1, wherein the cobot motion processor circuitry is further operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of the at least one human joint.
Matijevich teaches the system of claim 1, wherein the cobot motion processor circuitry is further operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of the at least one human joint (See at least Para [0023] “In one embodiment, the processing unit is further configured to alert the user when the musculoskeletal loading and/or microdamage accumulation is greater than a threshold that has been predefined or a threshold that has been calibrated for the specific user.”).
Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chao with the teachings of Matijevich and include the feature of the cobot motion processor circuitry being operable to issue a strain alert if it is unable to determine a suitable position or orientation for the destination container that does not exacerbate the strain level of the at least one human joint, thereby provide notification in case of an emergency (See at least Para [0265] “…The investigators also have considerable experience in developing custom embedded systems for wearable exoskeletons that provide compact, energy efficient, self-contained operation.”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHEDA HOQUE whose telephone number is (571)270-5310. The examiner can normally be reached Monday-Friday 8:00 am- 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at 571-270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHAHEDA HOQUE/
Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658