DETAILED ACTION
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6, 8-13, 15-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over
U.S. Patent Application Publication No. 2019/0147655 (Galera) in view of
U.S. Patent Application Publication No. 2021/0216773 (Bohannon) and further in view of
U.S. Patent Application Publication No. 2021/0069907 (Vu).
Claim 1:
The cited prior art describes a computer-implemented method for identifying critical zones of machines, the computer-implemented method comprising: (Galera: “The subject matter disclosed herein relates generally to industrial automation systems, and, more particularly, to management of monitored industrial safety zones.” Paragraph 0002; “FIG. 3 is a block diagram of an example virtual and augmented reality presentation system 302 (also referred to herein as VR/AR presentation system) according to one or more embodiments of this disclosure. Aspects of the systems, apparatuses, or processes explained in this disclosure can constitute machine-executable components embodied within machine(s), e.g., embodied in one or more computer-readable mediums (or media) associated with one or more machines. Such components, when executed by one or more machines, e.g., computer(s), computing device(s), automation device(s), virtual machine(s), etc., can cause the machine(s) to perform the operations described.” Paragraph 0067)
performing, by a computer, a digital twin simulation of a machine of a plurality of machines in an industrial machine environment using a digital twin computing system; (Galera: see the simulation component 318 as illustrated in figure 3 and as described in paragraphs 0173, 0174; “Simulation component 318 can be configured to simulate execution of multiple control devices, including but not limited to industrial controllers, motor drives, and other such control devices. As more simulated control devices are integrated with the VR/AR model 1804, a digital twin of the physical automation system can be realized. This digital twin can be used to test new control programs on virtualized equipment analogous to their real-world counterparts, perform predictive analytics to estimate asset maintenance or replacement schedules, or other such functions.” Paragraph 0175)
Galera does not explicitly describe historical incident records or identifying machine data as described below. However, Bohannon teaches the historical incident records and Vu teaches identifying machine data as described below.
performing, by the computer, an analysis of a set of historical incident records corresponding to the machine while previously operating in the industrial machine environment to determine areas surrounding the machine in the industrial machine environment that will be impacted by propagation of different types of incidents corresponding to the machine; (Bohannon: see the determination of information for the worker on the AR display 188 based on the safety data repository 48B as illustrated in figure 10 and as described in paragraph 0158; see the safety data repository 48B as illustrated in figure 2 and as described in paragraph 0067; “Based on the information relating to the field of view, such as the information from safety data repository 48B and/or worker data repository 48C, information processor 40B may determine if there are any safety events, hazards, worker information, environment information, machine information, PPE information, or the like to indicate to worker 10 via an AR display of safety glasses 14 (188).” Paragraph 0188; “For example, safety data repository 48B may include data relating to recorded safety events, sensed environmental conditions, worker indicated hazards, machine or equipment statuses, emergency exit information, safe navigation paths, proper PPE use instructions, service life or condition of articles of PPE, horizon or ground level indicators, boundaries, hidden structure information, or the like.” Paragraph 0067)
identifying, by the computer, (Vu: see the object monitoring system 410 as illustrated in figure 4 and as described in paragraph 0051) (Galera: “In some embodiments, the VR/AR representation of an industrial factory generated by VR/AR presentation system 302 can be used as the basis for a digital twin of the factory. In such embodiments, the plant models 524 can not only model the physical appearance of industrial assets, but can also model certain behaviors of those assets (e.g., responses to control inputs in terms of movement, speed, temperatures, flows, fill levels, etc.).” paragraph 0173)
a task performed by the machine, (Vu: see the robot state determination 420 as illustrated in figure 4; “A robot state determination module (RSDM) 420 is responsive to data from sensors 102 and signals from the robot 402 and/or robot controller 407 to determine the instantaneous state of the robot. In particular, RSDM 420 determines the pose and location of robot 402 within workspace 400; this may be achieved using sensors 102, signals from the robot and/or its controller, or data from some combination of these sources.” Paragraph 0053)
an operating context of the machine, (Vu: see the object analysis 415 as illustrated in figure 4; “The sensors 102 provide real-time image information that is analyzed by an object-analysis module 415 at a fixed frequency in the manner discussed above; in particular, at each cycle, object analysis module 415 identifies the precise 3D location and extent of all objects in workspace 400 that are either within the robot's reach or that could move into the robot's reach at conservative expected velocities.” Paragraph 0052)
an aggregate energy of the machine, and (Vu: “RSDM 420 may also determines the instantaneous velocity of robot 402 or any appendage thereof; in addition, knowledge of the robot's instantaneous joint accelerations or torques, or planned future trajectory may be needed in order to determine safe motion constraints for the subsequent cycle as described below.” Paragraph 0053)
an area in the industrial machine environment affected by propagation of released energy from the machine (Vu: see the safe action determination 425 as illustrated in figure 4; “One approach to achieving this is to modulate the robot's maximum velocity (by which is meant the velocity of the robot itself or any appendage thereof) proportionally to the minimum distance between any point on the robot and any point in the relevant set of sensed objects to be avoided. The robot is allowed to operate at maximum speed when the closest object is further away than some threshold distance beyond which collisions are not a concern, and the robot is halted altogether if an object is within a certain minimum distance. Sufficient margin can be added to the specified distances to account for movement of relevant objects or humans toward the robot at some maximum realistic velocity. This is illustrated in FIG. 5. An outer envelope or 3D zone 502 is generated computationally by SADM 425 around the robot 504.” Paragraph 0060)
based on
the digital twin simulation of the machine and (Vu: see the training and use of a model to determine robot motions as described in paragraph 0058) (Galera: see the simulation component 318 as illustrated in figure 3 and as described in paragraphs 0173, 0174; “Simulation component 318 can be configured to simulate execution of multiple control devices, including but not limited to industrial controllers, motor drives, and other such control devices. As more simulated control devices are integrated with the VR/AR model 1804, a digital twin of the physical automation system can be realized. This digital twin can be used to test new control programs on virtualized equipment analogous to their real-world counterparts, perform predictive analytics to estimate asset maintenance or replacement schedules, or other such functions.” Paragraph 0175; “In some embodiments, the VR/AR representation of an industrial factory generated by VR/AR presentation system 302 can be used as the basis for a digital twin of the factory. In such embodiments, the plant models 524 can not only model the physical appearance of industrial assets, but can also model certain behaviors of those assets (e.g., responses to control inputs in terms of movement, speed, temperatures, flows, fill levels, etc.).” paragraph 0173)
the analysis of the set of historical incident records corresponding to the machine; and (Vu: see the use of tests and state analysis to train the system for expected operations as described in paragraphs 0057, 0058) (Bohannon: see the determination of information for the worker on the AR display 188 based on the safety data repository 48B as illustrated in figure 10 and as described in paragraph 0158; see the safety data repository 48B as illustrated in figure 2 and as described in paragraph 0067; “Based on the information relating to the field of view, such as the information from safety data repository 48B and/or worker data repository 48C, information processor 40B may determine if there are any safety events, hazards, worker information, environment information, machine information, PPE information, or the like to indicate to worker 10 via an AR display of safety glasses 14 (188).” Paragraph 0188; “For example, safety data repository 48B may include data relating to recorded safety events, sensed environmental conditions, worker indicated hazards, machine or equipment statuses, emergency exit information, safe navigation paths, proper PPE use instructions, service life or condition of articles of PPE, horizon or ground level indicators, boundaries, hidden structure information, or the like.” Paragraph 0067)
identifying, by the computer, (Galera: “In yet another example, safety zone definition data 2314 may be defined on presentation system 302 (e.g., as part of plant model data 524).” Paragraph 0210) (Vu: see the zones 502, 508, 508 for robot operation based on the determinations by the SADM 425 using object analysis 415, robot state determination 420, and safe action determination 425 as illustrated in figures 4, 5; “The safe-action constraints identified by SADM 425 may be communicated by OMS 410 to robot controller 407 on each cycle via a robot communication module 430.” Paragraph 0064)
a set of critical zones corresponding to the machine and (Galera: see the fields 2402a, 2402b, 2402c for the machine 2408 as illustrated in figure 24 and as described in paragraph 0204: “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412. In general, a monitored safety zone comprises the collection of warning fields and/or safety fields that make up the monitored zone. Warning fields (e.g., warning field 2402c) define areas within which detection of an unauthorized object (e.g., a person or a vehicle) will cause a notification or warning to be issued, while safety fields (e.g., safety fields 2404a and 2402b) define areas within which detection of an unauthorized object will cause the controlled equipment to transition to a safe state (e.g., a slow mode, a de-energized state, etc.).” paragraph 0204)
an intensity level of each respective critical zone of the set of critical zones (Galera: “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412. In general, a monitored safety zone comprises the collection of warning fields and/or safety fields that make up the monitored zone. Warning fields (e.g., warning field 2402c) define areas within which detection of an unauthorized object (e.g., a person or a vehicle) will cause a notification or warning to be issued, while safety fields (e.g., safety fields 2404a and 2402b) define areas within which detection of an unauthorized object will cause the controlled equipment to transition to a safe state (e.g., a slow mode, a de-energized state, etc.).” paragraph 0204)
based on the task performed by the machine, the operating context of the machine, the aggregate energy of the machine, and the area in the industrial machine environment affected by propagation of released energy from the machine. (Vu: see the zones 502, 508, 508 for robot operation based on the determinations by the SADM 425 using object analysis 415, robot state determination 420, and safe action determination 425 as illustrated in figures 4, 5; “The safe-action constraints identified by SADM 425 may be communicated by OMS 410 to robot controller 407 on each cycle via a robot communication module 430.” Paragraph 0064)
One of ordinary skill in the art would have recognized that applying the known technique of Galera, namely, an augmented reality safety automation zone system, with the known techniques of Bohannon, namely, an augmented reality safety event detection and visualization system, and the known techniques of Vu, namely, determining workspace safe zones, would have yielded predictable results and resulted in an improved system. Accordingly, applying the teachings of Galera to identify safety zones for industrial facilities and visualize the safety zones using augmented reality with the teachings of Bohannon to use various data repositories to determine safety zones and display data about the safety zones using augmented reality and the teachings of Vu to determine safe zones for machinery using various data would have been recognized by those of ordinary skill in the art as resulting in an improved safety zone system for machines. In other words, the combination of the references provides for safety zone identification for machines based on a digital twin simulation, recorded safety data, and machine movement data based on the teachings of safety zone identification for machines based on a digital twin simulation in Galera, the teachings of safety zone identification based on recorded safety data in Bohannon and the teachings of safety zone identification based on machine movement data in Vu).
Claim 2:
The cited prior art describes the computer-implemented method of claim 1 further comprising:
detecting, by the computer, using a set of sensors, that a worker is in a particular critical zone of the set of critical zones corresponding to the machine; (Galera: see the safety sensing device 2412 as illustrated in figure 24; “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412.” Paragraph 0204; “although only one safety sensor device 2412 is depicted in FIG. 25, in some implementations multiple safety sensor devices may be installed and configured to monitor a given hazardous machine or area, with each safety sensor device having one or more defined safety and/or warning fields which may or may not overlap with safety and/or warning fields of other safety sensor devices” paragraph 0210)
displaying, by the computer, the particular critical zone corresponding to the machine to the worker via an augmented reality device worn by the worker; and (Galera: see the wearable device 206 as illustrated in figure 24; “To assist with configuration and identification of safety or warning fields within the plant facility, some embodiments of the presentation system 302 described herein can render defined graphical representations of safety and warning fields via wearable appliance 206 or a hand-held client device as graphical overlays of an AR presentation.” Paragraph 0212; “Also, in addition to the safety and warning actions described above, intrusion into a warning or safety field by a user can cause the system to render the graphical representation of the safety or warning field at a different brightness level relative to the default brightness level applied when the user is not located within the field, indicating to the user that he or she is currently within a defined safety or warning field. Additionally, some embodiments can be configured to deliver additional information to the user's wearable appliance or client device in response to determining that the user is currently within a warning or safety field, where the additional information is relevant to the particular field within which the user intrudes. This information can include, for example, information regarding a safety action being performed on the machine in response to the user's intrusion into a safety field, an instruction to vacate the monitored safety zone, a haptic signal directed to the wearable appliance intended to draw the user's attention to the fact that he or she has entered a monitored zone, or other such information.” Paragraph 0208)
Galera does not explicitly describe safety equipment as described below. However, Bohannon teaches the safety equipment as described below.
alerting, by the computer, using the augmented reality device, the worker of a type of safety equipment to be worn by the worker in the particular critical zone of the set of critical zones corresponding to the machine based on (Bohannon: see the alert for gloves (i.e., PPE) as illustrated in figure 5 and as described in paragraph 0117)
the intensity level of that particular critical zone, (Bohannon: “For example, computing device 32 of the safety manager may send a message that defines or specifies the one or more articles of PPE required for a specific job function, for a specific environment 8, for a specific worker 10A, or the like. As another example, computing device 32 of the safety manager may send a message that defines or specifies when certain information should be determined to pertain to the field of view. For instance, the message may define or specify a distance threshold that a worker 10 is from a safety event or potential hazard in which the safety event or potential hazard becomes relevant to the field of view.” Paragraph 0069)
worker restrictions as to different types of worker activities in that particular critical zone, and (Bohannon: “Worker data repository 48C may include identification information of workers 10, PPE required for workers 10, PPE required for various work environments 8, articles of PPE that workers 10 have been trained to use, information pertaining to various sizes of one or more articles of PPE for workers 10, locations of workers, paths workers 10 have followed, gestures or annotations input by workers 10, machine or equipment training of workers 10, location restrictions of workers 10, task lists for specific workers 10, PPE compliance information of workers 10, physiological information of workers 10, motions of workers 10, or the like.” Paragraph 0067)
worker level of attention needed to perform job duties in that particular critical zone. (Bohannon: “Context data of a worker may include, but is not limited to, a unique identifier of a worker, type of worker, role of worker, physiological or biometric properties of a worker, experience of a worker, training of a worker, time worked by a worker over a particular time interval, location of the worker, or any other data that describes or characterizes a worker.” Paragraph 0070; “As yet another example, computing device 32 of the safety manager may send a message that defines or specifies severities, rankings, or priorities of different types of information relating to the field of view. WSMS 6 may receive the message at interface layer 36 which forwards the message to information processor 40B, which may additionally be configured to provide a user interface to specify conditions and actions of rules, receive, organize, store, and update rules included in safety data repository 48B and/or worker data repository 48C, such as rules indicating what information is relevant to a field of view in various cases.” Paragraph 0069)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 3:
Galera does not explicitly describe safety equipment as described below. However, Bohannon teaches the safety equipment as described below.
The cited prior art describes the computer-implemented method of claim 2 further comprising: displaying, by the computer, using the augmented reality device, a safe boundary around the machine for the worker based on the type of safety equipment worn by the worker. (Bohannon: see the indicators 94a, 94b, 94c indicating if the proper PPE is worn by the workers in the area as illustrated in figure 5 and as described in paragraph 0113; “In some examples, an indicator image including a task list may be used for PPE compliance, to enter a work environment or area of a work environment, or the like.” Paragraph 0131) (Galera: “If the user's line of sight encompasses at least a portion of a defined safety or warning field, the rendering component 308 can render the defined safety or warning field (or the portion determined to be within the user's line of sight) as a colored geometric overlay on wearable appliance 206, such that the graphically rendered safety or warning field is located within the user's line of sight to correspond to its actual location and dimensions relative to the protected equipment or sensor.” Paragraph 0212; see the fields 2402a, 2402b, 2402c for the machine 2408 as illustrated in figure 24 and as described in paragraph 0204)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 4:
The cited prior art describes the computer-implemented method of claim 2 further comprising:
monitoring, by the computer, influential factors in the industrial machine environment; (Galera: “Monitoring component 316 can analyze this information to determine whether a temperature of a machine or a mechanical component is excessive.” Paragraph 0147; “In another example, a video capture device 1414 may be mounted at a location near a critical machine so that the machine can be visibly and/or audibly monitored. The monitoring component 316 can be trained to recognize one or more critical performance events based on analysis of the video data 1412 captured for the machine.” Paragraph 0145)
determining, by the computer, that the influential factors will affect an impact of a predicted incident corresponding to the machine; and (Galera: “Monitoring component 316 can be configured to monitor selected subsets of data collected by device interface component 314 according to defined monitoring rules, and to deliver notifications and/or workflow recommendations in response to detecting a maintenance or performance issue based on a result of the monitoring.” Paragraph 0074; “In another example, a video capture device 1414 may be mounted at a location near a critical machine so that the machine can be visibly and/or audibly monitored. The monitoring component 316 can be trained to recognize one or more critical performance events based on analysis of the video data 1412 captured for the machine.” Paragraph 0145)
Galera does not explicitly describe adjusting a zone as described below. However, Bohannon teaches the adjusting a zone as described below.
adjusting, by the computer, at least one of a size or a shape of the particular critical zone of the set of critical zones corresponding to the machine in response to determining that the influential factors will affect the impact of the predicted incident corresponding to the machine. (Bohannon: see the addition of the safety event 86 as illustrated in figure 4; “With the use of AR display device 49 configured to present AR display 80, however, worker 10 may be notified of safety event 86 even if the gas leakage includes a colorless and odorless gas.” Paragraph 0110; “In the example of FIG. 6A, worker 10 may see a safety hazard 126 (e.g., a gas leak) that is not indicated in field of view 122a. For instance, in field of view 122a of FIG. 6a, safety hazard 126 does not include an indicator image or any other information to alert worker 10 of the potentially dangerous situation. AR display 120a may enable worker 10 to use gesture inputs to add indicator images to alert other workers and/or WSMS 6 of safety event 126.” Paragraph 0122; “Although described with respect to safety event 124, gesture inputs may be able to be used for a wide range of scenarios or preform multiple different functions. For example, a gesture input may be used to open information box 98 of FIG. 5. Moreover, gesture inputs may be used add additional or alternative information relating to a field of view in general, another worker, a machine, a potential hazard, an article of PPE, or the like. Moreover, the information added by worker 10 using gesture input 124 may include any suitable information, such as, for example, presence of a safety event or potential hazard, notes about an indicator image or a portion of field of view 122a, 122b, a severity, priority, and/or rank, whether inspection is required, an update to previously added information or indicator image, a status, or the like.” Paragraph 0125)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 6:
Galera and Bohannon does not explicitly describe machine data as described below. However, Vu teaches machine data as described below.
The cited prior art describes the computer-implemented method of claim 1, wherein the aggregate energy includes at least one of potential energy or kinetic energy corresponding to the machine that can cause an incident. (Vu: see the velocity analysis of the various robot components as described in paragraphs 0053, 0060; “RSDM 420 may also determines the instantaneous velocity of robot 402 or any appendage thereof; in addition, knowledge of the robot's instantaneous joint accelerations or torques, or planned future trajectory may be needed in order to determine safe motion constraints for the subsequent cycle as described below.” Paragraph 0053)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 8:
Galera and Bohannon does not explicitly describe machine data as described below. However, Vu teaches machine data as described below.
The cited prior art describes the computer-implemented method of claim 6, wherein the potential energy of the machine is determined based on identifying any component part of the machine that has an ability to release energy. (Vu: see the velocity analysis of the various robot components as described in paragraphs 0053, 0060; “RSDM 420 may also determines the instantaneous velocity of robot 402 or any appendage thereof; in addition, knowledge of the robot's instantaneous joint accelerations or torques, or planned future trajectory may be needed in order to determine safe motion constraints for the subsequent cycle as described below.” Paragraph 0053)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 9:
The cited prior art describes the computer-implemented method of claim 1, wherein a critical zone is a defined area surrounding a component part of the machine within which an incident can occur potentially causing worker injury when the machine is performing its task. (Galera: “One or more embodiments of the presentation system 302 can also be used to view and manage monitored industrial safety zones that are configured to prevent injury by industrial equipment. FIG. 23 is an illustration of an example safety system architecture that uses a defined monitored safety zone 2308 to render an area surrounding an industrial machine 2306 safe.” Paragraph 0202; “If the user subsequently proceeds to the first safety field 2402a, the safety system 502 may disconnect power from the machine 2408 to prevent injury.” Paragraph 0209)
Claim 10:
The cited prior art describes a computer system for identifying critical zones of machines, the computer system comprising: (Galera: “The subject matter disclosed herein relates generally to industrial automation systems, and, more particularly, to management of monitored industrial safety zones.” Paragraph 0002; “FIG. 3 is a block diagram of an example virtual and augmented reality presentation system 302 (also referred to herein as VR/AR presentation system) according to one or more embodiments of this disclosure. Aspects of the systems, apparatuses, or processes explained in this disclosure can constitute machine-executable components embodied within machine(s), e.g., embodied in one or more computer-readable mediums (or media) associated with one or more machines. Such components, when executed by one or more machines, e.g., computer(s), computing device(s), automation device(s), virtual machine(s), etc., can cause the machine(s) to perform the operations described.” Paragraph 0067)
a communication fabric; (Galera: see the networks as illustrated in figures 1, 2)
a storage device connected to the communication fabric, wherein the storage device stores program instructions; and (Galera: see the memory 322 as illustrated in figure 3; “In some embodiments, components 304, 306, 308, 310, 312, 314, and 316 can comprise software instructions stored on memory 322 and executed by processor(s) 320.” Paragraph 0068)
a processor connected to the communication fabric, wherein the processor executes the program instructions to: (Galera: see the processor 320 as illustrated in figure 3; “In some embodiments, components 304, 306, 308, 310, 312, 314, and 316 can comprise software instructions stored on memory 322 and executed by processor(s) 320.” Paragraph 0068)
perform a digital twin simulation of a machine of a plurality of machines in an industrial machine environment using a digital twin computing system; (Galera: see the simulation component 318 as illustrated in figure 3 and as described in paragraphs 0173, 0174; “Simulation component 318 can be configured to simulate execution of multiple control devices, including but not limited to industrial controllers, motor drives, and other such control devices. As more simulated control devices are integrated with the VR/AR model 1804, a digital twin of the physical automation system can be realized. This digital twin can be used to test new control programs on virtualized equipment analogous to their real-world counterparts, perform predictive analytics to estimate asset maintenance or replacement schedules, or other such functions.” Paragraph 0175)
Galera does not explicitly describe historical incident records or identifying machine data as described below. However, Bohannon teaches the historical incident records and Vu teaches identifying machine data as described below.
perform an analysis of a set of historical incident records corresponding to the machine while previously operating in the industrial machine environment to determine areas surrounding the machine in the industrial machine environment that will be impacted by propagation of different types of incidents corresponding to the machine; (Bohannon: see the determination of information for the worker on the AR display 188 based on the safety data repository 48B as illustrated in figure 10 and as described in paragraph 0158; see the safety data repository 48B as illustrated in figure 2 and as described in paragraph 0067; “Based on the information relating to the field of view, such as the information from safety data repository 48B and/or worker data repository 48C, information processor 40B may determine if there are any safety events, hazards, worker information, environment information, machine information, PPE information, or the like to indicate to worker 10 via an AR display of safety glasses 14 (188).” Paragraph 0188; “For example, safety data repository 48B may include data relating to recorded safety events, sensed environmental conditions, worker indicated hazards, machine or equipment statuses, emergency exit information, safe navigation paths, proper PPE use instructions, service life or condition of articles of PPE, horizon or ground level indicators, boundaries, hidden structure information, or the like.” Paragraph 0067)
identify (Vu: see the object monitoring system 410 as illustrated in figure 4 and as described in paragraph 0051) (Galera: “In some embodiments, the VR/AR representation of an industrial factory generated by VR/AR presentation system 302 can be used as the basis for a digital twin of the factory. In such embodiments, the plant models 524 can not only model the physical appearance of industrial assets, but can also model certain behaviors of those assets (e.g., responses to control inputs in terms of movement, speed, temperatures, flows, fill levels, etc.).” paragraph 0173)
a task performed by the machine, (Vu: see the robot state determination 420 as illustrated in figure 4; “A robot state determination module (RSDM) 420 is responsive to data from sensors 102 and signals from the robot 402 and/or robot controller 407 to determine the instantaneous state of the robot. In particular, RSDM 420 determines the pose and location of robot 402 within workspace 400; this may be achieved using sensors 102, signals from the robot and/or its controller, or data from some combination of these sources.” Paragraph 0053)
an operating context of the machine, (Vu: see the object analysis 415 as illustrated in figure 4; “The sensors 102 provide real-time image information that is analyzed by an object-analysis module 415 at a fixed frequency in the manner discussed above; in particular, at each cycle, object analysis module 415 identifies the precise 3D location and extent of all objects in workspace 400 that are either within the robot's reach or that could move into the robot's reach at conservative expected velocities.” Paragraph 0052)
an aggregate energy of the machine, and (Vu: “RSDM 420 may also determines the instantaneous velocity of robot 402 or any appendage thereof; in addition, knowledge of the robot's instantaneous joint accelerations or torques, or planned future trajectory may be needed in order to determine safe motion constraints for the subsequent cycle as described below.” Paragraph 0053)
an area in the industrial machine environment affected by propagation of released energy from the machine (Vu: see the safe action determination 425 as illustrated in figure 4; “One approach to achieving this is to modulate the robot's maximum velocity (by which is meant the velocity of the robot itself or any appendage thereof) proportionally to the minimum distance between any point on the robot and any point in the relevant set of sensed objects to be avoided. The robot is allowed to operate at maximum speed when the closest object is further away than some threshold distance beyond which collisions are not a concern, and the robot is halted altogether if an object is within a certain minimum distance. Sufficient margin can be added to the specified distances to account for movement of relevant objects or humans toward the robot at some maximum realistic velocity. This is illustrated in FIG. 5. An outer envelope or 3D zone 502 is generated computationally by SADM 425 around the robot 504.” Paragraph 0060)
based on
the digital twin simulation of the machine and (Vu: see the training and use of a model to determine robot motions as described in paragraph 0058) (Galera: see the simulation component 318 as illustrated in figure 3 and as described in paragraphs 0173, 0174; “Simulation component 318 can be configured to simulate execution of multiple control devices, including but not limited to industrial controllers, motor drives, and other such control devices. As more simulated control devices are integrated with the VR/AR model 1804, a digital twin of the physical automation system can be realized. This digital twin can be used to test new control programs on virtualized equipment analogous to their real-world counterparts, perform predictive analytics to estimate asset maintenance or replacement schedules, or other such functions.” Paragraph 0175; “In some embodiments, the VR/AR representation of an industrial factory generated by VR/AR presentation system 302 can be used as the basis for a digital twin of the factory. In such embodiments, the plant models 524 can not only model the physical appearance of industrial assets, but can also model certain behaviors of those assets (e.g., responses to control inputs in terms of movement, speed, temperatures, flows, fill levels, etc.).” paragraph 0173)
the analysis of the set of historical incident records corresponding to the machine; and (Vu: see the use of tests and state analysis to train the system for expected operations as described in paragraphs 0057, 0058) (Bohannon: see the determination of information for the worker on the AR display 188 based on the safety data repository 48B as illustrated in figure 10 and as described in paragraph 0158; see the safety data repository 48B as illustrated in figure 2 and as described in paragraph 0067; “Based on the information relating to the field of view, such as the information from safety data repository 48B and/or worker data repository 48C, information processor 40B may determine if there are any safety events, hazards, worker information, environment information, machine information, PPE information, or the like to indicate to worker 10 via an AR display of safety glasses 14 (188).” Paragraph 0188; “For example, safety data repository 48B may include data relating to recorded safety events, sensed environmental conditions, worker indicated hazards, machine or equipment statuses, emergency exit information, safe navigation paths, proper PPE use instructions, service life or condition of articles of PPE, horizon or ground level indicators, boundaries, hidden structure information, or the like.” Paragraph 0067)
identify a set of critical zones corresponding to the machine and (Galera: see the fields 2402a, 2402b, 2402c for the machine 2408 as illustrated in figure 24 and as described in paragraph 0204: “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412. In general, a monitored safety zone comprises the collection of warning fields and/or safety fields that make up the monitored zone. Warning fields (e.g., warning field 2402c) define areas within which detection of an unauthorized object (e.g., a person or a vehicle) will cause a notification or warning to be issued, while safety fields (e.g., safety fields 2404a and 2402b) define areas within which detection of an unauthorized object will cause the controlled equipment to transition to a safe state (e.g., a slow mode, a de-energized state, etc.).” paragraph 0204)
an intensity level of each respective critical zone of the set of critical zones (Galera: “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412. In general, a monitored safety zone comprises the collection of warning fields and/or safety fields that make up the monitored zone. Warning fields (e.g., warning field 2402c) define areas within which detection of an unauthorized object (e.g., a person or a vehicle) will cause a notification or warning to be issued, while safety fields (e.g., safety fields 2404a and 2402b) define areas within which detection of an unauthorized object will cause the controlled equipment to transition to a safe state (e.g., a slow mode, a de-energized state, etc.).” paragraph 0204)
based on the task performed by the machine, the operating context of the machine, the aggregate energy of the machine, and the area in the industrial machine environment affected by propagation of released energy from the machine. (Vu: see the zones 502, 508, 508 for robot operation based on the determinations by the SADM 425 using object analysis 415, robot state determination 420, and safe action determination 425 as illustrated in figures 4, 5; “The safe-action constraints identified by SADM 425 may be communicated by OMS 410 to robot controller 407 on each cycle via a robot communication module 430.” Paragraph 0064)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 11:
The cited prior art describes the computer system of claim 10, wherein the processor further executes the program instructions to:
detect, using a set of sensors, that a worker is in a particular critical zone of the set of critical zones corresponding to the machine; (Galera: see the safety sensing device 2412 as illustrated in figure 24; “In this example, a safety sensing device 2412 (e.g., a laser scanner, a 3D or 2D optical safety sensor, etc.) has been installed in front of a hazardous machine 2408, and multiple nested fields—one warning field 2402c and two safety fields 2402a and 2402b—have been defined relative to the safety sensing device 2412.” Paragraph 0204; “although only one safety sensor device 2412 is depicted in FIG. 25, in some implementations multiple safety sensor devices may be installed and configured to monitor a given hazardous machine or area, with each safety sensor device having one or more defined safety and/or warning fields which may or may not overlap with safety and/or warning fields of other safety sensor devices” paragraph 0210)
display the particular critical zone corresponding to the machine to the worker via an augmented reality device worn by the worker; and (Galera: see the wearable device 206 as illustrated in figure 24; “To assist with configuration and identification of safety or warning fields within the plant facility, some embodiments of the presentation system 302 described herein can render defined graphical representations of safety and warning fields via wearable appliance 206 or a hand-held client device as graphical overlays of an AR presentation.” Paragraph 0212; “Also, in addition to the safety and warning actions described above, intrusion into a warning or safety field by a user can cause the system to render the graphical representation of the safety or warning field at a different brightness level relative to the default brightness level applied when the user is not located within the field, indicating to the user that he or she is currently within a defined safety or warning field. Additionally, some embodiments can be configured to deliver additional information to the user's wearable appliance or client device in response to determining that the user is currently within a warning or safety field, where the additional information is relevant to the particular field within which the user intrudes. This information can include, for example, information regarding a safety action being performed on the machine in response to the user's intrusion into a safety field, an instruction to vacate the monitored safety zone, a haptic signal directed to the wearable appliance intended to draw the user's attention to the fact that he or she has entered a monitored zone, or other such information.” Paragraph 0208)
Galera does not explicitly describe safety equipment as described below. However, Bohannon teaches the safety equipment as described below.
alert, using the augmented reality device, the worker of a type of safety equipment to be worn by the worker in the particular critical zone of the set of critical zones corresponding to the machine based on (Bohannon: see the alert for gloves (i.e., PPE) as illustrated in figure 5 and as described in paragraph 0117)
the intensity level of that particular critical zone, (Bohannon: “For example, computing device 32 of the safety manager may send a message that defines or specifies the one or more articles of PPE required for a specific job function, for a specific environment 8, for a specific worker 10A, or the like. As another example, computing device 32 of the safety manager may send a message that defines or specifies when certain information should be determined to pertain to the field of view. For instance, the message may define or specify a distance threshold that a worker 10 is from a safety event or potential hazard in which the safety event or potential hazard becomes relevant to the field of view.” Paragraph 0069)
worker restrictions as to different types of worker activities in that particular critical zone, and (Bohannon: “Worker data repository 48C may include identification information of workers 10, PPE required for workers 10, PPE required for various work environments 8, articles of PPE that workers 10 have been trained to use, information pertaining to various sizes of one or more articles of PPE for workers 10, locations of workers, paths workers 10 have followed, gestures or annotations input by workers 10, machine or equipment training of workers 10, location restrictions of workers 10, task lists for specific workers 10, PPE compliance information of workers 10, physiological information of workers 10, motions of workers 10, or the like.” Paragraph 0067)
worker level of attention needed to perform job duties in that particular critical zone. (Bohannon: “Context data of a worker may include, but is not limited to, a unique identifier of a worker, type of worker, role of worker, physiological or biometric properties of a worker, experience of a worker, training of a worker, time worked by a worker over a particular time interval, location of the worker, or any other data that describes or characterizes a worker.” Paragraph 0070; “As yet another example, computing device 32 of the safety manager may send a message that defines or specifies severities, rankings, or priorities of different types of information relating to the field of view. WSMS 6 may receive the message at interface layer 36 which forwards the message to information processor 40B, which may additionally be configured to provide a user interface to specify conditions and actions of rules, receive, organize, store, and update rules included in safety data repository 48B and/or worker data repository 48C, such as rules indicating what information is relevant to a field of view in various cases.” Paragraph 0069)
Galara, Bohannon, and Vu are combinable for the same rationale as set forth above with respect to claim 1.
Claim 12:
Galera does not explicitly describe safety equipment as described below. However, Bohannon teaches the safety equipment as described below.
The cited prior art describes the computer system of claim 11, wherein the processor further executes the program instructions to: display, using the augmented reality device, a safe boundary around the machine for the worker based on the type of safety equipment worn by the worker. (Bohannon: see the indicators 94a, 94b, 94c indicating if the proper PPE is worn by the workers in the area as illustrated in figure 5 and as described in paragraph 0113; “In some examples, an indicator image including a task list may be used for PPE compliance, to enter a work environment or area of a work environment, or the like.” Paragraph 0131) (Galera: “If the user's line of sight encompasses at least a portion of a defined safety or warning field, the rendering component 308 can render the defined safety or warning field (or the portion determined to be within the user's line of sight) as a colored geometric overlay on wearable appliance 206, such that the graphically rendered safety or warning field is located within the user's line of sight to correspond to its actual location and dimensions relative to the protected equipment or sensor.” Paragraph 0212; see the fields 240