DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Comments
The Amendment – After Non-Final Rejection filed on July 10, 2026 has been entered and made of record.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (U.S. Pub. No. 2023/0196895) in view of Whelan et al. (U.S. Pub. No. 2018/0033276).
As to claims 1, 15 and 16, Lee et al. teaches a management apparatus (i.e., “site monitoring device 100”, Paragraph [0043]) comprising:
a memory configured to store instructions (i.e., “medium may include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and hardware devices specially configured to store and execute program instructions”, Paragraph [0123]); and
a processor configured to execute the instructions to (i.e., “computer program that can be executed on a computer”, Paragraph [0123])/a management method (i.e., “method for monitoring a state of wearing safety protective equipment”, Abstract) comprising causing a computer to execute/a non-transitory computer-readable medium (i.e., “computer program that can be executed on a computer through various components, and such a computer program may be recorded on a computer-readable medium”, Paragraph [0123]) storing a program for causing a computer to execute a management method including:
detect/detecting a movement (i.e., “The dangerous work is separately classifying certain types of work with a high risk of accidents, such as a work handling chemical products, and information about the type of work classified as the dangerous work may be set in advance and stored in the database 250”, Paragraph [0070]) of a target (i.e., “In the present specification, “workers” refer to all workers identified in an image unless otherwise specified, and a “target worker” refers to a worker to be monitored among the workers”, Paragraph [0046]) from an image obtained by capturing a predetermined place (i.e., “site monitoring device 100 may obtain images of a worker in a monitoring area of an operation site and an operation environment in real time. The images of the operation environment may be, for example, photographed images of a surrounding environment in which the worker is working, safety protective equipment worn by the worker, buildings and work facilities located at the work site, etc.”, Paragraph [0045]); and
specify/specifying a related image showing a predetermined object or area related to safety of the target from the image (i.e., “The monitoring area is a target area for the control module 230 to determine whether a dangerous task is performed, and may be set by dividing into an area entering a dangerous work and an area where the dangerous work is performed”, Paragraph [0067]; and “The monitoring area may be set in advance and stored in the database 250”, Paragraph [0069]).
However, Lee et al. does not explicitly disclose the processor configured to execute the instructions to: determine/determining that the target performing the movement is in a safe situation in a case where the movement in a caution area associated with the related image is a predetermined motion; determine/determining that the target performing the movement is not in a safe situation in a case where the movement in the caution area is not the predetermined motion; and output/outputting determination information including a result determining whether or not the target performing the movement is in a safe situation.
Whelan et al. teaches a processor that is configured to execute instructions to (i.e., “Processor unit 1704 serves to execute instructions for software that may be loaded into memory 1706”, Paragraph [0135]): determine/determining that the target performing the movement (i.e., “sensor 106, sensor 108, and sensor 110 are used to detect humans and human movement in proximity to machine 102 or machine 104”, Paragraph [0036]) is in a safe situation in a case where the movement in a caution area associated with the related image is a predetermined motion (See for example, “the system determines whether the objects movements detected by the sensor are justified (operation 518)”, Paragraph [0062]; and “A movement is considered justified if the computer has been pre-programmed to recognize a movement as being justified. For example, the detected skeletal positions of a human body may indicate that the person is performing a feeding motion with respect to feeding a machine. This motion is considered part of the normal operation of the machine and thus is justified”, Paragraph [0063]);
determine/determining that the target performing the movement is not in a safe situation (i.e., “the term “undesirable” means a movement which places the human at risk for interacting with the machinery in a manner which could result in injury to the human or damage to the machinery”, Paragraph [0063]) in a case where the movement in the caution area is not the predetermined motion (i.e., “All motions are considered unjustified except for those motions defined as justified, such as for example inserting a hand or other body part into an operating part of the machine or walking into an unauthorized area. Furthermore, just as the system can be trained to recognize justified movements, it can also be training to recognize “undesirable” movements, which like unjustified movements”, Paragraph [0063]); and
output/outputting determination information including a result determining whether or not the target performing the movement is in a safe situation (See for example, “recognize “undesirable” movements, which like unjustified movements, would trigger an alarm (operation 520)”, Paragraph [0063]; and “notifying the system that the motion is justified at operation 522”, Paragraph [0064]).
Lee et al. and Whelan et al. are analogous art because they are from the field of digital image processing for monitoring working environments.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Lee et al. by incorporating the determining that the target performing the movement is in a safe situation in a case where the movement in a caution area associated with the related image is a predetermined motion, determining that the target performing the movement is not in a safe situation in a case where the movement in the caution area is not the predetermined motion, and outputting determination information including a result determining whether or not the target performing the movement is in a safe situation, as taught by Whelan et al.
The suggestion/motivation for doing so would have been to improve the safety of both, humans and equipment, when humans are physically within the environment in which machines operate.
Therefore, it would have been obvious to combine Whelan et al. with Lee et al. to obtain the invention as specified in claims 1, 15 and 16.
As to claim 2, Lee et al. teaches wherein the processor is configured to execute the instructions to detect the movement similar to a predetermined registered movement (i.e., “database 250 may store various information necessary to check the safety of the operation site, such as image data received from the site monitoring device 100, information about the monitoring area, information about types of tasks classified as the dangerous task”, Paragraph [0080]; and “central management server 200 may learn images when the dangerous work is performed and when the dangerous work is not performed through the artificial intelligence network, and determine whether the dangerous work is being performed using the learned images”, Paragraph [0088]).
As to claim 3, Lee et al. does not explicitly disclose wherein the processor is configured to execute the instructions to detect the movement from skeleton data regarding a structure of a body of the target extracted from an image including the target.
Whelan et al. teaches the processor is configured to execute the instructions to detect the movement from skeleton data regarding a structure of a body of the target extracted from an image including the target (i.e., “human motion sensor 300 detects the three-dimensional coordinates of the subject's movements in the form of a stick man or skeleton 304. This type of identification may be referred-to as skeletal tracking”, Paragraph [0048]).
Therefore, in view of Whelan et al., it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee et al. by incorporating the detecting of the movement from skeleton data regarding a structure of a body of the target extracted from an image including the target, as taught by Whelan et al., in order to provide a system that is flexible and relatively inexpensive to use.
As to claim 4, Whelan et al. teaches wherein the processor is configured to execute the instructions to detect the movement by comparing the skeleton data related to the movement with the skeleton data as the predetermined registered movement based on a form of each element forming the skeleton data (See for example, “when the computer is detecting either human positions or human movements or both, the computer compares specific patterns of these text descriptions to known patterns for which the computer or the software are trained. In a more specific example, when HAND_LEFT 612 is “NEAR_HEAD” and “HIGH” the computer is trained to return a result that the human may be holding a mobile phone to his or her head, and take action accordingly”, Paragraph [0076]).
As to claim 5, Lee et al. teaches wherein the processor is configured to execute the instructions to: detect a type of the detected movement based on the predetermined registered movement (See for example, “The dangerous work is separately classifying certain types of work with a high risk of accidents, such as a work handling chemical products, and information about the type of work classified as the dangerous work may be set in advance and stored in the database 250”, Paragraph [0070]); and determine whether or not the target is in a safe situation based on the type of the motion detected movement and a positional relationship between the target and the predetermined object or the area indicated by the related image (i.e., “detects a target worker when the work performed in the monitoring area corresponds to the dangerous work (S305) as a result of determining whether the work performed in the monitoring area corresponds to the dangerous work”, Paragraph [0089]; and “determine the type of safety protective equipment required for the target worker based on the type of the dangerous work determined in step S303”, Paragraph [0094]; and “determines the state of the target worker wearing the safety protective equipment (S311)”, Paragraph [0097]).
As to claim 6, Lee et al. does not explicitly disclose wherein the processor is configured to execute the instructions to detect the movement from a posture change extracted in time series from each of a plurality of images captured at a plurality of different times.
Whelan et al. teaches the processor is configured to execute the instructions to detect the movement from a posture change extracted in time series from each of a plurality of images captured at a plurality of different times (i.e., “manufacturing environment 100 may be characterized as a human motion sensor or a sensor system used to detect the change in position of a human relative to his or her surroundings”, Paragraph [0038]; and “human motion sensor 300 detects the three-dimensional coordinates of the subject's movements in the form of a stick man or skeleton 304. This type of identification may be referred-to as skeletal tracking”, Paragraph [0048]).
Therefore, in view of Whelan et al., it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee et al. by incorporating the detecting of the movement from a posture change extracted in time series from each of a plurality of images captured at a plurality of different times, as taught by Whelan et al., in order to learn a safe or acceptable movement, and also learn or recognize an undesirable movement.
As to claim 7, Lee et al. teaches wherein the memory is configured to store safety standard data related to a positional relationship between the target related to the detected movement and the related image (See for example, “information about the safety protective equipment required for the head area and the body area according to the type of the dangerous task”, Paragraph [0080]), wherein the processor is configured to execute the instructions to determine whether or not the target is safe with reference to the safety standard data (i.e., “types of safety protective equipment required for the target worker are not all included in the safety protective equipment worn by the target worker and/or when the safety protective equipment is detected in an incorrect location”, Paragraph [0078]).
As to claim 8, Lee et al. teaches wherein the processor is configured to execute the instructions to: specify a predetermined object worn on a body of the target as the related image (See for example, Paragraph [0072]; “safety protective equipment images of different sizes”, Paragraph [0073]; and “detect the safety protective equipment worn by the worker by using human body modeling data of the worker stored in the database 250 and information about the safety protective equipment detectable from each of the head and body areas”, Paragraph [0075]); and
determine that the target is not safe in a case where a position of the specified predetermined object does not correspond to a predetermined position of the target related to the detected movement (i.e., “types of safety protective equipment required for the target worker are not all included in the safety protective equipment worn by the target worker and/or when the safety protective equipment is detected in an incorrect location”, Paragraph [0078]).
As to claim 9, Whelan et al. does not explicitly disclose wherein the processor is configured to execute the instructions to: specify an object having a predetermined dangerous area as the related image; and determine that the target is not safe in a case where there is the target related to a motion different from a predetermined motion permitted in the dangerous area.
Whelan et al. teaches wherein the processor is configured to execute the instructions to: specify an object having a predetermined dangerous area as the related image (i.e., “computer allows the user to adjust the scan area of the motion detector or motion sensor, as shown at arrow 904. In this manner, skeletal tracker software 906 only picks up objects at a certain proximity of an operating machine”, Paragraph [0089]); and determine that the target is not safe in a case where there is the target related to a motion different from a predetermined motion permitted in the dangerous area (i.e., “Once the system is notified that the detected object, which is not static, is not at an acceptable distance from the machine, the system determines whether the objects movements detected by the sensor are justified”, Paragraph [0062]; and Paragraph [0063]).
Therefore, in view of Whelan et al., it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee et al. by incorporating the specifying of an object having a predetermined dangerous area as the related image, and determining that the target is not safe in a case where there is the target related to a motion different from a predetermined motion permitted in the dangerous area, as taught by Whelan et al., in order to prevent a human from being hurt while creating a safe work environment.
As to claim 10, Lee et al. teaches wherein the processor is configured to execute the instructions to: specify a predetermined determination area as the related image (i.e., “The monitoring area is a target area for the control module 230 to determine whether a dangerous task is performed, and may be set by dividing into an area entering a dangerous work and an area where the dangerous work is performed”, Paragraph [0067]; and “The monitoring area may be set in advance and stored in the database 250”, Paragraph [0069]); and
determine whether or not the target is safe (See for example, Paragraphs [0097]-[0103]) based on a positional relationship between the target related to the detected movement and the predetermined determination area (See for example, “determines whether a work performed in a monitoring area corresponds to a dangerous work by analyzing the image received from the site monitoring device 100 (S303)”, Paragraph [0085]).
As to claim 11, Lee et al. does not explicitly disclose wherein the processor is configured to execute the instructions to: detect the movement of each of a plurality of the targets from an image obtained by capturing the predetermined place including the plurality of targets; and determine whether or not the detected target is safe based on a positional relationship between each of the plurality of targets and the related image.
Whelan et al. teaches the processor is configured to execute the instructions to: detect the movement of each of a plurality of the targets from an image obtained by capturing the predetermined place including the plurality of targets (See for example, “identify a plurality of movements of each of the persons in the manufacturing environment”, Paragraph [0068]; and FIG. 10, Paragraphs [0092]-[0096]); and
determine whether or not the detected target is safe based on a positional relationship between each of the plurality of targets and the related image (i.e., “workers may need to feed parts to a machine, as shown in FIG. 10 at arrow 1002. In order to address this concern, the computer uses “movement recognition” to identify movements that are predetermined to be justified or safe … if an unauthorized movement, such as that of a human walking at an unauthorized space near a machine as indicated at arrow 1004, was detected, then the computer would activate an alarm or modify operation of the machine. In the same way, the computer can learn a safe or acceptable movement; it can also learn or recognize an undesirable movement, as indicated by arrow 1006. An example of an undesirable movement may be walking while talking on a mobile phone, or perhaps carrying something, or any other movement predetermined to be undesirable”, Paragraphs [0093]-[0094]).
Therefore, in view of Whelan et al., it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee et al. by incorporating the detecting of the movement of each of a plurality of the targets from an image obtained by capturing the predetermined place including the plurality of targets, and determining whether or not the detected target is safe based on a positional relationship between each of the plurality of targets and the related image, as taught by Whelan et al., in order to provide a safe work environment in which a plurality of people simultaneously work with a plurality of machines.
As to claim 12, Lee et al. teaches wherein the processor is configured to execute the instructions to output a predetermined warning signal in a case where it is determined that the target is not safe (i.e., “When the state of wearing the safety protective equipment is inappropriate, the central management server 200 transmits an alarm to the output device 300 (S313)”, Paragraph [0103]).
As to claim 13, Lee et al. does not explicitly disclose wherein the apparatus has a plurality of safety levels for determining whether or not the target is safe, and the processor is configured to execute the instructions to output the warning signal corresponding to a safety level in the plurality of safety levels.
Whelan et al. teaches the apparatus has a plurality of safety levels for determining whether or not the target is safe, and the processor is configured to execute the instructions to output the warning signal corresponding to a safety level in the plurality of safety levels (See for example, “users can assign a severity, as indicated by scale 908, to the scanner’s range as well. The computer can cause the severity to increase the closer an object gets. In these cases, the computer could issue a warning or modify operation of the machinery when the severity reaches a predetermined range. Additionally, different actions could be taken at different measurements of severity, from issuing an alert at a first severity, to modification of the machine at a second higher severity, to halting of the machine at a third even higher severity. Furthermore, the computer could initiate proximity alerts as a subject nears an operating machine”, Paragraph [0090]).
Therefore, in view of Whelan et al., it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee et al. by incorporating the apparatus has a plurality of safety levels for determining whether or not the target is safe, and the processor is configured to execute the instructions to output the warning signal corresponding to a safety level in the plurality of safety levels, as taught by Whelan et al., in order to provide different actions to be taken according to the target proximity while operating a machine.
As to claim 14, Lee et al. teaches wherein the processor is configured to execute the instructions to: specify the target included in an image (i.e., “the object detection network may be used for the control module 230 to detect the target worker, a head area, and a body area of the target worker when a dangerous task is performed in a monitoring area”, Paragraph [0062]); and
output the warning signal corresponding to the specified target in a case where the specified target is not safe (i.e., “the central management server 200 may determine the state of the target worker wearing the safety protective equipment at every moment when the target worker performs the work using an artificial intelligence network”, Paragraph [0100]; and “When the state of wearing the safety protective equipment is inappropriate, the central management server 200 transmits an alarm to the output device 300”, Paragraph [0103]).
Response to Arguments
Claim Objections
With respect to claims 1-7, 10, 11, 15, and 16, Applicant has amended the claims in order to correct for minor informalities. Therefore, the objections have been withdrawn.
Claim Rejections - 35 USC § 112
With respect to claims 8, 9, 11, and 13, Applicant has amended the claims in order to particularly point out and distinctly claim the subject matter which the inventor regards as the invention. Therefore, the rejections have been withdrawn.
Claim Rejections - 35 USC §§ 102 and 103
With respect to claims 1-16, Applicant’s arguments (Remarks dated July 10, 2026, pages 9-10) have been fully considered. With respect to claim 1, Applicant respectfully submits that “the cited references fail to disclose or suggest each and every element of claim 1. For example, Lee, taken individually or in combination with the additionally cited references, fail to disclose or suggest the above-amended features of claim 1, when considered with the other elements of the claim” (Remarks dated July 10, 2026, pages 9-10). Examiner respectfully disagrees.
With respect to claim 1, a new ground(s) of rejection is made in view of Lee et al. and Whelan et al. as stated above (Refer to Claim Rejections - 35 USC § 103 Section above). Therefore, the claims are not patentable over cited art of record.
Conclusion
Applicant’s amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE M TORRES whose telephone number is (571)270-1356. The examiner can normally be reached Monday thru Friday; 10:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSE M TORRES/Examiner, Art Unit 2664 09/18/2026
/PING Y HSIEH/Primary Examiner, Art Unit 2664
Prosecution Insights