DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over El Essaili et al. (U.S. Publication No. 2023/0105603 A1) in view of Anderson et al. (U.S. Publication No. 2017/0352282 A1).
Regarding Claim 1:
El Essaili et al. teaches a work management apparatus for managing work performed by a worker, comprising: a motion recognition section configured to recognize a motion of the worker by using a measurement result of a sensor; (Paragraph [0032], human behavior is detected in the environment using for example hand tracking)
an object recognition section configured to recognize, by using the measurement result of the sensor or data acquired from an object used by the worker, an operation status of the object; (Paragraph [0034], object detection and recognition)
a work recognition section configured to recognize content of work of the worker by using a recognition result of each of the motion recognition section and the object recognition section; (Paragraph [0035], database has the exact sequence of manipulating steps and uses the tracking to determine if the user is manipulating the object correctly)
a control section configured to control the motion recognition section, the object recognition section, and the work recognition section; (Paragraph [0034], monitoring unit)
El Essaili et al. does not teach and a start/completion judgment section configured to identify a start time at which the worker has started the work and a completion time at which the worker has completed the work, according to the content of the work identified by the work recognition section, wherein the control section confirms whether or not there is any missing content of the work on a basis of the start time identified by the start/completion judgment section and the completion time identified by the start/completion judgment section and outputs a confirmation result thereof.
However, Anderson et al. teaches and a start/completion judgment section configured to identify a start time at which the worker has started the work and a completion time at which the worker has completed the work, according to the content of the work identified by the work recognition section, (Paragraph [0031] and [0040], time taken to perform an assembly step is tracked. If the time taken to perform a step is tracked, a start and end time must have been collected)
wherein the control section confirms whether or not there is any missing content of the work on a basis of the start time identified by the start/completion judgment section and the completion time identified by the start/completion judgment section and outputs a confirmation result thereof. (Paragraph [0032] and [0040], time taken can be used to determine if there was an issue during an assembly step; Additionally, issue can be that a long time was taken to complete the step or the step was never completed)
It would have been obvious to one of ordinary skill in the art, at the effective filing date of the claimed invention, to modify El Essaili et al.’s tracking of a user and object during assembly with Anderson et al.’s tracking of time taken to compete a step in order to use the time taken to ensure no problems arise during the assembly. One would be motivated to combine these teachings in order to apply a known technique (time tracking during assembly) to a known device (system that tracks a user and object during assembly) ready for improvement to yield predictable results (track the time taken to ensure no problems arise during the assembly).
Regarding Claim 9:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 1, further comprising: a sensor switching section configured to output a sensor switching instruction for switching a type of the sensor, wherein the control section acquires a work instruction for the worker, and the sensor switching section configures the sensor switching instruction according to the work instruction. (El Essaili et al. Paragraph [0033], sensor may be located on a headset, in an area around the user, or connected to gloves that the user is wearing)
Regarding Claim 10:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 9, wherein, in a case where the sensor type is specified by the work instruction, the sensor switching section configures the sensor switching instruction according to the specification, and in a case where the sensor type is not specified by the work instruction, the sensor switching section configures the sensor switching instruction in such a manner that a default sensor is specified. (El Essaili et al. Paragraph [0033], sensor may be located on a headset, in an area around the user, or connected to gloves that the user is wearing)
Regarding Claim 11:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 10, wherein the sensor switching section acquires a past work history of the worker, and in a case where there is, among sensors used in the work history, a past sensor for which a rate of accurate acquisition of the start time and the completion time by the start/completion judgment section is equal to or larger than a reference value, the sensor switching section configures the sensor switching instruction in such a manner that the past sensor is used instead of a sensor of the type specified by the work instruction. (Anderson et al. Paragraph [0019], intelligent hypotheses can be made about the type of objects that are likely to be captured for a given assembly step)
Regarding Claim 12:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 1, further comprising: a model switching section configured to output a model switching instruction for switching recognition models that prescribe respective pieces of recognition processing performed by the motion recognition section, the object recognition section, and the work recognition section, wherein the control section acquires a work instruction for the worker, and in a case where a type of the sensor is specified by the work instruction, the model switching section configures the model switching instruction in such a manner that the recognition model corresponding to the sensor type is used. (Anderson et al. Paragraph [0019], intelligent hypotheses can be made about the type of objects that are likely to be captured for a given assembly step; Additionally, El Essaili et al. Paragraph [0033], sensor may be located on a headset, in an area around the user, or connected to gloves that the user is wearing and El Essaili et al. Paragraph [0034], artificial intelligence can be used for object detection and recognition, a different model would be required depending on what is being detected and where the sensors are located)
Regarding Claim 13:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 12, wherein the model switching section creates a schedule for performing the model switching instruction, in such a manner that the recognition models are switched synchronously between the motion recognition section, the object recognition section, and the work recognition section. (Anderson et al. Paragraph [0019], intelligent hypotheses can be made about the type of objects that are likely to be captured for a given assembly step; Additionally, El Essaili et al. Paragraph [0033], sensor may be located on a headset, in an area around the user, or connected to gloves that the user is wearing and El Essaili et al. Paragraph [0034], artificial intelligence can be used for object detection and recognition, a different model would be required depending on what is being detected and where the sensors are located)
Regarding Claim 14:
The combination of El Essaili et al. and Anderson et al. additionally teaches the work management apparatus according to claim 12, wherein, when a new recognition model is acquired according to the model switching instruction, the model switching section creates a schedule for performing processing of acquiring the new recognition model so as to ensure that there is no lack of free space in a storage device for storing the recognition model. (El Essaili et al. Paragraph [0034], artificial intelligence can be used for object detection and recognition, a different model would be required depending on what is being detected and where the sensors are located; Additionally, Anderson et al. Paragraph [0019], local storage is used to store assembly step information)
Claims 2-8 are rejected under 35 U.S.C. 103 as being unpatentable over El Essaili et al. (U.S. Publication No. 2023/0105603 A1) in view of Anderson et al. (U.S. Publication No. 2017/0352282 A1) as applied to claims 1 and 9-14 above, and further in view of Nishimura et al. (U.S. Publication No. 2018/0204148 A1).
Regarding Claim 2:
The combination of El Essaili et al. and Anderson et al. teaches the work management apparatus according to claim 1.
The combination of El Essaili et al. and Anderson et al. does not teach wherein, in a case where at least one of the start time identified by the start/completion judgment section and the completion time identified by the start/completion judgment section is lost, the control section complements the lost time, and the control section writes and outputs the complemented time as part of the confirmation result.
However, Nishimura et al. teaches wherein, in a case where at least one of the start time identified by the start/completion judgment section and the completion time identified by the start/completion judgment section is lost, the control section complements the lost time, and the control section writes and outputs the complemented time as part of the confirmation result. (Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
It would have been obvious to one of ordinary skill in the art, at the effective filing date of the claimed invention, to modify El Essaili et al. and Anderson et al.’s tracking of assembly steps and time with Nishimura et al.’s correcting of the time taken in order to correct the time tracked for the assembly steps. One would be motivated to combine these teachings in order to correct abnormal work times when the actual work time is not abnormal (Nishimura et al. Paragraph [0063]).
Regarding Claim 3:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 2, wherein the control section acquires the start time and the completion time for each of content of work for a first work process and content of work for a second work process that follows the first work process, and in a case where the start time of the first work process is acquired, the completion time of the first work process is lost, and then the start time of the second work process is acquired, the control section complements the lost completion time of the first work process. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Regarding Claim 4:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 2, wherein the control section acquires the start time and the completion time for each of content of work for a first work process and content of work for a second work process that follows the first work process, and in a case where the completion time of the first work process is acquired, the start time of the second work process is lost, and then the completion time of the second work process is acquired, the control section complements the lost start time of the second work process. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Regarding Claim 5:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 2, wherein the control section acquires the start time and the completion time for each of content of work for a first work process and content of work for a second work process that follows the first work process, in a case where the completion time of the first work process is lost, the control section complements the lost completion time of the first work process in such a manner that the completion time of the first work process is equal to or earlier than the start time of the second work process, and in a case where the start time of the second work process is lost, the control section complements the lost start time of the second work process in such a manner that the completion time of the first work process is equal to or earlier than the start time of the second work process. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Regarding Claim 6:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 2, wherein the control section acquires the start time and the completion time for each of content of work for a first work process and content of work for a second work process that is similar in type to the first work process and follows the first work process, the control section calculates a first time length of the first work process on a basis of the start time and the completion time of the first work process, the control section calculates a second time length of the second work process on a basis of the start time and the completion time of the second work process, and in a case where a difference between the first time length and the second time length is equal to or larger than a threshold, the control section complements at least one of the start time and the completion time for whichever is longer of the first work process and the second work process. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Regarding Claim 7:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 6, wherein, in a case where the difference between the first time length and the second time length is equal to or larger than the threshold, the control section complements a loss by substituting the start time of a next process for the lost completion time or by substituting the completion time of the next process for the lost start time, for whichever is longer of the first work process and the second work process. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Regarding Claim 8:
The combination of El Essaili et al., Anderson et al. and Nishimura et al. additionally teaches the work management apparatus according to claim 2, wherein the control section acquires an order of the content of the work, and the control section judges whether or not both the start time identified by the start/completion judgment section and the completion time identified by the start/completion judgment section are lost, by comparing the content of the work recognized by the work recognition section with the order. (Nishimura et al. Paragraph [0040], if the break time was 12:00 to 13:00 but a process ended at 12:01, break start time can be corrected to 12:01; Additionally, Nishimura et al. Paragraph [0042], if end time of process A and start of process B has a space between, system knows that the break occurred then)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Shirakawa (U.S. Publication No. 2021/0287010 A1) discloses outputting the likelihood of occurrence for an action from time-series images.
Suzuki (U.S. Publication No. 2021/0110165 A1) discloses gathering object information from a plurality of videos.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER DEAN HEDRICK whose telephone number is (571)272-5803. The examiner can normally be reached Monday-Friday: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Lee can be reached at (571) 272-3667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.D.H./Examiner, Art Unit 2115
/THOMAS C LEE/Supervisory Patent Examiner, Art Unit 2115