DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/05/2024 and 11/05/2024 is/are compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Office Action Summary
Claim(s) 1-13 and 15-17 is/are interpreted under 35 USC 112(f).
Claim(s) 1-4, 6-7, 9, and 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shinya et al (JP2021086218A; See translation provided by Examiner) in view of Itoh et al (US 2022/0012514 A1).
Claim(s) 5, 8, and 10-12 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “acquisition unit” in claim(s) 1, 13, 15, and 17, “determination unit” in claim(s) 1-7, 15 and 17, “display control unit” in claim(s) 7-12, “transmission unit” in claim(s) 12, “imaging unit” in claim(s) 16-17, “detection unit” in claim(s) 16-17, and “output control unit” in claim(s) 16-17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6-7, 9, and 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shinya et al (JP2021086218A; See translation provided by Examiner) in view of Itoh et al (US 2022/0012514 A1).
Regarding claim(s) 1 and 15, Shinya teaches an action analysis program causing a computer (Paragraph [0015]: “The analysis program uses a computer to analyze the behavior of the worker and the collaborative work robot to calculate the work time required for the work of each process, and between the worker and the collaborative work robot at an arbitrary position in the process”; and Paragraph [0072] – Paragraph [0073]: “the analysis device 200 includes […] work time calculation unit that calculates the work time required […] by analyzing the behaviors […]”) to function as:
an acquisition unit that acquires behavior data indicating a behavior of an object during an operation process, which has been recognized by a model (Paragraph [0069] – Paragraph [0071]: “the entire process 40 composed of a plurality of processes is regarded as the highest unit, and the process 41 of each process is composed of a plurality of operations. Each work of 41 in the process can be decomposed into one or a plurality of element work 42. Each of the elemental operations 42 can be decomposed into one or more operations 43. Each of the movements 43 can be decomposed into one or more unit movements 44”; and Paragraph [0077] – Paragraph [0082]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed […] The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time”); and
a determination unit that determines a required time for a process corresponding to the behavior data based on the behavior data acquired by the acquisition unit (Paragraph [0073]: “The worker element work recognition unit 250 and the robot element work recognition unit 260 correspond to a work time calculation unit that calculates the work time required for the work of each process by analyzing the behaviors of the worker and the robot”; Paragraph [0075]: “robot element work recognition unit 260 analyzes the start and end timings of each element work performed by the robot in the process, and the time required for the element work”; Paragraph [0102]: “Based on the movement of the worker 20 or the status and position of the robot 300, the start and end timings for each element work are determined (step S116), and the work time for each element work is calculated (step S118)”; and Paragraph [0106]: “The start time column 704 and the end time column 705 indicate the start and end times of the corresponding element work. The working time 706 indicates the time required for the corresponding element work”).
Shinya fails to teach to recognized by a model preliminarily learned for recognizing the object. However, Itoh teach to recognized by a model preliminarily learned for recognizing the object (Figure 1; Figure 4; and Paragraph [0106]: “An identification information assignment apparatus of the present disclosure includes an acquirer configured to acquire a plurality of pieces of image data, an assigner configured to assign identification information to image data selected from the plurality of pieces of image data by using a learning model after learning, and an updater configured to update the learning model using the image data to which the identification information is assigned, in which the assigner assigns identification information to the rest of the image data acquired by the acquirer using the learning model that has been updated”).
Shinya discloses an analysis device that acquires behavior data of a worker and/or a collaborative robot during execution of a manufacturing process and determines a required work time corresponding to each process based on the analyzed behavior, including determining start and end timings of element work and calculating the time required for the process. Shinya further teaches recognizing behavior using a trained model generated in advance by machine learning. Itoh teaches employing a pretrained learning model as a core component for recognizing posture or behavior from image data in a work analysis system.
Therefore, it would have been obvious to one of ordinary skill in the art to combine Shinya and Itoh before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the accuracy and reliability of behavior acquisition, thereby arriving at the claimed action analysis device configured to determine a required time for a process based on behavior data recognized using a preliminarily learned model. This motivation for the combination of Shinya and Itoh is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 2, Shinya as modified by Itoh teaches the action analysis device according to claim 1, where Shinya teaches wherein, based on time information set as the required time for the process (Figure 10; Figure 11; Paragraph [0076]: “the type of element work included in each process is determined in advance in the process design”; and Paragraph [0084]: “The process database 254 includes information on the element work included in each process and the operation corresponding to each element work […]”), the determination unit determines a required time for a process corresponding to the behavior data (Paragraph [0073]; Paragraph [0075]; Paragraph [0102]; and Paragraph [0106]).
Regarding claim(s) 3, Shinya as modified by Itoh teaches the action analysis device according to claim 2, where Shinya teaches wherein the determination unit determines a process break corresponding to behavior data at timing when the behavior data in which the object exhibits a predetermined behavior is observed between a minimum time and a maximum time set as the time information (Paragraph [0075]; Paragraph [0102]; Paragraph [0082]: “The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning”; and Paragraph [0083]: “The element work determination unit 255 refers to the process database 254 and determines the type of element work in the process and the start and end timings based on the operator's motion recognized by the motion recognition unit 253 […]”).
Regarding claim(s) 4, Shinya as modified by Itoh teaches the action analysis device according to claim 3, where Shinya teaches wherein the determination unit determines whether or not the object has exhibited a predetermined behavior based on region information set as an operation region of the process (Paragraph [0080] – Paragraph [0082]: “The feature point extraction unit 251 extracts the feature points (skeleton, joints, etc.) of the worker based on the sensing result (typically, an image) of the region including at least the worker by the sensing device […] The motion recognition unit 253 recognizes the operator's motion based on the feature points of the worker extracted by the feature point extraction unit 251 and the position and type of the object extracted by the object recognition unit 252 […]”), and determines a process break corresponding to the behavior data (Paragraph [0075]; Paragraph [0102]; and Paragraph [0083]).
Regarding claim(s) 6, Shinya as modified by Itoh teaches the action analysis device according to claim 1, where Shinya teaches wherein the determination unit determines a required time for a process corresponding to the behavior data by learning a feature observed in the behavior data (Paragraph [0082]: “The motion recognition unit 253 recognizes the operator's motion based on the feature points of the worker extracted by the feature point extraction unit 251 and the position and type of the object extracted by the object recognition unit 252. The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic”; Paragraph [0073]; Paragraph [0075]; and Paragraph [0102]).
Regarding claim(s) 7, Shinya as modified by Itoh teaches the action analysis device according to claim 1, where Shinya teaches further comprising a display control unit that lists required times for processes corresponding to the behavior data determined by the determination unit a plurality of times on a user interface along a time axis (Figure 14; Paragraph [0109] – Paragraph [0110]: “the analysis result may be displayed by using a graph capable of expressing the variation in the working time required for each step [...] the process, the work included in the process, and the element work included in the work It provides the analysis result that can grasp how much time is required”).
Regarding claim(s) 9, Shinya as modified by Itoh teaches the action analysis device according to claim 1, where Shinya teaches further comprising a display control unit that displays a result obtained by comparing first behavior data optionally selected from a plurality of pieces of behavior data with second behavior data used as a comparison target on a user interface in a graph (Figure 14; Paragraph [0109] – Paragraph [0110]: “the analysis result may be displayed by using a graph capable of expressing the variation in the working time required for each step [...] the process, the work included in the process, and the element work included in the work It provides the analysis result that can grasp how much time is required”; and Paragraph [0120] – Paragraph [0121]: “The switching loss score is the behavior of the worker 20 (or robot 300) in charge of the work immediately before the arbitrary switching point and the robot 300 (or the worker 20) in charge of the work immediately after the switching point. It is determined by calculating the degree of reduction in productivity when the switching point is set by comparing with the behavior of […] by comparing the locus data for the work immediately before the switching point of interest with the locus data for the work immediately after the switching point of interest, the worker 20 and the robot 300 Evaluate the degree of approach during work. Here, the locus data includes information indicating the position of the worker 20 or the robot 300 at each time”).
Regarding claim(s) 13, Shinya as modified by Itoh teaches the action analysis device according to claim 1, where Shinya teaches wherein the acquisition unit acquires behavior data on the object detected by an image sensor from the image sensor by using a model incorporated in a chip integrated with the image sensor (Paragraph [0077]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed”).
Regarding claim(s) 14, Shinya teaches an action analysis method comprising:
a computer (Paragraph [0015]: “The analysis program uses a computer to analyze the behavior of the worker and the collaborative work robot to calculate the work time required for the work of each process, and between the worker and the collaborative work robot at an arbitrary position in the process”) acquiring behavior data indicating a behavior of an object during an operation process, which has been recognized by a model (Paragraph [0069] – Paragraph [0071]: “the entire process 40 composed of a plurality of processes is regarded as the highest unit, and the process 41 of each process is composed of a plurality of operations. Each work of 41 in the process can be decomposed into one or a plurality of element work 42. Each of the elemental operations 42 can be decomposed into one or more operations 43. Each of the movements 43 can be decomposed into one or more unit movements 44”; and Paragraph [0077] – Paragraph [0082]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed […] The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time”); and
determining a required time for a process corresponding to the behavior data based on the behavior data that has been acquired (Paragraph [0073]: “The worker element work recognition unit 250 and the robot element work recognition unit 260 correspond to a work time calculation unit that calculates the work time required for the work of each process by analyzing the behaviors of the worker and the robot”; Paragraph [0075]: “robot element work recognition unit 260 analyzes the start and end timings of each element work performed by the robot in the process, and the time required for the element work”; Paragraph [0102]: “Based on the movement of the worker 20 or the status and position of the robot 300, the start and end timings for each element work are determined (step S116), and the work time for each element work is calculated (step S118)”; and Paragraph [0106]: “The start time column 704 and the end time column 705 indicate the start and end times of the corresponding element work. The working time 706 indicates the time required for the corresponding element work”).
Shinya fails to teach to recognized by a model preliminarily learned for recognizing the object. However, Itoh teach to recognized by a model preliminarily learned for recognizing the object (Figure 1; Figure 4; and Paragraph [0106]: “An identification information assignment apparatus of the present disclosure includes an acquirer configured to acquire a plurality of pieces of image data, an assigner configured to assign identification information to image data selected from the plurality of pieces of image data by using a learning model after learning, and an updater configured to update the learning model using the image data to which the identification information is assigned, in which the assigner assigns identification information to the rest of the image data acquired by the acquirer using the learning model that has been updated”).
Shinya discloses an analysis device that acquires behavior data of a worker and/or a collaborative robot during execution of a manufacturing process and determines a required work time corresponding to each process based on the analyzed behavior, including determining start and end timings of element work and calculating the time required for the process. Shinya further teaches recognizing behavior using a trained model generated in advance by machine learning. Itoh teaches employing a pretrained learning model as a core component for recognizing posture or behavior from image data in a work analysis system.
Therefore, it would have been obvious to one of ordinary skill in the art to combine Shinya and Itoh before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the accuracy and reliability of behavior acquisition, thereby arriving at the claimed action analysis device configured to determine a required time for a process based on behavior data recognized using a preliminarily learned model. This motivation for the combination of Shinya and Itoh is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 16, Shinya teaches a photographing device comprising:
an imaging unit that captures an image including an object (Paragraph [0078]: “The sensing device may be a normal camera (2D camera) or a stereo camera (3D camera). Alternatively, a sensing device such as a laser scanner or a distance measuring sensor that can acquire a distance profile to an object may be used”);
a detection unit that detects a behavior of the object included in the image by using a (Paragraph [0069] – Paragraph [0071]: “the entire process 40 composed of a plurality of processes is regarded as the highest unit, and the process 41 of each process is composed of a plurality of operations. Each work of 41 in the process can be decomposed into one or a plurality of element work 42. Each of the elemental operations 42 can be decomposed into one or more operations 43. Each of the movements 43 can be decomposed into one or more unit movements 44”; and Paragraph [0077] – Paragraph [0082]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed […] The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time”); and
an output control unit that selectively outputs, to outside, at least one of behavior data indicating the behavior of the object detected by the detection unit and the image (Paragraph [0082]: “The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time; Paragraph [0073]; Paragraph [0075]; Paragraph [0102]; and Paragraph [0106]).
Shinya fails to teach to recognized by a model preliminarily learned for recognizing the object. However, Itoh teach to recognized by a model preliminarily learned for recognizing the object (Figure 1; Figure 4; and Paragraph [0106]: “An identification information assignment apparatus of the present disclosure includes an acquirer configured to acquire a plurality of pieces of image data, an assigner configured to assign identification information to image data selected from the plurality of pieces of image data by using a learning model after learning, and an updater configured to update the learning model using the image data to which the identification information is assigned, in which the assigner assigns identification information to the rest of the image data acquired by the acquirer using the learning model that has been updated”).
Shinya discloses an analysis device that acquires behavior data of a worker and/or a collaborative robot during execution of a manufacturing process and determines a required work time corresponding to each process based on the analyzed behavior, including determining start and end timings of element work and calculating the time required for the process. Shinya further teaches recognizing behavior using a trained model generated in advance by machine learning. Itoh teaches employing a pretrained learning model as a core component for recognizing posture or behavior from image data in a work analysis system.
Therefore, it would have been obvious to one of ordinary skill in the art to combine Shinya and Itoh before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the accuracy and reliability of behavior acquisition, thereby arriving at the claimed action analysis device configured to determine a required time for a process based on behavior data recognized using a preliminarily learned model. This motivation for the combination of Shinya and Itoh is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 17, Shinya teaches an action analysis system comprising:
a photographing device (Figure 4; and Paragraph [0037]: “the PLC 100, the analysis device 200, the robot 300, and the display operation device 400 are connected via the field network 2. One or more cameras 30 are connected to the field network 2. Based on the image captured by the camera 30, the behaviors of the worker 20 and the robot 300 are analyzed, and the state of each process and the work in each process is estimated”) including:
an imaging unit that captures an image including an object (Paragraph [0078]: “The sensing device may be a normal camera (2D camera) or a stereo camera (3D camera). Alternatively, a sensing device such as a laser scanner or a distance measuring sensor that can acquire a distance profile to an object may be used”);
a detection unit that detects a behavior of the object included in the image by using a (Paragraph [0069] – Paragraph [0071]: “the entire process 40 composed of a plurality of processes is regarded as the highest unit, and the process 41 of each process is composed of a plurality of operations. Each work of 41 in the process can be decomposed into one or a plurality of element work 42. Each of the elemental operations 42 can be decomposed into one or more operations 43. Each of the movements 43 can be decomposed into one or more unit movements 44”; and Paragraph [0077] – Paragraph [0082]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed […] The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time”); and
an output control unit that selectively outputs, to outside, at least one of behavior data indicating the behavior of the object detected by the detection unit and the image (Paragraph [0082]: “The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time; Paragraph [0073]; Paragraph [0075]; Paragraph [0102]; and Paragraph [0106]); and
an action analysis device (Paragraph [0015]: “The analysis program uses a computer to analyze the behavior of the worker and the collaborative work robot to calculate the work time required for the work of each process, and between the worker and the collaborative work robot at an arbitrary position in the process”; and Paragraph [0072] – Paragraph [0073]: “the analysis device 200 includes […] work time calculation unit that calculates the work time required […] by analyzing the behaviors […]”) including:
an acquisition unit that acquires behavior data output from the output control unit (Paragraph [0069] – Paragraph [0071]: “the entire process 40 composed of a plurality of processes is regarded as the highest unit, and the process 41 of each process is composed of a plurality of operations. Each work of 41 in the process can be decomposed into one or a plurality of element work 42. Each of the elemental operations 42 can be decomposed into one or more operations 43. Each of the movements 43 can be decomposed into one or more unit movements 44”; and Paragraph [0077] – Paragraph [0082]: “The worker element work recognition unit 250 analyzes the movement of the worker and the like based on the detection result by one or a plurality of sensing devices such as the camera 30, and identifies which element work is being performed […] The motion recognition unit 253 may be realized by using, for example, a trained model generated in advance by machine learning, or may be realized by using a rule-based determination logic. The motion recognition unit 253 may output locus data including information for specifying the position of the recognized worker at each time. The locus data includes information indicating the position of the worker at each time”); and
a determination unit that determines a required time for a process corresponding to the behavior data based on the behavior data acquired by the acquisition unit (Paragraph [0073]: “The worker element work recognition unit 250 and the robot element work recognition unit 260 correspond to a work time calculation unit that calculates the work time required for the work of each process by analyzing the behaviors of the worker and the robot”; Paragraph [0075]: “robot element work recognition unit 260 analyzes the start and end timings of each element work performed by the robot in the process, and the time required for the element work”; Paragraph [0102]: “Based on the movement of the worker 20 or the status and position of the robot 300, the start and end timings for each element work are determined (step S116), and the work time for each element work is calculated (step S118)”; and Paragraph [0106]: “The start time column 704 and the end time column 705 indicate the start and end times of the corresponding element work. The working time 706 indicates the time required for the corresponding element work”).
Shinya fails to teach to recognized by a model preliminarily learned for recognizing the object. However, Itoh teach to recognized by a model preliminarily learned for recognizing the object (Figure 1; Figure 4; and Paragraph [0106]: “An identification information assignment apparatus of the present disclosure includes an acquirer configured to acquire a plurality of pieces of image data, an assigner configured to assign identification information to image data selected from the plurality of pieces of image data by using a learning model after learning, and an updater configured to update the learning model using the image data to which the identification information is assigned, in which the assigner assigns identification information to the rest of the image data acquired by the acquirer using the learning model that has been updated”).
Shinya discloses an analysis device that acquires behavior data of a worker and/or a collaborative robot during execution of a manufacturing process and determines a required work time corresponding to each process based on the analyzed behavior, including determining start and end timings of element work and calculating the time required for the process. Shinya further teaches recognizing behavior using a trained model generated in advance by machine learning. Itoh teaches employing a pretrained learning model as a core component for recognizing posture or behavior from image data in a work analysis system.
Therefore, it would have been obvious to one of ordinary skill in the art to combine Shinya and Itoh before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the accuracy and reliability of behavior acquisition, thereby arriving at the claimed action analysis device configured to determine a required time for a process based on behavior data recognized using a preliminarily learned model. This motivation for the combination of Shinya and Itoh is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Allowable Subject Matter
Claim(s) 5, 8, and 10-12 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Relevant Prior Art Directed to State of Art
Mackenzie et al (US 2018/0247560 A1) is/are relevant prior art not applied in the rejection(s) above. Mackenzie discloses a method comprising: dressing an operator with colored surgical gloves wherein a dominant hand of the operator wears a first surgical glove having a first color on at least a portion of the first glove and a non-dominant hand of the operator wears a second surgical glove having a different second color on at least a portion of the second glove; after dressing the operator, capturing video data that views the operator's hands during a surgical procedure on a subject; for each of a plurality of frames of the video data, automatically determining, on a processor, a minimum rectangle of pixels, called a first rectangle, that encloses pixels having the first color; determining automatically on a processor a first time series for a representative property of the first rectangle at the plurality of frames; determining automatically on a processor a first value for a first measure of entropy based on the first time series; and storing, on a computer-readable medium, a metric of operator performance based at least in part on the first value for the first measure of entropy.
Gorek et al (US 2021/0006047 A1) is/are relevant prior art not applied in the rejection(s) above. Gorek discloses the data from several sensors can be measured to provide improved measurement of surgical workflow. The data may comprise times at which needles are removed from suture packs and placed in receptacles. The surgical workflow data may comprise data from several instruments such as removal and placement time of surgical instruments and electrocautery devices. The data from several sensors can indicate vital statistics of a patient or environmental conditions of an operating room. The data from several sensors can indicate the presence, absence, arrival, or departure of one or more actors in a surgical workflow. The data from several sensors can be registered with a common time base and a report generated. The report can indicate a performance of individuals and groups of participants in a surgical workflow.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONGBONG NAH whose telephone number is (571) 272-1361. The examiner can normally be reached M - F: 9:00 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL MISTRY can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONGBONG NAH/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674