DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-10, as amended, are currently pending and have been considered below.
Claim Objections
Claim 6 is objected to because of the following informalities: Claim 6 recites the limitation, “The task management device claim 5” in line 1 of the claim. This limitation appears to be missing a word or words and should recite, i.e., “The task management device according to claim 5.” Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “image obtaining section configured to obtain,” “image extracting section configured to extract,” “data generating section configured to generate,” “linear data extracting section configured to extract,” “task inferring section configured to infer,” “frame setting section configured to set,” “index outputting section configured to output,” “weighting processing section configured to assign,” “output section that outputs” and “notification section that outputs” in claims 1-9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5 and 7-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kishi et al., Japanese Publication No. 2020 107341A, hereinafter, “Kishi”, and further in view of Imai et al., U.S. Publication No. 2022/0237533, hereinafter, “Imai”.
As per claim 1, Kishi discloses a task management device comprising:
an image obtaining section configured to obtain, over time, captured images each including a task target region in waterworks construction (Kishi, ¶0006, an image detection unit that detects characteristic images from a captured image that includes the work area in the waterworks work; Kishi, ¶0023, The photographing unit 3 photographs at least the area to be worked on in the plumbing work. The image capturing unit 3 may capture still images or moving images. Multiple photographs may be taken within the same plumbing work. The work area (and thus the camera 1) may be fixed, or may be movable for each image capture. In other words, the photographing unit 3 creates a photographed image including the area to be worked on each time it photographs an object);
an image extracting section configured to extract, from each of the captured images that have been obtained over time, an image of at least one target object which is involved in the waterworks construction (Kishi, ¶0006, an image detection unit that detects characteristic images from a captured image that includes the work area in the waterworks work; Kishi, ¶0007, including an image detection process for detecting characteristic images from a captured image that includes a work area in the waterworks work; Kishi, ¶0033, For example, the image detection unit 6 detects an image of an asphalt cutter 18 from the captured image 17 as a characteristic image);
a data generating section configured to, for each image of the at least one object that is inferred to be identical in each of the captured images, by extracting over time coordinates in a region containing the each image, generate time series data of the coordinates (Kishi, ¶0006, a work state estimation unit that estimates the current work state in the waterworks work from the characteristic images detected by the image detection unit, and a notification content determination unit that determines the content of notifications to those involved in the waterworks work in accordance with the current work state estimated; Kishi, ¶0013, it is possible to accurately detect characteristic images whose positions, etc. change over time, and to optimize the upper or lower limit of the work time corresponding to the current work state; Kishi, ¶0023, The image capturing unit 3 may capture still images or moving images. Multiple photographs may be taken within the same plumbing work. The work area (and thus the camera 1) may be fixed, or may be movable for each image capture. In other words, the photographing unit 3 creates a photographed image including the area to be worked on each time it photographs an object; Kishi, ¶0054, If the captured image 45 is a moving image, an image of the crane 46 lowering the tunneling machine 47 and the tunneling machine 47 descending into the launch tunnel 44 may be used as the characteristic image. In this case, the work status estimation unit 7 may estimate that the current work status in the waterworks construction work is the transportation and installation of the tunneling machine based on the tension of the crane46's hoisting equipment 48 and the way the tunneling machine 47 is being lowered; Kishi, ¶0058, FIG. 14 is an image diagram showing an example of a photographed image 50 during a jack return operation … The captured image 50 may be a still image, but is assumed to be a moving image here. The captured image 50 shows that the time during which the tunneling machine 47 is visible is less than a predetermined time, the space 51 behind the tunneling machine 47 is sufficiently large, and the stroke of the main jack 43 is short. The image detection unit 6 detects, for example, the situation from the photographed image 50 as a characteristic image. The work state estimation unit 7 estimates from the state as a characteristic image that the current work state in the plumbing work is jack return work 1 [the current work state is obtained by comparing identical objects / positions over time]);
a task inferring section configured to infer, a task being carried out at a time point at which each of the captured images is obtained (Kishi, ¶0054, If the captured image 45 is a moving image, an image of the crane 46 lowering the tunneling machine 47 and the tunneling machine 47 descending into the launch tunnel 44 may be used as the characteristic image. In this case, the work status estimation unit 7 may estimate that the current work status in the waterworks construction work is the transportation and installation of the tunneling machine based on the tension of the crane46's hoisting equipment 48 and the way the tunneling machine 47 is being lowered; Kishi, ¶0058, FIG. 14 is an image diagram showing an example of a photographed image 50 during a jack return operation … The captured image 50 may be a still image, but is assumed to be a moving image here. The captured image 50 shows that the time during which the tunneling machine 47 is visible is less than a predetermined time, the space 51 behind the tunneling machine 47 is sufficiently large, and the stroke of the main jack 43 is short. The image detection unit 6 detects, for example, the situation from the photographed image 50 as a characteristic image. The work state estimation unit 7 estimates from the state as a characteristic image that the current work state in the plumbing work is jack return work 1).
Kishi does not explicitly disclose the following limitations as further recited however Imai discloses
a linear data extracting section configured to extract, from the time series data of the coordinates, linear data that is a part in which a value is continuously and regularly changed (Imai, ¶0007-10, (1) A work analyzing system, includes: a measurement unit that measures an inside of a work area and acquires measurement data of time series; an object recognizing unit that recognizes an object including a work machine or a person on a basis of the acquired measurement data and determines position information on the recognized object and a feature amount with regard to a shape of the object; and a determination unit that determines a work having been performed in the work area on a basis of a position of the object recognized by the object recognizing unit, a positional relationship relative to other objects, and the feature amount; Imai, ¶0028, (12) The work analyzing system described in any one of the above-described (1) to the above-described (11), the determination unit determines the work by using a moving speed of the object detected on a basis of a change of time series of a position of the object; Imai, ¶0144, The control unit 22 records the determined work in a detection list and the like. FIG. 6 shows an example of the detected object list. As shown in the same diagram (FIG. 6), the detected object list includes a detection ID of an object recognized by the above-mentioned processing, detection coordinates at each time, size information, determination result of moving/stop, feature (feature amount), and determined work contents); and
on the basis of the linear data and the image of the at least one target object, the image being included in each of the captured images, a task being carried out at a time point (Imai, ¶0028, (12) The work analyzing system described in any one of the above-described (1) to the above-described (11), the determination unit determines the work by using a moving speed of the object detected on a basis of a change of time series of a position of the object; Imai, ¶0144, The control unit 22 records the determined work in a detection list and the like).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Imai and Kishi because they are in the same field of endeavor. One skilled in the art would have been motivated to include the time-series data as taught by Imai in the system of Kishi in order to perform work analysis and work process improvement to increase productivity (Imai, ¶0002-0003).
As per claim 2, Kishi and Imai disclose the task management device according to claim 1. Imai discloses further comprising a frame setting section configured to set, around the image of the at least one target object or around a part of the image of the at least one target object, a frame that is in accordance with a size of the image of the at least one target object, the data generating section extracting, as the coordinates in the region, coordinates of a position predetermined in the frame (Imai, ¶0115, The determination unit 223 determines (classifies) a work performed in the work area 90 from the position of an object recognized by the object recognizing unit 222, a positional relationship relative to other objects, and a feature amount. Here, as a positional relationship (relative positional relationship) relative to other objects, a distance between the respective center positions of multiple objects may be used, or a distance between the respective outlines of objects, i.e., a “gap”, may be used. The calculation of this positional relationship relative to other objects may be calculated from the center position of a bounding box surrounding a recognized object or this positional relationship may be calculated from the closest distance between apexes, or sides (or faces) that constitute two bounding boxes. This bounding box is, for example, one rectangular parallelepiped that becomes a minimum area (volume) to surround an object. The positions of apexes, and sides (faces) of a bounding box can be obtained on the basis of coordinates (center position), sizes (width, height, depth), and a rotation angle θ (rotation angle (orientation of an object) on a top view) of each bounding box). The motivation would be the same as above in claim 1.
As per claim 3, Kishi and Imai disclose the task management device according to claim 1. Imai discloses wherein the data generating section normalizes the coordinates with use of a size of the image of the at least one target object from which image the coordinates have been extracted (Imai, ¶0214, The recognizing unit 222 extracts feature points corresponding to the shape and outline of an object in a photographed image from each of both images by performing contrast adjustment and binarization processing to a pair of picture image data photographed at the same time by both cameras and calculates a distance up to each of the feature points on the basis of a positional relationship within images of the matched feature points and the data of a base length. With this, it is possible to obtain a distance value for each pixel in the picture image data. By such processing, the control unit 22 of the work analyzing apparatus 20 creates distance measurement point group data from photographed data (measurement data) in the work area 90). The motivation would be the same as above in claim 1.
As per claim 4, Kishi and Imai disclose the task management device according to claim 1. Imai discloses wherein the data generating section removes an outlier from the time series data of the coordinates, and the linear data extracting section extracts the linear data from the time series data of the coordinates from which time series data the outlier has been removed (Imai, ¶0110, the object recognizing unit 222 determines whether or not the calculated size is a predetermined size threshold to specify a moving body of an analytical target of an extraction target or less. The size threshold can be set arbitrarily. For example, it can be set on the basis of the size of the moving body assumed in the work area 90. In the case of analyzing movement (trajectory) by tracing the worker 85 or the work machine 80, it may be permissible that the minimum value of the worker 85 or the work machine 80 is set to a size threshold in the case of clustering. With this, garbage, such as fallen leaves and a plastic bag, or small animals can be excluded from a detection target; Imai, ¶0144, FIG. 6 shows an example of the detected object list. As shown in the same diagram (FIG. 6), the detected object list includes a detection ID of an object recognized by the above-mentioned processing, detection coordinates at each time, size information, determination result of moving/stop, feature (feature amount), and determined work contents). The motivation would be the same as above in claim 1.
As per claim 5, Kishi and Imai disclose the task management device according to claim 1. Imai discloses further comprising an index outputting section configured to output at least one index that is inferred to appear in the task at the time point at which each of the captured images is obtained, the at least one index being inferred by inputting each of the captured images to an index inference model that is among a plurality of indices, including the at least one target object, for evaluating a task state in the waterworks construction and that is constructed, by machine learning, so as to infer at least one index that appears in a task at a specified time point, the task inferring section inferring, on the basis of the linear data and an output result from the index outputting section, the task being carried out at the time point at which each of the captured images is obtained (Imai, ¶0099, In the memory unit 21, a detected object list (also referred to as a detected thing list), position information history data, a work determining criterion, a work plan, and the like are memorized; Imai, ¶0100, In the “detected object list”, a detection ID for inner management is provided for an object (work machine 80, worker 85, etc.) recognized by recognition processing (mentioned later) executed by the control unit 22 (object recognizing unit 222), and on the basis of the detection ID, tracing of an object is performed. Moreover, in the detected object list, at each time for each detection ID, a position, the kind of an object (the kind of a work machine), and a work specified (classified) by later-mentioned processing, are described; Imai, ¶0101, The “position information history data” is history data that shows transition of the position of an object (work machine 80, worker 85, etc.) recognized continuously during predetermined time; Imai, ¶0102, The determination unit 223 of the control unit 22 determines a work process performed in the work area 90; Imai, ¶0112, with regard to the recognition of the kind of an object and a feature amount, the recognition may be made by using a learned model. By using a large number of learning-sample data provided with a correct answer label (the kind of a work machine or the kind of a work machine with the kind and feature amount of a work machine) with regard to an object recognized from distance measurement point group data; Imai, ¶0113, as the work determining criterion, a learned model according to machine learning may be used. In this learned model, an input is information on the position of an object recognized by the object recognizing unit 222, a positional relationship relative to other objects, and a feature amount. Then, the learned model is one having supervised learned by setting an output to a correct answer level of work classification; Imai, ¶0218, the recognition may be performed by using a learned model. The learned model can be machine-learned by supervised learning by using a large number of learning sample data provided with a correct answer label (“an arm has been being raised upward”, “an arm has been being lowered downward”, “be conveying a load”, and the like) with regard to a picture image obtained by the camera 13 and a feature amount of an object existing in the picture image). The motivation would be the same as above in claim 1.
As per claim 7, Kishi and Imai disclose the task management device according to claim 1, further comprising a camera configured to capture an image of the task target region, the data generating section correcting the coordinates on the basis of a predetermined reference position of the camera and a position of the camera when each of the captured images is obtained (Kishi, ¶0022, The plumbing work management system 101 includes a camera 1 and a management device 2. The camera 1 has a photographing unit 3 and a communication unit 4; Kishi, ¶0023, The photographing unit 3 photographs at least the area to be worked on in the plumbing work. The image capturing unit 3 may capture still images or moving images. Multiple photographs may be taken within the same plumbing work. The work area (and thus the camera 1) may be fixed, or may be movable for each image capture. In other words, the photographing unit 3 creates a photographed image including the area to be worked on each time it photographs an object; Kishi, ¶0045, the waterworks management system 101 may perform each of the above estimations using machine learning such as neural networks. This makes it possible to accurately detect feature images whose positions, etc. change over time, and to optimize the upper or lower limit of the work time corresponding to the current work state depending on factors that may affect the work time. For example, the upper limit estimation unit 10 can update the upper limit of the task time to a more appropriate value every time the element is newly acquired. Similarly, the lower limit estimation unit 12 can update the lower limit value of the task time to a more appropriate value every time the element is newly acquired [positions and displacements of elements are tracked over time]).
As per claim 8, Kishi and Imai disclose the task management device according to claim 1, wherein the task inferring section infers the task being carried out at the time point at which each of the captured images is obtained, the task being inferred by inputting (i) at least one of a plurality of indices, including the at least one target object, for evaluating a task state in the waterworks construction and (ii) the linear data to an implemented-task inference model that is constructed, by machine learning, so as to infer the task being carried out at the time point at which each of the captured images is obtained (Kishi, ¶0007, including an image detection process for detecting characteristic images from a captured image that includes a work area in the waterworks work, a work state estimation process for estimating the current work state in the waterworks work from the characteristic images detected in the image detection process, and a notification content determination process for determining the content of notifications to those involved in the waterworks work in accordance with the current work state estimated in the work state estimation process; Kishi, ¶0045, the waterworks management system 101 may perform each of the above estimations using machine learning such as neural networks. This makes it possible to accurately detect feature images whose positions, etc. change over time, and to optimize the upper or lower limit of the work time corresponding to the current work state depending on factors that may affect the work time. For example, the upper limit estimation unit 10 can update the upper limit of the task time to a more appropriate value every time the element is newly acquired. Similarly, the lower limit estimation unit 12 can update the lower limit value of the task time to a more appropriate value every time the element is newly acquired; Imai, ¶0046, A work analyzing system according to the present invention includes an object recognizing unit that recognizes an object including a work machine or a person on a basis of measurement data obtained by measuring a work area by a measurement unit and determines position information on the recognized object and a feature amount with regard to a shape of the object, and a determination unit that determines a work performed in a work area on a basis of a position of the object recognized by the object recognizing unit, a positional relationship relative to other objects, and the feature amount. With this, it becomes possible to record a work history even in a work site in which surrounding situations changes day by day; Imai, ¶0077, In the work area 90, multiple work machines 80 (801 to 803) and workers 85 move and work; Imai, ¶0078, In the work machine 80, for example, an arti-damp, a wheel loader, a backhoe, a power shovel, a breaker, a mixer truck, and a spray machine for spraying concrete, and the like are included ... The data of respective sizes and forms of these work machines 80 are registered beforehand in the memory unit 21; Imai, ¶0099, In the memory unit 21, a detected object list (also referred to as a detected thing list), position information history data, a work determining criterion, a work plan, and the like are memorized; Imai, ¶0100, In the “detected object list”, a detection ID for inner management is provided for an object (work machine 80, worker 85, etc.) recognized by recognition processing (mentioned later) executed by the control unit 22 (object recognizing unit 222), and on the basis of the detection ID, tracing of an object is performed. Moreover, in the detected object list, at each time for each detection ID, a position, the kind of an object (the kind of a work machine), and a work specified (classified) by later-mentioned processing, are described; Imai, ¶0101, The “position information history data” is history data that shows transition of the position of an object (work machine 80, worker 85, etc.) recognized continuously during predetermined time; Imai, ¶0111, the object recognizing unit 222 recognizes the kind of a recognized object and recognizes a feature amount with regard to the shape of an object; Imai, ¶0112, with regard to the recognition of the kind of an object and a feature amount, the recognition may be made by using a learned model; Imai, ¶0113, as the work determining criterion, a learned model according to machine learning may be used. In this learned model, an input is information on the position of an object recognized by the object recognizing unit 222, a positional relationship relative to other objects, and a feature amount; Imai, ¶0115, The determination unit 223 determines (classifies) a work performed in the work area 90 from the position of an object recognized by the object recognizing unit 222, a positional relationship relative to other objects, and a feature amount; Imai, ¶0118, The output creating unit 224 creates work analysis information by analyzing and processing the data of the determination result of the determination unit 223. The work analysis information includes the analysis result of the determination result and the display data in which this analysis result is visualized. The display data created by analysis includes a Gantt chart; Imai, ¶0132, The control unit 22 records the movement trajectory of the recognized object. This record is stored as position information history data in the memory unit 21; Imai, ¶0169; Imai, ¶0218; Imai, ¶0231, FIG. 22 shows an output example of a Gantt chart and shows the working time for each of work machines and workers).
As per claim 9, Kishi and Imai disclose the task management device according to claim 1, further comprising an output section that is connected with a notification section so as to be capable of communicating with the notification section and that outputs, from the notification section, at least one of an inference result from the task inferring section and the time series data of the coordinates (Kishi, ¶0007, including an image detection process for detecting characteristic images from a captured image that includes a work area in the waterworks work, a work state estimation process for estimating the current work state in the waterworks work from the characteristic images detected in the image detection process, and a notification content determination process for determining the content of notifications to those involved in the waterworks work in accordance with the current work state estimated in the work state estimation process).
As per claim 10, Kishi discloses a task management method comprising:
an image obtaining step of obtaining, over time, captured images each including a task target region in waterworks construction (Kishi, ¶0006, an image detection unit that detects characteristic images from a captured image that includes the work area in the waterworks work; Kishi, ¶0023, The photographing unit 3 photographs at least the area to be worked on in the plumbing work. The image capturing unit 3 may capture still images or moving images. Multiple photographs may be taken within the same plumbing work. The work area (and thus the camera 1) may be fixed, or may be movable for each image capture. In other words, the photographing unit 3 creates a photographed image including the area to be worked on each time it photographs an object);
an image extracting step of extracting, from each of the captured images that have been obtained over time, an image of at least one target object which is involved in the waterworks construction (Kishi, ¶0006, an image detection unit that detects characteristic images from a captured image that includes the work area in the waterworks work; Kishi, ¶0007, including an image detection process for detecting characteristic images from a captured image that includes a work area in the waterworks work; Kishi, ¶0033, For example, the image detection unit 6 detects an image of an asphalt cutter 18 from the captured image 17 as a characteristic image);
a data generating step of, for each image of the at least one object that is inferred to be identical in each of the captured images, by extracting over time coordinates in a region containing the each image, generating time series data of the coordinates (Kishi, ¶0006, a work state estimation unit that estimates the current work state in the waterworks work from the characteristic images detected by the image detection unit, and a notification content determination unit that determines the content of notifications to those involved in the waterworks work in accordance with the current work state estimated; Kishi, ¶0013, it is possible to accurately detect characteristic images whose positions, etc. change over time, and to optimize the upper or lower limit of the work time corresponding to the current work state; Kishi, ¶0023, The image capturing unit 3 may capture still images or moving images. Multiple photographs may be taken within the same plumbing work. The work area (and thus the camera 1) may be fixed, or may be movable for each image capture. In other words, the photographing unit 3 creates a photographed image including the area to be worked on each time it photographs an object; Kishi, ¶0054, If the captured image 45 is a moving image, an image of the crane 46 lowering the tunneling machine 47 and the tunneling machine 47 descending into the launch tunnel 44 may be used as the characteristic image. In this case, the work status estimation unit 7 may estimate that the current work status in the waterworks construction work is the transportation and installation of the tunneling machine based on the tension of the crane46's hoisting equipment 48 and the way the tunneling machine 47 is being lowered; Kishi, ¶0058, FIG. 14 is an image diagram showing an example of a photographed image 50 during a jack return operation … The captured image 50 may be a still image, but is assumed to be a moving image here. The captured image 50 shows that the time during which the tunneling machine 47 is visible is less than a predetermined time, the space 51 behind the tunneling machine 47 is sufficiently large, and the stroke of the main jack 43 is short. The image detection unit 6 detects, for example, the situation from the photographed image 50 as a characteristic image. The work state estimation unit 7 estimates from the state as a characteristic image that the current work state in the plumbing work is jack return work 1 [the current work state is obtained by comparing identical objects / positions over time]);
a task inferring step of inferring, a task being carried out at a time point at which each of the captured images is obtained (Kishi, ¶0054, If the captured image 45 is a moving image, an image of the crane 46 lowering the tunneling machine 47 and the tunneling machine 47 descending into the launch tunnel 44 may be used as the characteristic image. In this case, the work status estimation unit 7 may estimate that the current work status in the waterworks construction work is the transportation and installation of the tunneling machine based on the tension of the crane46's hoisting equipment 48 and the way the tunneling machine 47 is being lowered; Kishi, ¶0058, FIG. 14 is an image diagram showing an example of a photographed image 50 during a jack return operation … The captured image 50 may be a still image, but is assumed to be a moving image here. The captured image 50 shows that the time during which the tunneling machine 47 is visible is less than a predetermined time, the space 51 behind the tunneling machine 47 is sufficiently large, and the stroke of the main jack 43 is short. The image detection unit 6 detects, for example, the situation from the photographed image 50 as a characteristic image. The work state estimation unit 7 estimates from the state as a characteristic image that the current work state in the plumbing work is jack return work 1).
Kishi does not explicitly disclose the following limitations as further recited however Imai discloses
a linear data extracting step of extracting, from the time series data of the coordinates, linear data that is a part in which a value is continuously and regularly changed (Imai, ¶0007-10, (1) A work analyzing system, includes: a measurement unit that measures an inside of a work area and acquires measurement data of time series; an object recognizing unit that recognizes an object including a work machine or a person on a basis of the acquired measurement data and determines position information on the recognized object and a feature amount with regard to a shape of the object; and a determination unit that determines a work having been performed in the work area on a basis of a position of the object recognized by the object recognizing unit, a positional relationship relative to other objects, and the feature amount; Imai, ¶0028, (12) The work analyzing system described in any one of the above-described (1) to the above-described (11), the determination unit determines the work by using a moving speed of the object detected on a basis of a change of time series of a position of the object; Imai, ¶0144, The control unit 22 records the determined work in a detection list and the like. FIG. 6 shows an example of the detected object list. As shown in the same diagram (FIG. 6), the detected object list includes a detection ID of an object recognized by the above-mentioned processing, detection coordinates at each time, size information, determination result of moving/stop, feature (feature amount), and determined work contents); and
on the basis of the linear data and the at least one target object from which the time series data of the coordinates from which time series data the linear data has been extracted is generated, a task being carried out (Imai, ¶0028, (12) The work analyzing system described in any one of the above-described (1) to the above-described (11), the determination unit determines the work by using a moving speed of the object detected on a basis of a change of time series of a position of the object; Imai, ¶0144, The control unit 22 records the determined work in a detection list and the like).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Imai and Kishi because they are in the same field of endeavor. One skilled in the art would have been motivated to include the time-series data as taught by Imai in the system of Kishi in order to perform work analysis and work process improvement to increase productivity (Imai, ¶0002-0003).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kishi et al., Japanese Publication No. 2020 107341A, hereinafter, “Kishi”, in view of Imai et al., U.S. Publication No. 2022/0237533, hereinafter, “Imai” as applied to claim 5 above, and further in view of Yagi et al., Internation Publication No. WO 2019/073615 A1, hereinafter, “Yagi”.
As per claim 6, Kishi and Imai disclose the task management device claim 5, but do not explicitly disclosed the following limitation as further recited however Yagi discloses further comprising a weighting processing section configured to, in a case where the linear data extracting section extracts the linear data, in the at least one index that has been output as the at least one target object from which the time series data of the coordinates from which time series data the linear data has been extracted is generated, assign more weights to a time period in which the linear data has been extracted than to another time period different from the time period, assuming that movement occurs in the at least one target object (Yagi, ¶0009, An object of the present invention is to make it possible to easily grasp the capabilities of workers performing work at a construction site; Yagi, ¶0010, The present invention is a construction site image judgment device characterized by comprising: a learning device that is capable of making good/bad judgments about construction site images by learning ¶0010, The present invention is a construction site image judgment device characterized by comprising: a learning device that is capable of making good/bad judgments about construction site images by learning; Yagi, ¶0011, the learning device is further capable of determining whether a construction site image is good or bad and classifying the types of defects by learning using the learning data including the types of defects identified for the learning construction site images … determines the type of defect for the construction site image; Yagi, ¶0012, a plurality of the learning devices are provided corresponding to a plurality of construction stages; Yagi, ¶0034, the learning device 36 uses a neural network; Yagi, ¶0036, Each input variable is input to each unit 50 in the input layer. In each unit 50, weights and bias b for each input variable are defined; Yagi, ¶0042, In this case, the learning processing unit 40 inputs learning construction site images whose pass/fail judgment is known into the learning device 36 as learning data, and adjusts the weights and biases of each unit 50 in each layer so that in the output data for the learning construction site image, the value of the output variable corresponding to the pass/fail judgment for the learning construction site image becomes larger and the values of the other output variables become smaller (i.e., learns the learning device 36). For example, if the learning construction site image is judged to be good, the learning processing unit 40 adjusts the weights and biases of each unit 50 in each layer so that the value of the output variable y1 increases and the value of the output variable y2 decreases in the output data for the learning construction site image; Yagi, ¶0047, the pass/fail judgments may be displayed in order of the time at which the construction site images to be judged were taken, based on the photographing time information of the plurality of construction site images to be judged. This provides the manager with statistical data showing whether the construction results are appropriate or not in chronological order. Such statistical data allows the manager to grasp the time-dependent changes in whether the construction results are appropriate for each on-site worker; Yagi, ¶0058, a group of new construction site images can be continuously sent to the server 16).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Yagi with Kishi and Imai because they are in the same field of endeavor. One skilled in the art would have been motivated to include the weighting of construction images as taught by Yagi in the system of Kishi and Imai in order to provide an automated assessment of and means of managing the progress and results of a construction project (Yagi, ¶0002).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Examiner, Art Unit 2668