DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the reply filed on June 11, 2025. Claims 1-5 are pending. Claim 1 is independent.
Response to Arguments
Applicants’ arguments have been fully considered. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Publication No. 2020/0399863 to Aizawa et al. (hereinafter “Aizawa”).
Claims 1-5 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Aizawa.
With respect to independent claim 1, Aizawa discloses remote operation system having a mutual communication function of communicating with each of plural work machines and with a remote operation device which remotely operates one work machine as a remote operation target among the plural work machines, wherein an actual machine image-capturing device being capable of capturing an image and actual machine identification information, an image of which is captured so as to be included in the image captured by the actual machine image-capturing device, are mounted on the work machine as a candidate of the remote operation target among the plural work machines, and the remote operation system comprises: a first assistance process element which determines whether or not one work machine as the remote operation target among the plural work machines is selected, connects a cooperating work machine with the remote operation device, the cooperating work machine being identified by an identifier corresponding to selection information as information of the one selected work machine, when a determination is made that the one work machine is selected, and acquires the image captured by the actual machine image-capturing device of the cooperating work machine (see paragraph [0172], [0173], [0175] and [0185]: an example of an operation screen (instruction transmission operation screen 700) for selecting the excavator 100 that is the destination of the site monitor instruction, that is, an operation screen for selecting the excavator 100 that is the target of a remote operation. FIG. 7B is a diagram illustrating an example of an operation screen (an instruction transmission operation screen 710) for making settings of a specific site monitor instruction. A candidate list 701 of the excavators 100 for which remote operations for surrounding monitoring (site monitoring) is possible through the support apparatus 200, is displayed in the instruction transmission operation screen 700. The excavator 100 to be the target of remote operation, i.e., the excavator 100 that is the destination of the site monitor instruction, may be selected. At this time, the excavator 100 selected may be one or more excavators. In this example, two excavators 100 including an “excavator A” and an “excavator B” are selected. The instruction unit 2101 transmits the site monitor instruction including the identification information of the excavator 100 selected in the instruction transmission operation screen 700 and the setting contents of the setting fields 711 to 715 for each of the selected excavators 100, to the management apparatus 300 through the communication device 220.); and
a second assistance process element which causes an output interface of the remote operation device to output the image captured by the actual machine image-capturing device of the cooperating work machine (see paragraphs [0170], [0180] : The display apparatus 240 displays various kinds of information images. The display apparatus 240 may be, for example, a liquid crystal display or an organic EL display. In a setting field 712 to which the text information of “camera image acquisition” is appended, a setting relating to the acquisition of a captured image captured by the imaging apparatus 40 of the excavator 100 is made. ).
With respect to dependent claim 2, Aizawa discloses wherein the second assistance process element causes the output interface of the remote operation device to output the selection information of the cooperating work machine to be identified by the first assistance process element (see paragraph [0176], [0185] and [0192]: the user operates a button icon 702, to which the text information of the “validate” is appended, at the lower end of the instruction transmission operation screen 700, through a touch panel and the like. This allows the user to validate the selection of the excavator 100 that is the target of the remote operation, that is, the excavator 100 that is the destination of the site monitor instruction, and to cause the display contents of the display apparatus 240 to transition to the instruction transmission operation screen 710. The user appropriately makes settings in the setting fields 711 to 715 on the excavator 100 (the excavator A and the excavator B in this example) selected on the instruction transmission operation screen 700, through a touch panel and the like. The user operates a button icon 716 to which the text information of “transmit” is appended at the lower end of the instruction transmission operation screen 710 through a touch panel and the like. Accordingly, the instruction unit 2101 transmits the site monitor instruction including the identification information of the excavator 100 selected in the instruction transmission operation screen 700 and the setting contents of the setting fields 711 to 715 for each of the selected excavators 100, to the management apparatus 300 through the communication device 220. Specifically, a captured image captured by the imaging apparatus 40 of the excavator 100 (the “excavator A”) is displayed in a display field 811 to which the text information of the “camera image” is appended. The captured image in the display field 811 may be, for example, a captured image representing the current appearance (substantially real-time) of the surroundings of the excavator 100, or a captured image representing the appearance of the surroundings of the excavator 100 at some point back in the past.).
With respect to dependent claim 3, Aizawa discloses wherein the first assistance process element acquires the selection information input to an input interface of the remote operation device, and the second assistance process element causes the output interface of the remote operation device to output a content of an input operation acquired by the first assistance process element (see paragraph [0176]: the user operates a button icon 702, to which the text information of the “validate” is appended, at the lower end of the instruction transmission operation screen 700, through a touch panel and the like. This allows the user to validate the selection of the excavator 100 that is the target of the remote operation, that is, the excavator 100 that is the destination of the site monitor instruction, and to cause the display contents of the display apparatus 240 to transition to the instruction transmission operation screen 710).
With respect to dependent claim 4, Aizawa discloses wherein the first assistance process element determines whether or not the actual machine identification information is included in the image captured by the actual machine image- capturing device of the cooperating work machine, and when a determination is made that the actual machine identification information is included in the image, the second assistance process element makes a determination about consistency between the actual machine identification information and the selection information and causes the output interface of the remote operation device to output a determination result about the consistency (see paragraphs [0180], [0192] and [0366]: In a setting field 712 to which the text information of “camera image acquisition” is appended, a setting relating to the acquisition of a captured image captured by the imaging apparatus 40 of the excavator 100 is made. Specifically, in the setting field 712, the presence or absence of acquisition of a captured image captured by the imaging apparatus 40 and the type of a captured image (moving image or still image) to be acquired, may be set. In the setting field 712, in this example, captured image acquisition (“ON”) to be captured by the imaging apparatus 40 is present, and the type of the captured image to be acquired is set to “moving image”. A captured image captured by the imaging apparatus 40 of the excavator 100 (the “excavator A”) is displayed in a display field 811 to which the text information of the “camera image” is appended. The captured image in the display field 811 may be, for example, a captured image representing the current appearance (substantially real-time) of the surroundings of the excavator 100, or a captured image representing the appearance of the surroundings of the excavator 100 at some point back in the past. The tracking unit 3013 determines whether the detecting unit 3012 has lost sight of the monitor target in the monitor area when the monitor target detected in the monitor area by the detecting unit 3012 is no longer detected. If the background of the captured image captured by the imaging apparatus 40 and the monitor target are assimilated or the monitor target is hidden by an obstacle, even when the monitor target is present in the monitor area, the monitor target may not be detected by the detecting unit 3012. When there is a difference in the height direction (i.e., the vertical direction as viewed from the excavator 100) between the position of the excavator 100 (i.e., the position of the work plane) and the position of the monitor target, the size of the monitor target included in the captured image captured by the imaging apparatus 40 may change as compared to the case where there is no difference in the height direction (i.e., the position of the monitor target is also on the work plane). Therefore, the detecting unit 3012 may determine that a monitor target, which is present in the monitor area, is outside the monitor area.).
With respect to dependent claim 5, Aizawa discloses wherein in a case where the actual machine identification information included in the image is different from the actual machine identification information corresponding to the selection information, the first assistance process element acquires a candidate image captured by the actual machine image-capturing device of another work machine than the cooperating work machine among the plural work machines and acquires, from the remote operation device, consistency information about whether or not the work machine on which the actual machine image-capturing device capturing the candidate image is mounted is a real cooperating work machine, and the second assistance process element causes the output interface of the remote operation device to output the candidate image acquired by the first assistance process element, and when the consistency information acquired by the first assistance process element indicates the real cooperating work machine, the second assistance process element rewrites the identifier corresponding to the cooperating work machine to an identifier of the work machine on which the actual machine image-capturing device capturing the candidate image is mounted (see paragraph (see paragraphs [0366] and [0424]: the tracking unit 3013 determines whether the detecting unit 3012 has lost sight of the monitor target in the monitor area when the monitor target detected in the monitor area by the detecting unit 3012 is no longer detected. If the background of the captured image captured by the imaging apparatus 40 and the monitor target are assimilated or the monitor target is hidden by an obstacle, even when the monitor target is present in the monitor area, the monitor target may not be detected by the detecting unit 3012. In a case where the detecting unit 3012 estimates the distance D in the horizontal direction from the excavator 100 to the monitor target based on the size of the monitor target included in the captured image captured by the imaging apparatus 40, even when the monitor target exists in the monitor area (horizontal monitor area), the detecting unit 3012 may determine that the monitor target is outside the monitor area. Specifically, when there is a difference in the height direction (i.e., the vertical direction as viewed from the excavator 100) between the position of the excavator 100 (i.e., the position of the work plane) and the position of the monitor target, the size of the monitor target included in the captured image captured by the imaging apparatus 40 may change as compared to the case where there is no difference in the height direction (i.e., the position of the monitor target is also on the work plane). The detecting unit 3012 may apply a known image recognition technology and, for example, by comparing the feature quantity of the monitor target that is no longer detected with the feature quantity of the monitor target that is detected, it is possible to identify that the monitor target detected again is the same target as the previously detected monitor target (the worker W2). Further, the tracking unit 3013 changes the number of the undetected monitor targets to “1”, because the only undetected monitor target is the worker W1 as the worker W2 is detected again by the detecting unit 3012.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEMETRA R SMITH-STEWART whose telephone number is (571)270-3965. The examiner can normally be reached 10am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Nolan can be reached at 571-270-7016. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEMETRA R SMITH-STEWART/Examiner, Art Unit 3661
/PETER D NOLAN/Supervisory Patent Examiner, Art Unit 3661
Prosecution Insights