DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 23 June 2026 has been entered.
Response to Amendment
Claim 1 has been amended. No claims have been newly added nor canceled. Claims 1-2, 4-7, and 9 remain pending in the present application.
Response to Arguments
Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an imaging apparatus configured to capture an image” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 9 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 9, Applicant claims: “wherein the controller outputs the captured image having the position information to a display device.” The examiner asserts that this limitation renders the claim indefinite, as it is unclear if the claimed “a display device” is the same display device as claimed in newly amended claim 1. Specifically, the examiner notes that if the “display device” of claim 9 is the same “display device” of claim 1, it is unclear why there would be a second, separate step of sending information input by a display device back to the same display device. As such, for the sake of examination, the examiner is interpreting the “a display device” of claim 9 to be a different display device from that of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, and 4-7 are rejected under 35 U.S.C. 103 as being unpatentable over Murata (JP2018170991A), hereafter Murata, in view of Ohtomo (US 20150220085 A1), hereafter Ohtomo, and further in view of Ackerman (US 20210018927 A1), hereafter Ackerman.
Regarding claim 1, Murata discloses a working robot system comprising:
A working robot configured to output a self-position on a field (0019, within the field H1, the unmanned flying device 70 communicates wirelessly with the tractor 1, receives the driving information of the tractor 1 along with position information and time information);
An imaging apparatus configured to capture an image of the field (0048, the unmanned flying device 70 comprises a camera 71 that photographs the field H1); and
A controller configured to acquire the image of the field captured by the imaging apparatus and the self-position information output by the working robot (0063, when communication is established with the unmanned tractor 1 that is running autonomously, the unmanned flying device 70 notifies the unmanned tractor 1 of a change in the work route when the monitoring control unit 76 detects an obstacle W1 on the work route of the unmanned tractor 1 from images captured by the camera 71, at this time the monitoring control unit 75 calculates the position information of the obstacle W1 from the relative position between the obstacle W1 and the unmanned tractor 1 calculated from the captured image and the position information of the unmanned tractor 1 received by the second wireless communication interface 73, and transmits the position information of the obstacle W1 to the unmanned tractor 1 along with a notification of a change in the work route, this allows the unmanned tractor 1 to confirm the position of the obstacle W1 on the work route and change the work route to avoid collision with the obstacle W1, thereby allowing the unmanned tractor 1 to continue autonomous driving),
Wherein, based on a position of the working robot on the captured image and the self-position information output by the working robot, the controller assigns position information to a target of the captured image (0063, when communication is established with the unmanned tractor 1 that is running autonomously, the unmanned flying device 70 notifies the unmanned tractor 1 of a change in the work route when the monitoring control unit 76 detects an obstacle W1 on the work route of the unmanned tractor 1 from images captured by the camera 71, at this time the monitoring control unit 75 calculates the position information of the obstacle W1 from the relative position between the obstacle W1 and the unmanned tractor 1 calculated from the captured image and the position information of the unmanned tractor 1 received by the second wireless communication interface 73, and transmits the position information of the obstacle W1 to the unmanned tractor 1 along with a notification of a change in the work route, this allows the unmanned tractor 1 to confirm the position of the obstacle W1 on the work route and change the work route to avoid collision with the obstacle W1, thereby allowing the unmanned tractor 1 to continue autonomous driving); and
Wherein the controller controls the autonomous travel of the working robot based on the position information (0063, when communication is established with the unmanned tractor 1 that is running autonomously, the unmanned flying device 70 notifies the unmanned tractor 1 of a change in the work route when the monitoring control unit 76 detects an obstacle W1 on the work route of the unmanned tractor 1 from images captured by the camera 71, at this time the monitoring control unit 75 calculates the position information of the obstacle W1 from the relative position between the obstacle W1 and the unmanned tractor 1 calculated from the captured image and the position information of the unmanned tractor 1 received by the second wireless communication interface 73, and transmits the position information of the obstacle W1 to the unmanned tractor 1 along with a notification of a change in the work route, this allows the unmanned tractor 1 to confirm the position of the obstacle W1 on the work route and change the work route to avoid collision with the obstacle W1, thereby allowing the unmanned tractor 1 to continue autonomous driving).
Murata fails to explicitly disclose, however, wherein the controller acquires self-position information of the working robot at least at two points.
Ohtomo, however, in an analogous field of endeavor, does teach wherein the controller acquires self-position information of the working robot at least at two points (0100, Step 01, A position as required during the flight of the flying vehicle system 2 is set as a point P1 and GPS coordinate A1 of the point P1 is acquired by the GPS device 8. The GPS coordinate A1 thus acquired is transmitted to the ground base station 4 via the remote controller 5, 0103, Step 03, The flying vehicle system 2 is moved to a point P2 at another position as required. Here, the moving distance is calculated based on the coordinates of the point P1 and the point P2, and the length of the moving distance is determined by taking the flying height of the flying vehicle system 2 and the accuracy needed for the measurement into consideration).
Murata and Ohtomo are analogous because they are in a similar field of endeavor, e.g., vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the acquiring of the self-position information at multiple points of Ohtomo in order to provide a means of increasing the accuracy of the self-position information. The motivation to combine is to ensure that the position information is relevant to the robot.
The combination of Murata and Ohtomo fails to explicitly teach, however, wherein the controller is configured to receive by a touch or a cursor, via input to a display input part of a display device, a position of a target on the captured image displayed on a screen of the display device.
Ackerman, however, in an analogous field of endeavor, does teach a controller configured to receive by a touch or a cursor, via input to a display input part of a display device, a position of a target on the captured image displayed on a screen of the display device (0058, An example operation of AR virtual boundary generation system 306 is discussed below with respect to FIG. 5. Briefly, however, system 306 performs image-based AR boundary generation using mobile device 224. System 306 includes image acquisition logic 326 configured to acquire images of the worksite (e.g., images 316 captured by image captured component(s) 267 of mobile device 224). This can include a still image, a series of still images, a video feed, etc. The images can be displayed to the user on a display device, such as display device 275 of mobile device 224, using user interface component 328. Based on user input detected by user interface component 328, virtual marker (e.g., flag) generator and identification logic 330 is configured to generate virtual markers corresponding to positions in the acquired images. Coordinate generation and translation logic 332 is configured to generate coordinates for the virtual markers and/or translate the coordinates between different coordinate systems. For example, coordinates can be translated between local coordinates in a local coordinate system and global coordinates in a global coordinate system (e.g., World Geodetic System (WGS), etc.) using reference points generated by reference point generator logic 334. One example of logic 334 uses ground control points to correlate local coordinates of the virtual markers to the global coordinates. Ground control points illustratively have known locations, defined with very high accuracy, on or around the worksite.).
Murata, Ohtomo, and Ackerman are analogous because they are in a similar field of endeavor, e.g., device localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the marker input of Ackerman in order to provide a means of affirmatively designating positions of interest in an image. The motivation to combine is to allow a user to have increased control in the operations of the device.
Regarding claim 2, the combination of Murata, Ohtomo, and Ackerman teaches the working robot system according to claim 1, and Murata further teaches wherein the self-position information is position information of an actual coordinate output by a self-position detector of the working robot (0018, the reference station 60 calculates position information from the satellite signal and the vehicle signal using the reference station communication device 62 by using the RTK positioning method or the like, and transmits it to the tractor 1 via the wireless communication antenna 64. The tractor 1 corrects the satellite positioning information measured by the positioning antenna 6 using correction information transmitted from the reference station 60 to obtain the current position information of the tractor 1, e.g., latitude information and longitude information).
Regarding claim 4, the combination of Murata, Ohtomo, and Ackerman teaches the working robot system according to claim 1, and Ohtomo further teaches wherein position information at the two points are acquired as the working robot moves (0100, Step 01, A position as required during the flight of the flying vehicle system 2 is set as a point P1 and GPS coordinate A1 of the point P1 is acquired by the GPS device 8. The GPS coordinate A1 thus acquired is transmitted to the ground base station 4 via the remote controller 5, 0103, Step 03, The flying vehicle system 2 is moved to a point P2 at another position as required. Here, the moving distance is calculated based on the coordinates of the point P1 and the point P2, and the length of the moving distance is determined by taking the flying height of the flying vehicle system 2 and the accuracy needed for the measurement into consideration).
Murata, Ohtomo, and Ackerman are analogous because they are in a similar field of endeavor, e.g., vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the position information at the two points being acquired while moving of Ohtomo in order to provide a means of capturing position information at multiple points. The motivation to combine is to ensure that the position information is relevant to the robot.
Regarding claim 5, the combination of Murata, Ohtomo, and Ackerman teaches the working robot system according to claim 1, and Ohtomo further teaches wherein the self-position information at the at least two points are acquired at different times (0100, Step 01, A position as required during the flight of the flying vehicle system 2 is set as a point P1 and GPS coordinate A1 of the point P1 is acquired by the GPS device 8. The GPS coordinate A1 thus acquired is transmitted to the ground base station 4 via the remote controller 5, 0103, Step 03, The flying vehicle system 2 is moved to a point P2 at another position as required. Here, the moving distance is calculated based on the coordinates of the point P1 and the point P2, and the length of the moving distance is determined by taking the flying height of the flying vehicle system 2 and the accuracy needed for the measurement into consideration, Examiner's note: if the position information is taken at two different points spaced apart in space, they must necessarily be taken at different times if the position is based off of the position of a single working vehicle).
Murata, Ohtomo, and Ackerman are analogous because they are in a similar field of endeavor, e.g., vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the position information at the two points being acquired at different times of Ohtomo in order to provide a means of capturing position information at multiple points. The motivation to combine is to ensure that the position information is relevant to the robot.
Regarding claim 6, the combination of Murata, Ohtomo, and Ackerman teaches the working robot system according to claim 1, and Ohtomo further teaches wherein the self-position information at the at least two points are acquired from one working robot or different working robots (0100, Step 01, A position as required during the flight of the flying vehicle system 2 is set as a point P1 and GPS coordinate A1 of the point P1 is acquired by the GPS device 8. The GPS coordinate A1 thus acquired is transmitted to the ground base station 4 via the remote controller 5, 0103, Step 03, The flying vehicle system 2 is moved to a point P2 at another position as required. Here, the moving distance is calculated based on the coordinates of the point P1 and the point P2, and the length of the moving distance is determined by taking the flying height of the flying vehicle system 2 and the accuracy needed for the measurement into consideration).
Murata, Ohtomo, and Ackerman are analogous because they are in a similar field of endeavor, e.g., vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the position information at the two points being acquired by one working robot of Ohtomo in order to provide a means of capturing position information at multiple points. The motivation to combine is to ensure that the position information is relevant to the robot.
Regarding claim 7, the combination of Murata, Ohtomo, and Ackerman teaches the working robot system according to claim 1, and Ohtomo further teaches wherein an imaging condition of the imaging apparatus can be adjusted (0056, A control box 31 is provided on a lower end of the shaft 6. Inside the control box 31, the control unit 35 is accommodated. A camera holder 32 is disposed on the lower surface of the control box 31, and the camera 7 is provided on the camera holder 32 via a horizontal axis 33. The camera 7 is rotatable around the horizontal shaft 33 as the center and an image pickup direction changing motor (not shown) for rotating the camera 7 is installed via the horizontal shaft 33. A reference posture of the camera 7 is maintained with an optical axis in vertical direction, and the image pickup direction changing motor rotates the camera 7 at an angle as required with respect to the vertical direction according to an instruction from the control unit 35).
Murata, Ohtomo, and Ackerman are analogous because they are in a similar field of endeavor, e.g., vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have included the imaging condition adjustment of Ohtomo in order to provide a means of changing imaging conditions. The motivation to combine is to ensure that the imaging device is able to obtain information relevant to the position of the vehicle.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Murata in view of Ohtomo, and further in view of Uemura (US 20190113928 A1), hereafter Uemura.
Regarding claim 9, the combination of Murata and Ohtomo teaches the working robot system according to claim 1, but fails to explicitly teach wherein the controller outputs the captured image having the position information to a display device.
Uemura, however, in an analogous field of endeavor, does teach wherein the controller outputs the captured image having the position information to a display device (0019-0024, a work area determination program for an autonomous traveling work vehicle, the program comprising: an image acquisition function for acquiring a photographic image photographed by a photographing device of a predetermined area including a work area, a photographing position acquisition function for acquiring position information indicative of a position where the photographic image was acquired, a map generation function for generating a map based on the photographic image and the position information, a displaying function for displaying the map in the displaying section, and a work area determination function for determining the work area where the autonomous traveling work vehicle is to work, based on an area designation for the map displayed by the displaying function).
Murata, Ohtomo, Ackerman, and Uemura are analogous because they are in a similar field of endeavor, e.g., work vehicle localization systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention, with a reasonable expectation of success, to have modified Murata to have included the outputting of the image to a display device of Uemura in order to provide a means for a user to view the captured image. The motivation to combine is to allow a user to monitor the environment of the working vehicle.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fukuda (US 20150296707 A1) teaches a robotic lawnmowing device having a display with means for input, wherein a map image is displayed of the location of a garden, and a user is able to draw a border on the map image.
Engle (US 20200064826 A1) teaches an autonomous vehicle which is able to receive image data from a mobile device, wherein the image data is indicative of a position surrounding the mobile device, wherein the position is selected by a user from an image captured by a camera of the mobile device.
Maekawa (US 20170217589 A1) teaches an aerial vehicle which captures an image of the ground, wherein a user is able to input positions of a plurality of markers by touching a display scene.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAKE A WOOD whose telephone number is (571)272-6830. The examiner can normally be reached M-F, 8:00 AM to 4:30 PM Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BLAKE A WOOD/ Examiner, Art Unit 3658
Prosecution Insights