DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to Applicant's Amendment and Remarks filed on 6/22/2026. This Action is made FINAL.
Claims 1-5, 7-9 are pending for examination.
Response to Arguments
(A) Applicant’s arguments, see page 5, filed “As discussed during the interview, absent specific and unambiguous identification of claim language that is allegedly indefinite, Applicant respectfully submits that the claims satisfy the requirements of 35 U.S.C. § 112(b). Nevertheless, a portion of the claims have been amended herein for which Applicant submits that the amendments satisfy the requirements of 35 U.S.C. §112(b).Withdrawal of the rejection is respectfully requested” on 6/22/2026, with respect to Claim Rejections - 35 USC § 112 have been fully considered and are persuasive.
As to point (A), the Claim Rejections - 35 USC § 112 of 1-9 has been withdrawn.
(B) Applicant's arguments filed “Applicant respectfully submits that Yamamoto fails to disclose the features of amended independent claim 1. In particular, Yamamoto fails to disclose the features of amended independent claim 1 regarding an adverbial operation of a motion of the autonomous traveling robot is input by using the handwritten input interface or the voice input interface, and as similarly recited in independent claims 8 and 9. A portion of these features is recited in dependent claim 6, which has been incorporated into independent claims 1, 8, and 9, and subsequently cancelled without prejudice or disclaimer.” on 6/22/2026 have been fully considered but they are not persuasive.
As to point (B), the examiner respectfully disagrees. The examiner further notes Yamamoto fully disclosed the claimed limitation in Para 73-79. In particular, Yamamoto disclosed a process of receiving input and controlling the robot to performing the operation which would fully encompass the amended limitation of “an adverbial operation of a motion of the autonomous traveling robot is input by using the handwritten input interface or the voice input interface”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5, 7-9 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yamamoto (US20210178581A1).
Regarding claim 1, Yamamoto teaches An autonomous traveling robot operation system comprising:
an autonomous traveling robot configured to shoot an environment near the
autonomous traveling robot and operates an object(Yamamoto: Fig. 1; Para 36 “The robot 100 shoots the first environment in which the robot 100 is located by a stereo camera 131 (an imaging unit), and transmits the shot image to the remote terminal 300 via the Internet 600”; Para 31 “The robot 100 performs a grasping motion or the like by a hand 124 in accordance with an operation of the remote terminal 300 by the user”);
a handwritten input interface configured to display an image shot by the autonomous traveling robot and receive a handwritten input to the displayed image(Yamamoto: Fig. 1; Para 37 “The remote terminal 300 is, for example, a tablet terminal, and includes a display panel 341 on which a touch panel is superimposed. The shot image received from the robot 100 is displayed on the display panel 341, and thus a user can indirectly view the first environment in which the robot 100 is located. Further, a user can input handwritten input information by handwriting to the shot image displayed on the display panel 341. The handwritten input information is, for example, information indicating an object to be grasped which has been requested to be grasped by the hand 124, a way of performing a grasping motion with regard to the object to be grasped, and the like.”); and
a voice input interface configured to receive a voice input related to the object (Yamamoto: Para 110: Para “a user may utter in a microphone or the like of the remote terminal 300, and the remote terminal 300 may recognize the content of this user's utterance by using a common voice recognition technique, convert it into text information, and use the converted text information as text information of a user's utterance”),
wherein the autonomous traveling robot operation system operates the autonomous traveling robot in such a way that the autonomous traveling robot operates the object in accordance with instructions of the handwritten input and the voice input(Yamamoto: Fig. 1, 10, and 12; Para 97: “Upon receiving the handwritten input information which a user has input to the shot imager from the remote terminal 300, the estimation unit 152 of the robot 100, based on this handwritten input information and a conversation history of the text chat, estimates an object to be grasped which has been requested to be grasped by the hand 124 and estimates a way of performing a grasping motion by the hand 124, the grasping motion having been requested to be performed with regard to the estimated object to be grasped (Step S13). At this time, the estimation unit 152 acquires from the recognition unit 151 the information of the objects that can be grasped shown in the shot image to which the handwritten input information is input, and estimates the object to be grasped from among the objects that can be grasped based on the handwritten input information and the conversation history of the text chat. Further, the estimation unit 152 analyzes the content of the handwritten input information and the content of the conversation history of the text chat, and performs the above-described estimation while at the same time confirming the analyzed contents with the remote terminal 300 using the text information of the text chat”; Para 98 “After that, the control unit 150 of the robot 100 generates a trajectory of the hand 124 for enabling the grasping motion that has been requested to be performed with regard to the object to be grasped (Step S14). When the control unit 150 has generated a trajectory of the hand 124, the control unit 150 controls the upper-body drive unit 146 in accordance with the generated trajectory, whereby the grasping motion is performed by the hand 124 with regard to the object to be grasped (Step S15)”), and
an adverbial operation of a motion of the autonomous traveling robot is input by using the handwritten input interface or the voice input interface(Yamamoto: Para 73 “the estimation unit 152 analyzes (i.e., determines) that a way of performing a grasping motion is to hold and lift the object to be grasped based on the text information of “Get this”. Further, the estimation unit 152 analyzes (i.e., determines) that the object to be grasped among the objects that can be grasped which the recognition unit 151 has recognized is the smartphone 403 located at the input position of the handwritten input information 931 based on the handwritten input information 931. Note that the estimation unit 152 can recognize the input position of the handwritten input information 931 on the shot image 311 by any method. For example, if the remote terminal 300 includes, in the handwritten input information 931, position information indicating the input position of the handwritten input information 931 on the shot image 311 and transmits this handwritten input information 931, the estimation unit 152 can recognize the input position of the handwritten input information 931 based on this position information. Alternatively, if the remote terminal 300 transmits the shot image 311 processed into a state in which the handwritten input information 931 has been input, the estimation unit 152 can recognize the input position of the handwritten input information 931 based on this shot image 311”; Para 74 “in order to confirm with a user that the object to be grasped is the smartphone 403, the estimation unit 152 generates text information (the text box 921) of a response utterance “Okay. Is it a smartphone?” and transmits the generated text information to the remote terminal 300”; Para 75 “the robot 100 receives text information (the text box 912) of a user's utterance “Yes. Bring it to me” from the remote terminal 300. Thus, the estimation unit 152 estimates that the object to be grasped which has been requested to be grasped by the hand 124 is the smartphone 403, and that a way of performing a grasping motion is to hold and lift the smartphone 403”; Para 77 “the estimation unit 152 analyzes (i.e., determines), based on the text information of “Bring it to me”, that an additionally requested motion of the robot 100 is to convey the smartphone 403 held by the grasping motion to “me”; Para 78 “in order to confirm where “me” is, the estimation unit 152 generates text information (the text box 923) of a response utterance “Are you in the living room?” and transmits the generated text information to the remote terminal 300”; Para 79 “the robot 100 receives text information (the text box 913) of a user's utterance “Yes, thank you” from the remote terminal 300. Thus, the estimation unit 152 estimates that the robot 100 has been additionally requested to convey the smartphone 403 to the living room. Consequently, the estimation unit 152 estimates that the overall motion which the robot 100 has been requested to perform is to hold the smartphone 403 and convey it to the living room”)
Regarding claim 2, Yamamoto teaches The autonomous traveling robot operation system according to claim 1,
wherein the voice input interface enables a user to have a conversation with the autonomous traveling robot using a voice recognition unit that receives a voice(Yamamoto: Para 110: Para “a user may utter in a microphone or the like of the remote terminal 300, and the remote terminal 300 may recognize the content of this user's utterance by using a common voice recognition technique, convert it into text information, and use the converted text information as text information of a user's utterance”),
recognizes the voice, and outputs the motion, the voice recognition unit being configured by performing machine learning, and inputs the motion of the autonomous traveling robot(Yamamoto: Para 110: Para “a user may utter in a microphone or the like of the remote terminal 300, and the remote terminal 300 may recognize the content of this user's utterance by using a common voice recognition technique, convert it into text information, and use the converted text information as text information of a user's utterance”; Para 68 “The learned model 181 is a neural network learned from teaching data which is a combination of an image showing objects that can be grasped by the hand 124 and a correct answer to which area of the image is the object that can be grasped. At this time, by preparing the teaching data so as to further indicate the names of the objects that can be grasped, the distances to the objects that can be grasped, and the directions in which the objects that can be grasped are located shown in the image, the learned model 181, which uses the shot image as an input image, can output not only the objects that can be grasped but also the names of the objects that can be grasped, the distances to the objects that can be grasped, and the directions in which the objects that can be grasped are located”; Para 97: “Upon receiving the handwritten input information which a user has input to the shot imager from the remote terminal 300, the estimation unit 152 of the robot 100, based on this handwritten input information and a conversation history of the text chat, estimates an object to be grasped which has been requested to be grasped by the hand 124 and estimates a way of performing a grasping motion by the hand 124, the grasping motion having been requested to be performed with regard to the estimated object to be grasped (Step S13). At this time, the estimation unit 152 acquires from the recognition unit 151 the information of the objects that can be grasped shown in the shot image to which the handwritten input information is input, and estimates the object to be grasped from among the objects that can be grasped based on the handwritten input information and the conversation history of the text chat. Further, the estimation unit 152 analyzes the content of the handwritten input information and the content of the conversation history of the text chat, and performs the above-described estimation while at the same time confirming the analyzed contents with the remote terminal 300 using the text information of the text chat”; Para 98 “After that, the control unit 150 of the robot 100 generates a trajectory of the hand 124 for enabling the grasping motion that has been requested to be performed with regard to the object to be grasped (Step S14). When the control unit 150 has generated a trajectory of the hand 124, the control unit 150 controls the upper-body drive unit 146 in accordance with the generated trajectory, whereby the grasping motion is performed by the hand 124 with regard to the object to be grasped (Step S15)”).
Regarding claim 3, Yamamoto teaches The autonomous traveling robot operation system according to claim 1,
wherein an instruction is sent to the object that is not displayed on the image using the voice input interface(Yamamoto: Fig. 12; Para 110: Para “a user may utter in a microphone or the like of the remote terminal 300, and the remote terminal 300 may recognize the content of this user's utterance by using a common voice recognition technique, convert it into text information, and use the converted text information as text information of a user's utterance”; Para 77 “the estimation unit 152 analyzes (i.e., determines), based on the text information of “Bring it to me”, that an additionally requested motion of the robot 100 is to convey the smartphone 403 held by the grasping motion to “me”; Para “Then, in order to confirm where “me” is, the estimation unit 152 generates text information (the text box 923) of a response utterance “Are you in the living room?” and transmits the generated text information to the remote terminal 300”; Para 79 “the robot 100 receives text information (the text box 913) of a user's utterance “Yes, thank you” from the remote terminal 300. Thus, the estimation unit 152 estimates that the robot 100 has been additionally requested to convey the smartphone 403 to the living room. Consequently, the estimation unit 152 estimates that the overall motion which the robot 100 has been requested to perform is to hold the smartphone 403 and convey it to the living room”; i.e. perform is to hold the smartphone and convey it to the living room is not displayed in the image).
Regarding claim 4, Yamamoto teaches The autonomous traveling robot operation system according to claim 1,
wherein the handwritten input interface inputs the object using an object estimation unit configured to perform machine learning so as to receive the image as an input, estimate an object in the image, and output a result of the estimation robot(Yamamoto: Para: 60 “The learned model 181 is a learned model that uses a shot image as an input image and outputs objects that can be grasped shown in the shot image”; Para 68 “The learned model 181 is a neural network learned from teaching data which is a combination of an image showing objects that can be grasped by the hand 124 and a correct answer to which area of the image is the object that can be grasped. At this time, by preparing the teaching data so as to further indicate the names of the objects that can be grasped, the distances to the objects that can be grasped, and the directions in which the objects that can be grasped are located shown in the image, the learned model 181, which uses the shot image as an input image, can output not only the objects that can be grasped but also the names of the objects that can be grasped, the distances to the objects that can be grasped, and the directions in which the objects that can be grasped are located”).
Regarding claim 5, Yamamoto teaches The autonomous traveling robot operation system according to claim 1,
wherein a trajectory of the autonomous traveling robot is input by using the handwritten input interface(Yamamoto: Fig. 1; Para 37 “The remote terminal 300 is, for example, a tablet terminal, and includes a display panel 341 on which a touch panel is superimposed. The shot image received from the robot 100 is displayed on the display panel 341, and thus a user can indirectly view the first environment in which the robot 100 is located. Further, a user can input handwritten input information by handwriting to the shot image displayed on the display panel 341. The handwritten input information is, for example, information indicating an object to be grasped which has been requested to be grasped by the hand 124, a way of performing a grasping motion with regard to the object to be grasped, and the like”).
Regarding claim 7, Yamamoto teaches The autonomous traveling robot operation system according to claim 1,
wherein the autonomous traveling robot is operated to perform grasping, cutting, moving, screw fixing, or welding(Yamamoto: Fig. 1, 10, and 12; Para 97 “Upon receiving the handwritten input information which a user has input to the shot imager from the remote terminal 300, the estimation unit 152 of the robot 100, based on this handwritten input information and a conversation history of the text chat, estimates an object to be grasped which has been requested to be grasped by the hand 124 and estimates a way of performing a grasping motion by the hand 124, the grasping motion having been requested to be performed with regard to the estimated object to be grasped (Step S13). At this time, the estimation unit 152 acquires from the recognition unit 151 the information of the objects that can be grasped shown in the shot image to which the handwritten input information is input, and estimates the object to be grasped from among the objects that can be grasped based on the handwritten input information and the conversation history of the text chat. Further, the estimation unit 152 analyzes the content of the handwritten input information and the content of the conversation history of the text chat, and performs the above-described estimation while at the same time confirming the analyzed contents with the remote terminal 300 using the text information of the text chat”; Para 98 “After that, the control unit 150 of the robot 100 generates a trajectory of the hand 124 for enabling the grasping motion that has been requested to be performed with regard to the object to be grasped (Step S14). When the control unit 150 has generated a trajectory of the hand 124, the control unit 150 controls the upper-body drive unit 146 in accordance with the generated trajectory, whereby the grasping motion is performed by the hand 124 with regard to the object to be grasped (Step S15)”).
As per claim 8, it recites An autonomous traveling robot operation method having limitations similar to those of claim 1 and therefore is rejected on the same basis.
As per claim 9, it recites A non-transitory computer readable medium storing a program for causing an information processing apparatus to operate an autonomous traveling robot having limitations similar to those of claim 1 and therefore is rejected on the same basis. Yamamoto further teaches non-transitory computer readable medium(Yamamoto: Para 59 “A memory 180 is a nonvolatile storage medium. For example, a solid-state drive is used for the memory 180. The memory 180 stores, in addition to a control program for controlling the robot 100, various parameter values, functions, lookup tables, and the like used for the control and the calculation. In particular, the memory 180 stores a learned model 181, an utterance DB 182, and a map DB 183”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
ROH (US20200130197A1) disclosed a voice input unit configured to receive a voice input of a user; a first display capable of receiving a touch input; a second display larger than the first display; and a controller configured to perform control such that a screen to be displayed in response to the voice input or the touch input is displayed on at least one of the first display or the second display based on a type and an amount of information included in the screen, and accordingly, it is possible to provide information and services more effectively using two displays.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WENYUAN YANG whose telephone number is (571)272-5455. The examiner can normally be reached Monday - Thursday 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hitesh Patel can be reached at (571) 270-5442. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.Y./Examiner, Art Unit 3667
/Hitesh Patel/Supervisory Patent Examiner, Art Unit 3667
7/13/26
Prosecution Insights