DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
Claim Rejections - 35 USC § 112(b)
The rejection of the claims provided in the Office Action dated 11/13/2025 (hereafter referred to as the previous Office Action) is withdrawn in light of Applicant’s amendments filed 03/12/2026 (hereafter referred to as Applicant’s amendments).
Claim Rejections - 35 USC § 102
Applicant’s arguments filed 03/12/2026 have been fully considered and one particular argument is found persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Joosten et al. (US 20210302922 A1), previously recited as a reference in the rejection of Claims 5, 19, and 21 – 23.
In specific, Examiner finds Applicant’s argument that Johnson does not explicitly disclose “prompting the operator by a controller” to be valid. Johnson appears to at most provide implications to how prior art may handle robot-operator/user interaction that may indicate as much, and as presently understood by the Examiner limits the disclosure to the operator/user providing commands and similar without a controller specifically prompting them under the explicit definition of “prompt” in the context of “by a controller” outlined in Applicant’s Remarks on Page 14.
Examiner notes with respect to other arguments related thereto found on Page 14, that Examiner made no such interpretations of the term as “To argue that a human's unilateral judgement or determination to intervene in Johnson's motion plan is equivalent to a controller-initiated prompt”. The claim did not previously recite “by a controller” and 12. of the previous Office Action was explicitly clear in that interpretations were made in said context. [0022] of Applicant’s specification first broadly states “If it is determined that the robot 18 was unsuccessful, the operator may be prompted for additional operator input (20) and the method may be repeated using the original operator input and the additional operator input (34)” before then providing that “the controller module 16 … may prompt the operator for additional inputs when needed” (emphasis added). It is therefore clear that the plain meaning BRI of the phrase “prompting” was not inconsistent with the specification, and interpreting it as requiring it to be performed by a controller would be to import limitations into the claims. MPEP 2111 relates.
With respect to Applicant’s remaining arguments, they are found unpersuasive. Similar to the discussion above with respect to MPEP 2111, Applicant’s arguments appear generally to rely on narrower interpretations of the claim limitations, terms, and phrases than is presently supported by the current claim language as the broadest reasonable interpretation thereof. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On page 12 of Applicant’s Remarks, Applicant states, “Par. [0032] discloses a task plan 130 and motion plans 132a-d, which may be considered analogous to the claimed "task" and "one or more robot movements" respectively”. This mapping of claim terms provided by Applicant is not the only appropriate mapping. Furthermore, the claims do not presently require the “task” and “one or more robot movements” to even be distinct entities. The task may readily be a particular action or other item of a greater whole. Applicant continues to use especially broad phrasing in their claim language which has been discussed in previous Office Actions. Note that Applicant still uses the phrase “an operator input representing”. The operator input itself, the task, the robot movements, or the nature of representing are not claimed with any particularity.
Consequently, the following arguments provided by Applicant with respect to [0032], [0046], Figures 3A and 3B or [0063] and Figure 4A, or [0065] and Figure 4B appear wholly reliant on Applicant’s chosen narrow interpretation of the claim terms which are not in alignment with the broadest reasonable interpretation thereof. The present claim language does not require the particular flow with respect to Johnson which Applicant believes renders these teachings as not reading on the claims and the claim is explicitly recited as a comprising claim. The operator input may presently comprise any number of tasks and robot movements, wherein only one or some of those correspond to a particular set of “software commands”. Relatedly, the nature of “translating … into software commands” is not claimed with particularity such that the software commands comprise the totality of the user input. Thus, completion of a particular desired task does not mean completion of everything that a particular operator input comprises such as every desired task and every desired action.
With respect to Joosten and Thomaz, Applicant’s arguments are wholly conclusory and consequently unpersuasive.
Claim 1 only recites “after the robot has been operated”. Applicant has made clear in the latest amendments that this is any kind of operation/operating whatsoever as the previously accompanying and further limiting statement of “using the software commands” has been deliberately removed without further replacement or equivalent substitution. Therefore, this may occur at any point after the robot may have been considered as operating in any capacity. Furthermore, the nature of “determining whether the software commands successfully directed the robot to complete the desired task” is not claimed with any particularity. Finally, the term “software commands” are not claimed with any particularity. This therefore might simply be an evaluation as in Claim 23 of if a whole or portion of a translated natural language operator input/instruction/command is recognized. An unrecognized command is clearly not a command which successfully directs anything or anyone to do anything. As is shown in Claim 23, the response may be of prompting for further input which then reads on the next limitation of prompting by a controller for said input.
Finally, and alternatively, as Johnson already discloses failure observance, the combination with Joosten may instead simply be of prompting by a controller upon detected failure.
In the interest of compact prosecution, see the Conclusion section for other references which disclose the basic concept of prompting for input from a user upon system detected faults, error, failure, etc.
Claim Rejections - 35 USC § 103
Applicant's arguments filed 10/07/2025 have been fully considered but they are not considered persuasive. The arguments appear conclusory and wholly reliant on the arguments directed towards Johnson alone.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 – 6 and 18 – 23 are rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. (US 20200086498 A1) in view of Joosten et al. (US 20210302922 A1).
Regarding Claim 1, Johnson teaches:
A method of programming a robot (See at least robot arm 110) by an operator (See at least [0010] “In an embodiment, the user input is speech, a gesture as detected by a camera, or an input from a control”), comprising:
receiving an operator input representing a desired task to be completed by the robot via one or more robot movements (Examiner notes that the “operator input” might be the beginning order disclosed, or could be a modification of actions taken by the robot of said order, or particular parts or actions thereof. For the beginning order disclosure, see at least [0028] “Tenth, the system should be able to receive an electronic order from a user, assemble the meal for the user, and place the meal for the user in a designated area for pickup automatically with minimal human involvement”, [0031] “To initiate an order, a patron in the patron area 120 enters an order 124 in an ordering station 122a-b, which is forwarded to a network 126. Alternatively, a patron on a mobile device 128 can, within or outside of the patron area 120, generate an optional order 132. Regardless of the source of the order, the network 126 forwards the order to a controller 114 of the robot arm 110. The controller generates a task plan 130 for the robot arm 110 to execute”. For the task plan and parts thereof corresponding to a particular order, see at least [0032] “The task plan 130 includes a list of motion plans 132a-d for the robot arm 110 to execute”. For the modification disclosure, see at least [0036] “Therefore, this disclosure enables these plans for selecting and manipulating material or objects to be subject to subsequent guidance and plan-modification via voice input during the manipulation. For example, when scooping ice cream, if self-adhesion begins pulling a partial scoop out of the right side of the disher tool, the scoop may be saved by altering the disher's trajectory so that it veers rightward. … Therefore, human intervention via human speech can improve the robotic system by recognizing voice commands in real-time, such as the command “move a little to the right”);
translating the operator input into software commands directing the robot to perform the one or more robot movements (See above citations. For further support, see [0040] “Based on the current state and executing plan, any recognized speech command is mapped to possible changes of state and plan. Current observed conditions and states may also inform the automatic speech recognition system. Recognized speech can be treated as an event and fed into a finite state machine (FSM) or other system controller. Some events may of course have no effect. In other words, the robotic system uses motion and physical feedback as inputs to determine the outside world and adjust in real time. However, in embodiments of the present disclosure, the system further includes voice or acoustic input in combination with other inputs” or Figure 2 and associated paragraphs [0054] – [0060]) to complete the desired task (Statement of intent, also disclosed, see preceding);
programming the robot with the software commands (See again above citations);
operating the robot by executing the software commands to perform the one or more robot movements (See again above citations) for completing the desired task (Statement of intent, lacks proper antecedent basis for anything else);
after the robot has been operated, determining whether the software commands successfully directed the robot to complete the desired task;
in response to determining that the software commands are unsuccessful, prompting the operator … for additional operator input;
receiving the additional operator input and translating the additional operator input into updated software commands directing the robot to perform the desired robot movement,
programming the robot with the updated software commands; and
operating the robot using the updated software commands to perform the desired robot movement (Johnson discloses modifying plans both as they are executed and after they are executed based on user and sensor input related to “success”. Examiner notes that a plan may comprise a plurality of motion plans or “desired robot movements” ([0032]). See at least [0046] “In an alternate approach, the system can allow re-tries/repeated attempts after failed plan executions. Instead of fixing failed plans or handling unforeseen contingencies in real-time, plans may be executed to completion, upon which the system or operator evaluates the result as deficient, and the system executes another plan” as well as Figure 3A and Figure 3B, or [0063] “FIG. 4A is a state diagram 400 illustrating an example embodiment of a robot decision making process according to the present disclosure. After providing an initial plan 402, during execution of the plan 404 and upon additional input, such as voice modification, the system transforms the plan, or interpolates a new plan 408” and Figure 4A, or “[0065] FIG. 4B is a flow diagram 400 illustrating an example embodiment of a process employed by the present disclosure. The process first creates a plan (452). The process then executes the plan (454) while monitoring sounds and sensory input (456). In this embodiment, if the process determines that the monitored sounds and sensory inputs require modification to the plan (458), the system modifies the plan (460) instead of creating a new one, and then executes the modified plan (454)” and Figure 4B).
Johnson does not explicitly teach, but in combination with Joosten teaches (the below recitations are with respect to Joosten):
…
[in response to determining that the software commands are unsuccessful, prompting the operator] by a controller [for additional operator input] (See at least [0114] “If, on the other hand, no device (the attached MAS, the attached user input device, and any other devices) can process the user request, then in a step 1052, the user may be prompted for clarification. For example, the user's original voice command may be in the form of incomprehensible speech or a vague command. The user may have input an audio command to “find Harry Potter” but the speech recognition feature recognized the phrase as “bind harri water”. The MAS or the attached user interacting device may then output a prompt to the user such as “did you mean ‘find Harry Potter’?” or “please repeat your command”. The MAS and all other devices may then return to step 1012 to monitor for another user input in response to the prompt”)
…
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to evaluate the sufficiency of user input for execution as taught by Joosten in the system or method of Johnson with a reasonable expectation of success. It is well understood and routine even in human interactions to verify the sufficiency of instructions given before execution. As illustrated by Joosten, this general concept might be similarly applied to robotics to verify that a given directive is processable that way the robot might ensure its ability to complete an assigned task and maintain user satisfaction and task completion. Furthermore, prompting a user for clarification or further directions through recognition of incomplete or insufficient instructions rather than after failed or unsuccessful execution of such instructions prevents wasted time and user dissatisfaction, and is likely to result in interaction with a user before they potentially become inattentive or unavailable.
Alternatively, Johnson teaches recognition of failure by the system itself of a completed action. It would thus have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to simply evaluate the success of actions at the particular “software commands” level and furthermore have the system (controller) prompt the user directly as disclosed by Joosten with a reasonable expectation of success
Regarding Claim 2, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson further teaches:
further comprising receiving identification and/or location information of one or more objects within a workspace of the robot from one or more environmental sensors (See at least [0030] “The robot arm 110 includes sensor elements/modules such as stereo vision systems (SVS), 3D vision sensors (e.g., Microsoft Kinect™ or an Intel RealSense™), LIDAR sensors, audio sensors (e.g., microphones), inertial sensors (e.g., internal motion unit (IMU), torque sensor, weight sensor, etc.) for sensing aspects of the environment, including pose (i.e., X, Y, Z coordinates and roll, pitch, and yaw angles) of tools for the robot to mate, shape and volume of foodstuffs in ingredient containers, shape and volume of foodstuffs deposited into food assembly container, moving or static obstacles in the environment, etc.”).
Regarding Claim 3, the combination of Johnson and Joosten teaches:
The method according to claim 2,
Johnson further teaches:
wherein the one or more environmental sensors includes a robot vision system (See at least [0030] “The robot arm 110 includes sensor elements/modules such as stereo vision systems (SVS), 3D vision sensors (e.g., Microsoft Kinect™ or an Intel RealSense™), LIDAR sensors … for sensing aspects of the environment … ”).
Regarding Claim 4, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson further teaches:
wherein the operator input is transcribed from a voice input (See at least [0038] “Before manipulation, the system stores a collection of possible motion plans with many variations and sequences of interchangeable components, as well as collection of recognizable speech commands and modifiers, as in verbs and adverbs. Mapping commands to plan components allows speech commands, when recognized by the system, to generate changes in the execution of motion and manipulation plans while in-progress”).
Regarding Claim 5, Johnson teaches:
The method according to claim 1,
Johnson does not explicitly teach, but Joosten explicitly teaches:
wherein the operator input is received from a display screen (Joosten teaches that a user input is not limited to voice commands received via a microphone or similar, and that such inputs are also processed as service requests. See at least [0101] “In a step 908, upon establishing communication between the user interacting device and the MAS, an interactive session may be initiated. … In one embodiment, any input device on the MAS, the user interacting device, and any other devices or in the cloud may be used to monitor for user input”, [0102] “In a step 912, the MAS may receive input from the user (such as voice command for speech recognition). In embodiments where more than one input device monitors for user input, such as other input devices may also receive input from the user”, [0127] “As part of this embodiment, the processor 1108 connects to a user interface 1116. The user interface 1116 may comprise any system or device configured to accept user input to control the mobile device 1100. The user interface 1116 may comprise one or more of the following: microphone, keyboard, roller ball, buttons, wheels, pointer key, touch pad, and touch screen”, and Figure 9).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to include the operational use of a display screen for user inputs as taught by Joosten in the method or system of Johnson with a reasonable expectation of success. The use of displays for inputs are well known and routine in the art and would allow for non-verbal inputs to be provided when a user is unwilling or unable to provide verbal inputs. Furthermore, and alternatively, the limitation might be interpreted as presently constructed as the microphone being located in/with a display screen, which is also well known and routine and would serve to allow for the use of common hardware.
Regarding Claim 6, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson further teaches:
wherein the operator input is received from one or more environmental sensors (See at least [0030] “The robot arm 110 includes sensor elements/modules such as … audio sensors (e.g., microphones) … for sensing aspects of the environment …”, [0042] “In an embodiment, the disclosure employs a speech sensing and recognition apparatus (e.g., a microphone array connected to a computer or controller) in conjunction with a robot system (e.g., a jointed arm, end-effector, and connected cameras) which is able to manipulate the chosen materials or objects”, and Figure 1B. Furthermore, and separately, Examiner notes that what qualifies as an “environmental sensor” is not claimed and can readily be considered as extending to the ordering stations 122 and mobile device 128 which receive user inputs in the form of orders).
Regarding Claim 18, the combination of Johnson and Joosten teaches:
The method according to claim 2,
Johnson further teaches:
wherein the operator input is transcribed from a voice input (See at least [0038] “Before manipulation, the system stores a collection of possible motion plans with many variations and sequences of interchangeable components, as well as collection of recognizable speech commands and modifiers, as in verbs and adverbs. Mapping commands to plan components allows speech commands, when recognized by the system, to generate changes in the execution of motion and manipulation plans while in-progress”).
Regarding Claim 19, Johnson teaches:
The method according to claim 2,
Johnson does not explicitly teach, but Joosten explicitly teaches:
wherein the operator input is received from a display screen (Joosten teaches that a user input is not limited to voice commands received via a microphone or similar, and that such inputs are also processed as service requests. See at least [0101] “In a step 908, upon establishing communication between the user interacting device and the MAS, an interactive session may be initiated. … In one embodiment, any input device on the MAS, the user interacting device, and any other devices or in the cloud may be used to monitor for user input”, [0102] “In a step 912, the MAS may receive input from the user (such as voice command for speech recognition). In embodiments where more than one input device monitors for user input, such as other input devices may also receive input from the user”, [0127] “As part of this embodiment, the processor 1108 connects to a user interface 1116. The user interface 1116 may comprise any system or device configured to accept user input to control the mobile device 1100. The user interface 1116 may comprise one or more of the following: microphone, keyboard, roller ball, buttons, wheels, pointer key, touch pad, and touch screen”, and Figure 9).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to include the operational use of a display screen for user inputs as taught by Joosten in the method or system of Johnson with a reasonable expectation of success. The use of displays for inputs are well known and routine in the art and would allow for non-verbal inputs to be provided when a user is unwilling or unable to provide verbal inputs. Furthermore, and alternatively, the limitation might be interpreted as presently constructed as the microphone being located in/with a display screen, which is also well known and routine and would serve to allow for the use of common hardware.
Regarding Claim 20, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson further teaches:
wherein the operator input, the additional operator input, and a data link between the operator input and the additional operator input are stored in a skill set library that recognizes operator inputs (In light of Applicant’s arguments provided in the Remarks filed 05/06/2025, Applicant appears to believe the limitations provided to be significantly narrower than either supported by Applicant’s specification, or the actual claim language provided.
Examiner notes that the term “skill set library” is not disclosed with particular detail within Applicant’s specification. [0022] of the published specification (US 20230373098 A1), which provides the only recitation of a “skill set library” reads “It may be desirable to store the operator inputs, additional operator inputs and/or translated software commands in a library storage with data links therebetween so that future operator inputs may be able to be autonomously linked to such additional operator inputs and/or software commands (28). The library storage may be included as part of the actuation module 14 and may be a skill set library that recognizes operator inputs”. The only other recitation of the phrase “skill set” is used in [0017] which recites “The set of operator commands that are recognized by the controller module 16 may be called a skill set and may be dynamically extended over the robot's lifetime”. Therefore, even using limitations from Applicant’s specification, a “skill set library that recognizes operator inputs” is a library that recognizes operator inputs comprised of operator commands that are recognized by a controller (or in other words the recognition appears redundant). More simply, the phrase appears to mean a collection of recognizable operator commands, wherein the nature of recognition is completely open to interpretation as Applicant does not describe what the verb “recognize” means in the context of their invention. Under the plain meaning of the verb, recognizing data might be accomplished simply through the act of storing or otherwise processing said data, including for evaluation of sufficiency. As such, storing operator inputs, additional or otherwise, would appear to make whatever they are stored in a “skill set library”.
Therefore, a “skill set library” without further limitation and in light of Applicant’s specification might amount to simply memory where an operator input and additional operator input are stored with some correspondence, and wherein such storing makes the stored data a “skill set library” as claimed, which is already shown as being taught in Claim 1. See also and alternatively [0038] of Johnson where a “collection of possible motion plans … as well as collection of recognizable speech commands and modifiers” are stored and commands are mapped with said collections when recognized, indicating simultaneous storage in memory and a “data link” between information.
Regarding Claim 21, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson has already been shown to teach:
wherein the operator input comprises an action (Claim 1 already recites that an operator input represents “a desired robot movement”, which is an action. See also [0039] “Upon recognizing the commands “turn right, softer”, or “softer, rightward” ”),
The combination of Johnson and Joosten has already been shown to teach:
and wherein said determining whether the operator input is sufficient to translate comprises determining whether the action is known to a system performing the method (Examiner could find no clear and explicit definition or description of “known” or the verb “knowing” within Applicant’s disclosure. Furthermore, “a system” is inclusive of any and all components. Therefore, see again [0038], [0040], and [0060] of Johnson and [0114] of Joosten which have already been provided with respect to Claim 1. An action must be “known” to be able to be recognized and prompting a user for additional input if no device (any system) is able to process (perform) a command requires that the system first make said determination that no device is able to process a command, or in other words see if the action is “known” by any system).
Regarding Claim 22, Johnson and Joosten teaches:
The method according to claim 21,
The combination of Johnson and Joosten has already been shown to teach:
wherein the operator input comprises information about an object associated with said action and wherein said determining whether the operator input is sufficient to translate comprises determining whether the object is known to the system (Examiner notes that the nature of the “object” is not claimed. The “object” might be under the broadest reasonable interpretation a physical object, a goal, or in the linguistic sense a noun or noun equivalent. Examiner notes that wherein the context of the invention is inclusive of speech recognition and translation of verbal commands to software commands, interpretations of the term in a linguistic sense are considered reasonable.
See again [0038], [0040], and [0060] of Johnson and [0114] of Joosten. See additionally that Johnson is directed towards “Manipulation of deformable materials and movable objects” ([0034]) wherein a human provides commands and feedback related to said manipulation, or that targets or nouns are provided for in the commands given to the robot in Joosten such as directions to face, things to track ([0034], [0050]), people or objects to follow ([0094], [0096]), etc.).
Regarding Claim 23, Johnson teaches:
The method according to Claim 1,
Examiner notes that Johnson, under the broadest reasonable interpretation of the claim limitations as presently claimed teaches the limitations as Johnson clearly indicates that only recognized commands are executed, which inherently means that there are unrecognized, or insufficient to translate commands. Johnson also discloses matching words and phrases to a database/library. See at least [0038], [0040], and [0060]. In other words, a sufficiency determination inherently must occur. However, in the interest of compact prosecution, Examiner provides Joosten which clearly and explicitly teaches the limitation:
further comprising determining whether the operator input is sufficient to translate the operator input into the software commands, and prompting the operator to provide an additional operator input if the operator input is determined to be insufficient to translate the operator input into the software commands (See at least [0114] “If, on the other hand, no device (the attached MAS, the attached user input device, and any other devices) can process the user request, then in a step 1052, the user may be prompted for clarification. For example, the user's original voice command may be in the form of incomprehensible speech or a vague command. The user may have input an audio command to “find Harry Potter” but the speech recognition feature recognized the phrase as “bind harri water”. The MAS or the attached user interacting device may then output a prompt to the user such as “did you mean ‘find Harry Potter’?” or “please repeat your command”. The MAS and all other devices may then return to step 1012 to monitor for another user input in response to the prompt”)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to evaluate the sufficiency of user input for execution as taught by Joosten in the system or method of Johnson with a reasonable expectation of success. It is well understood and routine even in human interactions to verify the sufficiency of instructions given before execution. As illustrated by Joosten, this general concept might be similarly applied to robotics to verify that a given directive is processable that way the robot might ensure its ability to complete an assigned task and maintain user satisfaction and task completion. Furthermore, prompting a user for clarification or further directions through recognition of incomplete or insufficient instructions rather than after failed or unsuccessful execution of such instructions prevents wasted time and user dissatisfaction, and is likely to result in interaction with a user before they potentially become inattentive or unavailable.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. in view of Joosten et al. and Thomaz et al. (WO 2018156952 A1).
Regarding Claim 24, the combination of Johnson and Joosten teaches:
The method according to claim 1,
Johnson does not explicitly teach but in combination with Thomaz clearly teaches:
storing the operator input, the additional operator input, and a data link between the operator input and the additional operator input (Examiner notes that the nature of a “data link” is not claimed, nor is it disclosed within Applicant’s specification in any particularity. It is therefore open to particularly broad interpretation under the plain meaning of the term. Examiner has interpreted “data link” to be any information/data which might relate in some manner to other information/data, specifically as presently claimed information/data which might be considered an operator input as well as additional and future operator input. Examiner furthermore notes that the limitations of the claim as a whole appear to be essentially that of machine learning, wherein past data is utilized in future actions), wherein future operator inputs corresponding to the operator input autonomously reference the additional operator input using the data link (Examiner notes that the nature of the terms/phrases “corresponding” and “autonomously reference … using the data link” are not claimed, nor are they disclosed within Applicant’s specification in any particularity. It is therefore open to particularly broad interpretation under the plain meaning of the term. See first at least [0055] which indicates that a plethora of information might be associated with a particular model including markers, marker information 335, models 334, sensory information 336, manipulating element information 337, and success criteria 339. All of this information associated with a model is recorded/stored during each training and execution of a particular model, for example see Figure 10, box 730 and 738 and Figure 11, box 774. This information is used to update/train a model (boxes 738 and 770) and a model might be selected during execution based on information of past run-throughs “At 754, the robotic device can determine whether to move closer to a selected marker prior to generating a trajectory and executing a skill with respect to the selected marker. For example, the robotic device can determine based on the selected set of markers and the selected model whether it should move to be better positioned to execute the skill (e.g., to be closer or more proximate to the marker, to be facing the marker from a certain angle). The robotic device may make this determination based on the sensory information that was recorded during a demonstration of the skill. For example, the robotic device may recognize that it was positioned closer to the marker when it was demonstrated the skill and accordingly adjust its position”. Therefore, Thomaz teaches a variety of “data” such as markers and sensory data which can “link” a particular user input, a selection of the markers or the markers themselves, to a particular model which has been iteratively approved and is linked to the iterations and said data which were used to train and update it. Examiner notes that the claim is not constructed such that the markers could not also be the data link, the data link and user inputs not being claimed with any particularity, and that other data might serve as a “link”. Alternatively, a model selection might be considered a user input and the markers as the data link. Again, Applicant does not claim or even disclose with particularity the claimed feature. Again, and also alternatively, see at least [0085] and [0086] and boxes 758 and 760 which describe calculating transformations based on past marker and keyframe information to compute how to execute a current action)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to include the use of linked data including operator inputs, additional and/or original, as disclosed by Thomaz in the machine learning of Johnson or Johnson in combination with Joosten. The use of machine learning is well understood and routine in the art of robotics. Furthermore, “data links” of the unspecified nature which perform a function of referencing in some manner prior and later corresponding information is likewise well understood and routine. These functions as presently claimed do not appear to be significantly more than typical standard machine learning operations necessary to the function of machine learning, which Johnson already explicitly describes as being utilized. Therefore, the combination with Thomaz merely serves to indicate with further clarity the flow of information in such a process as being inclusive of the claimed limitations. Consequently, the motivation to combine Thomaz and Johnson or Johnson in combination with Joosten is simply to facilitate or illustrate the operation of such a function.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nix et al. (US 20180365913 A1), Wu et al. (US 20180103238 A1), Liu et al. (US 6170067 B1), and Takeuchi (US 5875421 A) which all disclose the common, well understood, and routine concept of prompting a user for input upon detection of an error in a system or executed operation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW C GAMMON whose telephone number is (571)272-4919. The examiner can normally be reached M - F 10:00 - 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ADAM MOTT can be reached on (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW C GAMMON/Examiner, Art Unit 3657 /ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights