DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 29th, 2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on June 29th, 2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments and Amendments
Applicant’s arguments and amendments, filed June 29th, 2026, with respect to the rejection(s) of claims 1-23 under 35 U.S.C. 103 have been fully considered but they are not persuasive. Upon further search and consideration of the amended limitations in the claims, a new ground(s) of rejection is made in view of the previously cited combination of Benaim et al. (US Patent Pub. No. 2019/0126487 A1), Paxton et al. (“Evaluating Methods for End-User Creation of Robot Task Plans”), Liberg et al. (WO Patent Pub. No. 2006/043873 A1), and Lu et al. (“Autonomous Obstacle Legipulation with a Hexapod Robot”). Examiner notes the additional limitations in the claims to describe the claimed robot and robot actions with varying terminology does not limit the scope of the original claim(s), and does not distinguish the robot’s structure or functionality over a combination of the previously cited prior art. Specifically, the Applicant amends certain limitations by removing them from previously presented claims and adding them to other independent or dependent claims. The Examiner notes that while the individual reference(s) used to support a single limitation may be different according to the revised 35 U.S.C. 103 rejections below, those limitations are still taught by the cited combination under broadest reasonable interpretation. Furthermore, a significant number of amended limitations are simply duplicating other essential working parts of the claimed combination, which does not amount to anything more than routine skill in the art.
For at least the reasons stated above, the 35 U.S.C. 103 rejections of claims 1-23 are maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-23 are rejected under 35 U.S.C. 103 as being obvious over Benaim et al. (US Patent Pub. No. 2019/0126487 A1), herein “Benaim”, in view of Paxton et al. (“Evaluating Methods for End-User Creation of Robot Task Plans”), herein “Paxton”, and further in view of Liberg et al. (WO Patent Pub. No. 2006/043873 A1), herein “Liberg”, and Lu et al. (“Autonomous Obstacle Legipulation with a Hexapod Robot”), herein “Lu”, published November 12th, 2020.
Regarding Claims 1, 16 and 19, Benaim discloses a computer-implemented method, a system and a legged robot comprising:
receiving, by data processing hardware of a legged robot, first sensor data from one or more sensors of the legged robot, the first sensor data corresponding to an environment of the robot (See 0069, “[…] at least one sensor collects a plurality of images in a given scene where the target object or objects are present. The sensor may be a digital camera, an analog camera, a depth map sensor and the like.”);
determining, by the data processing hardware, based on the first sensor data, a first object in the environment and a second object in the environment (See 0003, “[…] robotic machine has to have capability of identifying objects in order to perform relevant tasks related to an object. Vision systems with different types of sensors combined with processing units executing image analysis software are widely used to identify objects in an operational scene.” See also 0134, “[…] to automatically identify different objects in a working environment scenario and to identify the objects’ location.”).
But does not explicitly disclose in response to determining the first object and the second object, determining, by the data processing hardware, that a first set of legged robot actions to interact with the first object are available for selection and a second set of legged robot actions to interact with the second object are available for selection, wherein the first set of legged robot actions are different from the second set of legged robot actions;
identifying, by the data processing hardware, a first legged robot action selected from the first set of legged robot actions to interact with the first object;
identifying, by the data processing hardware, a second legged robot action selected from the second set of legged robot actions to interact with the second object;
in response to identifying the first legged robot action and identifying the second legged robot action, generating, by the data processing hardware, a mission to traverse the environment using two or more legs of the legged robot and perform the first legged robot action and the second legged robot action; and
instructing, by the data processing hardware, performance of the mission by the legged robot.
Paxton, in a similar field of endeavor, teaches identifying, by the data processing hardware, a first action from the first set of actions associated with the first object (See Section III Reference B, “allows the end user to combine and parameterize operations […] can switch the robot into a compliant mode by pressing a TEACH button, and can enable autonomous execution by pressing the SERVO button […] task editor is accompanied by a 3D visualization of the robot, detected objects, and coordinate frames […]” Examiner notes the method shows detected objects are presented in the same interface where operations and actions are parameterized and selectable, meaning that the robot’s first selected action(s) are from a first set of actions associated with a target object);
identifying, by the data processing hardware, a second action from the second set of actions associated with the second object (See Section III Reference B as explained above, and also, “The resulting sorted list of grasp poses is used to generate motion plans in order of preference […] users can then frame the task plan as a sequence of high-level commands […]” Examiner notes each task plan is differentiable based on the target operation of different objects, thus including a second action from a different set of actions regarding a different object).
Liberg, in a similar field of endeavor, teaches in response to identifying the first legged robot action and identifying the second legged robot action, generating, by the data processing hardware, a mission to perform the first action and the second action (See Pg. 11 Lines 15-22, “[…] robot program generator 8 adapted for generating a robot program for performing the work cycle based on the stored information about the workstation, including the preprogrammed robot code, the workstations selected by the user, and the order for which the robot shall assist the workstation […]”); and
instructing, by the data processing hardware, performance of the mission by the robot (See Pg. 6 Lines 16-23, “[…] one or more predefined movement paths to be followed by the robot when performing work at the workstation […] includes predefined movement path for carrying out the task.”).
Lu, in a similar field of endeavor, teaches in response to determining the first object and the second object, determining, by the data processing hardware, that a first set of legged robot actions to interact with the first object are available for selection and a second set of legged robot actions to interact with the second object are available for selection, wherein the first set of legged robot actions are different from the second set of legged robot actions (See Section 3.1, “[…] selected so that the robot was capable of seeing and interacting with them.” See also Section 5, “[…] compared the behavior of the robot when faced with the same object shape and size but different weight.” Examiner notes that because the legged robot’s actions vary depending on the physical parameters of the object, there must necessarily be different robot actions when interacting with them);
a legged robot executing legged robot actions (See Section 2.2, “Through the use of control points, unique sequences of motion can be created for legged robots. The leg motion can be altered to allow for a modified leg end-effector, such as a gripper.”); and
a mission to traverse the environment using two or more legs of the legged robot (See Section 1, “[…] ability to traverse these challenging environments […]”).
In view of Paxton, Liberg and Lu’ teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data as disclosed by Benaim, different sets of actions that may be parameterized and selectable as taught by Paxon, generating a robot program to be executed based on predefined parameters as taught by Liberg and applied to a legged robot as taught by Lu, with a reasonable expectation of success, since the robot system already includes the necessary components to conduct the environmental visualization, and the sensor perception and mission-based planning frameworks both operate within autonomous robotic systems that rely on environmental perception. Furthermore, this combination would predictably improve mission generation by using more accurate sensory inputs, which is a standard motivation in robotics to reduce uncertainty in various environments, and it has been held that making a machine portable or movable without producing new and unexpected results involves only skill routine in the art.
Regarding Claim 2, Benaim discloses the computer-implemented method of claim 1, further comprising:
instructing display of the graphical representation based on a location of the legged robot corresponding to a location of the first object (See 0046, “A display and/or a user input device […]” See also 0003, “[…] sensors and the dedicated software are designed and set to determine the physical location of the object relative to the sensors and/or to other objects or equipment in the scene.”).
But does not explicitly disclose in response to determining that the first set of legged robot actions to interact with the first object are available for selection, generating a graphical representation of the first set of legged robot actions.
Liberg, in a similar field of endeavor, teaches in response to determining that the first set of legged robot actions to interact with the first object are available for selection, generating a graphical representation of the first set of legged robot actions (See Abstract, “[…] a graphical generator (7), generating one or more graphical user interfaces on said display device […]” See also Pgs. 6-7 Lines 33-8, “[…] generate one or more graphical user interfaces for entering configuration data for the robot […] to generate said robot program based on said received configurations data for the robot […] define safety zones, robot tool, and robot and work object coordinate systems.”).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to generate a graphical representation of a first set of actions, with a reasonable expectation of success, since graphical user interfaces (GUIs) in robotics often visualize planned missions relative to robot position, and would increase user comprehension and monitoring of the robot’s operations, thus improving usability without altering core functions.
Regarding Claim 3, Benaim discloses the computer-implemented method of claim 1, further comprising:
instructing display of the graphical representation (See 0046 and 0003 as referenced above).
But does not explicitly disclose generating a graphical representation based on performance of the first legged robot action, wherein the graphical representation indicates a status of performance of the mission by the legged robot.
Liberg, in a similar field of endeavor, teaches generating a graphical representation based on performance of the first legged robot action, wherein the graphical representation indicates a status of performance of the mission by the legged robot (See Abstract and Pgs. 6-7 Lines 33-8 as referenced above. See also Pg. 8 Lines 10-24, “[…] generating a graphical user interface including graphical information about the status of the selected workstations during operation […]” Examiner notes status of a workstation during operation includes robot performance).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to generate a graphical representation based on performance and status of an operation, with a reasonable expectation of success, since displaying progress or performance of a task is a well-known feedback mechanism, and would provide predictable user feedback that may aid debugging and situational awareness.
Regarding Claim 4, Benaim does not explicitly disclose the computer-implemented method of claim 1, further comprising:
receiving an input indicating selection of the first legged robot action, wherein identifying the first legged robot action is based on the input, wherein the input indicates two or more missions of a set of missions associated with the legged robot to include the first legged robot action, and wherein the two or more missions comprise the mission and at least one additional mission.
Liberg, in a similar field of endeavor, teaches receiving an input indicating selection of the first legged robot action, wherein identifying the first legged robot action is based on the input, wherein the input indicates two or more missions of a set of missions associated with the legged robot to include the first legged robot action, and wherein the two or more missions comprise the mission and at least one additional mission (See Abstract, “[…] allowing a user to select one or more predefined workstations and to specify the order in which the robot shall visit the selected workstations […] generating a robot program for performing the work cycle based on said predefined workstations […]” Examiner notes the robot inputs are already capable of indicating an action or mission based on an input, and including multiple actions or missions based on the same input is a mere duplication of the essential working parts of the device).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to receive an input indicating the selection of an action, with a reasonable expectation of success, since this enables user input and increases user interaction with the robot’s operations to improve precision towards desired outcomes.
Regarding Claim 5, Benaim does not explicitly disclose the computer-implemented method of claim 1, further comprising:
in response to determining that the first set of legged robot actions to interact with the first object are available for selection, generating a graphical representation based on the environment, wherein the graphical representation indicates the first set of legged robot actions, a pictorial representation of the environment, and a pictorial representation of the first object;
instructing display of the graphical representation via a user computing device; and
receiving, from the user computing device, via an interaction with the graphical representation, an input indicating selection of the first legged robot action, wherein identifying the first legged robot action is based on the input.
Liberg, in a similar field of endeavor, teaches in response to determining that the first set of legged robot actions to interact with the first object are available for selection, generating a graphical representation based on the environment, wherein the graphical representation indicates the first set of legged robot actions a pictorial representation of the environment, and a pictorial representation of the first object (See Abstract, and Pgs. 6-8 as referenced above. See also Pg. 5 Lines 13-15, “[…] graphical interface is defined as a view, or a part of a view, adapted for interaction with a user and displayed on the display device.” Examiner notes the graphical representation includes display and selection of predefined workstations in the environment);
instructing display of the graphical representation via a user computing device (See Abstract and Pgs. 5-8 as referenced above); and
receiving, from the user computing device, via an interaction with the graphical representation, an input indicating selection of the first legged robot action, wherein identifying the first legged robot action is based on the input (See Abstract as referenced above).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to receive an input indicating the selection of an action and generate a graphical representation of the environment, with a reasonable expectation of success, since creating a GUI to show objects and their actions sets while accepting a selection via interaction is an anticipated and conventional integration for improved usability in robotic systems.
Regarding Claim 6, Benaim does not explicitly disclose the computer-implemented method of claim 1, further comprising:
generating, using second sensor data obtained from a sensor located on the legged robot, a graphical representation based on the environment, wherein the graphical representation indicates the first object; and
receiving, via an interaction with the graphical representation, an input defining the first set of legged robot actions.
Liberg, in a similar field of endeavor, teaches generating, using second sensor data obtained from a sensor located on the legged robot, a graphical representation based on the environment, wherein the graphical representation indicates the first object (See Abstract, and Pgs. 6-8 as referenced above. Examiner notes each workstation includes objects involved in the robot’s work cycle); and
receiving, via an interaction with the graphical representation, an input defining the first set of legged robot actions (See Abstract as referenced above).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to receive an input indicating the selection of an action and generate a graphical representation indicating an object in the environment, with a reasonable expectation of success, since creating a GUI to show objects and their actions sets while accepting a selection via interaction is an anticipated and conventional integration for improved usability in robotic systems.
Regarding Claim 7, Benaim further discloses the computer-implemented method of claim 1, wherein instructing performance of the mission by the legged robot comprises:
instructing performance of the first legged robot action by the legged robot at the first location using at least one of an arm of the legged robot or a leg of the two or more legs (See 0151, “Step 309 is performed when another iteration is needed. In such case, the Robot assisted Object-Learning vision system instructs the mechanical device to move the object to a next position in space.”).
But does not explicitly disclose instructing the legged robot to navigate from a docking station of the legged robot to a first location using the two or more legs.
Lu, in a similar field of endeavor, teaches instructing the legged robot to navigate from a docking station of the legged robot to a first location using the two or more legs (See Section 1 as referenced above and, “To successfully traverse these unstructured terrains with unknown obstacles in the robot’s path, the robot is required to manipulate obstacles out of its way. Thus, if a legged robot platform is able to autonomously identify and manipulate an obstacle in its path, the robot can progress further in the environment […] to achieve autonomous docking […]” See also Section 2.1, “[…] location of the object influences the final position of the intermediate and last control points for trajectory generation […]”).
In view of Lu’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, navigation of the robot using two or more legs from some stationary position to another location, with a reasonable expectation of success, since it has been held that making a machine portable or movable without producing new and unexpected results involves only routine skill in the art.
Regarding Claim 8, Benaim further discloses the computer-implemented method of claim 1, wherein instructing performance of the mission by the legged robot comprises:
instructing determination of a location of the legged robot along a path of the legged robot within the environment (See 0120, “[…] the positional information previously stored by the mechanical device is retrieved from the record and an estimation is performed as to the relative X,Y,Z position of the object with regards to the sensor […] if in a training stage the object was originally placed exactly in front of the center of the camera, and the match found in the training set corresponds to the image that was taken after either the sensor or object was displaced i, j ,k cm in space, then this relative position can also be retrieved […]”); and
instructing performance of the first legged robot action by the legged robot based on the location of the legged robot along the path of the legged robot within the environment corresponding to a location of the first object within the environment (See 0120 as referenced above).
Regarding Claim 9, Benaim does not explicitly disclose the computer-implemented method of claim 1, further comprising:
instructing iterative performance of the mission by the legged robot at different time periods, wherein multiple performances of the first legged robot action by the legged robot are scheduled based on instructing iterative performances of the mission.
Liberg, in a similar field of endeavor, teaches instructing iterative performance of the mission by the legged robot at different time periods, wherein multiple performance of the first legged robot action by the legged robot are scheduled based on instructing iterative performances of the mission (See Abstract, “[…] allowing a user to select one or more of the predefined workstations and to specify the order in which the robot shall visit the selected workstations […]” See also Pg. 14 Lines 28-29, “[…] through one workstation at a time and a movement path for each workstation is defined.” Examiner notes a sequential order for performance of the robot at each designated workstation necessarily includes different time periods for its iterated performances, considering the robot cannot perform its mission at each workstation at the same time).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the mission generation stage to include scheduling of various actions, with a reasonable expectation of success, since planning for timed sequences to perform certain operations is a conventional extension to add temporal parameters that would increase system efficiency and reduce undesired behaviors in robot operations. Furthermore, it has been held that mere duplication of the essential working parts of a device or steps in a process involves only routine skill in the art.
Regarding Claim 10, Benaim does not explicitly disclose the computer-implemented method of claim 1, further comprising:
receiving, from a user computing device, an input, wherein the legged robot navigates the environment based on the input, and wherein an order of performance of the first legged robot action and performance of the second legged robot action associated with the mission is based on the input.
Liberg, in a similar field of endeavor, teaches receiving, from a user computing device, an input, wherein the legged robot navigates the environment based on the input, and wherein an order of performance of the first legged robot action and performance of the second legged robot action associated with the mission is based on the input (See Abstract and Pgs. 5-8 as referenced above).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the mission generation stage to indicate an order of performance between two actions, with a reasonable expectation of success, since Liberg directly teaches ordering of workstation visits that map to ordered actions in a mission. It would be obvious to use that existing teaching to indicate the order of performance for actions in a mission.
Regarding Claim 11, Benaim does not explicitly disclose the computer-implemented method of claim 1, wherein instructing performance of the mission by the legged robot comprises:
instructing the legged robot to navigate to a first location using the two or more legs;
in response to determining that the legged robot navigated to the first location, instructing the legged robot to perform the first legged robot action at the first location using at least one leg of the two or more legs;
in response to determining that the legged robot performed the first legged robot action, instructing the legged robot to navigate to a second location from the first location using the two or more legs; and
in response to determining that the legged robot navigated to the second location, instructing the legged robot to perform the second legged robot action at the second location using the at least one leg.
Lu, in a similar field of endeavor, instructing performance of the mission by the legged robot comprises:
instructing the legged robot to navigate to a first location using the two or more legs (See Section 1 and 2.1 as referenced above);
in response to determining that the legged robot navigated to the first location, instructing the legged robot to perform the first legged robot action at the first location using at least one leg of the two or more legs (See Sections 2.1-2.2, “[…] extracts the height and width of the detected object and provides key points where the robot can interact with the object […] Additional key contact points can be specified for complex interaction motions such as combining lifting and pushing […] Spatial control of the leg allows unique leg movements for interacting with different objects […] key contact point from the object is fed into the control points of the curves, guiding the leg to interact with the object.”);
in response to determining that the legged robot performed the first legged robot action, instructing the legged robot to navigate to a second location from the first location using the two or more legs (See Section 1 and 2.1 as referenced above); and
in response to determining that the legged robot navigated to the second location, instructing the legged robot to perform the second legged robot action at the second location using the at least one leg (See Sections 2.1-2.2 as referenced above).
In view of Lu’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, navigation to multiple locations and performance of robot actions at each location, since Lu directly teaches interaction actions that are tied to objects in proximity to the robot during navigation, and it has been held that making a machine portable or movable without producing new and unexpected results involves only routine skill in the art.
Regarding Claim 12, Benaim does not explicitly disclose the computer-implemented method of claim 1, wherein determining that the first set of legged robot actions to interact with the first object are available for selection and the second set of legged robot actions to interact with the second object are available for selection comprises:
determining, from a third set of legged robot actions, that the first set of legged robot actions to interact with the first object are available for selection, wherein the third set of legged robot actions comprises the first set of legged robot actions and one or more additional first legged robot actions; and
determining, from a fourth set of legged robot actions, that the second set of legged robot actions to interact with the second object are available for selection, wherein the fourth set of legged robot actions comprises the second set of legged robot actions and one or more additional second legged robot actions.
Lu, in a similar field of endeavor, teaches determining that the first set of legged robot actions to interact with the first object are available for selection and the second set of legged robot actions to interact with the second object are available for selection comprises:
determining, from a third set of legged robot actions, that the first set of legged robot actions to interact with the first object are available for selection, wherein the third set of legged robot actions comprises the first set of legged robot actions and one or more additional first legged robot actions (See Sections 3.1 and 5 as referenced above. Examiner notes that the sets of legged robot actions already include a plurality of potential actions dependent on the physical parameters of the object, and determining further arbitrary sets of legged robot actions is a mere duplication of the essential working parts of the robot); and
determining, from a fourth set of legged robot actions, that the second set of legged robot actions to interact with the second object are available for selection, wherein the fourth set of legged robot actions comprises the second set of legged robot actions and one or more additional second legged robot actions (See Sections 3.1 and 5 as referenced above. Examiner notes that the sets of legged robot actions already include a plurality of potential actions dependent on the physical parameters of the object, and determining further arbitrary sets of legged robot actions is a mere duplication of the essential working parts of the robot).
In view of Lu’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, multiple sets of robot actions, with a reasonable expectation of success, since it is obvious that different sets of actions can be applied to different objects or tasks, and it is only routine in the art to associate different objects with different action sets. Furthermore, a mere duplication of the essential working parts of the robot involves only routine skill in the art.
Regarding Claim 13, Benaim further discloses the computer-implemented method of claim 1, wherein the first object and the second object are a same type of object, wherein the first set of legged robot actions is based on an output of a machine learning model and an output of a user computing device (See 0120 as referenced above. See also 0108, “[…] other classification models are used for the above stage for achieving same goal as described above. Such other classification models may be neural networks and deep learning models or the like.”).
But does not explicitly disclose wherein the second set of legged robot actions is based on the output of the user computing device.
Liberg, in a similar field of endeavor, teaches the second set of legged robot actions is based on the output of the user computing device (See Abstract and Pgs. 5-8 as referenced above).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, outputs from a machine learning model as a basis for determining robot action(s), with a reasonable expectation of success, since Liberg directly teaches actions in a mission dependent on outputs from the industrial robot control module. It would be obvious to use that existing teaching to be fed through a generic machine learning model to produce a predictable improvement on efficient robot action determinations.
Regarding Claim 14, Benaim further discloses the computer-implemented method of claim 1, further comprising:
providing data associated with the first object to a machine learning model, wherein determining that the first set of legged robot actions to interact with the first object are available for selection is based on an output of the machine learning model (See 0120 as referenced above. See also 0108, “[…] other classification models are used for the above stage for achieving same goal as described above. Such other classification models may be neural networks and deep learning models or the like.” Examiner notes the robot system and method derive positional and feature data from various sensors).
Regarding Claim 15, Benaim discloses the computer-implemented method of claim 1, further comprising:
matching the first object to a respective object of a set of objects wherein, for each respective object of the set of objects, the set of objects indicates a respective set of legged robot actions to interact with the respective object are available for selection (See 0153, “[…] system searches through the Objects Database 112 to identify a record that best matches the features extracted from the currently inspected object […] system determines when the match found is statistically significant. When the decision is that the match is significant, it is assumed that the object being inspected is identified as similar to the object in the Database, with the matching feature.”).
But does not explicitly disclose in response to matching the first object to the respective object, identifying the first set of legged robot actions based on the set of objects, wherein the set of objects indicates the first set of legged robot actions to interact with the first object are available for selection.
Liberg, in a similar field of endeavor, teaches in response to matching the first object to the respective object, identifying the first set of legged robot actions based on the set of objects, wherein the set of objects indicates the first set of legged robot actions to interact with the first object are available for selection (See Abstract as referenced above).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, the system to match objects to a respective set of objects, with a reasonable expectation of success, since matching a detected object to a stored object or workstation entry in memory allows a corresponding action or program to be retrieved as well. This is an expected, routine system for mapping observed objects and stored templates and further retrieving the associated action set.
Regarding Claim 17, Benaim further discloses the system of claim 16, wherein the execution of the instructions by the data processing hardware further causes the data processing hardware to:
identify the first set of legged robot actions using a machine learning model (See 0108, “[…] other classification models are used for the above stage for achieving same goal as described above. Such other classification models may be neural networks and deep learning models or the like.”).
Regarding Claim 18, Benaim further discloses the system of claim 16, wherein to determine the first object, the execution of the instructions by the data processing hardware further causes the data processing hardware to:
identify the first object using an object detection model (See 0108, “[…] other classification models are used for the above stage for achieving same goal as described above. Such other classification models may be neural networks and deep learning models or the like.”).
Regarding Claim 20, Benaim does not explicitly disclose the legged robot of claim 19, wherein the first legged robot action comprises a navigational behavior of the legged robot.
Liberg, in a similar field of endeavor, teaches the first legged robot action comprises a navigational behavior of the legged robot (See Pg. 6 Lines 17-23, “[…] default data includes one or more predefined movement paths to be followed by the robot when performing work at the workstation […] the predefined workstations include predefined movement path for carrying out the task.”).
In view of Liberg’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, an operational action of the robot to include navigational behavior, with a reasonable expectation of success, since it is an expected, routine functions for robots to be able to navigate within an environment, especially when they are further capable of mapping observed objects and determining actions with respect to the object(s).
Regarding Claims 21-23, Benaim does not explicitly disclose the computer-implemented method of claim 1, system of claim 16 and legged robot of claim 19, wherein each legged robot action of the first set of legged robot actions and the second set of legged robot actions comprises at least one of:
a particular action to move at least one leg of the two or more legs to adjust a pose of the legged robot relative to a respective object (See Section 2.2 excerpt below);
PNG
media_image1.png
302
424
media_image1.png
Greyscale
a particular action to move the at least one leg to actuate the respective object (See Sections 2.1-2.2 as referenced above); or
a particular action to move a manipulator arm of the legged robot to at least one of actuate or grasp the respective object.
Lu, in a similar field of endeavor, teaches each legged robot action of the first set of legged robot actions and the second set of legged robot actions comprises at least one of:
a particular action to move at least one leg of the two or more legs to adjust a pose of the legged robot relative to a respective object;
a particular action to move the at least one leg to actuate the respective object; or
a particular action to move a manipulator arm of the legged robot to at least one of actuate or grasp the respective object.
In view of Lu’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the robot system and method of determining various objects in an environment of the robot using received sensor data and displaying a relative location of the robot and the object(s) as disclosed by Benaim, particular action(s) involving object manipulation or interaction using one or more legs of the robot, since Lu directly teaches interaction actions of the robot legs involving manipulation of objects in proximity to the robot during navigation, and it has been held that making a machine portable or movable without producing new and unexpected results involves only routine skill in the art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bryant Tang whose telephone number is (571)270-0145. The examiner can normally be reached M-F 8-5 CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571)272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYANT TANG/Examiner, Art Unit 3658
/THOMAS E WORDEN/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights