DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 9-13, 16-18, 20-32 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Oleynik et al (US 20170348854, hereinafter Oleynik).
Regarding Claim 1, Oleynik teaches:
a tool for skill retrieval and skill adaptation (SRSA) (see at least standard object library module in par. 0845 ) , comprising:
an interface configured to receive data, wherein the data includes target task information associated with a target task (see at least " The real time adjustment module 112 is configured to provide real-time adjustments to the variables associated with a particular kitchen operation or a mini operation to produce a resulting process that is a precise replication of the chef movement or a precise replication of the sensory curve." in par. 0412 and “Retrieving one or more stored cases whose problems bear strong similarity to the new problem, optionally adjusting the parameters from the retrieved case(s) to apply to the current case (e.g. an item may weigh somewhat more, and hence a somewhat stronger force is needed to lift it), and using the same methods and steps from the case(s) with the adjusted parameters (if needed) at least in part to solve the new problem.” In par. 0460 and “The Central RoboticControl both accesses the Case Library to determine if has a known sequence of actions for a current task, and updates the Case Library with outcome information upon executing the task.” In par. 0613) ; and
one or more processors to perform one or more operations (see at least "processor" in par. 0085) that include:
retrieving, using the target task information, a relevant skill policy from a skill library to perform the target task (see at least “Retrieving one or more stored cases whose problems bear strong similarity to the new problem, optionally adjusting the parameters from the retrieved case(s) to apply to the current case (e.g. an item may weigh somewhat more, and hence a somewhat stronger force is needed to lift it), and using the same methods and steps from the case(s) with the adjusted parameters (if needed) at least in part to solve the new problem.” In par. 0460),
wherein the retrieving is based on a predicted success of the relevant skill policy (see at least "First, a potentially very large library of pre-defined/pre-learned sensing-and-action sequences called minimanipulations. Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation). " in par. 0141 and “This software module uses data from the 3D world configuration modeler 262, which creates a new 3D world model at every sampling step from sensory data supplied by the multimodal sensor(s) unit(s), in order to ascertain that the configuration of the robotic kitchen systems and process matches that required by the recipe script (database); if not, it enacts modifications to the commanded system-configuration values to ensure the task is completed successfully.” In par. 0429 and “The creation module 60 of the minimanipulation database library is a process of creating, testing various possible combinations, and selecting an optimal minimanipulation to achieve a specific functional result.” In par. 0511 and “In recipe execution 780, the robotic hands 72 execute the minimanipulations 770 of cracking an egg with a knife, where the optimal way to execute each movement in the cracking an egg operation 772, the holding a knife operation 774, the striking the egg with a knife operation 776, and opening the cracked egg operation 778 is selected from the minimanipulations library database. The process of executing the optimal way to carry out each of the movements 772, 774, 776, 778 ensures that the minimanipulation 770 will achieve the same (or guarantee of), or substantially the same, outcome for that specific minimanipulation. The multimodal three-dimensional sensor 20 provides real-time adjustment capabilities 112 as to the possible variations in one or more ingredients, such as the dimension and weight of an egg.” In par. 0536) and
the relevant skill policy includes a source geometry of at least one source object (see at least "The validator module 314 is configured to receive standard object data 318 from a standard object library module 318A which is, for instance, a database stored in a memory. The standard object data 318 comprises one or more of 2D or 3D shape data, visual signatures and/or image samples of standard objects which are used in the kitchen module 1. The standard objects are, for instance, objects that are to be expected to be present within the robotic kitchen module 1, such as dishes, tools, utensils and appliances." in par. 0845 ) and
at least one task trajectory of the at least one source object, where the at least one source object is matched to a target object of the target task (see at least "As an example, process 3165-1 might identify a motion-sequence through a dataset that indicates object-grasping and repetitive back-and-forth motion related to a studio-chef grabbing a knife and proceeding to cut a food item into slices. The motion-sequence is then broken down in 3165-2 into associated actions of several physical elements (fingers and limbs/joints) shown in FIG. 109 with a set of transitions between multiple manipulation phases for one or more arm(s) and torso (such as controlling the fingers to grasp the knife, orienting it properly, translating arms and hands to line up the knife for the cut, controlling contact and associated forces during cutting along a cut-plane, re-setting the knife to the beginning of the cut along a free-space trajectory and then repeating the contact/force-control/trajectory-following process of cutting the food-item indexed for achieving a different slice width/angle). The parameters associated with each portion of the manipulation-phase are then extracted and assigned numerical values in 3165-3, and associated with a particular action-primitive offered by 3165-5 with mnemonic descriptors such as ‘grab’, ‘align utensil’, ‘cut’, ‘index-over’, etc." in par. 0595 and “In the block 5113 the Action Data 5071 is retrieved from the Action Storage 5018. The Action Data 5071 contains the Pick Action Parameters, stored in the variable PAP. The Pick Action Parameters define completely the pickup action to perform and is different based on the ingredient 4080 and the internal state, the carrier 4060, the required quantity RQ to collect, the Recipe Step Info RS, the environment data (stored in the variable E) like humidity, temperature, atmospheric pressure. So in order to get the correct Pick Action Parameters a query is sent to the Action Storage 5018, using Input Data 5060 and Environment Data E.” in par. 1027) , and
modifying the relevant skill policy for the target task using the target task information. (see at least "The real time adjustment module 112 is configured to provide real-time adjustments to the variables associated with a particular kitchen operation or a mini operation to produce a resulting process that is a precise replication of the chef movement or a precise replication of the sensory curve." in par. 0412 and “However, in the case of the non-standardized kitchen, the chances are very high that the system will have to modify and adapt the actual recipe itself and its execution, via a recipe script modification module 204, to suit the available tools/appliances 192 which differ from those in the chef studio 44 or the measured deviations from the recipe script (meat cooking too slowly, hot-spots in pot burning the roux, etc.).” in par. 0418)
Regarding Claim 2, Oleynik teaches:
the tool as recited in claim 1,
wherein the target task is an assembly task. (see assembly tasks in par. 0519-0521)
Regarding Claim 3, Oleynik teaches:
the tool as recited in claim 1,
wherein the retrieving utilizes a task feature learning and a transfer success prediction (see at least " Abstraction motion-commands (e.g. “crack an egg into the pan”, “sear to a golden color on both sides”, etc.) can be generated from the raw data, refined, and optimized through a multitude of iterative learning processes, carried out live and/or off-line, allowing the robotic kitchen systems to successfully deal with measurement-uncertainties, ingredient variations, etc., enabling complex (adaptive) minimanipulation motions using fingered-hands mounted to robot-arms and wrists, based on fairly abstraction/high-level commands (e.g. “grab the pot by the handle”, “pour out the contents”, “grab the spoon off the countertop and stir the soup”, etc.)." in par. 0137 and “Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation).” In par. 0141 and “In a preferred embodiment each POST result is associated with a probability of obtaining the desired result if the MM is executed. The Central Robotic Control both accesses the MM library to retrieve and execute MM's and updates it, e.g. in learning mode to add new MMs.” In par. 0612) , wherein the task feature learning comprises:
extracting target geometry features from point cloud reconstruction of the target object involved in the target task (see at least “The data-reduction and abstraction engine (set of software routines) 226 is intended to reduce the larger three-dimensional data sets and extract from them key geometric and associative information. A first step is to extract from the large three-dimensional data point-cloud only the specific workspace area of importance to the recipe at that particular point in time. Once the data set has been trimmed, key geometric features will be identified by a process known as template matching. This allows for the identification of such items as horizontal tabletops, cylindrical pots and pans, arm and hand locations, etc.” in par. 0423) ;
extracting dynamics features by predicting a next state of the target object given a current state and an action of the target object (see at least " More generally, a generalized minimanipulation M comprises triple <PRE, ACT, POST>, where PRE={(s.sub.1, s.sub.2, . . . , s.sub.n} is a set of items in the world state that must be true before the actions ACT=[a.sub.1, a.sub.2, . . . , a.sub.k] can take place, and result in a set of changes to the world state denoted as POST={p.sub.1, p.sub.2 . . . , p.sub.m}. Note that [square brackets] mean sequences, and {curly brackets} mean unordered sets. Each post condition may also have a probability in case the outcome is less than certain. For instance the minimanipulation for grasping an egg may have a 0.99 probability that the egg is in the hand of the robot (the remaining 0.01 probability may correspond to inadvertently breaking the egg while attempting to grasp it, or other unwanted consequence)." in par. 0515) ; and
extracting expert action features by predicting the action from an observed state transition using inverse dynamics prediction. (see at least " Fifth, minimanipulations may be acquired by repeated observation of a human tutor (e.g. an expert chef) to determine the sensing-and-action sequence, and to determine the range of acceptable values for the variables." in par. 0141 and “FIG. 34 depicts a block diagram illustrating the process of how a remote robotic system would utilize the minimanipulation (MM) library(ies) to carry out a remote replication of a particular task (cooking, painting, etc.) carried out by an expert in a studio-setting, where the expert's actions were recorded, analyzed and translated into machine-executable sets of hierarchically-structured minimanipulation datasets (commands, parameters, metrics, time-histories, etc.) which when downloaded and properly parsed, allow for a robotic system (in this case a dual-arm torso/humanoid system) to faithfully replicate the actions of the expert with sufficient fidelity to achieve substantially the same end-result as that of the expert in the studio-setting.” In par. 0599 and “This is achieved by taking the parameter-set from the ‘offending’ minimanipulation action-step and using one or more of multiple techniques for parameter-optimization common in the field of machine-learning, to rebuild a specific minimanipulation step or sequence MM.sub.i into a revised minimanipulation step or sequence MM.sub.i*. The revised step or sequence MM.sub.i* is then used to rebuild a new command-0sequence that is passed back to the command executor 3175 for re-execution. The revised minimanipulation step or sequence MM.sub.i* is then fed to a re-build function that re-assembles the final version of the minimanipulation dataset, that led to the successful achievement of the required functional result, so it may be passed to the task- and parameter monitoring process 3179.” In par. 0604)
Regarding Claim 9, Oleynik teaches:
the tool as recited in claim 1,
wherein the skill library is generated using previous tasks at a previous time interval. (see at least "Updated parameter data is then used to rebuild the modified minimanipulation parameter set for re-execution as well as for updating/rebuilding a particular minimanipulation routine, which is provided back to the original library routines as a modified/re-tuned library for future use by other robotic systems." in par. 0600 )
Regarding Claim 10, Oleynik teaches:
the tool as recited in claim 1, wherein the relevant skill policy is retrieved using a combination of object geometries of the at least one source object, object dynamics of the at least one source object, and expert actions on the at least one source object to represent a previous task in the skill library, and the combination is compared to the target task information. (see at least “Another kind is case-based learning, where previous solutions, e.g. sequences of actions by a human teacher or by the robot itself are remembered, together with any constraints or reasons for the solutions, and then are applied or reused in new settings.” In par. 0354 and “Retrieve one or more stored cases with the transformed exact values (now ranges, or calculations for new values depending on values of the input parameters), but still whose initial problems bear strong similarity to the new problem, including parameter values and value ranges, and use the transformed methods and steps from the case(s) at least in part to solve the new problem.” In par. 0460 and “A predefined minimanipulation is available to achieve each functional result (e.g., the egg is cracked). Each minimanipulation comprises of a collection of action primitives which act together to accomplish the functional result. For example, the robot may begin by moving its hand towards the egg, touching the egg to localize its position and verify its size, and executing the movements and sensing actions necessary to grasp and lift the egg into the known and predetermined configuration.” In par. 0479)
Regarding Claim 11, Oleynik teaches:
the tool as recited in claim 1,
wherein the modifying the relevant skill policy uses a self-imitation learning process. (see at least "Another kind is case-based learning, where previous solutions, e.g. sequences of actions by a human teacher or by the robot itself are remembered, together with any constraints or reasons for the solutions, and then are applied or reused in new settings." in par. 0354)
Regarding Claim 12, Oleynik teaches:
the tool as recited in claim 1,
wherein the one or more operations are performed within a simulated environment and the modified relevant skill policy is stored in the skill library as a previous task. (see at least " Computer 2722 may also control and capture 3-D modeling feedback for simulation model calibration and real time adjustments. Minimanipulation library 2723 stores the captured minimanipulations that have been downloaded from the creator's recording system 2710 to the commercial robotic system 2720 via communications link 2701. Minimanipulation library 2723 may store the minimanipulations locally or remotely and may store them in a predetermined or relational basis." in par. 0548 and “Skill movement emulator 2822 is coupled to the robotic human-skill replication engine 2800 and may be used to emulate creator skills without actual sensor input. Skill movement emulator 2822 provides alternate input to robotic human-skill replication engine 2800 to allow for the creation of a skill execution program without the use of a creator 2711 providing sensor input. Extended simulation validation and calibration module 2823 may be coupled to robotic human-skill replication engine 2800 and provides for extended creator input and provides for real time adjustments to the robotic movements based on 3-D modeling and real time feedback.” In par. 0554 and “The robotic cooking engine 237 then generates 256 a simulation program based on the recorded cooking parameter data (temperature, humidity, pressure) and generates curve profiles for each container 250 and all cooking wares. The curve profiles indicate the cooking parameters within the containers 250 and the appliances within the robotic kitchen as the recipe is followed. The computer 235 records any adjustments made by the chef 234 to the cooking parameters during the process.” In par. 0797)
Regarding Claim 13, Oleynik teaches:
the tool as recited in claim 1,
wherein the one or more operations further include directing one or more robots using the modified relevant skill policy to perform the target task. (see at least " The robotic arms 70 and the robotic hands 72 operate autonomously with the same xyz coordinates 438 and with possible real-time adjustment on the size and shape of a particular carrot by creating a temporary three-dimensional model 440 of the carrot from the real-time adjustment devices 112" in par. 0447)
Regarding Claim 16, Oleynik also teaches:
A robotic assembly system comprising the tool of Claim 1 (see Claim 1 analysis for rejection of the tool)
Regarding Claim 17, Oleynik also teaches:
A robotic assembly system comprising the tool of Claim 13 (see Claim 13 analysis for rejection of the tool)
Regarding Claim 18, Oleynik also teaches:
the robotic assembly system as recited in claim 16,
wherein the target task is an assembly task of two objects, one of which is the target object. (see assembly of parts in a factory and welding in par. 0519-0521)
Regarding Claim 20, Oleynik also teaches:
the robotic assembly system as recited in claim 16,
wherein the one or more processors are part of or executing on a central processor unit (CPU) or a graphics processor unit (GPU). (see at least " he example computer system 3624 includes a processor 3626 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both)," in par. 1179)
Regarding Claim 21, Oleynik also teaches:
the robotic assembly system as recited in claim 16, wherein the modified relevant skill policy is used by a robotic planner or a robotic controller system to direct the movement of a robotic assembler. (see at least " The robotic arms 70 and the robotic hands 72 operate autonomously with the same xyz coordinates 438 and with possible real-time adjustment on the size and shape of a particular carrot by creating a temporary three-dimensional model 440 of the carrot from the real-time adjustment devices 112" in par. 0447 and assembly of parts in a factory and welding in par. 0519-0521)
Regarding Claim 22, Oleynik also teaches:
the robotic assembly system as recited in claim 16,
wherein the relevant skill policy includes the source geometry of at least one source object. (see at least "The validator module 314 is configured to receive standard object data 318 from a standard object library module 318A which is, for instance, a database stored in a memory. The standard object data 318 comprises one or more of 2D or 3D shape data, visual signatures and/or image samples of standard objects which are used in the kitchen module 1. The standard objects are, for instance, objects that are to be expected to be present within the robotic kitchen module 1, such as dishes, tools, utensils and appliances." in par. 0845 )
Regarding Claim 23, Oleynik also teaches:
the robotic assembly system as recited in claim 16,
wherein the relevant skill policy includes at least one task trajectory of the at least one source object. (see at least "As an example, process 3165-1 might identify a motion-sequence through a dataset that indicates object-grasping and repetitive back-and-forth motion related to a studio-chef grabbing a knife and proceeding to cut a food item into slices. The motion-sequence is then broken down in 3165-2 into associated actions of several physical elements (fingers and limbs/joints) shown in FIG. 109 with a set of transitions between multiple manipulation phases for one or more arm(s) and torso (such as controlling the fingers to grasp the knife, orienting it properly, translating arms and hands to line up the knife for the cut, controlling contact and associated forces during cutting along a cut-plane, re-setting the knife to the beginning of the cut along a free-space trajectory and then repeating the contact/force-control/trajectory-following process of cutting the food-item indexed for achieving a different slice width/angle). The parameters associated with each portion of the manipulation-phase are then extracted and assigned numerical values in 3165-3, and associated with a particular action-primitive offered by 3165-5 with mnemonic descriptors such as ‘grab’, ‘align utensil’, ‘cut’, ‘index-over’, etc." in par. 0595 and “In the block 5113 the Action Data 5071 is retrieved from the Action Storage 5018. The Action Data 5071 contains the Pick Action Parameters, stored in the variable PAP. The Pick Action Parameters define completely the pickup action to perform and is different based on the ingredient 4080 and the internal state, the carrier 4060, the required quantity RQ to collect, the Recipe Step Info RS, the environment data (stored in the variable E) like humidity, temperature, atmospheric pressure. So in order to get the correct Pick Action Parameters a query is sent to the Action Storage 5018, using Input Data 5060 and Environment Data E.” in par. 1027)
Regarding Claim 24, Oleynik also teaches:
A method for using the tool of Claim 1 (see Claim 1 analysis for rejection of the tool)
Regarding Claim 25, Oleynik also teaches:
the method as recited in claim 24, further comprising:
directing operations of a robotic assembly system using the modified relevant skill policy. (see at least " The robotic arms 70 and the robotic hands 72 operate autonomously with the same xyz coordinates 438 and with possible real-time adjustment on the size and shape of a particular carrot by creating a temporary three-dimensional model 440 of the carrot from the real-time adjustment devices 112" in par. 0447)
Regarding Claim 26, Oleynik also teaches:
26. The method as recited in claim 24, further comprising:
applying the modified relevant skill policy to a set of previously unseen target tasks with unknown object geometries, wherein the modified relevant skill policy enables zero-shot assembly in real-world environments. (see at least " The robotic arms 70 and the robotic hands 72 operate autonomously with the same xyz coordinates 438 and with possible real-time adjustment on the size and shape of a particular carrot by creating a temporary three-dimensional model 440 of the carrot from the real-time adjustment devices 112" in par. 0447 and assembly and welding in par. 0519-0521)
Regarding Claim 27, Oleynik also teaches:
the method as recited in claim 24, further comprising:
using the modified relevant skill policy by a robotic planner or a robotic controller system. (see at least " The robotic arms 70 and the robotic hands 72 operate autonomously with the same xyz coordinates 438 and with possible real-time adjustment on the size and shape of a particular carrot by creating a temporary three-dimensional model 440 of the carrot from the real-time adjustment devices 112" in par. 0447)
Regarding Claim 28, Oleynik also teaches:
the method as recited in claim 24, further comprising:
using the modified relevant skill policy as an input for a machine learning process or a robotic learning process. (see at least "Updated parameter data is then used to rebuild the modified minimanipulation parameter set for re-execution as well as for updating/rebuilding a particular minimanipulation routine, which is provided back to the original library routines as a modified/re-tuned library for future use by other robotic systems." in par. 0600 )
Regarding Claim 29, Oleynik also teaches:
the method as recited in claim 24, further comprising:
simulating a robotic assembly system, the target task, and the skill library within a machine learning system. (see at least " Computer 2722 may also control and capture 3-D modeling feedback for simulation model calibration and real time adjustments. Minimanipulation library 2723 stores the captured minimanipulations that have been downloaded from the creator's recording system 2710 to the commercial robotic system 2720 via communications link 2701. Minimanipulation library 2723 may store the minimanipulations locally or remotely and may store them in a predetermined or relational basis." in par. 0548 and “Skill movement emulator 2822 is coupled to the robotic human-skill replication engine 2800 and may be used to emulate creator skills without actual sensor input. Skill movement emulator 2822 provides alternate input to robotic human-skill replication engine 2800 to allow for the creation of a skill execution program without the use of a creator 2711 providing sensor input. Extended simulation validation and calibration module 2823 may be coupled to robotic human-skill replication engine 2800 and provides for extended creator input and provides for real time adjustments to the robotic movements based on 3-D modeling and real time feedback.” In par. 0554 and “The robotic cooking engine 237 then generates 256 a simulation program based on the recorded cooking parameter data (temperature, humidity, pressure) and generates curve profiles for each container 250 and all cooking wares. The curve profiles indicate the cooking parameters within the containers 250 and the appliances within the robotic kitchen as the recipe is followed. The computer 235 records any adjustments made by the chef 234 to the cooking parameters during the process.” In par. 0797)
Regarding Claim 30, Oleynik also teaches:
a non-transitory computer program product having a series of operating instructions stored on a non-transitory computer-readable medium that directs a robotic assembly system when executed thereby to perform operations, the operations comprising: (see at least " The disk drive unit 3640 includes a machine-readable medium 244 on which is stored one or more sets of instructions (e.g., software 3646) embodying any one or more of the methodologies or functions described herein." in par. 1180)
receiving a target task, wherein the target task is an assembly task of at least two objects; (see assembly of parts in a factory and welding in par. 0519-0521)
determining target task information associated with the target task; (see at least " The real time adjustment module 112 is configured to provide real-time adjustments to the variables associated with a particular kitchen operation or a mini operation to produce a resulting process that is a precise replication of the chef movement or a precise replication of the sensory curve." in par. 0412 and “Retrieving one or more stored cases whose problems bear strong similarity to the new problem, optionally adjusting the parameters from the retrieved case(s) to apply to the current case (e.g. an item may weigh somewhat more, and hence a somewhat stronger force is needed to lift it), and using the same methods and steps from the case(s) with the adjusted parameters (if needed) at least in part to solve the new problem.” In par. 0460 and “The Central RoboticControl both accesses the Case Library to determine if has a known sequence of actions for a current task, and updates the Case Library with outcome information upon executing the task.” In par. 0613)
retrieving, using the target task information, a relevant skill policy from a skill library to perform the target task, (see at least “Retrieving one or more stored cases whose problems bear strong similarity to the new problem, optionally adjusting the parameters from the retrieved case(s) to apply to the current case (e.g. an item may weigh somewhat more, and hence a somewhat stronger force is needed to lift it), and using the same methods and steps from the case(s) with the adjusted parameters (if needed) at least in part to solve the new problem.” In par. 0460),
wherein the retrieving is based on a predicted success of the relevant skill policy (see at least "First, a potentially very large library of pre-defined/pre-learned sensing-and-action sequences called minimanipulations. Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation). " in par. 0141 and “This software module uses data from the 3D world configuration modeler 262, which creates a new 3D world model at every sampling step from sensory data supplied by the multimodal sensor(s) unit(s), in order to ascertain that the configuration of the robotic kitchen systems and process matches that required by the recipe script (database); if not, it enacts modifications to the commanded system-configuration values to ensure the task is completed successfully.” In par. 0429 and “The creation module 60 of the minimanipulation database library is a process of creating, testing various possible combinations, and selecting an optimal minimanipulation to achieve a specific functional result.” In par. 0511 and “In recipe execution 780, the robotic hands 72 execute the minimanipulations 770 of cracking an egg with a knife, where the optimal way to execute each movement in the cracking an egg operation 772, the holding a knife operation 774, the striking the egg with a knife operation 776, and opening the cracked egg operation 778 is selected from the minimanipulations library database. The process of executing the optimal way to carry out each of the movements 772, 774, 776, 778 ensures that the minimanipulation 770 will achieve the same (or guarantee of), or substantially the same, outcome for that specific minimanipulation. The multimodal three-dimensional sensor 20 provides real-time adjustment capabilities 112 as to the possible variations in one or more ingredients, such as the dimension and weight of an egg.” In par. 0536);
modifying the relevant skill policy for the target task. (see at least "The real time adjustment module 112 is configured to provide real-time adjustments to the variables associated with a particular kitchen operation or a mini operation to produce a resulting process that is a precise replication of the chef movement or a precise replication of the sensory curve." in par. 0412 and “However, in the case of the non-standardized kitchen, the chances are very high that the system will have to modify and adapt the actual recipe itself and its execution, via a recipe script modification module 204, to suit the available tools/appliances 192 which differ from those in the chef studio 44 or the measured deviations from the recipe script (meat cooking too slowly, hot-spots in pot burning the roux, etc.).” in par. 0418)
Regarding Claim 31, Oleynik teaches:
the non-transitory computer program product recited in claim 30, wherein the retrieving further comprises:
utilizing task feature learning and transfer success prediction to select the target task. (see at least " Abstraction motion-commands (e.g. “crack an egg into the pan”, “sear to a golden color on both sides”, etc.) can be generated from the raw data, refined, and optimized through a multitude of iterative learning processes, carried out live and/or off-line, allowing the robotic kitchen systems to successfully deal with measurement-uncertainties, ingredient variations, etc., enabling complex (adaptive) minimanipulation motions using fingered-hands mounted to robot-arms and wrists, based on fairly abstraction/high-level commands (e.g. “grab the pot by the handle”, “pour out the contents”, “grab the spoon off the countertop and stir the soup”, etc.)." in par. 0137 and “Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation).” In par. 0141 and “In a preferred embodiment each POST result is associated with a probability of obtaining the desired result if the MM is executed. The Central Robotic Control both accesses the MM library to retrieve and execute MM's and updates it, e.g. in learning mode to add new MMs.” In par. 0612)
Regarding Claim 32, Oleynik teaches:
the non-transitory computer program product recited in claim 31,
wherein the task feature learning includes capturing geometry features from point cloud reconstruction, (see at least “The data-reduction and abstraction engine (set of software routines) 226 is intended to reduce the larger three-dimensional data sets and extract from them key geometric and associative information. A first step is to extract from the large three-dimensional data point-cloud only the specific workspace area of importance to the recipe at that particular point in time. Once the data set has been trimmed, key geometric features will be identified by a process known as template matching. This allows for the identification of such items as horizontal tabletops, cylindrical pots and pans, arm and hand locations, etc.” in par. 0423)
capturing dynamics features from next state prediction (see at least " More generally, a generalized minimanipulation M comprises triple <PRE, ACT, POST>, where PRE={(s.sub.1, s.sub.2, . . . , s.sub.n} is a set of items in the world state that must be true before the actions ACT=[a.sub.1, a.sub.2, . . . , a.sub.k] can take place, and result in a set of changes to the world state denoted as POST={p.sub.1, p.sub.2 . . . , p.sub.m}. Note that [square brackets] mean sequences, and {curly brackets} mean unordered sets. Each post condition may also have a probability in case the outcome is less than certain. For instance the minimanipulation for grasping an egg may have a 0.99 probability that the egg is in the hand of the robot (the remaining 0.01 probability may correspond to inadvertently breaking the egg while attempting to grasp it, or other unwanted consequence)." in par. 0515), and
capturing expert action features from inverse dynamics prediction. (see at least " Fifth, minimanipulations may be acquired by repeated observation of a human tutor (e.g. an expert chef) to determine the sensing-and-action sequence, and to determine the range of acceptable values for the variables." in par. 0141 and “FIG. 34 depicts a block diagram illustrating the process of how a remote robotic system would utilize the minimanipulation (MM) library(ies) to carry out a remote replication of a particular task (cooking, painting, etc.) carried out by an expert in a studio-setting, where the expert's actions were recorded, analyzed and translated into machine-executable sets of hierarchically-structured minimanipulation datasets (commands, parameters, metrics, time-histories, etc.) which when downloaded and properly parsed, allow for a robotic system (in this case a dual-arm torso/humanoid system) to faithfully replicate the actions of the expert with sufficient fidelity to achieve substantially the same end-result as that of the expert in the studio-setting.” In par. 0599 and “This is achieved by taking the parameter-set from the ‘offending’ minimanipulation action-step and using one or more of multiple techniques for parameter-optimization common in the field of machine-learning, to rebuild a specific minimanipulation step or sequence MM.sub.i into a revised minimanipulation step or sequence MM.sub.i*. The revised step or sequence MM.sub.i* is then used to rebuild a new command-0sequence that is passed back to the command executor 3175 for re-execution. The revised minimanipulation step or sequence MM.sub.i* is then fed to a re-build function that re-assembles the final version of the minimanipulation dataset, that led to the successful achievement of the required functional result, so it may be passed to the task- and parameter monitoring process 3179.” In par. 0604)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5-7, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oleynik et al (US 20170348854, hereinafter Oleynik) in view of Nasiriany et al (Learning and Retrieval from Prior Data for Skill-based Imitation Learning, published November 2022, see attached, hereinafter Nasiriany).
Regarding Claim 5, Oleynik teaches:
the tool as recited in claim 3,
Oleynik does not appear to explicitly teach all of the following, but Nasiriany does teach:
wherein retrieving the relevant skill policy includes computing the transfer success prediction between each source policy in the skill library and the target task, and selecting the source policy with a highest predicted transfer success score as the relevant skill policy for the target task. (see at least " We then calculate the pairwise 2 distances between the prior and target dataset skill embeddings, i.e., D[i][j] = Zi prior − Zj target 2. Next, for each prior dataset skill embedding zi ∈ Zprior, we find the closest corresponding target dataset skill, D_min[i] = min(D[i][:]). Finally, we retrieve the top-n sub-trajectories in Dprior with the smallest distance argsort(D_min)[:n], resulting in the retrieval dataset Dret." On page 5 )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the tool taught by Oleynik to incorporate the teachings of Nasiriany wherein the highest similarity prior skill embedding relative to the target skill embedding is determined and retrieved for fine-tuning. The motivation to incorporate the teachings of Nasiriany would be to improve the data-efficiency and robustness of the policy training (see page 2)
Regarding Claim 6, Oleynik as modified by Nasiriany teaches:
the tool as recited in claim 5,
Oleynik further teaches: wherein the retrieved relevant skill policy further utilizes fine-tuning for the target task. (see at least " The interim library data 3165-4 is fed into a learning-and-tuning engine 3166, where data from other multiple studio-sessions 3168 is used to extract similar minimanipulation actions and their outcomes 3166-1 and comparing their data sets 3166-2, allowing for parameter-tuning 3166-3 within each minimanipulation group using one or more of standard machine-learning/-parameter-tuning techniques in an iterative fashion 3166-5." in par. 0596 and “Updated parameter data is then used to rebuild the modified minimanipulation parameter set for re-execution as well as for updating/rebuilding a particular minimanipulation routine, which is provided back to the original library routines as a modified/re-tuned library for future use by other robotic systems.” In par. 0600)
Regarding Claim 7, Oleynik as modified by Nasiriany teaches:
the tool as recited in claim 6,
Oleynik further teaches: wherein the fine-tuning utilizes a proximal policy optimization. (see at least “The learning module 114 is configured to provide learning capabilities to the robotic cooking engine 56 to optimize the precise replication in preparing a food dish by robotic arms 70 and the robotic hands 72, as if the food dish was prepared by a chef, using a method such as case-based (robotic) learning.” In par. 0413 and "Machine learning in the context of robotic manipulation of relevance to the disclosure can involve well known methods for parameter adjustment, such as reinforcement learning. An alternate and preferred embodiment for this disclosure is a different and more appropriate learning technique for repetitive complex actions such as preparing and cooking a meal with multiple steps over time, namely case-based learning. Case-based reasoning, also known as analogical reasoning, has been developed overtime." in par. 0455)
Regarding Claim 19, Oleynik also teaches:
the robotic assembly system as recited in claim 16, further comprising:
Oleynik does not appear to explicitly teach all of the following, but Nasiriany does teach:
a machine learning system configured to work with the one or more processors to analyze the target task information and to select the relevant skill policy using a machine learning process of the machine learning system. (see at least " We train the policy on a dataset Dpolicy = {(oi fs,zi = µ(qφ(τi))}, where τi is an H-length sub trajectory, zi is the mean encoding of that sub-trajectory, and oi fs is the frame-stacked history of F observations preceding the sub-trajectory. We train the policy to predict zi from oi fs using a standard behavioral cloning loss. During execution, we roll out the LSTM skill decoder pψ in a closed-loop manner, i.e., at each timestep we observe ot and execute the next action at = µ(pψ(z,ot)). After rolling out the skill for H timesteps we repeat the process by sampling a new skill from the policy. " on page 5)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the tool taught by Oleynik to incorporate the teachings of Nasiriany wherein a machine learning model is used to select the most relevant skill to use for a task. The motivation to incorporate the teachings of Nasiriany would be to improve the data-efficiency and robustness of the policy training (see page 2)
Claim(s) 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oleynik et al (US 20170348854, hereinafter Oleynik) in view of Lynch et al (US 20260126804, hereinafter Lynch).
Regarding Claim 14, Oleynik teaches:
the tool as recited in claim 1,
Oleynik does not appear to explicitly teach all of the following, but Lynch does teach:
wherein at least one skill policy in the skill library includes disassembly trajectory information enabling representation learning of dynamics and expert actions. (see at least " Other processing, refining, or structuring of the training data may include… temporal order/reversal for sequence understanding" in par. 0221 and “If the robot fails to complete the task, a process for collecting corrective demonstrations may be initiated. In this process, an operator may take control of the robot from the failure state and provide a new, expert demonstration showing the correct sequence of actions to recover and complete the task.” In par. 0234)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the tool taught by Oleynik to incorporate the teachings of Lynch wherein the expert demonstration training data trajectories are temporally reversed. The motivation to incorporate the teachings of Lynch would be to improve the model’s sequence understanding (see par. 0221)
Regarding Claim 15, Oleynik as modified by Lynch teaches:
the tool as recited in claim 14,
Oleynik does not appear to explicitly teach all of the following, but Lynch does teach:
wherein the disassembly trajectory information is stored with disassembly actions, representing expert actions for task feature learning. (see at least " Other processing, refining, or structuring of the training data may include… temporal order/reversal for sequence understanding" in par. 0221 and “If the robot fails to complete the task, a process for collecting corrective demonstrations may be initiated. In this process, an operator may take control of the robot from the failure state and provide a new, expert demonstration showing the correct sequence of actions to recover and complete the task.” In par. 0234)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the tool taught by Oleynik to incorporate the teachings of Lynch wherein the expert demonstration training data trajectories are temporally reversed. The motivation to incorporate the teachings of Lynch would be to improve the model’s sequence understanding (see par. 0221)
Allowable Subject Matter
Claims 4, 8, 33 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
For Claims 4 and 33, the closest prior art comes from Oleynik and Nasiriany. These references teach robot control systems with skill libraries used to adapt previously learned skills to the current task but the prior art does not appear to teach using a neural network to predict a score indicating the relevancy of a skill policy to the target task in combination with all of the other limitations in the claim.
For Claim 8, the closest prior art comes from Oleynik and Nasiriany. These references teach robot control systems with skill libraries used to adapt previously learned skills to the current task but the prior art does not appear to teach training a transfer success predictor with labels obtained from previous experience applying source policies in the library to the target task in combination with all of the other limitations in the claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN M KATZ whose telephone number is (571)272-2776. The examiner can normally be reached Mon-Thurs. 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DYLAN M KATZ/Primary Examiner, Art Unit 3657
Prosecution Insights