DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
General Note:
While Examiner appreciates the support pointed out for the amendment of Claim 1, Applicant is requested to point out support for all amendments in accordance with the guidance of MPEP 2163. Lack thereof may be considered as an indication that Applicant considers the scope of a claim unaltered from the previously provided version of the claim.
Specification
Applicant’s amendments to the specification are accepted in light of the typographical nature of the error corrected. Examiner appreciates Applicant’s review of the specification.
Claim Rejections - 35 USC § 112(b)
The rejections of the claims are generally withdrawn, except for those related to Claims 12 – 13 and 15. Applicant does not appear to have rectified the issue presented with respect to Claim 12 and its dependent claims.
Double Patenting
The double patenting rejection is withdrawn in light of the terminal disclaimer filed by Applicant 05/07/2026 and accepted 05/13/2026.
Claim Rejections - 35 USC § 101
The rejections of the claims are withdrawn in light of Applicant’s amendments.
Claim Rejections - 35 USC § 102
Applicant's arguments filed 05/07/2026 have been fully considered but they are not persuasive.
Applicant’s arguments appear wholly conclusory. They appear to be a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references or otherwise do not clearly point out the patentable novelty which he or she thinks the claims present in view of the state of the art disclosed by the references cited or the objections made. Further, they do not show how the amendments avoid such references or objections.
More specifically, Applicant’s arguments appear to be limited to a statement stating that Oleynik does not teach an amended limitation. See the updated rejection below which contradicts this statement.
Claim Interpretation
The term “workflow” holds an ordinary and customary meaning to those of ordinary skill in the art. The term has been interpreted as some combination of activities which seek to accomplish an objective. In light of [0089] of Applicant’s originally filed specification, the activities are not inherently constrained by any particular chronological or other order
The term “work primitive” holds an ordinary and customary meaning to those of ordinary skill in the art. The term has been interpreted as any activity (or “step”, “sub-task”, etc.) that may be performed in combination with other activities to form a larger activity, etc. [0089] of Applicant’s originally filed specification relates.
The term “reusable” holds an ordinary and customary meaning to those of ordinary skill in the art. The term has been interpreted as meaning something capable of being used again or repeatedly. Examiner notes that [0090] of Applicant’s originally filed specification is not considered to amount to a special definition.
The term “percept” holds an ordinary and customary meaning to those of ordinary skill in the art. The term has been interpreted as meaning an impression of an object obtained by use of the senses. Examiner notes that [0099] of Applicant’s originally filed specification is not considered to amount to a special definition.
The term “access” holds multiple ordinary and customary meanings to those of ordinary skill in the art. One might mean to actually open or load (a computer file, etc.), but another might mean to be able to use, enter, or get near.
The term “goal state” holds an ordinary and customary meaning to those of ordinary skill in the art. Furthermore, the term is always relative to another state, etc. Therefore, the term does not appear to hold any inherent chronological or similar meaning without explicit further definition within the claims. The term has been interpreted as meaning “desired or intended state” or similar.
The term Boolean function holds an ordinary and customary meaning to those of ordinary skill in the art. However, while a Boolean function might generally be considered as requiring a binary input as well as a binary output, Applicant explicitly discloses a function that checks for a value within a threshold or range as a Boolean function. See [0128] of Applicant’s originally filed specification, “Such a Boolean function could entail … checking whether the position and/or orientation of the head is within a threshold”. Therefore “Boolean function” only appears to refer to the outputs being in a binary format. Furthermore, the additional recitation of “which returns either true or false” does not appear to further define the term. The Boolean function should return a binary value and therefore always have what could be considered a “true” or “false” value.
MPEP 2111.01 relates.
The nature of “satisfying” and not “satisfying” a metric, whether something is truly based on another, and what exactly constitutes a metric, particularly a combined metric, appears to be more fluid than implied by the claims in light of Applicant’s disclosure.
For example, in [0171] Applicant indicates that “a combined metric may not require that each metric be satisfied. With reference to the above exemplary scenario, the percept “is_self_looking_down()” can be replaced by the percept “is_self_looking_up()”. Given the right configuration and/or nature of sensors (e.g. image sensors with a wide field of view), the robot can be determined as sufficiently “looking down” by determining that the robot is NOT “looking up”.”
This clearly indicates that the metric of interest remains that of “is_self_looking_down()”, and is merely being determined in another manner. Furthermore, it indicates that the true metric is not “is_self_looking_up()”, but is “NOT is_self_looking_up()”. This furthermore indicates that the “combined metric” only requires a loose association with the supposedly combined metrics, as these metrics might be freely manipulated in the consideration of the combined metric, for example through arbitrary use of “NOT”.
For example, the disclosure does not recite retaining is_self_looking_down() and a separate combined metric function wherein 0 = true and 1 = false instead or similar, but instead makes it explicitly clear that it is a sufficiently equivalent function and uses the common NOT programming operator.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 8 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding Claim 8, the claim recites the limitations “each precept” and “at least one precept”. There does not appear to be any disclosure supporting these greater limitations to which they belong, as there does not appear to be any disclosure whereby all precepts are considered. Examiner notes that Applicant has amended away the limitation of “applied to the respective aspect of the first state representation” which previously limited the percepts referred to, to a particular subset of potential percepts rather than any and all percepts as presently claimed. The claims are presently comprising claims and the percepts available are not limited in scope.
Applicant has not pointed out support for these amendments ([0146] does not appear related), and Examiner was unable to find support for this limitation in the claims. Instead, the opposite was found as it is highly unclear why percepts of movement primitives in the library but not of relevancy to the current task would have their related percepts evaluated, or in other words, why every single percept known to a given system would always be evaluated.
This relates to some degree to Claim 7 which only holds support wherein the “plurality of percepts” which are “associated with a respective reusable work primitive” is only supported inasmuch as “respective reusable work primitive” refers to only a single work primitive and the “plurality” is understood to not refer to all included in a given system (any existing instead belonging to one or more other pluralities).
Therefore, the claim contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
laim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12 – 13 and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 12, the claim recites the limitation “the percept”. There is insufficient antecedent basis for this limitation in the claim. Furthermore, the claim recites “for each reusable work primitive … apply the percept”. The claim previously only recites “each percept in the plurality of percepts associated with a respective reusable work primitive”. This means that each percept must have a work primitive it is associated with, but does not mean that each work primitive must have a percept associated with it, or even that each percept is associated with a different work primitive. Consequently, it is not known which percept is referred to, as there may be multiple or none, and in the case of none, it is not possible to perform the final claim limitation.
In the interest of compact prosecution, the claim has been interpreted as reciting “each reusable work primitive having an associated percept in the plurality of percepts” previously and “the percept” to instead read “a percept of the respective reusable work primitive” or similar.
Regarding Claims 13 and 15, the claims depend from claim(s) rejected above and inherit the deficiencies of said claim(s) as described above. Therefore, Claims 13 and 15 are rejected under the same logic presented above.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 – 9, 11 – 13, and 15 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Oleynik (US 20230031545 A1).
Regarding Claim 1, Oleynik discloses:
A robot system comprising:
a robot body (See at least “robotic dual-arm system” ([0306]) in Figure 7D);
at least one sensor (See at least sensor units 66 ([0306]) in Figure 7D); and
a robot controller (See at least computer (14) ([0306]) in Figure 7D) which includes at least one processor and (See at least onboard processing unit 16 ([0306]) in Figure 7D) at least one non-transitory processor-readable storage medium (See at least media 18 ([0306]) in Figure 7D) communicatively coupled to the at least one processor (See at least Figure 7D), the at least one processor-readable storage medium storing processor-executable instructions (See at created recipe script 19 ([0306]) in Figure 7D) which when executed by the at least one processor cause the robot controller to perform operations comprising:
accessing a first reusable work primitive (See at least [0208] disclosing action primitives and [0224] disclosing minimanipulations) from a library of reusable work primitives (See at least [0017] “A minimanipulation library provides a large suite of higher-level sensing-and-execution sequences that are common building blocks for complex tasks, such as cooking, taking care of the infirm, or other tasks performed by the next generation of humanoid robots. More specifically, unlike the previous art, the present disclosure provides the following distinctive features. First, a potentially very large library of pre-defined/pre-learned sensing-and-action sequences called minimanipulations”);
accessing a first percept associated with the first reusable work primitive, wherein the first percept comprises a first metric for evaluating a state representation in relation to the first reusable work primitive (See at least [0017] “Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation) … Seventh, the assembly of minimanipulations into end-to-end-tasks is performed by robotic planning, taking into account the preconditions and postconditions of the component minimanipulations”);
accessing first sensor data captured by the at least one sensor at a first time (See at least [0279] “The quality check refers to three-dimensional vision sensors in the standardized robotic kitchen 50, which monitor and adjust in real time each manipulation action during the food preparation process to correct any deviation and avoid a flawed result”);
determineing a first state representation for the first time based on the first sensor data (See at least [0636] “A first database (database 1) 3194 contains the library of all minimanipulations (MM) known to the robot, including for each MM, a triple <PRE, ACT, POST>, where PRE={s.sub.1, s.sub.2, . . . , s.sub.n} is a set of items in the world state that must be true before the actions ACT=[a.sub.1, a.sub.2, . . . , a.sub.k] can take place, and result in a set of changes to the world state denoted as POST={p.sub.1, p.sub.2, . . . , p.sub.m}. In a preferred embodiment, the MMs are index by … by sensors and actuators they involved”); and
applying the first percept to the first state representation to obtain a first comparison result;
determining whether or not the first state representation satisfies the first metric based in part on the first comparison result (See again at least [0636], [0381] “Generalized Minimanipulations: A generalized minimanipulation comprises a well-defined sequence of sensing and actuator actions with an expected functional outcome. Associated with each minimanipulation we have a set of pre-conditions and a set of post-conditions. The pre-conditions assert what must be true in the world state in order to enable the minimanipulation to take place. The postconditions are changes to the world state brought about by the minimanipulations”, and the below recitations.
Examiner notes that as presently claimed there is no clear distinction between applying the first precept and determining metric satisfaction, particularly and especially given the claimed nature of the first precept, i.e. “a first metric for evaluating a state representation in relation to the first reusable work primitive”. In other words, determination of metric satisfaction and the comparison result are the same. As necessary, see also [388] – [0390] providing three examples of condition review, wherein the result is a binary success or unsuccessful, or similar); and
in response to determining that the first state representation does not satisfy the first metric, causing the robot body to perform the first reusable work primitive (See variously at least [0333] “A stage in preparing a food dish comprises one or more minimanipulations, where each minimanipulation comprises one or more robotic actions leading to a well-defined intermediate result. For instance, slicing a vegetable can be a minimanipulation comprising grasping the vegetable with one hand, grasping a knife with the other, and applying repeated knife movements until the vegetable is sliced. A stage in preparing a dish can comprise one or multiple slicing minimanipulations” or
[0399] “FIG. 18D is a simplified flow diagram illustrating one embodiment on taxonomy of manipulation actions for food preparation in kneading dough 740. Kneading dough 740 may be a minimanipulation that has been previously predefined in the library database of minimanipulations. The process of kneading dough 740 comprises a sequence of actions (or short minimanipulations), including grasping the dough 742, placing the dough on a surface 744, and repeating the kneading action until one obtains a desired shape 746” or,
[0613] “Note that should a minimanipulation (MM) sub-routine action fail (such as needing to re-grasp), all the minimanipulation sequencer has to do is to jump back backwards to a prior phase and repeat the same actions (possibly with a modified set of parameters to ensure success, if needed)’, or
[0336] “In another embodiment, more than one alternative method is provided for each stage, wherein, if one alternative fails, another alternative is tried”).
Regarding Claim 2, Oleynik discloses:
The robot system of claim 1 wherein the operations further comprise:
if the first metric is satisfied, outputting an indication of metric satisfaction as the first comparison result; and
if the first metric is not satisfied, outputting an indication of metric non-satisfaction as the first comparison result (Examiner notes that “an indication” is particularly broad, and “satisfaction” is similarly broad. Effectively, the metric simply appears to need to be processed, applied, etc.
See again at least [0636])
Regarding Claim 3, Oleynik discloses:
The robot system of claim 1, wherein, prior to the first time, the operations further comprise:
accessing second sensor data captured by the at least one sensor at a second time before the first time (See again at least [0279]);
determining a goal state based on the second sensor data (The nature of “based on” is not claimed with any particularity. Therefore, any association appears to satisfy this requirement. See already disclosed portions of Claim 1.
See alternatively also that the “recipe” to be replicated is based on demonstrated data. The conditions of minimanipulations, etc. are based on prior recorded data);
selecting the first percept and the associated first reusable work primitive based on the goal state (The nature of “selecting” and “based on” are not claimed with any particularity. Any use of a work primitive and its associated percept inherently requires selection, and each is inherently associated with conditions. All conditions are evaluated, see again at least [0636]. See also alternatively various discussions of high-level, intermediate, and low-level commands.); and
causing the robot body to perform the first reusable work primitive to (See again Claim 1).
Regarding Claim 4, Oleynik discloses:
The robot system of claim 1, wherein the operations further comprise:
determining a goal state (See already disclosed portions of Claim 1, for example [0636].
See alternatively also that the “recipe” to be replicated is based on demonstrated data. The conditions of minimanipulations, etc. are based on prior recorded data);
selecting the first percept and the associated first reusable work primitive by based on the goal state (All conditions are evaluated, see again at least [0636]).
Regarding Claim 5, Oleynik discloses:
The robot system of claim 1, wherein the first percept is a Boolean function which returns either true or false (See at least [388] – [0390] providing three examples of condition review, wherein the result is a binary success or unsuccessful, or similar. Examiner notes that these appear to match Applicant’s disclosure in [0128] of Applicant’s originally filed specification)
Regarding Claim 6, Oleynik discloses:
The robot system of claim 1, wherein the first metric indicates a success state representation, and the first metric is satisfied if the first state representation matches the success state representation (See at least [388] “In one embodiment, when specifying the threshold performance of a minimanipulation, whether generalized or basic, the measurements are performed on the POST conditions, comparing the actual result to the optimal result. For instance, in the task of assembly if a part is positioned within 1% of its desired orientation and location and the threshold of performance was 2%, then the minimanipulation is successful. Similarly, if the threshold were 0.5% in the above example, then the minimanipulation is unsuccessful”).
Regarding Claim 7, Oleynik discloses:
The robot system of claim 1, wherein:
accessing the first reusable work primitive from the library of reusable work primitives comprises accessing a plurality of reusable work primitives including the first reusable work primitive from the library of reusable work primitives (See again at least [0017] noting the plurality of “minimanipulations” and “primitives”);
accessing the first percept associated with the first reusable work primitive comprises accessing a plurality of percepts including the first percept, each percept of the plurality of percepts associated with a respective reusable work primitive of the plurality of reusable work primitives, wherein each percept comprises a respective metric for evaluating a state representation in relation to the respective reusable work primitive (See again at least [0636], in particular “for each MM, a triple <PRE, ACT, POST>” (emphasis added));
applying the first percept to the first state representation to obtain the comparison result comprises, for each percept in the plurality of percepts, applying the percept to at least one aspect of the first state representation to obtain a respective comparison result (See again at least [0636] and [0381]. Each condition is evaluated, and each minimanipulation may have a plurality of pre and post conditions); and
determining whether or not the first state representation satisfies the first metric based at least in part on the first comparison result comprises determining whether or not the first state representation satisfies a combined metric represented by the plurality of percepts based on a combination of the comparison results (See at least “PRE={s.sub.1, s.sub.2, . . . , s.sub.n}” and “POST={p.sub.1, p.sub.2, . . . , p.sub.m}” of [0636]. Applicant does not define “combined metric” beyond the vague phrasing of “represented by the plurality of percepts” wherein what constitutes “representing” is not provided).
Regarding Claim 8, Oleynik discloses:
The robot system of claim 7, wherein determining whether or not the first state representation satisfies the combined metric represented by the plurality of percepts comprises:
determining that the first state representation satisfies the combined metric if the first state representation satisfies the respective metric of each percept; and
determining that the first state representation does not satisfy the combined metric if the first state representation does not satisfy the respective metric of at least one percept (See again at least “a set of items in the world state that must be true” of [0636]).
Regarding Claim 9, Oleynik discloses:
The robot system of claim 7, wherein:
applying the percept to at least one aspect of the first state representation to obtain the respective comparison result comprises:
if the first state representation satisfies the respective metric of the percept, outputting an indication of metric satisfaction as the respective comparison result; and
if the first state representation does not satisfy the respective metric of the percept, outputting an indication of metric non-satisfaction as the respective comparison result (See again at least “a set of items in the world state that must be true” of [0636]).
Regarding Claim 11, Oleynik discloses:
The robot system of claim 1, wherein the operations further comprise:
accessing second sensor data captured by the at least one sensor at a second time subsequent to the first time;
determining a second state representation for the second time based on the second sensor data (Sensor data and world state data are constantly taken in real-time and updated for use in the system. See at least [0302], e.g. “Output data from both engines 316 and 318 are then used to feed the scene modeler and content classifier 320, where the 3D world model is created with all the key content required for executing the robotic cooking script executor”); and
applying the first percept to the second state representation to obtain a second comparison result; and
determining whether or not the second state representation satisfies the first metric based at least in part on the second comparison result (This claim may be satisfied a number of ways.
First, a given recipe process may use the same condition more than once. For example, a pre and post condition might be shared, or a given minimanipulation or action repeated without failure (e.g. [0288], [0399], or [0440]) or repeated due to failure (e.g. [0613]).
Second, any repetition of the entire recipe would read on this limitation (e.g. [0411] for testing or [0029] for the capacity to produce the same recipe on command)).
Regarding Claim 12, Oleynik discloses:
A robot system comprising:
a robot body (See at least “robotic dual-arm system” ([0306]) in Figure 7D);
at least one sensor (See at least sensor units 66 ([0306]) in Figure 7D);
a robot controller (See at least computer (14) ([0306]) in Figure 7D) which includes at least one processor (See at least onboard processing unit 16 ([0306]) in Figure 7D) and at least one non-transitory processor-readable storage medium (See at least media 18 ([0306]) in Figure 7D) communicatively coupled to the at least one processor (See at least Figure 7D), the at least one processor-readable storage medium storing processor-executable instructions (See at created recipe script 19 ([0306]) in Figure 7D) which when executed by the at least one processor cause the robot system to perform operations comprising:
identifying a workflow to complete a work objective (See at least [0240] “Recipe Script—refers to a recipe script as a sequence in time containing a structure and a list of commands and execution primitives (simple to complex command software) that, when executed by the robotic kitchen elements (robot-arm, automated equipment, appliances, tools, etc.) in a given sequence, should result in the proper replication and creation of the same dish as prepared by the human chef in the studio-kitchen”), the workflow comprising a plurality of reusable work primitives available in a library of reusable work primitives (See at least [0017] “A minimanipulation library provides a large suite of higher-level sensing-and-execution sequences that are common building blocks for complex tasks, such as cooking, taking care of the infirm, or other tasks performed by the next generation of humanoid robots. More specifically, unlike the previous art, the present disclosure provides the following distinctive features. First, a potentially very large library of pre-defined/pre-learned sensing-and-action sequences called minimanipulations”);
accessing each reusable work primitive in the plurality of reusable work primitives (See again at least [0240]));
accessing a plurality of percepts, each percept in the plurality of percepts associated with a respective reusable work primitive in the plurality of reusable work primitives, each percept in the plurality of percepts comprising a respective metric for evaluating a state representation in relation to the respective reusable work primitive (See at least [0017] “Second, each mini-manipulation encodes preconditions required for the sensing-and-action sequences to produce successfully the desired functional results (i.e. the postconditions) with a well-defined probability of success (e.g. 100% or 97% depending on the complexity and difficulty of the minimanipulation) … Seventh, the assembly of minimanipulations into end-to-end-tasks is performed by robotic planning, taking into account the preconditions and postconditions of the component minimanipulations”); and
performing the workflow (See at least abstract “a robotic execution module configured for executing the minimanipulation steps by the robotic platform to accomplish a functional result”),
wherein, for each reusable work primitive in the plurality of reusable work primitives, performing the workflow comprises (This appears to be a statement of purpose of the following, otherwise this is an improperly constructed limitation as the work primitives as subsidiary to the workflow, not the other way around):
accessing sensor data captured by the at least one sensor at a respective time (See at least [0279] “The quality check refers to three-dimensional vision sensors in the standardized robotic kitchen 50, which monitor and adjust in real time each manipulation action during the food preparation process to correct any deviation and avoid a flawed result”);
determining a state representation for the respective time based on the sensor data (See at least [0636] “A first database (database 1) 3194 contains the library of all minimanipulations (MM) known to the robot, including for each MM, a triple <PRE, ACT, POST>, where PRE={s.sub.1, s.sub.2, . . . , s.sub.n} is a set of items in the world state that must be true before the actions ACT=[a.sub.1, a.sub.2, . . . , a.sub.k] can take place, and result in a set of changes to the world state denoted as POST={p.sub.1, p.sub.2, . . . , p.sub.m}. In a preferred embodiment, the MMs are index by … by sensors and actuators they involved”);
applying the percept to the state representation for the respective time to determine whether or not the respective metric is satisfied (See at least [0381] “Generalized Minimanipulations: A generalized minimanipulation comprises a well-defined sequence of sensing and actuator actions with an expected functional outcome. Associated with each minimanipulation we have a set of pre-conditions and a set of post-conditions. The pre-conditions assert what must be true in the world state in order to enable the minimanipulation to take place. The postconditions are changes to the world state brought about by the minimanipulations”);
and in response to determining that the respective metric is not satisfied, causing the robot body to perform the reusable work primitive (See variously at least [0333] “A stage in preparing a food dish comprises one or more minimanipulations, where each minimanipulation comprises one or more robotic actions leading to a well-defined intermediate result. For instance, slicing a vegetable can be a minimanipulation comprising grasping the vegetable with one hand, grasping a knife with the other, and applying repeated knife movements until the vegetable is sliced. A stage in preparing a dish can comprise one or multiple slicing minimanipulations” or
[0399] “FIG. 18D is a simplified flow diagram illustrating one embodiment on taxonomy of manipulation actions for food preparation in kneading dough 740. Kneading dough 740 may be a minimanipulation that has been previously predefined in the library database of minimanipulations. The process of kneading dough 740 comprises a sequence of actions (or short minimanipulations), including grasping the dough 742, placing the dough on a surface 744, and repeating the kneading action until one obtains a desired shape 746” or,
[0613] “Note that should a minimanipulation (MM) sub-routine action fail (such as needing to re-grasp), all the minimanipulation sequencer has to do is to jump back backwards to a prior phase and repeat the same actions (possibly with a modified set of parameters to ensure success, if needed)’, or
[0336] “In another embodiment, more than one alternative method is provided for each stage, wherein, if one alternative fails, another alternative is tried”).
Regarding Claim 13, Oleynik discloses:
The robot system of claim 12, wherein for each reusable work primitive in the plurality of reusable work primitives, prior to each respective time, the operations further comprise:
determining a goal state for the reusable work primitive based on the workflow (See again at least [0636]. Conditions are known before taking sensor data); and
causing the robot body to perform the reusable work primitive (Appears inherent to the limitations of Claim 12 of “perform the workflow” and “causing the robot body to perform the reusable work primitive”).
Regarding Claim 15, Oleynik discloses:
The robot system of claim 12, wherein each percept is a Boolean function which returns either true or false (See at least [388] – [0390] providing three examples of condition review, wherein the result is a binary success or unsuccessful, or similar. Examiner notes that these appear to match Applicant’s disclosure in [0128] of Applicant’s originally filed specification).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Examiner notes that PDDL (Planning Domain Definition Language) and associated works appears to generally encompass the general concepts presented within the claims.
Ijiri et al. (US 20210283771 A1) which has been used as a primary reference in combination with Natarajan et al. (US 20210229281 A1) and Oleynik to reject similar claims in copending Application No. 18/655,623.
Jeon et al. (Jeon, Jeongmin, et al. "Primitive action based combined task and motion planning for the service robot." Frontiers in Robotics and AI 9 (2022): 713470) which discloses “a CTAMP system in which a symbolic action sequence is generated in task planning, and each action is verified geometrically in motion planning using the off-the-shelf planners and reasoners. The approach is that a set of action models is defined with PDDL in the interface module (action library) and the required information to each planner is automatically provided by the interface module”. See in particular Section 3 and Figure 2.
Kattepur et al. (US 20210049037 A1) which discloses the use of conditioned basic actions. [0043] “For this purpose, the system 100 may also maintain a domain file consisting: (a) Action description, describing a possible action that may be performed in relation to an environment, (b) Precondition, a conjunction of atoms which specify what must be true before an operator may be applied, (c) Effects, is a conjunction of literals which define situation changes after application of an operator”.
Spies et al. (US 20230321826 A1) which discloses sequences of primitives having conditions including a clear combined condition set. See for example Figure 3 and [0026] “Symbolic states 320 that are determined by each sensor modality 330 to be “true” are indicated by a box with a cross-hatched pattern. Symbolic states 320 that are determined to be “false” by a sensor modality 330 are indicated by a box with an empty fill pattern. No box is shown when a sensor modality 330 does not evaluate the corresponding symbolic state 320. In this example, predicates are defined to maximize the performance of each sensor modality 330”
González et al. (González-Santamarta, Miguel Á., et al. "SAILOR: Perceptual Anchoring For Robotic Cognitive Architectures." arXiv preprint arXiv:2303.08204 (2023).) which discloses the use of PDDL which uses conditioned actions and provides perceptual anchoring such that the conditioned actions can be grounded in the perception.
Kroemer et al. (Kroemer, Oliver, Scott Niekum, and George Konidaris. "A review of robot learning for manipulation: Challenges, representations, and algorithms." Journal of machine learning research 22.30 (2021): 1-82.) which discusses primitive actions, pre-conditions, post-conditions, effects, predicates, etc. See for example Section 7.
Zhang et al. (Zhang, Xiaohan, et al. "Grounding classical task planners via vision-language models." arXiv preprint arXiv:2304.08587 (2023).) which uses a vision language model to evaluate the precondition and effects during execution of a task using actions having said features.
Fox et al. (Fox, Maria, Jonathan Gough, and Derek Long. "Detecting execution failures using learned action models." Proceedings of The National Conference on Artificial Intelligence. Vol. 22. No. 2. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2007.) which discloses means for detecting when planned behavior has diverged from expected behavior.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW C GAMMON whose telephone number is (571)272-4919. The examiner can normally be reached M - F 10:00 - 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ADAM MOTT can be reached on (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW C GAMMON/Examiner, Art Unit 3657 /ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights