DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on February 23, 2026 has been entered.
Response to Amendment
This correspondence is in response to amendments filed on February 23, 2026 following a Request for Continued Examination filed on March 17, 2026. Claims 1, 10, 17, and 19 are amended. Claims 2-9, 11-16, 18, and 21-23 are filed as originally or previously presented. Claim 20 is cancelled. Claim 24 is new. Arguments regarding the prior art have been addressed below.
Response to Arguments
Applicant argues that Sun does not disclose a cobot physically providing tools/materials to an operator in a shared workspace and thus the robot of Sun cannot perform the assistance subtask as described (see Remarks Page 9). Applicant’s arguments with respect to the assistance subtask and corresponding features of Sun have been considered but are moot because the new ground of rejection does not rely on the same combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Specification
The disclosure is objected to because of the following informalities:
Throughout the specification, Applicant continually refers to an “atomic action”. Where applicant acts as his or her own lexicographer to specifically define a term of in the disclosure contrary to its ordinary meaning, the written description must clearly redefine the term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The term “atomic” is not clearly defined in the disclosure while the accepted meaning is most commonly related to nuclear bombs or nuclear energy. Merriam-Webster discloses associated definitions of atomic as “of, related to, or concerned with atoms”, “nuclear”, “marked by acceptance of the theory of atomism”, or “minute”. The use of atomic is most closely aligned with the less common meaning defined as “minute”, but Examiner cannot ascertain whether this is the intended meaning regarding an atomic action. Examiner found Paragraphs [0054-0056] to be most helpful for describing such atomic actions. However, given the disparate meanings of atomic as listed above, this renders some level of confusion as to what is truly meant.
Appropriate correction is required.
Drawings
The drawings are objected to because Fig. 4 includes labels regarding “atomic action” which has been objected to above with regards to the specification. Upon decision of a corrective response to the above specification objection, similar corrections should be made to the drawings.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “…an executable identifier indicative of whether the human operator and/or the cobot is configured to perform the corresponding subtask…” in lines 5-6. It is unclear how a human operator may be configured to perform a subtask. Thus, the metes and bounds of the claim are unclear when considering the human operator and cobot as alternatives. Based on Applicant’s description in [0065-0066], Examiner best understands this to mean the subtask is either human-executable or cobot-executable. As such, Examiner will read the claim instead as “… an executable identifier indicative of a human-executable identifier and/or cobot-executable identifier which represents that the subtask may be performed by either the human operator or the cobot…” as described in [0065].
Claim 15 is rejected for having a similar limitation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-16 and 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Sedlmayr (US 2018/0345497 A1) in view of Heckmann et al. (US 2020/0012670 A1; hereinafter “Heckmann”).
Regarding claim 1, Sedlmayr teaches a control device (“The object is furthermore achieved by means of a computer program having program commands which, when they are loaded on a computer and/or microcontroller, cause the computer and/or microcontroller to execute the method described. Such computer programs can be loaded on devices configured to control manipulator systems” [0051]. Thus, there is a control device of the manipulator system comprising a computer program which executes the method of the disclosure.), comprising:
an error detector configured to detect an error in a performance of a manipulator task (”FIG. 2 shows a schematic illustration 2 of a semantic module of a computer program for carrying out a method for correcting errors in a manipulator system. In this case, the semantic module 200 is subdivided into an operation structure 210 and into a reaction structure 220. If an error occurs during the execution of the operation structure 210, then the operation structure 210 is interrupted and the process continues with the reaction structure 220” [0060]. Thus, the computer program is configured to determine if an error has occurred in the manipulator task such that the process continues from an operation structure to a reaction structure, thus including such a module to act as an error detector. Examiner will consistently refer to the operation structure as the task.)…; and
an error corrector (The computer program additionally includes the reaction structure which will be considered as the error corrector.) including:
correction planner configured to determine an error correction plan based on the detected error, the error correction plan including one or more corrective subtasks configured to control a manipulator to correct the detected error (“The reaction structure 220 includes reaction operations 221 to 225. In a first reaction operation 221, the type of error that occurred is determined and a decision is made regarding with which of the reaction operations 222-224 the process is intended to continue” [0061]. Thus, the reaction structure comprises reaction operations, i.e., error correction plans, which are chosen based on the type of error that has occurred, i.e., the detected error. Provided that the reaction structure decides the reaction operation, i.e., correction plan, the reaction structure additionally functions as a correction planner configured to make such a determination. Each of reaction operations 222-224 are corrective subtasks configured to control the manipulator to correct the detected error, as exemplified in the remainder of paragraph [0061] which is not directly cited.); and
a facilitator configured to determine a facilitation plan based on the determined error correction plan (“The reaction operation 225 comprises an instruction that defines the rerun point and the following operation structure with which the manipulator program is intended to be continued” [0062]. Thus, the reaction operation 225 performs as a facilitator since it determines the rerun point and following operation structure, i.e., facilitation plan, based on the determined reaction operations. See additional examples in Figs. 3 and 4 which show different operations and their associated reaction operations.), the determination of the facilitation plan including:
determining an independent correctability, by the manipulator, of the detected error (“This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request the intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like. Once the operator confirms to the manipulator system that the task has been performed successively, then it is possible to continue with the reaction structure or an operation structure” [0044]. Thus, there is a function of the system which requires a determination of whether the error can be rectified automatically by the manipulator, or if operator intervention will be required.); and
… wherein the error corrector is configured to generate a control signal to control the manipulator based on the correction plan and the facilitation plan (“The reaction operation 225 comprises an instruction that defines the rerun point and the following operation structure with which the manipulator program is intended to be continued. Such a rerun point can be defined for example at the beginning of the operation structure 210, such that the operation structure 210 can be executed anew. The renewed execution of the operation structure 210 can be effected, if appropriate, with a changed parameter set. Likewise, other rerun points can also be used for continuing the manipulator program. A branching and diverse reactions to errors are thus possible. Since both the operation structure and the reaction structure are implemented in the same semantic module, the manipulator program remains clear and allows an individual adaptation of the operation structure and/or of the reaction structure” [0062]. Thus, the reaction structure, i.e., error corrector, is configured to generate an instruction, i.e., control signal, which causes the manipulator to resume the intended operation structure at the defined rerun point based on a branching and diverse reaction to errors as defined in the correction plan and facilitation plan defined above.).
However, Sedlmayr does not explicitly teach the task as a human-robot collaborative task within a human-robot collaborative workspace,
the manipulator as a collaborative robot, and
the determination of the facilitation plan including:
…based on the independent correctability of the detected error:
determining one or more tools and/or materials usable, by a human operator, to correct the detected error; and
generating an assistance subtask configured to control the cobot to pick up and deliver the one or more tools and/or materials to the human operator within the workspace to assist the human operator in correcting the detected error using the delivered one or more tools and/or materials…
Heckmann, pertinent to the problem at hand, teaches a human-robot collaborative task within a human-robot collaborative workspace (“The inventive approach is particularly suited to support cooperation between the user and the robot in a workshop environment and concerning frequently re-occurring, but flexibly structured tasks” [0018]. Thus, there is a cooperative workshop environment in which a user and human perform cooperative tasks, i.e., human-robot collaborative task and workspace.),
a collaborative robot (cobot) (“The invention provides an assistance system that is able to cooperate with the user in a large set of activities” [0019]. “The assistance system 1 may form part of a machine capable of carrying out a complex series of actions in an automated manner (robot)” [0054]. Thus, there is an assistance system formed as part of a robot which cooperates with the user thus functioning as a collaborative robot (cobot).),
…determining one or more tools and/or materials usable, by a human operator, to advance a task (“The current task status provided by the task status determination unit 12 may in particular enable the assistance system 1 to determine a currently requested object in the unit for determining a requested object 13. The requested object may be, for example, a spare part or a required tool” [0128]. “The current task status determined in the task status determination unit 12 may further serve to predict the next needed object in the unit for predicting a required object 14. The unit for predicting a required object 14 enables the assistance system to prepare a next subtask, step, or next sub-step while the user and the robot are performing the current subtask, step or sub-step” [0067]. Thus, the assistance system determines one or more tools or spare parts which are requested by the user or predicted to be needed by the user to perform the next step or sub-step as determined by the task status.); and
generating an assistance subtask configured to control the cobot to pick up and deliver the one or more tools and/or materials to the human operator within the workspace to assist the human operator in advancing the task using the delivered one or more tools and/or materials (“The processor of the system, according to an embodiment of the invention, is configured to generate the support signal including information on manipulating the object or part of the object comprising handing the at least one object to the user or fetching the at least one object from the user. Such an object may in particular be a tool or a spare part, and it is evident that a plurality of objects may be handed or fetched” [0027]. Thus, the system the system controls the manipulator, i.e., cobot, to pick up and deliver the one or more tools or spare parts to the user within the workspace in order to assist the human operator as evidenced by the support signal. As evidenced in the rejection above, and additionally based on description of [0028-0029], the system determines the tools and objects/parts to be handed based on the intended task/sub-task and corresponding progression, thus handing the user a tool to be used in executing said task.)…
Examiner acknowledges that Heckmann does not explicitly determine the assistance sub-task based on a detected error. However, given that the system monitors task progress to output a support signal (see [0011]), it would be obvious to one of ordinary skill in the art that task progress would additionally include any such errors encountered throughout the execution of the task. Exemplary to this implied teaching, Heckmann recites “Based on this task knowledge, the assistance system is able to resolve the requests of the user by narrowing down a search using the knowledge on the most likely involved tools and objects at each state of execution of a task. A temporal order in which the tools and objects are used my enable the assistance system to estimate the current state of the task, for example, task progress and/or a degree of success in completing the task” [0014]. Thus, it would be obvious to one of ordinary skill in the art that Heckmann, although absent any explicit teaching of a detected error requiring correction, implies such teachings of possible encountered errors and uses the assistance subtask as a means to facilitate task completion and success based on the progress of the task.
Therefore, it would additionally be obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the error detection system of Sedlmayr to include the collaborative robot system and associated assistance subtasks as taught by Heckmann. One of ordinary skill in the art would have been motivated to combine such teachings because such a combination encourages successful cooperation between user and robot by minimizing human effort and maximizing task progress (see Heckmann, [0016]). Additionally, for less experienced users, the inclusion of an assistance system allows the user to perform a guided approach to any such corrective tasks while maintaining flexibility for the order in which corrective tasks may be executed (see Heckmann, [0119]).
Regarding claim 2, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the correction planner is configured to determine the error correction plan further based on a task description (Figs. 3 and 4 are examples of such task descriptions in which the reaction operation, i.e., error correction plan, “Rx” (x being an integer) are determined based on task operations “Ox” (x being an integer).).
Regarding claim 3, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the facilitator is configured to determine the facilitation plan further based on a task description (Figs. 3 and 4 are examples of such task descriptions which decide which reaction operations should be performed such that the task resumes at the appropriate rerun point “APx” (x being an integer). As acknowledged in the rejection of claim 1, the facilitation plan is the determination of the rerun point.).
Regarding claim 4, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the correction planner is configured to determine the error correction plan further based on a task description (Figs. 3 and 4 are examples of such task descriptions in which the reaction operation, i.e., error correction plan, “Rx” (x being an integer) are determined based on task operations “Ox” (x being an integer).) and the facilitator is configured to determine the facilitation plan further based on the task description (Figs. 3 and 4 are examples of such task descriptions which decide which reaction operations should be performed such that the task resumes at the appropriate rerun point “APx” (x being an integer). As acknowledged in the rejection of claim 1, the facilitation plan is the determination of the rerun point.).
Regarding claim 5, Sedlmayr as modified by Heckmann teaches the control device of claim 4,
with Sedlmayr further teaching wherein the task description comprises one or more subtask actions defining one or more respective subtasks of the human-robot collaborative task (Each operation “Ox” (x being an integer) and reaction operation “Rx” (x being an integer) corresponds to a subtask action which defines the one or more subtask of the task description. In the case of the modification with Heckman, these subtasks would correspond to the sub-tasks of the human-robot collaborative task.).
Regarding claim 6, Sedlmayr as modified by Heckmann teaches the control device of claim 5,
with Sedlmayr further teaching wherein the one or more subtask actions comprise:
an inverse subtask configured to reverse a corresponding subtask of the human-robot collaborative task (“The reaction structure preferably comprises reaction operations which can reverse individual operations if the relevant operation structure describes a reversible process” [0030]. Thus, there are reaction structures which reverse operations, i.e., inverse subtask which reverses a corresponding operation subtask.);
an executable identifier indicative of whether the human operator and/or the cobot is configured to perform the corresponding subtask (“Preferably, the reaction structure includes at least one reaction operation whose execution interrupts the interrupted operation structure until an operator input has been effected, preferably by means of the graphical user interface. This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like. Once the operator confirms to the manipulator system that the task has been performed successively, then it is possible to continue with the reaction structure or an operation structure” [0043-0044]. Thus, in the case of an error which is not able to be automatically rectified, there will be a notification sent out which notifies an operator that assistance is necessary in performing an aspect of the subtask. This notification, or lack thereof, serves as the “executable identifier” which indicates whether the human operator or the robot performs the subtask.); and
with Heckmann further teaching … tool and/or material information defining the one or more tools and/or materials usable to perform the corresponding subtask (“To identify the tools and parts in each repair step it is in many cases sufficient to parse the list of tools and parts and then go from repair step to repair step and identify these tools and parts in each step. For parsing, the assistance system 1 may apply a simple pattern matching, i.e., keywords indicating the list of tools and parts and the segmentation into steps can be searched and then in each step a pattern matching between the tools and parts in the list and the words in each step can be performed” [0139]. Thus, there is a tool list to be parsed by the cobot which relates each tool or part to a corresponding step in the task or subtask.).
Regarding claim 7, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the control signal controls the cobot to perform the one or more corrective subtasks in response to the detected error being correctable by the cobot (“Preferably, the execution of an operation structure is interrupted upon the occurrence of an error and the execution of a reaction structure is continued afterward. Consequently, an operation structure of the manipulator program can be repeatedly directly after the occurrence of an error or the manipulator program can be continued for a different rerun point. This makes it possible to minimize stoppage times of the manipulator system and to continue preferably in an automated manner, i.e. without intervention by an operator” [0036]. Thus, to minimize stoppage times of the system, the manipulator executes the reaction structure to perform the corrective subtasks in response to such subtasks being correctable by the manipulator.).
Regarding claim 8, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the control signal controls the cobot to perform the assistance subtask in response to the cobot being unable to correct the detected error (“This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request the intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like” [0044]. Thus, in the event that the error is not correctable by the manipulator, an operator is requested for intervention. The assistance subtask which has been modified by Heckmann would thus correspond to such a support signal as evidenced by Heckmann.).
Regarding claim 9, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr teaching wherein the one or more corrective subtasks includes one or more inverted subtasks configured to reverse one or more corresponding subtasks of the human-robot collaborative task (“The reaction structure preferably comprises reaction operations which can reverse individual operations if the relevant operation structure describes a reversible process” [0030]. Thus, there are reaction structures which reverse operations, i.e., inverse subtask which reverses a corresponding operation subtask.).
Regarding claim 10, Sedlmayr teaches a control device (“The object is furthermore achieved by means of a computer program having program commands which, when they are loaded on a computer and/or microcontroller, cause the computer and/or microcontroller to execute the method described. Such computer programs can be loaded on devices configured to control manipulator systems” [0051]. Thus, there is a control device of the manipulator system comprising a computer program which executes the method of the disclosure.), comprising:
one or more processors (The computer and/or microcontroller stated above are best understood to be one or more processors.); and
memory storing instructions that, when executed by the one or more processors, configure the control device (The computer programs, i.e., instructions, executed by the microcontroller and/or computer are loaded, i.e., stored, on the microcontroller and/or computer. These microcontrollers are thus comprised by the devices configured to control the manipulator system which further load, i.e., store, the computer programs.) to:
detect an error in a performance of a manipulator task (”FIG. 2 shows a schematic illustration 2 of a semantic module of a computer program for carrying out a method for correcting errors in a manipulator system. In this case, the semantic module 200 is subdivided into an operation structure 210 and into a reaction structure 220. If an error occurs during the execution of the operation structure 210, then the operation structure 210 is interrupted and the process continues with the reaction structure 220” [0060]. Thus, the computer program is determines if an error has occurred in the manipulator task such that the process continues from an operation structure to a reaction structure, thus including such a module to act as an error detector. Examiner will consistently refer to the operation structure as the task.)…;
determine an error correction plan, based on the detected error, configured to control a manipulator to correct the detected error (“The reaction structure 220 includes reaction operations 221 to 225. In a first reaction operation 221, the type of error that occurred is determined and a decision is made regarding with which of the reaction operations 222-224 the process is intended to continue” [0061]. Thus, the reaction structure comprises reaction operations, i.e., error correction plans, which are chosen based on the type of error that has occurred, i.e., the detected error. Each of reaction operations 222-224 are corrective tasks configured to control the manipulator to correct the detected error, as exemplified in the remainder of paragraph [0061] which is not directly cited.);
determine a facilitation plan, based on the determined error correction plan (“The reaction operation 225 comprises an instruction that defines the rerun point and the following operation structure with which the manipulator program is intended to be continued” [0062]. Thus, the reaction operation 225 performs as a facilitator since it determines the rerun point and following operation structure, i.e., facilitation plan, based on the determined reaction operations. See additional examples in Figs. 3 and 4 which show different operations and their associated reaction operations.), the determination of the facilitation plan including:
determining an independent correctability, by the manipulator, of the detected error (“This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request the intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like. Once the operator confirms to the manipulator system that the task has been performed successively, then it is possible to continue with the reaction structure or an operation structure” [0044]. Thus, there is a function of the system which requires a determination of whether the error can be rectified automatically by the manipulator, or if operator intervention will be required.); and
…generate a control signal to control the manipulator based on the correction plan and the facilitation plan (“The reaction operation 225 comprises an instruction that defines the rerun point and the following operation structure with which the manipulator program is intended to be continued. Such a rerun point can be defined for example at the beginning of the operation structure 210, such that the operation structure 210 can be executed anew. The renewed execution of the operation structure 210 can be effected, if appropriate, with a changed parameter set. Likewise, other rerun points can also be used for continuing the manipulator program. A branching and diverse reactions to errors are thus possible. Since both the operation structure and the reaction structure are implemented in the same semantic module, the manipulator program remains clear and allows an individual adaptation of the operation structure and/or of the reaction structure” [0062]. Thus, the reaction structure, i.e., error corrector, is configured to generate an instruction, i.e., control signal, which causes the manipulator to resume the intended operation structure at the defined rerun point based on a branching and diverse reaction to errors as defined in the correction plan and facilitation plan defined above.).
However, Sedlmayr does not explicitly teach the task as a human-robot collaborative task within a human-robot collaborative workspace,
the manipulator as a collaborative robot, and
the determination of the facilitation plan including:
…based on the independent correctability of the detected error:
determining one or more tools and/or materials usable, by a human operator, to correct the detected error; and
generating an assistance subtask configured to control the cobot to pick up and deliver the one or more tools and/or materials to the human operator within the workspace to assist the human operator in correcting the detected error using the delivered one or more tools and/or materials…
Heckmann, pertinent to the problem at hand, teaches a human-robot collaborative task within a human-robot collaborative workspace (“The inventive approach is particularly suited to support cooperation between the user and the robot in a workshop environment and concerning frequently re-occurring, but flexibly structured tasks” [0018]. Thus, there is a cooperative workshop environment in which a user and human perform cooperative tasks, i.e., human-robot collaborative task and workspace.),
a collaborative robot (cobot) (“The invention provides an assistance system that is able to cooperate with the user in a large set of activities” [0019]. “The assistance system 1 may form part of a machine capable of carrying out a complex series of actions in an automated manner (robot)” [0054]. Thus, there is an assistance system formed as part of a robot which cooperates with the user thus functioning as a collaborative robot (cobot).),
…determining one or more tools and/or materials usable, by a human operator, to advance a task (“The current task status provided by the task status determination unit 12 may in particular enable the assistance system 1 to determine a currently requested object in the unit for determining a requested object 13. The requested object may be, for example, a spare part or a required tool” [0128]. “The current task status determined in the task status determination unit 12 may further serve to predict the next needed object in the unit for predicting a required object 14. The unit for predicting a required object 14 enables the assistance system to prepare a next subtask, step, or next sub-step while the user and the robot are performing the current subtask, step or sub-step” [0067]. Thus, the assistance system determines one or more tools or spare parts which are requested by the user or predicted to be needed by the user to perform the next step or sub-step as determined by the task status.); and
generating an assistance subtask configured to control the cobot to pick up and deliver the one or more tools and/or materials to the human operator within the workspace to assist the human operator in advancing the task using the delivered one or more tools and/or materials (“The processor of the system, according to an embodiment of the invention, is configured to generate the support signal including information on manipulating the object or part of the object comprising handing the at least one object to the user or fetching the at least one object from the user. Such an object may in particular be a tool or a spare part, and it is evident that a plurality of objects may be handed or fetched” [0027]. Thus, the system the system controls the manipulator, i.e., cobot, to pick up and deliver the one or more tools or spare parts to the user within the workspace in order to assist the human operator as evidenced by the support signal. As evidenced in the rejection above, and additionally based on description of [0028-0029], the system determines the tools and objects/parts to be handed based on the intended task/sub-task and corresponding progression, thus handing the user a tool to be used in executing said task.)…
Examiner acknowledges that Heckmann does not explicitly determine the assistance sub-task based on a detected error. However, given that the system monitors task progress to output a support signal (see [0011]), it would be obvious to one of ordinary skill in the art that task progress would additionally include any such errors encountered throughout the execution of the task. Exemplary to this implied teaching, Heckmann recites “Based on this task knowledge, the assistance system is able to resolve the requests of the user by narrowing down a search using the knowledge on the most likely involved tools and objects at each state of execution of a task. A temporal order in which the tools and objects are used my enable the assistance system to estimate the current state of the task, for example, task progress and/or a degree of success in completing the task” [0014]. Thus, it would be obvious to one of ordinary skill in the art that Heckmann, although absent any explicit teaching of a detected error requiring correction, implies such teachings of possible encountered errors and uses the assistance subtask as a means to facilitate task completion and success based on the progress of the task.
Therefore, it would additionally be obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the error detection system of Sedlmayr to include the collaborative robot system and associated assistance subtasks as taught by Heckmann. One of ordinary skill in the art would have been motivated to combine such teachings because such a combination encourages successful cooperation between user and robot by minimizing human effort and maximizing task progress (see Heckmann, [0016]). Additionally, for less experienced users, the inclusion of an assistance system allows the user to perform a guided approach to any such corrective tasks while maintaining flexibility for the order in which corrective tasks may be executed (see Heckmann, [0119]).
Regarding claim 11, Sedlmayr as modified by Heckmann teaches the control device of claim 10,
with Sedlmayr further teaching wherein the error correction plan includes one or more corrective subtasks configured to control the cobot to correct the detected error (“Preferably, the execution of an operation structure is interrupted upon the occurrence of an error and the execution of a reaction structure is continued afterward. Consequently, an operation structure of the manipulator program can be repeatedly directly after the occurrence of an error or the manipulator program can be continued for a different rerun point. This makes it possible to minimize stoppage times of the manipulator system and to continue preferably in an automated manner, i.e. without intervention by an operator” [0036]. Thus, to minimize stoppage times of the system, the manipulator is autonomously controlled to execute the reaction structure itself without operator intervention to perform the corrective subtasks. In the above combination, the manipulator is best understood as the cobot.).
Regarding claim 12, Sedlmayr as modified by Heckmann teaches the control device of claim 11,
with Sedlmayr further teaching wherein the one or more corrective subtasks includes one or more inverted subtasks configured to reverse one or more corresponding subtasks of the human-robot collaborative task (“The reaction structure preferably comprises reaction operations which can reverse individual operations if the relevant operation structure describes a reversible process” [0030]. Thus, there are reaction structures which reverse operations, i.e., inverse subtask which reverses a corresponding operation subtask.).
Regarding claim 13, Sedlmayr as modified by Heckmann teaches the control device of claim 10,
with Sedlmayr further teaching wherein determining the error correction plan is further based on a task description (Figs. 3 and 4 are examples of such task descriptions in which the reaction operation, i.e., error correction plan, “Rx” (x being an integer) are determined based on task operations “Ox” (x being an integer).) and
determining the facilitation plan is further based on the task description (Figs. 3 and 4 are examples of such task descriptions which decide which reaction operations should be performed such that the task resumes at the appropriate rerun point “APx” (x being an integer). As acknowledged in the rejection of claim 1, the facilitation plan is the determination of the rerun point.).
Regarding claim 14, Sedlmayr as modified by Heckmann teaches the control device of claim 13,
with Sedlmayr further teaching wherein the task description comprises one or more subtask actions defining one or more respective subtasks of the human-robot collaborative task (Each operation “Ox” (x being an integer) and reaction operation “Rx” (x being an integer) corresponds to a subtask action which defines the one or more subtask of the task description. In the case of the modification with Heckman, these subtasks would correspond to the sub-tasks of the human-robot collaborative task.).
Regarding claim 15, Sedlmayr as modified by Heckmann teaches the control device of claim 14,
with Sedlmayr further teaching wherein the one or more subtask actions comprise:
an inverse subtask configured to reverse a corresponding subtask of the human-robot collaborative task (“The reaction structure preferably comprises reaction operations which can reverse individual operations if the relevant operation structure describes a reversible process” [0030]. Thus, there are reaction structures which reverse operations, i.e., inverse subtask which reverses a corresponding operation subtask.);
an executable identifier indicative of whether the human operator and/or the cobot is configured to perform the corresponding subtask (“Preferably, the reaction structure includes at least one reaction operation whose execution interrupts the interrupted operation structure until an operator input has been effected, preferably by means of the graphical user interface. This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like. Once the operator confirms to the manipulator system that the task has been performed successively, then it is possible to continue with the reaction structure or an operation structure” [0043-0044]. Thus, in the case of an error which is not able to be automatically rectified, there will be a notification sent out which notifies an operator that assistance is necessary in performing an aspect of the subtask. This notification, or lack thereof, serves as the “executable identifier” which indicates whether the human operator or the robot performs the subtask.); and
with Heckmann further teaching …tool and/or material information defining the one or more tools and/or materials usable to perform the corresponding subtask (“To identify the tools and parts in each repair step it is in many cases sufficient to parse the list of tools and parts and then go from repair step to repair step and identify these tools and parts in each step. For parsing, the assistance system 1 may apply a simple pattern matching, i.e., keywords indicating the list of tools and parts and the segmentation into steps can be searched and then in each step a pattern matching between the tools and parts in the list and the words in each step can be performed” [0139]. Thus, there is a tool list to be parsed by the cobot which relates each tool or part to a corresponding step in the task or subtask.).
Regarding claim 16, Sedlmayr as modified by Heckmann teaches the control device of claim 10,
with Sedlmayr further teaching wherein the control signal controls the cobot to:
correct the detected error in response to the detected error being correctable by the cobot (“Preferably, the execution of an operation structure is interrupted upon the occurrence of an error and the execution of a reaction structure is continued afterward. Consequently, an operation structure of the manipulator program can be repeatedly directly after the occurrence of an error or the manipulator program can be continued for a different rerun point. This makes it possible to minimize stoppage times of the manipulator system and to continue preferably in an automated manner, i.e. without intervention by an operator” [0036]. Thus, to minimize stoppage times of the system, the manipulator executes the reaction structure to perform the corrective subtasks in response to such subtasks being correctable by the manipulator.); and
assist the human operator in their performance of one or more corrective actions to correct the detected error in response to the cobot being unable to correct the detected error (“This makes it possible, if the error that occurred cannot automatically be rectified by the manipulator system, to request the intervention of an operator. By way of example, the operator can be asked to check a specific system state, to remove or exchange faulty parts or workpieces from the manipulator system, and/or the like” [0044]. Thus, in the event that the error is not correctable by the manipulator, an operator is requested for intervention. The assistance subtask for assisting the human operator in their performance of task actions which has been modified by Heckmann would thus correspond to such a support signal as evidenced by Heckmann.).
Regarding claim 21, Sedlmayr as modified by Heckmann teaches the control device of claim 1,
with Sedlmayr further teaching wherein the determination, by the facilitator, of the facilitation plan further comprises:
based on the independent correctability of the detected error, generating another assistance subtask configured to control the cobot to correct the detected error (“Preferably, the execution of an operation structure is interrupted upon the occurrence of an error and the execution of a reaction structure is continued afterward. Consequently, an operation structure of the manipulator program can be repeatedly directly after the occurrence of an error or the manipulator program can be continued for a different rerun point. This makes it possible to minimize stoppage times of the manipulator system and to continue preferably in an automated manner, i.e. without intervention by an operator” [0036]. Thus, to minimize stoppage times of the system, the manipulator executes the reaction structure to perform the corrective subtasks in response to such subtasks being correctable by the manipulator. These corrective tasks which are performed independently by the manipulator may be considered as self-assistive subtasks, based on the manipulator’s independent correctability, wherein the manipulator attempts to correct the detected error without human intervention.).
Regarding claim 22, Sedlmayr as modified by Heckmann teaches the control device of claim 21.
The combination of Sedlmayr and Heckmann implicitly teaches …wherein:
the determining the one or more tools and/or materials usable to correct the detected error is in response to the determined independent correctability indicating that the cobot is unable to independently correct the detected error (Regarding Sedlmayr, in response to the determined independent correctability indicating that the cobot is unable to independently correct the detected error, the manipulator calls for operator assistance in correcting the error (see Sedlmayr, [0044]). This request for the operator is the equivalent to the support signal as described by Heckmann. In Heckmann, the response to the support signal is a determination of which tools and/or materials the operator may need to perform the designated task (see Heckman, [0067] and [0027-0029]). When such teachings are combined with Sedlmayr, this task would be the corrective task as notified by the original request.); and
the generating the other assistance subtask configured to control the cobot to correct the detected error is in response to the determined independent correctability indicating that the cobot is able to independently correct the detected error (Regarding Sedlmayr, the manipulator performs any such corrective subtask according to the reaction structure when it is determined that autonomous correction is possible, thus performing the additional assistance subtask in which the manipulator independently corrects the detected error. See rejection of claim 22 as identified above.).
Further evidence for the combination can be determined by the teachings of Heckmann as they pertain to [0011] and [0014]. These teachings, as identified in the rejection of claim 1, imply that Heckmann would implicitly account for encountered errors in an evaluation of the progress of the task and task success. Additionally, the assistance subtasks of Heckmann is a means to facilitate such task completion and success based on the progress of the task. See rejection of claim 1 for further clarifications, as the associated rationale for such implicit teachings remains the same for this rejection of claim 22.
Regarding claim 23, Sedlmayr as modified by Heckmann teaches the control device of claim 1.
The combination of Sedlmayr and Heckmann implicitly teaches wherein the determining of one or more tools and/or materials usable to correct the detected error is in response to the determined independent correctability indicating that the cobot is unable to independently correct the detected error (Regarding Sedlmayr, in response to the determined independent correctability indicating that the cobot is unable to independently correct the detected error, the manipulator calls for operator assistance in correcting the error (see Sedlmayr, [0044]). This request for the operator is the equivalent to the support signal as described by Heckmann. In Heckmann, the response to the support signal is a determination of which tools and/or materials the operator may need to perform the designated task (see Heckman, [0067] and [0027-0029]). When such teachings are combined with Sedlmayr, this task would be the corrective task as notified by the original request.).
Further evidence for the combination can be determined by the teachings of Heckmann as they pertain to [0011] and [0014]. These teachings, as identified in the rejection of claim 1, imply that Heckmann would implicitly account for encountered errors in an evaluation of the progress of the task and task success. Additionally, the assistance subtasks of Heckmann is a means to facilitate such task completion and success based on the progress of the task. See rejection of claim 1 for further clarifications, as the associated rationale for such implicit teachings remains the same for this rejection of claim 23.
Claims 17-19 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Heckmann in view of Karl et al. (US 2025/0291333 A1; hereinafter “Karl”; priority to DE 10 2022 131 925 A1 attached in file).
Regarding claim 17, Heckmann teaches a collaborative robot (cobot) (“The invention provides an assistance system that is able to cooperate with the user in a large set of activities” [0019]. “The assistance system 1 may form part of a machine capable of carrying out a complex series of actions in an automated manner (robot)” [0054]. Thus, there is an assistance system formed as part of a robot which cooperates with the user thus functioning as a collaborative robot (cobot).), comprising:
a movable manipulator arm configured to perform one or more subtasks of a human-robot collaborative task within a human-robot collaborative workspace (“The robot may comprise at least one robotic arm with an effector (manipulator) adapted to perform physical actions, for example, gripping, grasping, reaching, or pointing” [0054-0055]. Thus, the robot defined above as the cobot comprising the assistance system comprises a robotic arm, i.e., manipulator, configured to perform the collaborative one or more subtasks of the human-robot collaborative task within the human-robot collaborative workspace. See [0018] for more information regarding the collaborative workspace.); and
a controller (Assistance system 1 shown in Fig. 4 comprises a processor which communicates support signals 33 to the manipulator and thus acts as a controller.) configured to:
…determining one or more tools and/or materials usable, by a human operator, to advance a task (“The current task status provided by the task status determination unit 12 may in particular enable the assistance system 1 to determine a currently requested object in the unit for determining a requested object 13. The requested object may be, for example, a spare part or a required tool” [0128]. “The current task status determined in the task status determination unit 12 may further serve to predict the next needed object in the unit for predicting a required object 14. The unit for predicting a required object 14 enables the assistance system to prepare a next subtask, step, or next sub-step while the user and the robot are performing the current subtask, step or sub-step” [0067]. Thus, the assistance system determines one or more tools or spare parts which are requested by the user or predicted to be needed by the user to perform the next step or sub-step as determined by the task status.); and
generating an assistance subtask configured to control the manipulator arm to physically pick up and deliver the one or more tools and/or materials to the human operator within the workspace to assist the human operator in advancing the task using the delivered one or more tools and/or materials (“The processor of the system, according to an embodiment of the invention, is configured to generate the support signal including information on manipulating the object or part of the object comprising handing the at least one object to the user or fetching the at least one object from the user. Such an object may in particular be a tool or a spare part, and it is evident that a plurality of objects may be handed or fetched” [0027]. Thus, the system the system controls the manipulator, i.e., cobot, to pick up and deliver the one or more tools or spare parts to the user within the workspace in order to assist the human operator as evidenced by the support signal. As evidenced in the rejection above, and additionally based on description of [0028-0029], the system determines the tools and objects/parts to be handed based on the intended task/sub-task and corresponding progression, thus handing the user a tool to be used in executing said task.); and
generate a control signal to control the manipulator arm based on the needed assistance (“The assistance system may form part of a robot that supports the user physically, for example, by handing objects like tools or spare parts to the user based on information contained in the support signal” [0015]. Thus, the assistance system causes the robot to hand tools or spare parts, i.e., generally control the robot, to the user based on the support signal, i.e., control signal, which contains the information of what the user may need based on the given description.).
However, Heckmann does not explicitly describe …detect an error in a performance of the human-robot collaborative task;
determine an error correction plan based on the detected error, the error correction plan including one or more corrective subtasks configured to control the cobot to correct the detected error;
determine a facilitation plan based on the determined error correction plan, the determination of the facilitation plan including:
determining an independent correctability, by the cobot, of the detected error…
Karl, pertinent to the problem at hand, teaches …detect an error in a performance of the human-robot collaborative task (“In a first step, the monitoring system 400 performs anomaly detection 410” [0102]. Thus, the monitoring system, which monitors a workpiece during the manufacturing process via a camera as shown in Fig. 4, detects an anomaly in the performance of the task.);
determine an error correction plan based on the detected error, the error correction plan including one or more corrective subtasks configured to control the cobot to correct the detected error (“In a second step, the monitoring system 400 carries out measures to eliminate the deviation. For this purpose, a necessity check 420 is first carried out to determine whether an action is necessary or not. If no action is necessary, a worker 441 is informed via a screen 442. If a measure is necessary, the type of measure is determined in a next step 421. The type of action is usually a corrective action on the workpiece 440. Once the type of action has been determined, the worker 441 is informed via the screen 442. The worker 441 then performs the corrective action on the workpiece 440. Alternatively or additionally, the monitoring system 400 may carry out further corrective actions on the workpiece 440 via a robot system 443. The further corrective actions are, for example, actions that the worker 441 cannot perform manually, for example. It may also be a purely robotic correction, so that the correction is carried out automatically only and without a worker” [0103-0104]. Thus, based on the detected deviation/anomaly, i.e., error, a corrective action is decided upon, including one or more corrective subtasks which are configured to control a cobot to correct the error automatically.);
determine a facilitation plan based on the determined error correction plan (“The monitoring system 400 includes databases 490, 491, 492, and 493 which are used in the monitoring process. The following information, or any combination thereof, may be stored in the databases: Patterns for detecting anomalies and/or irregularities (491), evaluation standards and/or metrics for quantifying a deviation (490), look-up tables for correcting specific deviations (492), tables or models for automated correction of deviations (493), that is, for performing the proposed corrective action. As a rule, the reference data described will also affect the respective databases and their information processing” [0106]. Thus, for any such proposed corrective action, there is a database which stores tables or models for performing the proposed corrective action. Such tables or models are a facilitation plan based on the determined error correction plan.), the determination of the facilitation plan including:
determining an independent correctability, by the cobot, of the detected error (As described above, some corrective actions are performed automatically only by the cobot and thus an independent correctability of the detected error is determined when making the decision of whether the user or cobot will perform the corrective action.)…
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Heckmann to include the error detection and correction as described by Karl with a reasonable expectation of success. One of ordinary skill in the art would have been motivated to make such a modification because by detecting such errors in a flexible and changing manufacturing environment, there is a reduced need for machine-specific programming and thus reduces effort and cost of the manufacturing system (Karl, [0006-0008]).
Regarding claim 18, Heckmann as modified by Karl teaches the cobot of claim 17,
with Karl further teaching wherein the detected error is an error performed by the human operator (“A user can be understood as a person who is involved in the manufacturing and/or assembly process. For example, the user may be a machine operator or a worker who processes the workpiece” [0049]. Thus, provided that the user may be the machine operator or worker processing the workpiece, it would be obvious that any such error detected by the monitoring system would be performed by the human operator who is processing the workpiece.).
Regarding claim 19, Heckmann as modified by Karl teaches the cobot of claim 17,
with Karl further teaching wherein the controller is configured to detect the error based on a comparison of a detected end state of the workspace following performance of the one or more subtasks and an expected end state of the workspace following the performance of the one or more subtasks (“In a fourth step, there is performed detecting 204 a deviation of the actual state from a target state. The target state has reference data for the manufacturing and/or assembly process” [0091]. Thus, the error, i.e., deviation, is detected by comparing an actual state of the workspace following the performance of a subtask and the target state of the workspace, i.e., expected state, following the performance of the task.).
Regarding claim 24, Heckmann as modified by Karl teaches the cobot of claim 17,
with Heckmann further teaching wherein the one or more tools and/or materials are determined based on:
a task description associated with the detected error (“The assistance system acquires this task knowledge by evaluating and interpreting unstructured data. Based on this task knowledge, the assistance system is able to resolve the requests of the user by narrowing down a search using the knowledge on the most likely involved tools and objects at each state of execution of a task. A temporal order in which the tools and objects are used my enable the assistance system to estimate the current state of the task, for example, task progress and/or a degree of success in completing the task” [0014]. Thus, there is knowledge and information regarding the one or more tools/materials needed to complete a task, such that when the task progress is evaluated, the system is able to determine the tools needed at the specific step. In combination with Karl, this would lead to information regarding the tools and materials required at the step in the task at which the error has been detected.),
NOTE: In making the rejection above, Examiner considers and/or language as suggesting the limitations as alternatives according to the broadest reasonable interpretation of and/or language. As such, the struck through limitation has not been considered during this round of prosecution as the references cited taught the task description limitation.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Examiner has included and attached other such documents considered in determining the best combination of references for the current rejections. These references may be additionally considered as pertinent art for future such evaluations of any amended claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY L MOLNAR whose telephone number is (571)272-2276. The examiner can normally be reached 9 A.M. to 4 P.M. EST Monday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jonathan (Wade) Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.L.M./Examiner, Art Unit 3656
/WADE MILES/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights