DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/25/2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/24/2025 was filed after the mailing date of the final Office action on 09/25/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments, see Remarks, filed 06/11/2025, with respect to the rejection(s) of claim(s) 1-3, 5, 8-14, 16-18, and 20-21 under 35 USC 102(a)(2) and 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made of in view of newly found prior art reference(s) and Applicant’s amendments changed the scope of the claims thereby necessitating a new ground(s) of rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 5, 8-10, 20, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Komoda et al. (US 20210078811 A1, hereinafter Komoda) in view of Mishra et al. (US 20160176043 A1, hereinafter Mishra) and Shaw et al. (US 20230182293 A1, hereinafter Shaw).
Regarding claim 1, Komoda discloses:
A robot system (Fig. 1, transfer system 1) comprising:
an exchangeable hand including a workpiece holder to hold a workpiece (at least as in paragraph 0026, wherein “The holder 200 includes a suction device 203 and a suction unit 205”; at least as in paragraph 0034, wherein “The holder 200 is a holding mechanism that holds the object O positioned at the movement source B1”; at least as in paragraph 0025, “The holder is attached to the arm and is configured to hold an object”);
a robot arm to which a hand is attached (at least as in paragraph 0026, wherein “The handling device 10 includes a moving mechanism 100, a holder 200, and a controller 300”; at least as in paragraph 0033, wherein “The moving mechanism 100 is a mechanism that moves the holder 200 to a desired position”);
an imager to image the workpiece (at least as in paragraph 0026, wherein “the transfer system 1 includes the handling device 10, one or more first detectors 11, one or more second detectors 12, and a management device 13”; at least as in paragraph 0035, wherein the first detector 11 is a sensor, such as a camera, configured to acquire information, such as image data, about the object O and output the information to management device 13); and
a controller (at least as in paragraph 0026, wherein “The handling device 10 includes a moving mechanism 100, a holder 200, and a controller 300”); wherein
the controller is configured or programmed to:
generate a plurality of holding posture candidates of at least one of the robot arm or the hand operable to hold the workpiece based on information on the hand with respect to a position of the workpiece imaged by the imager (at least as in paragraph 0117, wherein “(Step S3) The model selector 352 selects a model that is a holding form of the holder 200 on the basis of the information about the object O. The selected model may include a plurality of patterns as described above”; at least as in paragraph 0118, wherein “(Step S4)… the holding position determiner 353 determines the suction position (holding position) of the holder 200… There may be a plurality of candidate holding positions… the holding position determiner 353 associates, for each holding position, candidates of the holding positions with information indicating holding postures of the candidate holding positions to set holding points”; at least as in paragraph 0102, wherein “the plan generator 350 causes the first detector 11 to capture an image including the movement source B1 via the management device 13 and generates a holding plan on the basis of a result of analyzing the captured image and the manipulability map”; at least as in paragraph 0103, wherein “The analyzer 351 acquires a detection result output by the acquirer 320 and analyzes information about a holding target object O on the basis of the acquired detection result”; at least as in paragraph 0105, wherein “The holding position determiner 353 determines suction positions (holding positions) of the holders 200 in the model selected as the candidate of the holding form on the basis of the calculated degree of superimposition… When there are a plurality of candidate holding positions, the holding position determiner 353 determines the holding posture for each candidate holding position”);
determine whether [there is interference between the holder and handling device and obstacles] (at least as in paragraph 0122, “(Step S8) The interference determiner 357 determines whether or not there is interference between the holder 200 and the handling device 10 and obstacles other than the holding target object O on the basis of Step S7” and exclude from candidates if not reachable);
evaluate (at least as in paragraph 0123, “The score calculator 358 calculates a score regarding the degree of manipulability for each of the candidate holding points for the selected pattern using Equation (4)”; at least as in paragraph 0089, “The variables include, for example, for each pattern … a distance from the holders 200 to an obstacle in the holding position and the holding posture … Each coefficient is set such that the score decreases as the distance from the holders 200 to the obstacle becomes shorter”); and
select one holding posture candidate from among the evaluated holding posture candidates based on the evaluation (at least as in paragraph 0127, “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”); and
generate a movement path of at least one of the robot arm or the hand for the selected holding posture candidate (at least as in paragraph 0121, wherein “(Step S7) The path calculator 356 calculates a moving path to the determined holding position and holding posture from the current position and current posture of the holder 200”; at least as in paragraph 0127, wherein “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”; at least as in paragraph 0128, wherein “(Step S13) The holding plan generator 359 generates a holding plan on the basis of the selected model and the information stored in the storage 340”; at least as in paragraph 0108, wherein “The path calculator 356 calculates a moving path to the holding position and the holding posture from the current position and the current posture of the holder 200”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340”),
wherein the information on the hand includes: (i) a type of the hand (at least as in paragraph 0116, wherein “(Step S2) The plan generator 350 causes the first detector 11 to capture an image… the acquirer 320 acquires the detection result detected by the first detector 11 [which] includes information (position, size, and the like) on the object O. The acquirer 320 acquires information (position and posture) on the handling device 10 from the moving mechanism 100 and the holder 200”; at least as in paragraph 0095, wherein “The acquirer 320 acquires an image output by the management device 13 or a detection value detected by a sensor. The acquirer 320 acquires a detection value detected by a sensor included in each of the moving mechanism 100 and the holder 200. The acquirer 320 outputs the acquired image and the detected value to the manipulability map generator 330, the plan generator 350, and the operation controller 360”; at least as in paragraph 0100, wherein “In the model pattern DB 343, model patterns related to the suction units 205 are registered. Information about the moving mechanism 100 and the holder 200 (shapes and models, etc.), information about the movement source B1 (inner sizes, models, etc.), information about the movement destination container B2 (inner sizes, models, etc.) are registered in the model pattern DB 343”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340),
wherein a table of the plurality of holding posture candidates is generated based on an association of: (i) the type of hand, (ii) the type of workpiece holder, and (iii) the position of the workpiece holder (at least as in paragraph 0161, “The manipulability map generator 330 reads image information of a selectable robot type from the visualization model DB 344 of the storage 340, and outputs the read robot type image information to the image generator 370”; at least as in paragraph 0172, “the image may configured to be switched to images of selectable robot types, images of selectable robot hands, image of selectable functions, images of manipulability maps, or the like in accordance with processing procedures”; at least as in paragraph 0160, “A reference numeral g311 is a model of a first type robot, and a reference numeral g312 is a model of a second type robot. A diagram of a region indicated by a reference sign g320 is an example of a model presented at the time of selecting a robot hand (holder 200). A reference sign g321 is a model of a first type robot hand, a reference sign g322 is a model of a second type robot hand, and a reference sign g323 is a model of a third type robot hand. A diagram of a region indicated by a reference sign g330 is an example of a model of the movement source B1 presented at the time of setting environmental information. A reference sigh g331 is a model of a first type movement source B1, and a reference sign g332 is a model of a second type movement source B1”), and
wherein the controller is configured to evaluate the plurality of holding posture candidates to select a type of hand and a type of workpiece holder associated with the hand, (at least as in paragraph 0106, “the manipulability evaluator 354 selects the manipulability map on the basis of types, shapes and sizes of the holder 200, positions and postures of the handling device 10 before moving, or the like. The manipulability evaluator 354 compares the holding point list determined by the holding position determiner 353 with the manipulability map to evaluate the manipulability for each candidate holding point”; at least as in paragraph 0131-0132, “(Step S101) The manipulability evaluator 354 extracts one holding point from the holding point list. The manipulability evaluator 354 repeats the processing of Steps S101 to S116 for all the holding points. (Step S102) The manipulability evaluator 354 selects a function in accordance with a type of the holder 200 attached to the moving mechanism 100”; at least as in paragraph 0148, “the manipulability map used when evaluating the holding position is selected in accordance with the function (for example, in accordance with a type of hand or the holding object O)”; at least as in paragraph 0162, “(Step S302) The manipulability map generator 330 selects a robot hand on the basis of the operation result of the user operating the operator 310”; see also claim 5).
But Komoda does not explicitly disclose “a hand exchange mechanism configured to exchange the exchangeable hand… each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held… only holding posture candidates determined not to cause interference… that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism.”
However, Mishra, in the same field of endeavor of robotic manipulator control systems, specifically teaches “a hand exchange mechanism configured to exchange the exchangeable hand… that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism” (at least as in paragraph 0038, “The robotic system 100 and control module 110 provide a framework and an environment that enable dynamic manipulator selection when a set of manipulators 158 is available for selection and operative coupling to the robot 150 [including] a base or base module 152, a manipulator controller 154, manipulator arm 156 and a set of manipulators 158, a member of which is shown operatively coupled to the manipulator arm 156… the robot 150 has three exemplary manipulators designated as manipulator A, manipulator B and manipulator C”; at least as in paragraph 0041, “the manipulator arm 156 includes a control port 155 coupled to the manipulator controller 154 and an interconnect 300 coupled to a member selected from the set of available manipulators 158 at an opposed end of the manipulator arm 156”; at least as in paragraph 0049, “In block 204, a manipulator is identified either using a predefined task/manipulator matrix (when one exists); parameter results when they exist; or in accordance with an operator override. In block 206, the select manipulator is coupled to the manipulator arm 156 of the robotic system 100 and a counter is reset”; at least as in paragraph 0056, “The generated associations between identified tasks and respective manipulators are stored in the task selection matrix 600, which may be implemented in one or more storage elements integrated with the selector engine 400 or in one or more storage elements external to the selector engine 400 but accessible to the logic 420… The robotic system 100 responds by automatically uncoupling a presently coupled manipulator and coupling the identified or select manipulator or providing an instruction for an operator to perform any necessary uncoupling of a manipulator before coupling the select manipulator”).
Furthermore, Shaw, in the same field of endeavor of robot control systems for determining grasp strategies to grasp an object with a gripper of a robotic device, specifically teaches:
each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held (at least as in paragraph 0080-0081, wherein “a grasp strategy for grasping the object on the selected grasp face is determined” which includes the generation of a set of grasp candidates; at least as in Fig. 9 and paragraph 0090, wherein the process for generating grasp candidates includes a collision check “to ensure that the gripper can be placed at the placement selected”; at least as in paragraph 0084, “a collision check between the gripper and the objects surrounding the target object may be performed to ensure that the gripper can be placed at the position specified by the determined grasp strategy”)…
only holding posture candidates determined not to cause interference (at least as in paragraph 0090, “If the gripper cannot be placed on the target object according to the selected placement, the grasp candidate is rejected and process 900 proceeds to act 910 to select a new gripper placement relative to the target object. Any suitable number of collision-free gripper placements may be used to generate grasp candidates, and embodiments are not limited in this respect”; at least as in paragraph 0092, “It should be appreciated that process 900 may be repeated any number of times to generate the set of grasp candidates to ensure backup grasping candidates are available if needed”; at least as in paragraph 0082, “each of the grasp candidates in the set is evaluated using a physics-based model describing physical interactions between the gripper and the object to determine an estimated grasp quality for the grasp candidate… the physics-based model is used to assign to each of the grasp candidates in the set, a grasp quality score… one of the grasp candidates is selected based on the determined grasp qualities for the set of grasp candidates”)…
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda, to include Mishra's teaching of a robotic system configured to select and couple a manipulator end from a set of manipulators and Shaw’s teaching of performing a collision check on the grasp candidates and scoring the grasp quality of the collision-free grasp candidates, since Mishra teaches wherein the system improves task completion efficiency and reduces operational costs by optimizing task completion through dynamic selection of a particular manipulator based on a defined task thus reducing energy consumption and required hardware and Shaw teaches wherein the system improves the robot’s capability to grasp various objects despite being surrounded by environmental obstacles thus reducing risk of damage to the robot and the target object, limiting the need for grasp-related interventions, and increasing the throughput of the robot.
Regarding claim 2, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 1, wherein the controller is configured or programmed to:
determine whether or not the plurality of generated holding posture candidates cause interference with the surrounding object (at least as in paragraph 0122, wherein “(Step S8) The interference determiner 357 determines whether or not there is interference between the holder 200 and the handling device 10 and obstacles other than the holding target object O on the basis of Step S7”); and
evaluate holding posture candidates (at least as in paragraph 0123, “The score calculator 358 calculates a score regarding the degree of manipulability for each of the candidate holding points for the selected pattern using Equation (4)”; at least as in paragraph 0089, “The variables include, for example, for each pattern … a distance from the holders 200 to an obstacle in the holding position and the holding posture … Each coefficient is set such that the score decreases as the distance from the holders 200 to the obstacle becomes shorter”).
But Komoda does not explicitly disclose “determined not to cause interference.”
However, Shaw, in the same field of endeavor of robot control systems for determining grasp strategies to grasp an object with a gripper of a robotic device, specifically teaches:
determined not to cause interference (at least as in paragraph 0080-0081, wherein “a grasp strategy for grasping the object on the selected grasp face is determined” which includes the generation of a set of grasp candidates; at least as in Fig. 9 and paragraph 0090, wherein the process for generating grasp candidates includes a collision check “to ensure that the gripper can be placed at the placement selected”; at least as in paragraph 0084, “a collision check between the gripper and the objects surrounding the target object may be performed to ensure that the gripper can be placed at the position specified by the determined grasp strategy”; at least as in paragraph 0090, “If the gripper cannot be placed on the target object according to the selected placement, the grasp candidate is rejected and process 900 proceeds to act 910 to select a new gripper placement relative to the target object. Any suitable number of collision-free gripper placements may be used to generate grasp candidates, and embodiments are not limited in this respect”; at least as in paragraph 0092, “It should be appreciated that process 900 may be repeated any number of times to generate the set of grasp candidates to ensure backup grasping candidates are available if needed”; at least as in paragraph 0082, “each of the grasp candidates in the set is evaluated using a physics-based model describing physical interactions between the gripper and the object to determine an estimated grasp quality for the grasp candidate… the physics-based model is used to assign to each of the grasp candidates in the set, a grasp quality score… one of the grasp candidates is selected based on the determined grasp qualities for the set of grasp candidates”).
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda, to include Shaw’s teaching of performing a collision check on the grasp candidates and scoring the grasp quality of the collision-free grasp candidates, since Shaw teaches wherein the system improves the robot’s capability to grasp various objects despite being surrounded by environmental obstacles thus reducing risk of damage to the robot and the target object, limiting the need for grasp-related interventions, and increasing the throughput of the robot.
Regarding claim 3, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 1, wherein the index includes a movement time of at least one of the robot arm or the hand until the workpiece is held in addition to the margin for interference of at least one of the robot arm or the hand with the surrounding object (at least as in paragraph at least as in paragraph 0089, wherein “The variables include, for example, for each pattern… an amount of movement (a moving length, a moving angle, an energy consumption) from a current position and a current posture to a holding position and a holding posture, a time required therefor from the current position and the current posture to the holding position and the holding posture, a distance from the holders 200 to an obstacle in the holding position and the holding posture”).
Regarding claim 5, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 3, wherein the controller is configured or programmed to evaluate the plurality of generated holding posture candidates while weighting the index (at least as in paragraph 0088, wherein “The controller 300 calculates, for each pattern (candidate), a score as an index for selecting the pattern (index for determining) The controller 300 sets a variable (a numerical value or a physical quantity) of each term of the score to x, (i=1, 2, . . . , n) and a coefficient (a weighting coefficient) of each term to a (i=1, 2, . . . , n), and calculates the score x using the following equation (4)”; at least as in paragraph 0089, wherein “Equation (4) may include at least one term, and the types and number of variables, a magnitude of the coefficient for each index, and the like can be appropriately adjusted in accordance with a usage environment, and the like”; at least as in paragraph 0146, wherein “the manipulability evaluator 354 calculates the manipulability score f by weighting each of the functions f1 to f4”).
Regarding claim 8, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 1, wherein
the workpiece holder is movable in the hand (at least as in paragraph 0039, wherein “The holder 200 includes, for example, a base 201, the rotator 202, the suction device 203, a switching valve 204, and the suction unit 205 (for example, suction pads)”; at least as in paragraph 0041, wherein “The rotator 202 is provided between the base 201 and the moving mechanism 100 and rotatably connects the base 201 to the moving mechanism 100”); and
the position of the workpiece holder includes a plurality of positions within a movable range of the workpiece holder in the hand (at least as in paragraph 0041, wherein “A rotational center axis C of the rotator 202 substantially coincides with a direction in which a tip portion of the moving mechanism 100 and the base 201 are arranged. The rotator 202 can rotate the base 201 of the holder 200 with respect to the moving mechanism 100 in a direction in the figure and a direction opposite thereto”; at least as in Fig. 3 & 4 and paragraphs 0050-0051, wherein a first and second holder types 200a and 200b enlarges the accessible range in various directions).
Regarding claim 9, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 1, wherein the hand includes at least one of a deformable configuration including a movable axis or a configuration including a plurality of the workpiece holders (at least as in paragraph 0041, wherein “A rotational center axis C of the rotator 202 substantially coincides with a direction in which a tip portion of the moving mechanism 100 and the base 201 are arranged. The rotator 202 can rotate the base 201 of the holder 200 with respect to the moving mechanism 100 in a direction in the figure and a direction opposite thereto”; at least as in Fig. 3 & 4 and paragraphs 0050-0051, wherein a first and second holder types 200a and 200b enlarges the accessible range in various directions; at least as in paragraph 0034, wherein “the holder 200 may be a holder that holds the object O by pinching the object O with a plurality of pinching members”; at least as in paragraph 0039, wherein “The holder 200 may include a plurality of switching valves 204 and a plurality of suction units 205”; at least as in paragraph 0042, wherein “The suction device 203 is provided inside the base 201 [and] communicates with each of the plurality of suction units 205”).
Regarding claim 10, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 1, wherein
the workpiece includes a plurality of workpieces (at least as in Fig. 1 and paragraph 0028, wherein “The transfer system 1 moves objects (holding targets or transfer targets) 0 positioned at a movement source B1 to a movement destination container B2”);
the workpiece holder includes a plurality of workpiece holders provided in the hand to hold the plurality of workpieces (at least as in paragraph 0039, wherein “The holder 200 may include a plurality of switching valves 204 and a plurality of suction units 205”; at least as in paragraph 0042, wherein “The suction device 203 is provided inside the base 201 [and] communicates with each of the plurality of suction units 205”); and
the controller is configured or programmed to generate the plurality of holding posture candidates (at least as in paragraph 0117, wherein “(Step S3) The model selector 352 selects a model that is a holding form of the holder 200 on the basis of the information about the object O. The selected model may include a plurality of patterns as described above”; at least as in paragraph 0118, wherein “(Step S4)… the holding position determiner 353 determines the suction position (holding position) of the holder 200… There may be a plurality of candidate holding positions… the holding position determiner 353 associates, for each holding position, candidates of the holding positions with information indicating holding postures of the candidate holding positions to set holding points”).
But Komoda does not explicitly disclose “with respect to the plurality of workpieces.”
However, Shaw, in the same field of endeavor of robot control systems for determining grasp strategies to grasp an object with a gripper of a robotic device, specifically teaches “with respect to the plurality of workpieces” (at least as in paragraph 0075, “a set of objects capable of being grasped by the robotic device (which may include all or a subset of objects in the environment near the robot) may be determined as candidates for grasping”; at least as in paragraph 0076, wherein “multiple candidate target objects may be selected in act 420 of process 400 and grasp strategies for each of the multiple candidate target objects may be evaluated using the techniques”).
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda to include Shaw’s teaching of determining candidate target objects, since Shaw teaches wherein the system improves the robot’s capability to grasp various objects despite being surrounded by environmental obstacles thus reducing risk of damage to the robot and the target object, limiting the need for grasp-related interventions, and increasing the throughput of the robot.
Regarding claim 20, Komoda discloses:
A movement path generation device for a robot system, the robot system including an exchangeable hand including a workpiece holder to hold a workpiece, a robot arm to which the hand is attached, (at least as in paragraph 0026, wherein “the transfer system 1 includes the handling device 10, one or more first detectors 11, one or more second detectors 12, and a management device 13. The handling device 10 includes a moving mechanism 100, a holder 200, and a controller 300… The holder 200 includes a suction device 203 and a suction unit 205”; at least as in paragraph 0035, wherein the first detector 11 is a sensor, such as a camera, configured to acquire information, such as image data, about the object O and output the information to management device 13), the movement path generation device comprising:
a controller (at least as in paragraph 0026, wherein “The handling device 10 includes a moving mechanism 100, a holder 200, and a controller 300”);
wherein the controller is configured or programmed to:
generate a plurality of holding posture candidates of at least one of the robot arm or the hand operable to hold the workpiece based on information on the hand with respect to a position of the workpiece imaged by the imager (at least as in paragraph 0117, wherein “(Step S3) The model selector 352 selects a model that is a holding form of the holder 200 on the basis of the information about the object O. The selected model may include a plurality of patterns as described above”; at least as in paragraph 0118, wherein “(Step S4)… the holding position determiner 353 determines the suction position (holding position) of the holder 200… There may be a plurality of candidate holding positions… the holding position determiner 353 associates, for each holding position, candidates of the holding positions with information indicating holding postures of the candidate holding positions to set holding points”; at least as in paragraph 0102, wherein “the plan generator 350 causes the first detector 11 to capture an image including the movement source B1 via the management device 13 and generates a holding plan on the basis of a result of analyzing the captured image and the manipulability map”; at least as in paragraph 0103, wherein “The analyzer 351 acquires a detection result output by the acquirer 320 and analyzes information about a holding target object O on the basis of the acquired detection result”; at least as in paragraph 0105, wherein “The holding position determiner 353 determines suction positions (holding positions) of the holders 200 in the model selected as the candidate of the holding form on the basis of the calculated degree of superimposition… When there are a plurality of candidate holding positions, the holding position determiner 353 determines the holding posture for each candidate holding position”);
determine whether [there is interference between the holder and handling device and obstacles] (at least as in paragraph 0122, “(Step S8) The interference determiner 357 determines whether or not there is interference between the holder 200 and the handling device 10 and obstacles other than the holding target object O on the basis of Step S7” and exclude from candidates if not reachable);
evaluate (at least as in paragraph 0123, “The score calculator 358 calculates a score regarding the degree of manipulability for each of the candidate holding points for the selected pattern using Equation (4)”; at least as in paragraph 0089, “The variables include, for example, for each pattern … a distance from the holders 200 to an obstacle in the holding position and the holding posture … Each coefficient is set such that the score decreases as the distance from the holders 200 to the obstacle becomes shorter”);
select one holding posture candidate from among the evaluated holding posture candidates based on the evaluation (at least as in paragraph 0127, “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”); and
generate a movement path of at least one of the robot arm or the hand for the selected holding posture candidate (at least as in paragraph 0121, wherein “(Step S7) The path calculator 356 calculates a moving path to the determined holding position and holding posture from the current position and current posture of the holder 200”; at least as in paragraph 0127, wherein “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”; at least as in paragraph 0128, wherein “(Step S13) The holding plan generator 359 generates a holding plan on the basis of the selected model and the information stored in the storage 340”; at least as in paragraph 0108, wherein “The path calculator 356 calculates a moving path to the holding position and the holding posture from the current position and the current posture of the holder 200”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340”),
wherein the information on the hand includes: (i) a type of the hand(at least as in paragraph 0116, wherein “(Step S2) The plan generator 350 causes the first detector 11 to capture an image… the acquirer 320 acquires the detection result detected by the first detector 11 [which] includes information (position, size, and the like) on the object O. The acquirer 320 acquires information (position and posture) on the handling device 10 from the moving mechanism 100 and the holder 200”; at least as in paragraph 0095, wherein “The acquirer 320 acquires an image output by the management device 13 or a detection value detected by a sensor. The acquirer 320 acquires a detection value detected by a sensor included in each of the moving mechanism 100 and the holder 200. The acquirer 320 outputs the acquired image and the detected value to the manipulability map generator 330, the plan generator 350, and the operation controller 360”; at least as in paragraph 0100, wherein “In the model pattern DB 343, model patterns related to the suction units 205 are registered. Information about the moving mechanism 100 and the holder 200 (shapes and models, etc.), information about the movement source B1 (inner sizes, models, etc.), information about the movement destination container B2 (inner sizes, models, etc.) are registered in the model pattern DB 343”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340),
wherein a table of the plurality of holding posture candidates is generated based on an association of: (i) the type of hand, (ii) the type of workpiece holder, and (iii) the position of the workpiece holder (at least as in paragraph 0105, “The holding position determiner 353 calculates the degree of superposition of the target region Ta and the pad region Pa of the model. The holding position determiner 353 determines suction positions (holding positions) of the holders 200 in the model selected as the candidate of the holding form on the basis of the calculated degree of superimposition. The holding position determiner 353 determines, for example, positions in which the degree of superimposition is equal to or more than a predetermined value (threshold) as the holding position. For this reason, there may be a plurality of candidate holding positions. The holding position determiner 353 generates a plurality of holding positions (holding points) as a holding point list and causes the storage 340 to store the generated holding point list. The holding position determiner 353 changes the rotation angle and performs a calculation of the degree of superposition for each angular position for the data including the pad region Pa, thereby determining the rotation angle (posture) having the highest degree of superposition as the holding posture. When there are a plurality of candidate holding positions, the holding position determiner 353 determines the holding posture for each candidate holding position”), and
wherein the controller is configured to evaluate the plurality of holding posture candidates to select a type of hand and a type of workpiece holder associated with the hand, with the hand to hold the workpiece (at least as in paragraph 0106, “the manipulability evaluator 354 selects the manipulability map on the basis of types, shapes and sizes of the holder 200, positions and postures of the handling device 10 before moving, or the like. The manipulability evaluator 354 compares the holding point list determined by the holding position determiner 353 with the manipulability map to evaluate the manipulability for each candidate holding point”; at least as in paragraph 0131-0132, “(Step S101) The manipulability evaluator 354 extracts one holding point from the holding point list. The manipulability evaluator 354 repeats the processing of Steps S101 to S116 for all the holding points. (Step S102) The manipulability evaluator 354 selects a function in accordance with a type of the holder 200 attached to the moving mechanism 100”; at least as in paragraph 0148, “the manipulability map used when evaluating the holding position is selected in accordance with the function (for example, in accordance with a type of hand or the holding object O)”; at least as in paragraph 0162, “(Step S302) The manipulability map generator 330 selects a robot hand on the basis of the operation result of the user operating the operator 310”; see also claim 5).
But Komoda does not explicitly disclose “a hand exchange mechanism… each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held… only holding posture candidates determined not to cause interference… that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism.”
However, Mishra, in the same field of endeavor of robotic manipulator control systems, specifically teaches “a hand exchange mechanism… that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism” (at least as in paragraph 0038, “The robotic system 100 and control module 110 provide a framework and an environment that enable dynamic manipulator selection when a set of manipulators 158 is available for selection and operative coupling to the robot 150 [including] a base or base module 152, a manipulator controller 154, manipulator arm 156 and a set of manipulators 158, a member of which is shown operatively coupled to the manipulator arm 156… the robot 150 has three exemplary manipulators designated as manipulator A, manipulator B and manipulator C”; at least as in paragraph 0041, “the manipulator arm 156 includes a control port 155 coupled to the manipulator controller 154 and an interconnect 300 coupled to a member selected from the set of available manipulators 158 at an opposed end of the manipulator arm 156”; at least as in paragraph 0049, “In block 204, a manipulator is identified either using a predefined task/manipulator matrix (when one exists); parameter results when they exist; or in accordance with an operator override. In block 206, the select manipulator is coupled to the manipulator arm 156 of the robotic system 100 and a counter is reset”; at least as in paragraph 0056, “The generated associations between identified tasks and respective manipulators are stored in the task selection matrix 600, which may be implemented in one or more storage elements integrated with the selector engine 400 or in one or more storage elements external to the selector engine 400 but accessible to the logic 420… The robotic system 100 responds by automatically uncoupling a presently coupled manipulator and coupling the identified or select manipulator or providing an instruction for an operator to perform any necessary uncoupling of a manipulator before coupling the select manipulator”).
Furthermore, Shaw, in the same field of endeavor of robot control systems for determining grasp strategies to grasp an object with a gripper of a robotic device, specifically teaches:
each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held (at least as in paragraph 0080-0081, wherein “a grasp strategy for grasping the object on the selected grasp face is determined” which includes the generation of a set of grasp candidates; at least as in Fig. 9 and paragraph 0090, wherein the process for generating grasp candidates includes a collision check “to ensure that the gripper can be placed at the placement selected”; at least as in paragraph 0084, “a collision check between the gripper and the objects surrounding the target object may be performed to ensure that the gripper can be placed at the position specified by the determined grasp strategy”)…
only holding posture candidates determined not to cause interference (at least as in paragraph 0090, “If the gripper cannot be placed on the target object according to the selected placement, the grasp candidate is rejected and process 900 proceeds to act 910 to select a new gripper placement relative to the target object. Any suitable number of collision-free gripper placements may be used to generate grasp candidates, and embodiments are not limited in this respect”; at least as in paragraph 0092, “It should be appreciated that process 900 may be repeated any number of times to generate the set of grasp candidates to ensure backup grasping candidates are available if needed”; at least as in paragraph 0082, “each of the grasp candidates in the set is evaluated using a physics-based model describing physical interactions between the gripper and the object to determine an estimated grasp quality for the grasp candidate… the physics-based model is used to assign to each of the grasp candidates in the set, a grasp quality score… one of the grasp candidates is selected based on the determined grasp qualities for the set of grasp candidates”)…
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda, to include Mishra's teaching of a robotic system configured to select and couple a manipulator end from a set of manipulators and Shaw’s teaching of performing a collision check on the grasp candidates and scoring the grasp quality of the collision-free grasp candidates, since Mishra teaches wherein the system improves task completion efficiency and reduces operational costs by optimizing task completion through dynamic selection of a particular manipulator based on a defined task thus reducing energy consumption and required hardware and Shaw teaches wherein the system improves the robot’s capability to grasp various objects despite being surrounded by environmental obstacles thus reducing risk of damage to the robot and the target object, limiting the need for grasp-related interventions, and increasing the throughput of the robot.
Regarding claim 21, Komoda discloses:
A movement path generation method comprising:
imaging a workpiece (at least as in paragraph 0116, wherein “(Step S2) The plan generator 350 causes the first detector 11 to capture an image including the movement source B1 via the management device 13, for example. Subsequently, the acquirer 320 acquires the detection result detected by the first detector 11 and the detection result detected by the second detector 12 from the management device 13. The detection result detected by the first detector 11 includes information (position, size, and the like) on the object O… the analyzer 351 performs image processing to acquire information on the object O. The analyzer 351 analyzes a curvature shape of the object O on the basis of the information output by the acquirer 320”);
generating, based on information on a hand including a workpiece holder to hold the workpiece, a plurality of holding posture candidates of at least one of a robot arm or the hand operable to hold the workpiece with respect to a position of the imaged workpiece (at least as in paragraph 0117, wherein “(Step S3) The model selector 352 selects a model that is a holding form of the holder 200 on the basis of the information about the object O. The selected model may include a plurality of patterns as described above”; at least as in paragraph 0118, wherein “(Step S4)… the holding position determiner 353 determines the suction position (holding position) of the holder 200… There may be a plurality of candidate holding positions… the holding position determiner 353 associates, for each holding position, candidates of the holding positions with information indicating holding postures of the candidate holding positions to set holding points”; at least as in paragraph 0026, wherein “The handling device 10 includes a moving mechanism 100, a holder 200, and a controller 300”; at least as in paragraph 0033, wherein “The moving mechanism 100 is a mechanism that moves the holder 200 to a desired position”; at least as in paragraph 0026, wherein “The holder 200 includes a suction device 203 and a suction unit 205”; at least as in paragraph 0034, wherein “The holder 200 is a holding mechanism that holds the object O positioned at the movement source B1”);
determining whether [there is interference between the holder and handling device and obstacles] (at least as in paragraph 0122, “(Step S8) The interference determiner 357 determines whether or not there is interference between the holder 200 and the handling device 10 and obstacles other than the holding target object O on the basis of Step S7” and exclude from candidates if not reachable);
evaluating (at least as in paragraph 0123, “The score calculator 358 calculates a score regarding the degree of manipulability for each of the candidate holding points for the selected pattern using Equation (4)”; at least as in paragraph 0089, “The variables include, for example, for each pattern … a distance from the holders 200 to an obstacle in the holding position and the holding posture … Each coefficient is set such that the score decreases as the distance from the holders 200 to the obstacle becomes shorter”); and
selecting one holding posture candidate from among the evaluated holding posture candidates based on the evaluation (at least as in paragraph 0127, “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”); and
generating a movement path of at least one of the robot arm or the hand for the selected holding posture candidate (at least as in paragraph 0121, wherein “(Step S7) The path calculator 356 calculates a moving path to the determined holding position and holding posture from the current position and current posture of the holder 200”; at least as in paragraph 0127, wherein “(Step S12) The pattern selector 355 selects a usage form (candidate) of the pattern having the highest score as a pattern used for control”; at least as in paragraph 0128, wherein “(Step S13) The holding plan generator 359 generates a holding plan on the basis of the selected model and the information stored in the storage 340”; at least as in paragraph 0108, wherein “The path calculator 356 calculates a moving path to the holding position and the holding posture from the current position and the current posture of the holder 200”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340”),
wherein the information on the hand includes: (i) a type of the hand(at least as in paragraph 0116, wherein “(Step S2) The plan generator 350 causes the first detector 11 to capture an image… the acquirer 320 acquires the detection result detected by the first detector 11 [which] includes information (position, size, and the like) on the object O. The acquirer 320 acquires information (position and posture) on the handling device 10 from the moving mechanism 100 and the holder 200”; at least as in paragraph 0095, wherein “The acquirer 320 acquires an image output by the management device 13 or a detection value detected by a sensor. The acquirer 320 acquires a detection value detected by a sensor included in each of the moving mechanism 100 and the holder 200. The acquirer 320 outputs the acquired image and the detected value to the manipulability map generator 330, the plan generator 350, and the operation controller 360”; at least as in paragraph 0100, wherein “In the model pattern DB 343, model patterns related to the suction units 205 are registered. Information about the moving mechanism 100 and the holder 200 (shapes and models, etc.), information about the movement source B1 (inner sizes, models, etc.), information about the movement destination container B2 (inner sizes, models, etc.) are registered in the model pattern DB 343”; at least as in paragraph 0111, wherein “The holding plan generator 359 generates a holding plan on the basis of the selected model and information stored in the storage 340),
wherein a table of the plurality of holding posture candidates is generated based on an association of: (i) the type of hand, (ii) the type of workpiece holder, and (iii) the position of the workpiece holder (at least as in paragraph 0105, “The holding position determiner 353 calculates the degree of superposition of the target region Ta and the pad region Pa of the model. The holding position determiner 353 determines suction positions (holding positions) of the holders 200 in the model selected as the candidate of the holding form on the basis of the calculated degree of superimposition. The holding position determiner 353 determines, for example, positions in which the degree of superimposition is equal to or more than a predetermined value (threshold) as the holding position. For this reason, there may be a plurality of candidate holding positions. The holding position determiner 353 generates a plurality of holding positions (holding points) as a holding point list and causes the storage 340 to store the generated holding point list. The holding position determiner 353 changes the rotation angle and performs a calculation of the degree of superposition for each angular position for the data including the pad region Pa, thereby determining the rotation angle (posture) having the highest degree of superposition as the holding posture. When there are a plurality of candidate holding positions, the holding position determiner 353 determines the holding posture for each candidate holding position”), and
wherein the plurality of holding posture candidates are evaluated to select a type of hand and a type of workpiece holder associated with the hand, (at least as in paragraph 0106, “the manipulability evaluator 354 selects the manipulability map on the basis of types, shapes and sizes of the holder 200, positions and postures of the handling device 10 before moving, or the like. The manipulability evaluator 354 compares the holding point list determined by the holding position determiner 353 with the manipulability map to evaluate the manipulability for each candidate holding point”; at least as in paragraph 0131-0132, “(Step S101) The manipulability evaluator 354 extracts one holding point from the holding point list. The manipulability evaluator 354 repeats the processing of Steps S101 to S116 for all the holding points. (Step S102) The manipulability evaluator 354 selects a function in accordance with a type of the holder 200 attached to the moving mechanism 100”; at least as in paragraph 0148, “the manipulability map used when evaluating the holding position is selected in accordance with the function (for example, in accordance with a type of hand or the holding object O)”; at least as in paragraph 0162, “(Step S302) The manipulability map generator 330 selects a robot hand on the basis of the operation result of the user operating the operator 310”; see also claim 5).
But Komoda does not explicitly disclose “each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held… only holding posture candidates determined not to cause interference… that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism.”
However, Mishra, in the same field of endeavor of robotic manipulator control systems, specifically teaches “that is exchangeable by a hand exchange mechanism… and cause the robot arm to exchange the hand using the hand exchange mechanism” (at least as in paragraph 0038, “The robotic system 100 and control module 110 provide a framework and an environment that enable dynamic manipulator selection when a set of manipulators 158 is available for selection and operative coupling to the robot 150 [including] a base or base module 152, a manipulator controller 154, manipulator arm 156 and a set of manipulators 158, a member of which is shown operatively coupled to the manipulator arm 156… the robot 150 has three exemplary manipulators designated as manipulator A, manipulator B and manipulator C”; at least as in paragraph 0041, “the manipulator arm 156 includes a control port 155 coupled to the manipulator controller 154 and an interconnect 300 coupled to a member selected from the set of available manipulators 158 at an opposed end of the manipulator arm 156”; at least as in paragraph 0049, “In block 204, a manipulator is identified either using a predefined task/manipulator matrix (when one exists); parameter results when they exist; or in accordance with an operator override. In block 206, the select manipulator is coupled to the manipulator arm 156 of the robotic system 100 and a counter is reset”; at least as in paragraph 0056, “The generated associations between identified tasks and respective manipulators are stored in the task selection matrix 600, which may be implemented in one or more storage elements integrated with the selector engine 400 or in one or more storage elements external to the selector engine 400 but accessible to the logic 420… The robotic system 100 responds by automatically uncoupling a presently coupled manipulator and coupling the identified or select manipulator or providing an instruction for an operator to perform any necessary uncoupling of a manipulator before coupling the select manipulator”).
Furthermore, Shaw, in the same field of endeavor of robot control systems for determining grasp strategies to grasp an object with a gripper of a robotic device, specifically teaches:
each of the plurality of generated holding posture candidates causes interference between at least one of the robot arm or the hand and a surrounding object at a time at which the workpiece is held (at least as in paragraph 0080-0081, wherein “a grasp strategy for grasping the object on the selected grasp face is determined” which includes the generation of a set of grasp candidates; at least as in Fig. 9 and paragraph 0090, wherein the process for generating grasp candidates includes a collision check “to ensure that the gripper can be placed at the placement selected”; at least as in paragraph 0084, “a collision check between the gripper and the objects surrounding the target object may be performed to ensure that the gripper can be placed at the position specified by the determined grasp strategy”)…
only holding posture candidates determined not to cause interference (at least as in paragraph 0090, “If the gripper cannot be placed on the target object according to the selected placement, the grasp candidate is rejected and process 900 proceeds to act 910 to select a new gripper placement relative to the target object. Any suitable number of collision-free gripper placements may be used to generate grasp candidates, and embodiments are not limited in this respect”; at least as in paragraph 0092, “It should be appreciated that process 900 may be repeated any number of times to generate the set of grasp candidates to ensure backup grasping candidates are available if needed”; at least as in paragraph 0082, “each of the grasp candidates in the set is evaluated using a physics-based model describing physical interactions between the gripper and the object to determine an estimated grasp quality for the grasp candidate… the physics-based model is used to assign to each of the grasp candidates in the set, a grasp quality score… one of the grasp candidates is selected based on the determined grasp qualities for the set of grasp candidates”)…
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda, to include Mishra's teaching of a robotic system configured to select and couple a manipulator end from a set of manipulators and Shaw’s teaching of performing a collision check on the grasp candidates and scoring the grasp quality of the collision-free grasp candidates, since Mishra teaches wherein the system improves task completion efficiency and reduces operational costs by optimizing task completion through dynamic selection of a particular manipulator based on a defined task thus reducing energy consumption and required hardware and Shaw teaches wherein the system improves the robot’s capability to grasp various objects despite being surrounded by environmental obstacles thus reducing risk of damage to the robot and the target object, limiting the need for grasp-related interventions, and increasing the throughput of the robot.
Claim(s) 11-14 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Komoda et al. (US 20210078811 A1, hereinafter Komoda) in view of Mishra et al. (US 20160176043 A1, hereinafter Mishra) and Shaw et al. (US 20230182293 A1, hereinafter Shaw), and further in view of Sun et al. (US 20210122586 A1, hereinafter Sun).
Regarding claim 11, in view of the above combination of Komoda, Mishra, and Shaw, Komoda further discloses:
The robot system according to claim 10, wherein the controller is configured or programmed to, when one workpiece holder of the plurality of workpiece holders is selected for one workpiece of the plurality of workpieces, not generate and evaluate the plurality of holding posture candidates for the one workpiece holder, but generate and evaluate the plurality of holding posture candidates for another workpiece holder of the plurality of workpiece holders, (at least as in paragraph 0043, wherein “When the object O is relatively small, the handling device 10 causes only one or a small number of the suction units 205 selected from the plurality of suction units 205 to function as the suction units 205 used for holding”; at least as in paragraph 0044, wherein “When the object O is relatively small and light, the handling device 10 sucks and holds the object O using only one or more effective suction units 205E selected from the plurality of suction units 205”; at least as in paragraph 0058, wherein “as indicated by reference signs g101 and g102, the model 1 and the model 2 are set as holding forms using one suction unit 205”; at least as in paragraph 0125, wherein “(Step S10) The pattern selector 355 excludes the selected pattern”; at least as in paragraph 0126, wherein “(Step S11) The pattern selector 355 determines whether or not there is another pattern candidate”).
But Komoda does not explicitly disclose “with respect to another workpiece of the plurality of workpieces.”
However, Sun discloses a robotic system wherein image data associated with a workspace is used to generate a three dimensional view of a workspace to determine grasp strategies for each item present in the workspace. Sun specifically teaches “with respect to another workpiece of the plurality of workpieces” (at least as in paragraph 0116, wherein “the respective pairs of suction cups have been actuated independently, each to grasp a corresponding one of the two items that are shown to have been grasped”; at least as in paragraph 0071, wherein “Items may be picked one at a time or, in some embodiments, multiple items may be grasped at once”; at least as in paragraph 0085, wherein “determining a grasp strategy includes determining whether to attempt to pick up one item or multiple items, whether to use all end effector actuators (e.g., use just one or two or more suction cups or sets of cups) and/or selecting a combined grasp technique and corresponding speed from a set of computed grasp technique+speed combinations under consideration (e.g., 1 suction cup @ 50% speed vs. 2 suction cups 80% speed)”; at least as in paragraph 0087, wherein “Another optimization implemented in some embodiments is to change grasp strategy based on double (or n-) item picking (also see next section) where a robot may adapt its strategy based on whether it's trying to reuse vision data while picking multiple items in a batch or sequentially. For example, a robot gripper with 4 suction cups can use 2 cups to pick up one object, and the others 2 to pick up a second object”; at least as in paragraph 0108, wherein “the system may take a photo and identify two (or more) objects to pick. The system picks and moves the first one; then, instead of doing a full scene re-compute to find a next package to pick, the system simply looks at whether the second package is disturbed”; at least as in paragraph 0115, wherein “the pairs of suction cups (i.e., a first pair comprising suction cups 804 and 812, and a second pair comprising suction cups 808 and 814) may be operated independently. For example, a single item, such as a smaller and/or lighter item, may be grasped using only one or the other of the suction cup pairs. In some embodiments, each pair may be used to grasp a separate item at the same time, enabling two (or in some embodiments more) items to be picked/placed simultaneously”)
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda to include Sun's teaching of a grasp strategy determining system, since Sun teaches wherein the system increases the likelihood of a successful grasp thus increasing efficiency.
Regarding claim 12, in view of the above combination of Komoda, Mishra, Shaw, and Sun, Komoda further discloses:
The robot system according to claim 11, wherein the controller is configured or programmed to:
generate the plurality of holding posture candidates for another workpiece holder of the plurality of workpiece holders (at least as in paragraph 0044, wherein “When the object O is relatively small and light, the handling device 10 sucks and holds the object O using only one or more effective suction units 205E selected from the plurality of suction units 205”; at least as in paragraph 0058, wherein “as indicated by reference signs g101 and g102, the model 1 and the model 2 are set as holding forms using one suction unit 205”; at least as in paragraph 0125, wherein “(Step S10) The pattern selector 355 excludes the selected pattern”; at least as in paragraph 0126, wherein “(Step S11) The pattern selector 355 determines whether or not there is another pattern candidate”).
But Komoda does not explicitly disclose “after causing the one workpiece holder to hold the workpiece… and cause the another workpiece holder to hold the workpiece.”
Sun discloses a robotic system wherein image data associated with a workspace is used to generate a three dimensional view of a workspace to determine grasp strategies for each item present in the workspace. Sun specifically teaches “after causing the one workpiece holder to hold the workpiece… and cause the another workpiece holder to hold the workpiece” (at least as in paragraph 0071, wherein “Items may be picked one at a time or, in some embodiments, multiple items may be grasped at once”; at least as in paragraph 0085, wherein “determining a grasp strategy includes determining whether to attempt to pick up one item or multiple items, whether to use all end effector actuators (e.g., use just one or two or more suction cups or sets of cups) and/or selecting a combined grasp technique and corresponding speed from a set of computed grasp technique+speed combinations under consideration (e.g., 1 suction cup @ 50% speed vs. 2 suction cups 80% speed)”; at least as in paragraph 0087, wherein “Another optimization implemented in some embodiments is to change grasp strategy based on double (or n-) item picking (also see next section) where a robot may adapt its strategy based on whether it's trying to reuse vision data while picking multiple items in a batch or sequentially. For example, a robot gripper with 4 suction cups can use 2 cups to pick up one object, and the others 2 to pick up a second object”; at least as in paragraph 0108, wherein “the system may take a photo and identify two (or more) objects to pick. The system picks and moves the first one; then, instead of doing a full scene re-compute to find a next package to pick, the system simply looks at whether the second package is disturbed”; at least as in paragraph 0109, wherein “a robot may use vision, depth, or other sensors to identify two graspable packages that are judged by the robot's control algorithm to be far enough apart so as not to be able to interfere with each other's picks (by destabilizing the pile and causing it to flow, or by hitting the other package etc.)”; at least as in paragraph 0115, wherein “the pairs of suction cups (i.e., a first pair comprising suction cups 804 and 812, and a second pair comprising suction cups 808 and 814) may be operated independently. For example, a single item, such as a smaller and/or lighter item, may be grasped using only one or the other of the suction cup pairs. In some embodiments, each pair may be used to grasp a separate item at the same time, enabling two (or in some embodiments more) items to be picked/placed simultaneously”)
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda to include Sun's teaching of a grasp strategy determining system, since Sun teaches wherein the system increases the likelihood of a successful grasp thus increasing efficiency.
Regarding claim 13, in view of the above combination of Komoda, Mishra, Shaw, and Sun, Komoda further discloses:
The robot system according to claim 12, wherein
the plurality of workpiece holders are movable in the hand (at least as in paragraph 0041, wherein “A rotational center axis C of the rotator 202 substantially coincides with a direction in which a tip portion of the moving mechanism 100 and the base 201 are arranged. The rotator 202 can rotate the base 201 of the holder 200 with respect to the moving mechanism 100 in a direction in the figure and a direction opposite thereto”; at least as in Fig. 3 & 4 and paragraphs 0050-0051, wherein a first and second holder types 200a and 200b enlarges the accessible range in various directions; at least as in paragraph 0034, wherein “the holder 200 may be a holder that holds the object O by pinching the object O with a plurality of pinching members”; at least as in paragraph 0039, wherein “The holder 200 may include a plurality of switching valves 204 and a plurality of suction units 205”; at least as in paragraph 0054, wherein “the number of the suction units 205 in the embodiment is five”); and
the controller is configured or programmed to, after causing the one workpiece holder to hold the workpiece, move the one workpiece holder to retract the held workpiece (at least as in Fig. 1 and paragraph 0028, wherein “The transfer system 1 moves objects (holding targets or transfer targets) 0 positioned at a movement source B1 to a movement destination container B2”; at least as in paragraph 0044, wherein “When the object O is relatively small and light, the handling device 10 sucks and holds the object O using only one or more effective suction units 205E selected from the plurality of suction units 205”; at least as in paragraph 0128, wherein “the operation controller 360 controls the moving mechanism 100 and the holder 200 to suck the object O at the movement source B1 on the basis of instructions of the holding plan output from the holding plan generator 359 and moves the sucked object O to the movement destination container B2”).
Regarding claim 14, in view of the above combination of Komoda, Mishra, Shaw, and Sun, Komoda further discloses:
The robot system according to claim 13, wherein the controller is configured or programmed to, after causing the one workpiece holder to hold the workpiece, move the one workpiece holder to retract the held workpiece away from a remaining workpiece of the plurality of workpieces (at least as in Fig. 1 and paragraph 0028, wherein “The transfer system 1 moves objects (holding targets or transfer targets) 0 positioned at a movement source B1 to a movement destination container B2”; at least as in paragraph 0044, wherein “When the object O is relatively small and light, the handling device 10 sucks and holds the object O using only one or more effective suction units 205E selected from the plurality of suction units 205”; at least as in paragraph 0128, wherein “the operation controller 360 controls the moving mechanism 100 and the holder 200 to suck the object O at the movement source B1 on the basis of instructions of the holding plan output from the holding plan generator 359 and moves the sucked object O to the movement destination container B2”).
Regarding claim 16, in view of the above combination of Komoda, Mishra, Shaw, and Sun, Komoda further discloses:
The robot system according to claim 12, wherein the controller is configured or programmed to, (at least as in paragraph 0117, wherein “(Step S3) The model selector 352 selects a model that is a holding form of the holder 200 on the basis of the information about the object O. The selected model may include a plurality of patterns as described above”; at least as in paragraph 0118, wherein “(Step S4)… the holding position determiner 353 determines the suction position (holding position) of the holder 200… There may be a plurality of candidate holding positions… the holding position determiner 353 associates, for each holding position, candidates of the holding positions with information indicating holding postures of the candidate holding positions to set holding points”; at least as in paragraph 0119, wherein “(Step S5)… the manipulability evaluator 354 evaluates the manipulability for each of the holding points using the selected manipulability map and gives a manipulability score related to the position and posture [and] excludes the holding points of the postures and positions that cannot be held on the basis of the evaluation result”; at least as in paragraph 0122, wherein “(Step S8) The interference determiner 357 determines whether or not there is interference between the holder 200 and the handling device 10 and obstacles other than the holding target object O on the basis of Step S7”; at least as in paragraph 0123, wherein “(Step S9) The score calculator 358 calculates a score regarding the degree of manipulability for each of the candidate holding points for the selected pattern”; at least as in paragraph 0126, wherein “(Step S11) The pattern selector 355 determines whether or not there is another pattern candidate”).
However, Komoda does not explicitly disclose “after causing the one workpiece holder to hold the workpiece… for the another workpiece holder of the plurality of workpiece holders such that the workpiece held by the one workpiece holder is included as a component of a robot… for the another workpiece holder of the plurality of workpiece holders in a state in which the workpiece held by the one workpiece holder is included as the component of the robot.”
Sun discloses a robotic system wherein image data associated with a workspace is used to generate a three dimensional view of a workspace to determine grasp strategies for each item present in the workspace. Sun specifically teaches “after causing the one workpiece holder to hold the workpiece… for the another workpiece holder of the plurality of workpiece holders such that the workpiece held by the one workpiece holder is included as a component of a robot… for the another workpiece holder of the plurality of workpiece holders in a state in which the workpiece held by the one workpiece holder is included as the component of the robot” (at least as in paragraph 0090, wherein “the process 500 of FIG. 5 is a continuous and ongoing process. As items are picked and placed from the pile, subsequently received image data is processed to identify and determine strategies to grasp more items (502, 504, and 506), and a plan to pick/place items is updated based on the grasp strategies and probabilities (508)”; at least as in paragraph 0071, wherein “Items may be picked one at a time or, in some embodiments, multiple items may be grasped at once”; at least as in paragraph 0085, wherein “determining a grasp strategy includes determining whether to attempt to pick up one item or multiple items, whether to use all end effector actuators (e.g., use just one or two or more suction cups or sets of cups) and/or selecting a combined grasp technique and corresponding speed from a set of computed grasp technique+speed combinations under consideration (e.g., 1 suction cup @ 50% speed vs. 2 suction cups 80% speed)”; at least as in paragraph 0087, wherein “Another optimization implemented in some embodiments is to change grasp strategy based on double (or n-) item picking (also see next section) where a robot may adapt its strategy based on whether it's trying to reuse vision data while picking multiple items in a batch or sequentially. For example, a robot gripper with 4 suction cups can use 2 cups to pick up one object, and the others 2 to pick up a second object”; at least as in paragraph 0108, wherein “the system may take a photo and identify two (or more) objects to pick. The system picks and moves the first one; then, instead of doing a full scene re-compute to find a next package to pick, the system simply looks at whether the second package is disturbed”; at least as in paragraph 0109, wherein “a robot may use vision, depth, or other sensors to identify two graspable packages that are judged by the robot's control algorithm to be far enough apart so as not to be able to interfere with each other's picks (by destabilizing the pile and causing it to flow, or by hitting the other package etc.)”; at least as in paragraph 0115, wherein “the pairs of suction cups (i.e., a first pair comprising suction cups 804 and 812, and a second pair comprising suction cups 808 and 814) may be operated independently. For example, a single item, such as a smaller and/or lighter item, may be grasped using only one or the other of the suction cup pairs. In some embodiments, each pair may be used to grasp a separate item at the same time, enabling two (or in some embodiments more) items to be picked/placed simultaneously”).
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Komoda to include Sun's teaching of a grasp strategy determining system, since Sun teaches wherein the system increases the likelihood of a successful grasp thus increasing efficiency.
Allowable Subject Matter
Claims 17 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICARDO ICHIKAWA VISCARRA whose telephone number is (571)270-0154. The examiner can normally be reached M-F 9-12 & 2-4 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached on (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICARDO I VISCARRA/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights