DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1-20 are now pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-12 and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chavez et al (US 10549928 B1), hereinafter Chavez in view of Menon et al. (US 20190339693 A1), hereinafter Menon.
Regarding claim 1, Chavez discloses:
A robotic system, comprising:
a communication interface and a processor coupled to the communication interface (see at least col. 11, lines 49-53: “In various embodiments, control module 908 is connected, e.g., via wireless and/or wired communication through communication interface 914 to a control computer external to end effector 900, e.g., control computer 118 of FIG. 1.”)
and configured to:
receive via the communication interface a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic system (see at least col. 11, lines 53-57: “The control module 908 includes electronic and/or electro-mechanical elements operable to supply suction force to suction cups 910, 912 comprising the end effector 900, e.g., to attach the end effector through suction to an item to be picked up, moved, and placed using end effector 900.”)
wherein the sensor reading associated with the force sensor occurs during a grasp of a receptacle that includes one or more items (see at least col. 12, lines 11-17: “In various embodiments, a plan to pick and stack items on a pallet or other receptacle and/or to depalletize items takes into consideration the trajectory through which each item is to be moved by the robotic arm. For example, a large box needs more clearance than a small item. Also, a later-placed item must be moved through a trajectory that avoids collisions with previously-placed items, etc.”)
determine based at least in part on the sensor reading that a condition requiring human intervention has been detected (see at least Fig. 5, item 512.)
wherein the sensor reading indicates a weight associated with the receptacle (see at least col. 3, lines 5-11: “Techniques are disclosed to programmatically use a robotic system comprising one or more robots (e.g., robotic arm with suction, gripper, and/or other end effector at operative end) to palletize/depalletize and/or to otherwise pack and/or unpack arbitrary sets of non-homogeneous items (e.g., dissimilar size, shape, weight, weight distribution, rigidity, fragility, etc.”)
wherein a change in the weight associated with the receptacle is used to detect that an item of the one or more items has dropped from the receptacle (see at least col. 6 lines 20-29: “Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Chavez does not explicitly disclose but Menon, in an analogous field of endeavor, teaches:
and in response to the change in the weight associated with the receptacle indicating the condition requiring human intervention, schedule a task to be performed by a human worker to correct the condition (see at least [0065]: “In some embodiments, the robot is configured to anticipate and preemptively avoid and/or schedule human assistance to resolve situations where it might otherwise get stuck. For example, assume the robot is tasked to pick up three items A, B, and C, and determines it can pick up A, may be able to pick up B, and cannot pick up C. In various embodiments, the robot implements a plan that anticipates the uncertainty over its ability to pick up B and its anticipated inability to pick up C. For example, in one approach, the robot will conclude it will need help with C and possibly B and schedules human assistance at the time it expects to need help, for example after it has had time to pick up A and make the configured number of attempts to pick up B. If when the time scheduled for human help the robot has picked up A and been successful in picking up B, the human is prompted to help with C. If the robot has not picked up B successfully by the time scheduled for human intervention, helped with B and C is requested, for example.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Chavez with the method of scheduling human intervention as taught by Menon. This is because, as stated in [0004] of Menon: “Kitting may be performed manually. For example, employees may collect items from shelves, bins, or other storage locations, each in a corresponding location within a warehouse or other facility. Aspects of kitting operations have been automated in part, such as box assembly. Use of robots or other machines to perform kitting operations has been proposed and explored, but challenges have been encountered, such as the relative complexity associated with using a robotic arm to find arbitrary quantities of an arbitrary set of items, and providing and programming a robot to perform tasks such as reaching into a bin or shelf, picking up items of arbitrary size, fragility, consistency, etc. As a result, large scale kitting operations have continued to be human labor intensive.”
Regarding claim 2, the combination of Chavez and Menon teaches:
The robotic system of claim 1.
Chavez further discloses wherein the processor is configured to determine that the condition requiring human intervention has been detected at least in part by detecting a change in a force measured by the force sensor (see at least col. 2, lines 13-31: “In various embodiments, 3D cameras, force sensors, and other sensors are used to detect and determine attributes of items to be picked and/or placed. Items the type of which is determined (e.g., with sufficient confidence, as indicated by a programmatically determined confidence score, for example) may be grasped and placed using strategies derived from an item type-specific model… In various embodiments, a library of item types, respective attributes, and grasp strategies is used to determine and implement a strategy to pick and place each item. The library is dynamic in some embodiments. For example, the library may be augmented to add newly-encountered items and/or additional attributes learned about an item, such as grasp strategies that worked or did not work in a given context. In some embodiments, human intervention may be invoked if the robotic system gets stuck.”
Regarding claim 3, the combination of Chavez and Menon teaches:
The robotic system of claim 2.
Chavez further discloses wherein the change in the force measured by the force sensor is associated with a weight of the one or more items in a grasp of the robotic instrumentality (see at least col. 4, lines 32-41: “In various embodiments, additional sensors not shown, e.g., weight or force sensors embodied in and/or adjacent to conveyor 104 and/or robotic arm 102, force sensors in the x-y plane and/or z-direction (vertical direction) of suction cups 110, etc. may be used to identify, determine attributes of, grasp, pick up, move through a determined trajectory, and/or place in a destination location on or in receptacle 106 items on conveyor 104 and/or other sources and/or staging areas in which items may be located and/or relocated, e.g., by system 100.”)
Regarding claim 4, the combination of Chavez and Menon teaches:
The robotic system of claim 3.
Chavez further discloses wherein the change in force measured by the force sensor is associated with a change in detected weight of said one or more items in the grasp of the robotic instrumentality (see at least col. 6, lines 12-29: “In the example shown, (re-)planning and plan implementation (204, 206) continue until the high level objective (202) is completed (208), at which the process 200 ends. In various embodiments, re-planning (204) may be triggered by conditions such as arrival of an items that is not expected and/or cannot be identified, a sensor reading indicating an attribute has a value other than what was expected based on item identification and/or associated item model information, etc. Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Regarding claim 5, the combination of Chavez and Menon teaches:
The robotic system of claim 4.
Chavez further discloses wherein the change in detected weight is from a first detected weight measured at a first time and a second non-zero detected weight computed based at least in part on the sensor reading associated with a second time after the first time (see at least col. 6, lines 12-29: “In the example shown, (re-)planning and plan implementation (204, 206) continue until the high level objective (202) is completed (208), at which the process 200 ends. In various embodiments, re-planning (204) may be triggered by conditions such as arrival of an items that is not expected and/or cannot be identified, a sensor reading indicating an attribute has a value other than what was expected based on item identification and/or associated item model information, etc. Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Regarding claim 6, the combination of Chavez and Menon teaches:
The robotic system of claim 1.
Chavez further discloses wherein the condition requiring human intervention is associated with instability of a stack of receptacles and the processor is configured to detect the instability of the stack of receptacles based at least in part on the sensor reading (see at least col. 7, lines 33-41: “In various embodiments, detection of unexpected instability triggers responsive action. For example, other items (e.g., items 314 as in FIG. 3A) may be placed in position to further support the item that was not stable. Or, the item may be shifted to a different nearby position (e.g., snugged left in the example shown), to see if the item would be stable in the new position. In various embodiments, if automated processing fails to determine a resolution, human intervention (e.g., via teleoperation, manually, etc.) may be triggered.”)
Regarding claim 7, the combination of Chavez and Menon teaches:
The robotic system of claim 6.
Chavez further discloses wherein the processor is configured to detect the instability of the stack of receptacles (see at least col. 7, lines 33-41: “In various embodiments, detection of unexpected instability triggers responsive action. For example, other items (e.g., items 314 as in FIG. 3A) may be placed in position to further support the item that was not stable. Or, the item may be shifted to a different nearby position (e.g., snugged left in the example shown), to see if the item would be stable in the new position. In various embodiments, if automated processing fails to determine a resolution, human intervention (e.g., via teleoperation, manually, etc.) may be triggered.”)
Regarding claim 8, the combination of Chavez and Menon teaches:
The robotic system of claim 7.
Chavez further discloses wherein the sensor reading is associated with an operation to place one or more receptacles in a grasp of the robotic instrumentality on top of the stack of receptacles (see at least col. 7, lines 12-25: “In various embodiments, a plan to stack items as shown in FIG. 3B may be generated by a control computer, such as control computer 118 of FIG. 1. In the example shown, an algorithmically generated plan may consider the item 318 stacked on items 310, 312 to be in a stable position. For example, there may be no computed net moment about any point or axis that would lead a control computer to determine the position as shown is not stable. In practice, however, slight deviation from plan in the placement of items on pallet 302 could result in the item 318 not being in a stable position once stacked on items 310, 312, for example. In some embodiments, a robotic arm and end effector (e.g., suction, gripper) used to place item 318 includes sensors and logic to detect instability.”)
Regarding claim 9, the combination of Chavez and Menon teaches:
The robotic system of claim 8.
Chavez further discloses wherein the processor is configured to control the robotic instrumentality to perform a slotting operation to place the one or more receptacles in the grasp of the robotic instrumentality on top of the stack of receptacles (see at least col. 7, lines 33-41: “In various embodiments, detection of unexpected instability triggers responsive action. For example, other items (e.g., items 314 as in FIG. 3A) may be placed in position to further support the item that was not stable. Or, the item may be shifted to a different nearby position (e.g., snugged left in the example shown), to see if the item would be stable in the new position. In various embodiments, if automated processing fails to determine a resolution, human intervention (e.g., via teleoperation, manually, etc.) may be triggered.”)
Regarding claim 10, the combination of Chavez and Menon teaches:
The robotic system of claim 9.
Chavez further discloses wherein the processor is configured to detect the instability at least in part by comparing an expected reading with the received sensor reading (see at least col. 8, lines 33-48: “At 506, one or more second order sensors (e.g., force sensors) are used to snug the item into place (as in FIG. 4C) and/or to detect instability (as in FIG. 3B) or other unexpected conditions. If the item is determined at 508 to have been placed successfully, at 510 processing continues with a next item, with respect to which a next iteration of the process 500 is performed. If the item is determined at 508 not to have been placed successfully (e.g., item not in expected location as determined by image data, item not stable, stack not stable, etc.), then at 512 the system tries again via automated processing to securely place the item, e.g., if a further strategy to attempt to do so is available, and/or human intervention is invoked (e.g., via on demand teleoperation) until the item is determined at 508 to have been placed successfully.”)
Regarding claim 11, the combination of Chavez and Menon teaches:
The robotic system of claim 10.
Chavez further discloses wherein the processor is further configured to learn an association between the expected reading and successful placement of the one or more receptacles in the grasp of the robotic instrumentality on top of the stack of receptacles in a condition in which the instability is not present (see at least col. 8, lines 33-48: “At 506, one or more second order sensors (e.g., force sensors) are used to snug the item into place (as in FIG. 4C) and/or to detect instability (as in FIG. 3B) or other unexpected conditions. If the item is determined at 508 to have been placed successfully, at 510 processing continues with a next item, with respect to which a next iteration of the process 500 is performed. If the item is determined at 508 not to have been placed successfully (e.g., item not in expected location as determined by image data, item not stable, stack not stable, etc.), then at 512 the system tries again via automated processing to securely place the item, e.g., if a further strategy to attempt to do so is available, and/or human intervention is invoked (e.g., via on demand teleoperation) until the item is determined at 508 to have been placed successfully.”)
Regarding claim 12, the combination of Chavez and Menon teaches:
The robotic system of claim 1.
Chavez further discloses the task to be performed by the human worker includes a task to do one or both of checking for and retrieving the dropped item (see first col. 5, lines 12-22: “In the example shown, control computer 118 is connected to an “on demand” teleoperation device 122. In some embodiments, if control computer 118 cannot proceed in a fully automated mode, for example, a strategy to grasp, move, and place an item cannot be determined and/or fails in a manner such that control computer 118 does not have a strategy to complete picking and placing the item in a fully automated mode, then control computer 118 prompts a human user 124 to intervene, e.g., by using teleoperation device 122 to operate the robotic arm 102 and/or end effector 108 to grasp, move, and place the item.” See further col. 6, lines 20-25: “Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item…”)
Regarding claim 15, the combination of Chavez, Menon and Mellinger teaches:
The robotic system of claim 1.
Chavez does not explicitly disclose, but Menon, in an analogous field of endeavor, teaches:
wherein the processor is further configured to notify the human worker of the task scheduled to be performed by the human worker (see at least [0065]: “For example, in one approach, the robot will conclude it will need help with C and possibly B and schedules human assistance at the time it expects to need help, for example after it has had time to pick up A and make the configured number of attempts to pick up B. If when the time scheduled for human help the robot has picked up A and been successful in picking up B, the human is prompted to help with C. If the robot has not picked up B successfully by the time scheduled for human intervention, helped with B and C is requested, for example. In another approach, the robot may pre-emptively trigger a direct request for task-related information to the teleoperator. For example, the robot may ask the teleoperator to indicate how the robot should grasp item C, and in the meanwhile, it picks up A and B. If the human teleoperator provides a strategy by the time the robot gets down to picking up item C, then the robot's motion is seamless. Otherwise, the robot requests help picking up C.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Chavez with the method of scheduling human intervention as taught by Menon. This is because, as stated in [0004] of Menon: “Kitting may be performed manually. For example, employees may collect items from shelves, bins, or other storage locations, each in a corresponding location within a warehouse or other facility. Aspects of kitting operations have been automated in part, such as box assembly. Use of robots or other machines to perform kitting operations has been proposed and explored, but challenges have been encountered, such as the relative complexity associated with using a robotic arm to find arbitrary quantities of an arbitrary set of items, and providing and programming a robot to perform tasks such as reaching into a bin or shelf, picking up items of arbitrary size, fragility, consistency, etc. As a result, large scale kitting operations have continued to be human labor intensive.”
Regarding claim 16, Chavez discloses:
A method to control a robotic system, comprising:
receiving a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic system (see at least col. 11, lines 53-57: “The control module 908 includes electronic and/or electro-mechanical elements operable to supply suction force to suction cups 910, 912 comprising the end effector 900, e.g., to attach the end effector through suction to an item to be picked up, moved, and placed using end effector 900.”)
wherein the sensor reading associated with the force sensor occurs during a grasp of a receptacle that includes one or more items (see at least col. 12, lines 11-17: “In various embodiments, a plan to pick and stack items on a pallet or other receptacle and/or to depalletize items takes into consideration the trajectory through which each item is to be moved by the robotic arm. For example, a large box needs more clearance than a small item. Also, a later-placed item must be moved through a trajectory that avoids collisions with previously-placed items, etc.”)
determining based at least in part on the sensor reading that a condition requiring human intervention has been detected (see at least Fig. 5, item 512.)
wherein the sensor reading indicates a weight associated with the receptacle (see at least col. 3, lines 5-11: “Techniques are disclosed to programmatically use a robotic system comprising one or more robots (e.g., robotic arm with suction, gripper, and/or other end effector at operative end) to palletize/depalletize and/or to otherwise pack and/or unpack arbitrary sets of non-homogeneous items (e.g., dissimilar size, shape, weight, weight distribution, rigidity, fragility, etc.”)
wherein a change in the weight associated with the receptacle is used to detect that an item of the one or more items has dropped from the receptacle (see at least col. 6 lines 20-29: “Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Chavez does not explicitly disclose but Menon, in an analogous field of endeavor, teaches:
and in response to the change in the weight associated with the receptacle indicating the condition requiring human intervention, scheduling a task to be performed by a human worker to correct the condition (see at least [0065]: “In some embodiments, the robot is configured to anticipate and preemptively avoid and/or schedule human assistance to resolve situations where it might otherwise get stuck. For example, assume the robot is tasked to pick up three items A, B, and C, and determines it can pick up A, may be able to pick up B, and cannot pick up C. In various embodiments, the robot implements a plan that anticipates the uncertainty over its ability to pick up B and its anticipated inability to pick up C. For example, in one approach, the robot will conclude it will need help with C and possibly B and schedules human assistance at the time it expects to need help, for example after it has had time to pick up A and make the configured number of attempts to pick up B. If when the time scheduled for human help the robot has picked up A and been successful in picking up B, the human is prompted to help with C. If the robot has not picked up B successfully by the time scheduled for human intervention, helped with B and C is requested, for example.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Chavez with the method of scheduling human intervention as taught by Menon. This is because, as stated in [0004] of Menon: “Kitting may be performed manually. For example, employees may collect items from shelves, bins, or other storage locations, each in a corresponding location within a warehouse or other facility. Aspects of kitting operations have been automated in part, such as box assembly. Use of robots or other machines to perform kitting operations has been proposed and explored, but challenges have been encountered, such as the relative complexity associated with using a robotic arm to find arbitrary quantities of an arbitrary set of items, and providing and programming a robot to perform tasks such as reaching into a bin or shelf, picking up items of arbitrary size, fragility, consistency, etc. As a result, large scale kitting operations have continued to be human labor intensive.”
Regarding claim 17, the combination of Chavez and Menon teaches:
The method of claim 16.
Chavez further discloses wherein the condition requiring human intervention has been detected at least in part by detecting a change in a force measured by the force sensor (see at least col. 2, lines 13-31: “In various embodiments, 3D cameras, force sensors, and other sensors are used to detect and determine attributes of items to be picked and/or placed. Items the type of which is determined (e.g., with sufficient confidence, as indicated by a programmatically determined confidence score, for example) may be grasped and placed using strategies derived from an item type-specific model… In various embodiments, a library of item types, respective attributes, and grasp strategies is used to determine and implement a strategy to pick and place each item. The library is dynamic in some embodiments. For example, the library may be augmented to add newly-encountered items and/or additional attributes learned about an item, such as grasp strategies that worked or did not work in a given context. In some embodiments, human intervention may be invoked if the robotic system gets stuck.”
Regarding claim 18, the combination of Chavez and Menon teaches:
The method of claim 17.
Chavez further discloses wherein in the change in force measured by the force sensor is associated with a change in detected weight of said one or more items in the grasp of the robotic instrumentality (see at least col. 6, lines 12-29: “In the example shown, (re-)planning and plan implementation (204, 206) continue until the high level objective (202) is completed (208), at which the process 200 ends. In various embodiments, re-planning (204) may be triggered by conditions such as arrival of an items that is not expected and/or cannot be identified, a sensor reading indicating an attribute has a value other than what was expected based on item identification and/or associated item model information, etc. Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Regarding claim 19, the combination of Chavez and Menon teaches:
The method of claim 16.
Chavez further discloses wherein the condition requiring human intervention is associated with instability of a stack of receptacles and the instability is detected based at least in part on the sensor reading (see at least col. 7, lines 33-41: “In various embodiments, detection of unexpected instability triggers responsive action. For example, other items (e.g., items 314 as in FIG. 3A) may be placed in position to further support the item that was not stable. Or, the item may be shifted to a different nearby position (e.g., snugged left in the example shown), to see if the item would be stable in the new position. In various embodiments, if automated processing fails to determine a resolution, human intervention (e.g., via teleoperation, manually, etc.) may be triggered.”)
Regarding claim 20, Chavez discloses:
A computer program product to control a robotic system, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic system (see at least col. 11, lines 53-57: “The control module 908 includes electronic and/or electro-mechanical elements operable to supply suction force to suction cups 910, 912 comprising the end effector 900, e.g., to attach the end effector through suction to an item to be picked up, moved, and placed using end effector 900.”)
wherein the sensor reading associated with the force sensor occurs during a grasp of a receptacle that includes one or more items (see at least col. 12, lines 11-17: “In various embodiments, a plan to pick and stack items on a pallet or other receptacle and/or to depalletize items takes into consideration the trajectory through which each item is to be moved by the robotic arm. For example, a large box needs more clearance than a small item. Also, a later-placed item must be moved through a trajectory that avoids collisions with previously-placed items, etc.”)
determining based at least in part on the sensor reading that a condition requiring human intervention has been detected (see at least Fig. 5, item 512.)
wherein the sensor reading indicates a weight associated with the receptacle (see at least col. 3, lines 5-11: “Techniques are disclosed to programmatically use a robotic system comprising one or more robots (e.g., robotic arm with suction, gripper, and/or other end effector at operative end) to palletize/depalletize and/or to otherwise pack and/or unpack arbitrary sets of non-homogeneous items (e.g., dissimilar size, shape, weight, weight distribution, rigidity, fragility, etc.”)
wherein a change in the weight associated with the receptacle is used to detect that an item of the one or more items has dropped from the receptacle (see at least col. 6 lines 20-29: “Other examples of unexpected conditions include, without limitation, determining that an expected item is missing, reevaluating item identification and determining an item is other than as originally identified, detecting an item weight or other attribute inconsistent with the item as identified, dropping or needing to re-grasp the item, determining that a later-arriving item is too heavy to be stacked on one or more other items as contemplated by the original and/or current plan, and detecting instability in the set of items as stacked on the receptacle.”)
Chavez does not explicitly disclose but Menon, in an analogous field of endeavor, teaches:
and in response to the change in the weight associated with the receptacle indicating the condition requiring human intervention, scheduling a task to be performed by a human worker to correct the condition (see at least [0065]: “In some embodiments, the robot is configured to anticipate and preemptively avoid and/or schedule human assistance to resolve situations where it might otherwise get stuck. For example, assume the robot is tasked to pick up three items A, B, and C, and determines it can pick up A, may be able to pick up B, and cannot pick up C. In various embodiments, the robot implements a plan that anticipates the uncertainty over its ability to pick up B and its anticipated inability to pick up C. For example, in one approach, the robot will conclude it will need help with C and possibly B and schedules human assistance at the time it expects to need help, for example after it has had time to pick up A and make the configured number of attempts to pick up B. If when the time scheduled for human help the robot has picked up A and been successful in picking up B, the human is prompted to help with C. If the robot has not picked up B successfully by the time scheduled for human intervention, helped with B and C is requested, for example.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Chavez with the method of scheduling human intervention as taught by Menon. This is because, as stated in [0004] of Menon: “Kitting may be performed manually. For example, employees may collect items from shelves, bins, or other storage locations, each in a corresponding location within a warehouse or other facility. Aspects of kitting operations have been automated in part, such as box assembly. Use of robots or other machines to perform kitting operations has been proposed and explored, but challenges have been encountered, such as the relative complexity associated with using a robotic arm to find arbitrary quantities of an arbitrary set of items, and providing and programming a robot to perform tasks such as reaching into a bin or shelf, picking up items of arbitrary size, fragility, consistency, etc. As a result, large scale kitting operations have continued to be human labor intensive.”
Claims 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Chavez and Menon further in view of Mellinger et al. (US 20200033865 A1), hereinafter Mellinger.
Regarding claim 13, the combination of Chavez and Menon teaches:
The robotic system of claim 1.
Chavez does not explicitly disclose, but Mellinger, in an analogous field of endeavor, teaches:
wherein the processor is further configured to mark an operating zone in which the task is to be performed as being in a state that prevents or modifies operation of the robotic instrumentality in the operating zone at a time when the task is scheduled to be performed (see at least [0062]: “In block 502, the processor of the cleaning robot may determine whether a person is present (or is not present) in the location. In some embodiments, the processor may determine based on whether a human is or is not present whether or not the structure is unoccupied (e.g., no one is home, all personnel have left work, etc.). In such embodiments, the processor may schedule an operation of the cleaning robot in response to determining that the structure is unoccupied. In some embodiments, the processor may schedule an operation of the cleaning robot in one or more locations where a person is not present. In some embodiments, the processor may not schedule an operation of the cleaning robot in a location where a person is present. In some embodiments, the processor may schedule an operation of the cleaning robot to avoid locations where a person is present. For example, the processor may schedule the cleaning robot operation in locations where a person is not present, and schedule the cleaning robot to stay away from, or pass by, or to travel around or away from, locations where a person is present.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Chavez to incorporate the dynamic robot task scheduling as taught by Mellinger, as such a modification would further improve the safety and efficiency of the system by ensuring that the robot does not harm or interfere with the human while they are completing a task.
Regarding claim 14, the combination of Chavez, Menon and Mellinger teaches:
The robotic system of claim 13.
Chavez does not explicitly disclose, but Mellinger, in an analogous field of endeavor, teaches:
wherein the processor is further configured to schedule the robotic instrumentality to perform, at the time when the task is scheduled to be performed, one or more other tasks in one or more operating zones other than the operating zone in which the task is to be performed (see at least [0062]: “In block 502, the processor of the cleaning robot may determine whether a person is present (or is not present) in the location. In some embodiments, the processor may determine based on whether a human is or is not present whether or not the structure is unoccupied (e.g., no one is home, all personnel have left work, etc.). In such embodiments, the processor may schedule an operation of the cleaning robot in response to determining that the structure is unoccupied. In some embodiments, the processor may schedule an operation of the cleaning robot in one or more locations where a person is not present. In some embodiments, the processor may not schedule an operation of the cleaning robot in a location where a person is present. In some embodiments, the processor may schedule an operation of the cleaning robot to avoid locations where a person is present.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Chavez to incorporate the dynamic robot task scheduling as taught by Mellinger, as such a modification would further improve the safety and efficiency of the system by ensuring that the robot does not harm or interfere with the human while they are completing a task.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658
/JASON HOLLOWAY/Primary Examiner, Art Unit 3658