Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office action is in response to the amendment filed on 07/20/2026. Claims 28-48 are currently pending with claims 28 and 38 being amended, and claim 48 being newly added.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/20/2026 has been received. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment
The amendments to the claims submitted on 07/20/2026 overcome the claim objections set forth in the previous Office action except for those set forth in the claim objection section.
Response to Arguments
Examiner notes wherein Applicant argues the newly amended limitations, which have not been addressed by the prior art of record. As such, Examiner has augmented the below rejection(s) in view of the prior art of record to address the newly amended limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 28-29, 32, 36, 38-39, 42, and 46 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun et al. (US 20220289502 A1), hereinafter Sun in view of Nakatani et al. (US 20140230581 A1), hereinafter Nakatani.
Regarding claim 28, Sun teaches:
28. (Currently Amended) A method of manipulating an object using a mobile manipulation robot, the method comprising:
planning a trajectory of the object; (Paragraph 0032, "According to various embodiments, a set of paths or trajectories for singulating an item is determined, and the path or trajectory along which the item is to be singulated is selected from the set of paths or trajectories. The path or trajectory can be selected based on various properties associated with the corresponding paths or trajectories within the set. Examples of the properties associated with the corresponding paths or trajectories that may be used in connection with selecting the path or trajectory include a speed with which the item is to be singulated, a location on the conveyor in which the item is to be singulated, a probability of success that the item is to be singulated according to the particular path or trajectory, an indication of whether another item or object within the workspace intersects with the particular path or trajectory, etc. In some embodiments, the probability of success that the item is to be singulated according to the particular path or trajectory is determined for at least a subset of the set of paths or trajectories, and the path or trajectory is selected based on the corresponding probability of success relative to the probability of success corresponding to other paths or trajectories. As an example, the path or trajectory is selected in response to determining that the probability of success corresponding to the path or trajectory exceeds a threshold probability, or exceeds a threshold number or percentage of other paths or trajectories within the subset.")
moving the object along the trajectory using the mobile manipulation robot; (Paragraph 0086, "Information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays).")
… along the trajectory;
estimating one or more mass characteristics of the object … (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") and
modifying an operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") based, at least in part, on the estimated one or more mass characteristics. (Paragraph 0085, "For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item.")
Sun does not specifically discuss determining acceleration of the object during movement or using that to estimate the mass characteristics. However, Nakatani, in the same field of endeavor of robotics, teaches:
… determining one or more accelerations of the object while the object is in motion … based, at least in part, on the one or more accelerations … (Paragraph 0099, “FIG. 1 is a schematic structural view of a mass measurement device 100. In FIG. 1, a force sensor 1 detects a force that acts on a moving article. A holding mechanism 2 holds an article Q. A movement mechanism 3 moves the holding mechanism 2 in three dimensions. An acceleration sensor 4 detects acceleration that acts on the article Q. The force sensor 1 is disposed between the holding mechanism 2 and the movement mechanism 3, and the acceleration sensor 4 is disposed adjacent to the holding mechanism 2.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Sun with the ability to detect acceleration of the object and utilize this value along with the force to determine the mass of the object as taught by Nakatani. This would allow the system to identify mass properties of unknown objects while the object is held. This would increase the efficiency of the system.
Regarding claim 29, where all the limitations of claim 28 are discussed above, Sun further teaches:
29. (Previously Presented) The method of claim 28 wherein:
planning the trajectory of the object comprises planning a first trajectory of the object, (Paragraph 0032, "According to various embodiments, a set of paths or trajectories for singulating an item is determined, and the path or trajectory along which the item is to be singulated is selected from the set of paths or trajectories. The path or trajectory can be selected based on various properties associated with the corresponding paths or trajectories within the set. Examples of the properties associated with the corresponding paths or trajectories that may be used in connection with selecting the path or trajectory include a speed with which the item is to be singulated, a location on the conveyor in which the item is to be singulated, a probability of success that the item is to be singulated according to the particular path or trajectory, an indication of whether another item or object within the workspace intersects with the particular path or trajectory, etc. In some embodiments, the probability of success that the item is to be singulated according to the particular path or trajectory is determined for at least a subset of the set of paths or trajectories, and the path or trajectory is selected based on the corresponding probability of success relative to the probability of success corresponding to other paths or trajectories. As an example, the path or trajectory is selected in response to determining that the probability of success corresponding to the path or trajectory exceeds a threshold probability, or exceeds a threshold number or percentage of other paths or trajectories within the subset.") and modifying the operation of the mobile manipulation robot comprises planning a second trajectory of the object different from the first trajectory of the object. (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).")
Regarding claim 32, where all the limitations of claim 28 are discussed above, Sun further teaches:
32. (Previously Presented) The method of claim 28, wherein modifying the operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") comprises adjusting a motion of a robotic arm of the mobile manipulation robot. (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.")
Regarding claim 36, where all the limitations of claim 28 are discussed above, Sun further teaches:
36. (Previously Presented) The method of claim 28, wherein estimating the one or more mass characteristics of the object (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") comprises estimating one or more of a mass of the object, a center of mass of the object, and one or more moments of inertia of the object. (Paragraph 0031, "The path or trajectory along which an item is to be singulated is determined according to various embodiments based at least in part on sensor data. The robotic system may obtain a plurality of sensors that output information pertaining to the workspace, including the items or objects within the workspace. The sensor data is obtained based on the information output from one or more sensors and used in connection with determining the path or trajectory. In some embodiments, the path or trajectory is determined based at least in part on one or more attributes of the item to be singulated. Examples of attributes of the item include a weight, a size (e.g., one or more dimensions), a type of packaging, an identifier on the item, a location of an identifier or label on the item, a location of the item relative to the chute and/or conveyor, information obtained from the identifier or label on the item, etc. Various other attributes can be used in connection with determining the path or trajectory. Determination of the path or trajectory of the item may be further based at least in part on a location on the conveyor at which the item is to be placed, an attribute of an item(s) already on the conveyor, an attribute of an item within the workspace (e.g., an item within the source pile/flow), a grip strength with which the robotic arm has grasped the item, a speed with which the robotic arm is to move the item, etc.")
Regarding claim 38, Sun further teaches:
38. (Currently Amended) A mobile manipulation robot, comprising:
a robotic arm; (Paragraph 0029, “A robotic system includes a robotic arm and end effector used to pick items from a source pile/flow and place them on a segmented conveyor or similar conveyance to be sorted and routed for transport to a downstream (e.g., ultimate addressed/physical) destination.”)
one or more sensors; (Paragraph 0028, “A robotic system to perform singulation is disclosed. In various embodiments, singulation is performed based on data associated with a workspace or an item within the workspace. A plan (e.g., to singulate an item) is determined based at least in part on an attribute of an item in the workspace. The attribute of the item may be determined based at least in part on the sensor data obtained with respect to the workspace. As used herein, a workspace may include a chute or other conveyance structure and/or receptacle on which a source pile/flow of items is disposed, a destination conveyance structure on which an item from the chute is to be singly placed, and a robotic structure that includes a robotic arm that picks one or more items from the chute (or other source) and places the one or more items singly, each in a corresponding location on the destination conveyance structure. The workspace can further include a control computer that obtains sensor data associated with the workspace, and/or an on-demand teleoperation device that a human operator can use to control an element within the workspace such as the robotic arm and/or the conveyance structure. As used herein, the term slot or tray may be used interchangeably in connection with describing a particular location on the conveyor.”) and
a controller (Paragraph 0026, “The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.”) configured to:
plan a trajectory of an object; (Paragraph 0032, "According to various embodiments, a set of paths or trajectories for singulating an item is determined, and the path or trajectory along which the item is to be singulated is selected from the set of paths or trajectories. The path or trajectory can be selected based on various properties associated with the corresponding paths or trajectories within the set. Examples of the properties associated with the corresponding paths or trajectories that may be used in connection with selecting the path or trajectory include a speed with which the item is to be singulated, a location on the conveyor in which the item is to be singulated, a probability of success that the item is to be singulated according to the particular path or trajectory, an indication of whether another item or object within the workspace intersects with the particular path or trajectory, etc. In some embodiments, the probability of success that the item is to be singulated according to the particular path or trajectory is determined for at least a subset of the set of paths or trajectories, and the path or trajectory is selected based on the corresponding probability of success relative to the probability of success corresponding to other paths or trajectories. As an example, the path or trajectory is selected in response to determining that the probability of success corresponding to the path or trajectory exceeds a threshold probability, or exceeds a threshold number or percentage of other paths or trajectories within the subset.")
control the robotic arm to move the object along the trajectory using the mobile manipulation robot; (Paragraph 0086, "Information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays).")
… along the trajectory;
estimate using information received from the one or more sensors, one or more mass characteristics of the object … (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") and
modify an operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") based, at least in part, on the estimated one or more mass characteristics. (Paragraph 0085, "For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item.")
Sun does not specifically discuss determining acceleration of the object during movement or using that to estimate the mass characteristics. However, Nakatani, in the same field of endeavor of robotics, teaches:
… determining one or more accelerations of the object while the object is in motion … based, at least in part, on the one or more accelerations … (Paragraph 0099, “FIG. 1 is a schematic structural view of a mass measurement device 100. In FIG. 1, a force sensor 1 detects a force that acts on a moving article. A holding mechanism 2 holds an article Q. A movement mechanism 3 moves the holding mechanism 2 in three dimensions. An acceleration sensor 4 detects acceleration that acts on the article Q. The force sensor 1 is disposed between the holding mechanism 2 and the movement mechanism 3, and the acceleration sensor 4 is disposed adjacent to the holding mechanism 2.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Sun with the ability to detect acceleration of the object and utilize this value along with the force to determine the mass of the object as taught by Nakatani. This would allow the system to identify mass properties of unknown objects while the object is held. This would increase the efficiency of the system.
Regarding claim 39, where all the limitations of claim 38 are discussed above, Sun further teaches:
39. (Previously Presented) The mobile manipulation robot of claim 38, wherein:
planning the trajectory of the object comprises planning a first trajectory of the object, (Paragraph 0032, "According to various embodiments, a set of paths or trajectories for singulating an item is determined, and the path or trajectory along which the item is to be singulated is selected from the set of paths or trajectories. The path or trajectory can be selected based on various properties associated with the corresponding paths or trajectories within the set. Examples of the properties associated with the corresponding paths or trajectories that may be used in connection with selecting the path or trajectory include a speed with which the item is to be singulated, a location on the conveyor in which the item is to be singulated, a probability of success that the item is to be singulated according to the particular path or trajectory, an indication of whether another item or object within the workspace intersects with the particular path or trajectory, etc. In some embodiments, the probability of success that the item is to be singulated according to the particular path or trajectory is determined for at least a subset of the set of paths or trajectories, and the path or trajectory is selected based on the corresponding probability of success relative to the probability of success corresponding to other paths or trajectories. As an example, the path or trajectory is selected in response to determining that the probability of success corresponding to the path or trajectory exceeds a threshold probability, or exceeds a threshold number or percentage of other paths or trajectories within the subset.") and modifying the operation of the mobile manipulation robot comprises planning a second trajectory of the object different from the first trajectory of the object. (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).")
Regarding claim 42, where all the limitations of claim 38 are discussed above, Sun further teaches:
42. (Previously Presented) The mobile manipulation robot of claim 38, wherein modifying the operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") comprises adjusting a motion of the robotic arm. (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.")
Regarding claim 46, where all the limitations of claim 38 are discussed above, Sun further teaches:
46. (Previously Presented) The mobile manipulation robot of claim 38, wherein estimating the one or more mass characteristics of the object (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") comprises estimating one or more of a mass of the object, a center of mass of the object, or one or more moments of inertia of the object. (Paragraph 0031, "The path or trajectory along which an item is to be singulated is determined according to various embodiments based at least in part on sensor data. The robotic system may obtain a plurality of sensors that output information pertaining to the workspace, including the items or objects within the workspace. The sensor data is obtained based on the information output from one or more sensors and used in connection with determining the path or trajectory. In some embodiments, the path or trajectory is determined based at least in part on one or more attributes of the item to be singulated. Examples of attributes of the item include a weight, a size (e.g., one or more dimensions), a type of packaging, an identifier on the item, a location of an identifier or label on the item, a location of the item relative to the chute and/or conveyor, information obtained from the identifier or label on the item, etc. Various other attributes can be used in connection with determining the path or trajectory. Determination of the path or trajectory of the item may be further based at least in part on a location on the conveyor at which the item is to be placed, an attribute of an item(s) already on the conveyor, an attribute of an item within the workspace (e.g., an item within the source pile/flow), a grip strength with which the robotic arm has grasped the item, a speed with which the robotic arm is to move the item, etc.")
Claim(s) 30 and 40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Yoshiura et al. (US 20210291371 A1), hereinafter Yoshiura.
Regarding claim 30, where all the limitations of claim 29 are discussed above, Sun further teaches:
30. (Previously Presented) The method of claim 29, wherein planning the second trajectory comprises planning the second trajectory (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).") … the estimated one or more mass characteristics. (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.")
Sun does not specifically discuss the use of inverse dynamics during the control process, however, Yoshiura, in the same field of endeavor of robotics, teaches:
… based, at least in part, on inverse dynamics computed using … (Paragraph 0042, "The motion control unit 34 calculates a target position of each of the motors 44 necessary for the movement of the end effector to the coordinate position command input from the work control unit 33 by a so-called inverse kinematic calculation, and sequentially outputs the target position as a motor position command to the corresponding servo 35.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Sun with the ability to identify control values using inverse kinematics as taught by Yoshiura. Inverse Dynamics are known to be a method of computing the forces on a body based on the motion and inertial characteristics of the body so it is inherent that the computation is based at least in part on the mass characteristics. This would allow for accurate control of the robotic arm during operation.
Regarding claim 40, where all the limitations of claim 39 are discussed above, Sun further teaches:
40. (Previously Presented) The mobile manipulation robot of claim 39, wherein planning the second trajectory comprises planning the second trajectory (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).") … the estimated one or more mass characteristics. (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.")
Sun does not specifically discuss the use of inverse dynamics during the control process, however, Yoshiura, in the same field of endeavor of robotics, teaches:
… based, at least in part, on inverse dynamics computed using … (Paragraph 0042, "The motion control unit 34 calculates a target position of each of the motors 44 necessary for the movement of the end effector to the coordinate position command input from the work control unit 33 by a so-called inverse kinematic calculation, and sequentially outputs the target position as a motor position command to the corresponding servo 35.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Sun with the ability to identify control values using inverse kinematics as taught by Yoshiura. Inverse Dynamics are known to be a method of computing the forces on a body based on the motion and inertial characteristics of the body so it is inherent that the computation is based at least in part on the mass characteristics. This would allow for accurate control of the robotic arm during operation.
Claim(s) 31 and 41 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Kalakrishnan et al. (US 9393693 B1), hereinafter Kalakrishnan.
Regarding claim 31, where all the limitations of claim 29 are discussed above, Sun further teaches:
31. (Previously Presented) The method of claim 29, wherein planning the second trajectory comprises planning the second trajectory (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).") …
Sun does not specifically discuss the adjustments to the trajectory being to limit the wrench experienced by the object. However, Kalakrishnan, in the same field of endeavor of robotics, teaches:
… to limit a wrench applied to the object by the mobile manipulation robot within a predetermined range. (Column 18 Line 65-Column 19 Line 21,"In some implementations, the computing device may alter the subsequent trajectory based on whether members of the subsequent set of force vectors are within the boundaries of the virtual model. For example, the computing device may receive an instruction for the robotic manipulator to move the subsequent physical object at one or more candidate trajectories corresponding to a candidate set of force vectors, the candidate set of force vectors representing forces to be exerted on the subsequent physical object at a plurality of points in time as the subsequent physical object is moved along the one or more candidate trajectories. In response to receiving the instruction, the computing device may determine whether members of the candidate set of force vectors are within the boundaries of the virtual model. The computing device may then, in turn, adjust members of the candidate set of force vectors that are outside of the boundaries of the virtual model, thereby updating the candidate set of force vectors. For instance, the computing device may adjust, for at least one force at one or more points in time along the subsequent trajectory, an acceleration and/or direction of the movement of the at least one force exerted on the subsequent physical object. As such, the adjusted/updated candidate set of force vectors may be used to determine/adjust the one or more subsequent trajectories.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods as taught by Sun with the ability to alter the trajectory to limit the forces on the object as taught by Kalakrishnan. This would allow the system to safely move objects which are fragile/delicate without risking damage during transport.
Regarding claim 41, where all the limitations of claim 39 are discussed above, Sun further teaches:
41. (Previously Presented) The mobile manipulation robot of claim 39, wherein planning the second trajectory comprises planning the second trajectory (Paragraph 0051, "The workspace environment state system produces output used by the robotic system to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each in a corresponding available defined location for machine identification and sorting, such as a partitioned section of segmented conveyor 208. In some embodiments, the workspace environment state system produces an output (e.g., sensor data or information otherwise characterizing the workspace and/or items within the workspace) used by the robotic system to detect a state, condition, and/or attribute associated with one or more items in the workspace, and/or a state or condition associated with the robotic arm or other element of the workspace. According to various embodiments, in response to detecting (e.g., determining) the state, condition, and/or attribute associated with one or more items in the workspace, the robotic system implements one or more active measures in connection with singulating an item. The active measure may include updating the plan to autonomously operate a robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure. In some embodiments, the active measure or the updating the plan can include operating the robotic structure to change or adapt to the detected state, condition, and/or attribute (e.g., implement a change or manner by which an item is singulated, change a path or trajectory along which the item is singulated, change a manner by which the item is grasped, change a location on the item at which the item is grasped, etc.).") …
Sun does not specifically discuss the adjustments to the trajectory being to limit the wrench experienced by the object. However, Kalakrishnan, in the same field of endeavor of robotics, teaches:
… to limit a wrench applied to the object by the mobile manipulation robot within a predetermined range. (Column 18 Line 65-Column 19 Line 21,"In some implementations, the computing device may alter the subsequent trajectory based on whether members of the subsequent set of force vectors are within the boundaries of the virtual model. For example, the computing device may receive an instruction for the robotic manipulator to move the subsequent physical object at one or more candidate trajectories corresponding to a candidate set of force vectors, the candidate set of force vectors representing forces to be exerted on the subsequent physical object at a plurality of points in time as the subsequent physical object is moved along the one or more candidate trajectories. In response to receiving the instruction, the computing device may determine whether members of the candidate set of force vectors are within the boundaries of the virtual model. The computing device may then, in turn, adjust members of the candidate set of force vectors that are outside of the boundaries of the virtual model, thereby updating the candidate set of force vectors. For instance, the computing device may adjust, for at least one force at one or more points in time along the subsequent trajectory, an acceleration and/or direction of the movement of the at least one force exerted on the subsequent physical object. As such, the adjusted/updated candidate set of force vectors may be used to determine/adjust the one or more subsequent trajectories.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods as taught by Sun with the ability to alter the trajectory to limit the forces on the object as taught by Kalakrishnan. This would allow the system to safely move objects which are fragile/delicate without risking damage during transport.
Claim(s) 33 and 43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Nielsen et al. (US 20210260757 A1), hereinafter Nielsen.
Regarding claim 33, where all the limitations of claim 32 are discussed above, Sun further teaches:
33. (Previously Presented) The method of claim 32, wherein adjusting the motion of the robotic arm (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.") …
Sun does not specifically discuss adjusting the torque applied to a joint. However, Nielsen, in the same field of endeavor of robotics, teaches:
… comprises adjusting one or more torques applied at one or more joints of the robotic arm. (Paragraph 0024, "The robot controller 115 comprises a processer 235 and memory 237 and is configured to control the joint motors of the robot joints by providing motor control signals 239a, 239b, 239f to the joint motors. The motor control signals 239a, 239b, 293f are indicative of the motor torque Tmotor,a, Tmotor,b, and Tmotor,f that each joint motor shall provide to the output flanges and the robot controller is configured to determine the motor torque based on a dynamic model of the robot arm as known in the prior art. The dynamic model makes it possible for the controller to calculate which torque the joint motors shall provide to each of the joint motors to make the robot arm perform a desired movement. The dynamic model of the robot arm can be stored in the memory 237 and be adjusted based on the joint sensor parameter Jsensor,a, Jsensor,b, Jsensor,f For instance, the joint motors can be provided as multiphase electromotors and the robot controller can be configured to adjust the motor torque provided by the joint motors by regulating the current through the phases of the multiphase motors as known in the art of motor regulation.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods as taught by Sun with the ability to control joint motors to a desired torque as taught by Nielsen. It is well known to utilize adjustments to the torque applied to a joint in order to control a robot to achieve a desired movement as demonstrated by Nielsen. This allows for precise control of the robotic system.
Regarding claim 43, where all the limitations of claim 42 are discussed above, Sun further teaches:
43. (Previously Presented) The mobile manipulation robot of claim 42, wherein adjusting the motion of the robotic arm (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.") …
Sun does not specifically discuss adjusting the torque applied to a joint. However, Nielsen, in the same field of endeavor of robotics, teaches:
… comprises adjusting one or more torques applied at one or more joints of the robotic arm. (Paragraph 0024, "The robot controller 115 comprises a processer 235 and memory 237 and is configured to control the joint motors of the robot joints by providing motor control signals 239a, 239b, 239f to the joint motors. The motor control signals 239a, 239b, 293f are indicative of the motor torque Tmotor,a, Tmotor,b, and Tmotor,f that each joint motor shall provide to the output flanges and the robot controller is configured to determine the motor torque based on a dynamic model of the robot arm as known in the prior art. The dynamic model makes it possible for the controller to calculate which torque the joint motors shall provide to each of the joint motors to make the robot arm perform a desired movement. The dynamic model of the robot arm can be stored in the memory 237 and be adjusted based on the joint sensor parameter Jsensor,a, Jsensor,b, Jsensor,f For instance, the joint motors can be provided as multiphase electromotors and the robot controller can be configured to adjust the motor torque provided by the joint motors by regulating the current through the phases of the multiphase motors as known in the art of motor regulation.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods as taught by Sun with the ability to control joint motors to a desired torque as taught by Nielsen. It is well known to utilize adjustments to the torque applied to a joint in order to control a robot to achieve a desired movement as demonstrated by Nielsen. This allows for precise control of the robotic system.
Claim(s) 34-35 and 44-45 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Tan et al. (US 20190321977 A1), hereinafter Tan.
Regarding claim 34, where all the limitations of claim 28 are discussed above, Sun further teaches:
34. (Previously Presented) The method of claim 28, wherein modifying the operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") …
Sun does not specifically discuss a mobile base unit of the robot. However, Tan, in the same field of endeavor of robotics, teaches:
… comprises adjusting a motion of a mobile base of the mobile manipulation robot. (Paragraph 0024, "In the depicted embodiment, the robot 12 includes a drive system 30A that may include components and functionality used to move the robot 12 in the environment, such as wheels or treads, motor(s) 32A, encoder 34A (configured to sense one or more parameters of the one or more motors 32A (e.g., rotational speed, linear displacement, etc.)), and so forth. The driving system 30A controls the position, the velocity, and the acceleration of the robot base. Such control may be implemented using feedback from one or more of localization/SLAM routines or modules 16, video detection and processing routines or modules 14, and encoders 34A. In one implementation the drive system 30A of the robot 12 includes or is controlled by a collaborative control mechanism, meaning humans can work closely with it. In such an implementation, the robot 12 can detect external forces and can adjust its motion and forces accordingly to avoid collisions and protect itself and people and objects in its environment. For example, in the depicted embodiment, a sonar 24 sensing modality is provided that can be used to detect obstacles in the environment, and, based upon sensed data to send signals to the motor(s) 32A of the robot base to avoid obstacles." as well as Paragraph 0049, "In this manipulation stage, there are two major sub-tasks for the robot 12. One is to ensure the successful manipulation of the task-related object. The other is to minimize energy consumption. Thus, the planning strategy in manipulation stage may involve moving the robot 12 (i.e., the robot base) to increase the working space and the success rate of manipulation and to optimize an energy consumption equation. For example, manipulation planning, in view of these objective, may involve moving the robot base so as to allow the robot to use less torque on its joints to perform manipulation tasks. In one implementation, the RRT-Connect algorithm may be employed in manipulation planning.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic arm and operation methods as taught by Sun with the drive system as taught by Tan. The incorporation of a drive system to which the arm is attached to would increase the versatility of the robot and allow it to travel throughout a space with a high level of control and the ability to adapt to a changing environment. This would allow the system to perform a greater number of tasks.
Regarding claim 35, where all the limitations of claim 28 are discussed above, Sun further teaches:
35. (Previously Presented) The method of claim 28, wherein modifying the operation of the mobile manipulation robot comprises (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") adjusting a motion of a robotic arm of the mobile manipulation robot (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.") and …
Sun does not specifically discuss a mobile base unit of the robot. However, Tan, in the same field of endeavor of robotics, teaches:
… adjusting a motion of a mobile base of the mobile manipulation robot, (Paragraph 0024, "In the depicted embodiment, the robot 12 includes a drive system 30A that may include components and functionality used to move the robot 12 in the environment, such as wheels or treads, motor(s) 32A, encoder 34A (configured to sense one or more parameters of the one or more motors 32A (e.g., rotational speed, linear displacement, etc.)), and so forth. The driving system 30A controls the position, the velocity, and the acceleration of the robot base. Such control may be implemented using feedback from one or more of localization/SLAM routines or modules 16, video detection and processing routines or modules 14, and encoders 34A. In one implementation the drive system 30A of the robot 12 includes or is controlled by a collaborative control mechanism, meaning humans can work closely with it. In such an implementation, the robot 12 can detect external forces and can adjust its motion and forces accordingly to avoid collisions and protect itself and people and objects in its environment. For example, in the depicted embodiment, a sonar 24 sensing modality is provided that can be used to detect obstacles in the environment, and, based upon sensed data to send signals to the motor(s) 32A of the robot base to avoid obstacles." as well as Paragraph 0049, "In this manipulation stage, there are two major sub-tasks for the robot 12. One is to ensure the successful manipulation of the task-related object. The other is to minimize energy consumption. Thus, the planning strategy in manipulation stage may involve moving the robot 12 (i.e., the robot base) to increase the working space and the success rate of manipulation and to optimize an energy consumption equation. For example, manipulation planning, in view of these objective, may involve moving the robot base so as to allow the robot to use less torque on its joints to perform manipulation tasks. In one implementation, the RRT-Connect algorithm may be employed in manipulation planning.") wherein the robotic arm is operatively coupled to the mobile base. (Paragraph 0029, "With the preceding sensing system and drive system discussion in mind, in the depicted example, the robot 12 and manipulator 44 are depicted as discrete components, with each having a separate and distinct sensing system (i.e., robot sensing system 18A and manipulator sensing system 18B) and driving system (i.e., robot driving system 30A and manipulator driving system 30B). As may be appreciated, the extent of integration between the manipulator 44 and robot 12 may determine the extent to which the respective sensing and drive systems components referenced are separate and distinct systems or are common or shared systems or components. For example, the cameras may be part of a sensing system 18 shared by both the robot 12 and manipulator 44 or may be distinct, with cameras 26A being used solely for robot functions and camera 26B (such as a camera positioned on or near an end-effector of the manipulator 44) being used solely for manipulator functions. For the sake of simplifying discussion the embodiment of FIG. 1 presents such systems and components as being separate, though it should be understood that other implementations may instead share sensing or drive functionality for the robot 12 and manipulator 44, and thus have combined sensing and/or driving functionality.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic arm and operation methods as taught by Sun with the drive system as taught by Tan. The incorporation of a drive system to which the arm is attached to would increase the versatility of the robot and allow it to travel throughout a space with a high level of control and the ability to adapt to a changing environment. This would allow the system to perform a greater number of tasks.
Regarding claim 44, where all the limitations of claim 38 are discussed above, Sun further teaches:
44. (Previously Presented) The mobile manipulation robot of claim 38, further comprising:
… wherein modifying the operation of the mobile manipulation robot … (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).")
Sun does not specifically discuss a mobile base unit of the robot. However, Tan, in the same field of endeavor of robotics, teaches:
… a mobile base, … comprises adjusting a motion of the mobile base. (Paragraph 0024, "In the depicted embodiment, the robot 12 includes a drive system 30A that may include components and functionality used to move the robot 12 in the environment, such as wheels or treads, motor(s) 32A, encoder 34A (configured to sense one or more parameters of the one or more motors 32A (e.g., rotational speed, linear displacement, etc.)), and so forth. The driving system 30A controls the position, the velocity, and the acceleration of the robot base. Such control may be implemented using feedback from one or more of localization/SLAM routines or modules 16, video detection and processing routines or modules 14, and encoders 34A. In one implementation the drive system 30A of the robot 12 includes or is controlled by a collaborative control mechanism, meaning humans can work closely with it. In such an implementation, the robot 12 can detect external forces and can adjust its motion and forces accordingly to avoid collisions and protect itself and people and objects in its environment. For example, in the depicted embodiment, a sonar 24 sensing modality is provided that can be used to detect obstacles in the environment, and, based upon sensed data to send signals to the motor(s) 32A of the robot base to avoid obstacles." as well as Paragraph 0049, "In this manipulation stage, there are two major sub-tasks for the robot 12. One is to ensure the successful manipulation of the task-related object. The other is to minimize energy consumption. Thus, the planning strategy in manipulation stage may involve moving the robot 12 (i.e., the robot base) to increase the working space and the success rate of manipulation and to optimize an energy consumption equation. For example, manipulation planning, in view of these objective, may involve moving the robot base so as to allow the robot to use less torque on its joints to perform manipulation tasks. In one implementation, the RRT-Connect algorithm may be employed in manipulation planning.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic arm and operation methods as taught by Sun with the drive system as taught by Tan. The incorporation of a drive system to which the arm is attached to would increase the versatility of the robot and allow it to travel throughout a space with a high level of control and the ability to adapt to a changing environment. This would allow the system to perform a greater number of tasks.
Regarding claim 45, where all the limitations of claim 44 are discussed above, Sun further teaches:
45. (Previously Presented) The mobile manipulation robot of claim 44, wherein modifying the operation of the mobile manipulation robot (Paragraph 0034, "According to various embodiments, a robotic singulation system performs an active measure in order to improve the singulation of an item (e.g., to successfully pick the item from a source pile/flow and place the item on the conveyor). The robotic system dynamically updates the path or trajectory of the item during singulation based on a context of the workspace (e.g., a state or condition of the item, a property of the item, another item within the workspace, etc.). For example, in response to determining that a detected state or condition impedes implementation of a current plan to autonomously operate the robotic structure to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyance structure, the robotic structure performs one or more active measures to improve the likelihood of successful singulation (e.g., the robotic structure can determine an active measure that is expected to improve the likelihood of successful singulation based at least in part on the detected state or condition). In some embodiments, the active measure includes using a robotic arm, an end effector of the robotic arm, a movement of a chute or other element of the workspace, or an air blower to reconfigure the source pile/flow or to reconfigure one or more items or debris on the workspace. The active measure can be performed to improve a scanning of a label or identifier on an item to be singulated, to improve the likelihood that an item can be picked up, to improve the grip on an item being singulated, to improve the release of an item from the robotic arm, or to improve operation of two robotic arms that are independently singulating items from the same workspace (e.g., the same source pile/flow).") further comprises adjusting a motion of the robotic arm. (Paragraph 0085, "Referring to FIG. 2A, in various embodiments, robotic arm 202 is to be driven by one or more motors, e.g., one or more motors at each movable joint or mount location. In some embodiments, the work required to drive robotic arm 202 (e.g., to move the robotic arm as the robotic arm attempts to singulate an item) is indicative of one or more characteristics of the item to be singulated. For example, in some embodiments, a weight of the item may be computed (or estimated) based on the work required to drive the robotic arm 202 while the item is in its grasp. In various embodiments, the work required to drive the robotic arm 202 is measured using a current sensor, a voltage sensor, a power sensor, and/or the like, or some combination thereof. In response to determining the weight of the item during singulation, the robotic system determines a path/trajectory of an item to be singulated based at least in part on the weight of the item. The robotic system may perform an active measure to adapt to the weight of the item such as, for example, updating the path or trajectory in response to determining the weight of the item. In some embodiments, in response to determining that the weight of the item is greater than a predefined threshold, robotic system 200 adjusts the plan to singulate the item via partially picking up the item and dragging the item to the corresponding location on the conveyance structure (e.g., in contrast to wholly picking up the item and moving the arm to place the item on the conveyance structure). In some embodiments, in response to determining the weight of the item, the robotic structure adjusts the speed at which the robotic arm (and the item) is moved. For example, the larger the weight of the item, the greater the shear forces are between the item and end effector 204 as the robotic arm 202 is moved. Further, the shear forces can increase as the speed at which the robotic arm is operated increases (e.g., the speed at which the robotic arm moves the item). Accordingly, robotic system 200 can control the speed of the robotic arm 202 based at least in part on the weight of the item to ensure that the item remains firmly grasped by the robotic arm. Although the description hereof describes the weight being measured based on using a current sensor, a voltage sensor, a power sensor, and/or the like, the weight can also be measured using a force sensor configured in the robotic arm 202 or the end effector 204. However, force sensors are relatively expensive and thus low-level hardware information, such as motor torque or a measure of the work used by the motor is an effective manner by which to determine (e.g., estimate) the weight of the item.")
Claim(s) 37 and 47 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Konstantinovich et al. (RU 2575184 C1), hereinafter Konstantinovich.
Regarding claim 37, where all the limitations of claim 28 are discussed above, Sun further teaches:
37. (Previously Presented) The method of claim 28, wherein estimating the one or more mass characteristics of the object (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") …
Sun does not specifically discuss estimating ten mass characteristics comprising one mass parameter, three center of mass parameters, and six moment of inertia parameters. However, Konstantinovich, in the same field of endeavor of identifying physical properties for use in robotics applications, teaches:
… comprises estimating at least ten mass characteristics, wherein the at least ten mass characteristics comprise one mass parameter, three center of mass parameters, and six moment of inertia parameters. (Page 4, Paragraph 1, "The determination of the body inertia tensor is carried out in stages: first, the mass and coordinates of the center of mass of the body in a given plane are determined by any known method, then six axial central moments of inertia are determined from the torques and corresponding angular accelerations, by which the tensor components are calculated inertia according to the method described in [3].")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods including mass property estimation as taught by Sun with the ability to estimate the mass, coordinates of the center of mass, and six axial moments of inertia as taught by Konstantinovich. This would ensure the system is able to adapt to the state of the object and plan an effective trajectory taking into account the each of these properties.
Regarding claim 47, where all the limitations of claim 38 are discussed above, Sun further teaches:
47. (Previously Presented) The mobile manipulation robot of claim 38, wherein estimating the one or more mass characteristics of the object (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") …
Sun does not specifically discuss estimating ten mass characteristics comprising one mass parameter, three center of mass parameters, and six moment of inertia parameters. However, Konstantinovich, in the same field of endeavor of identifying physical properties for use in robotics applications, teaches:
… comprises estimating at least ten mass characteristics, wherein the at least ten mass characteristics comprise one mass parameter, three center of mass parameters, and six moment of inertia parameters. (Page 4, Paragraph 1, "The determination of the body inertia tensor is carried out in stages: first, the mass and coordinates of the center of mass of the body in a given plane are determined by any known method, then six axial central moments of inertia are determined from the torques and corresponding angular accelerations, by which the tensor components are calculated inertia according to the method described in [3].")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operation methods including mass property estimation as taught by Sun with the ability to estimate the mass, coordinates of the center of mass, and six axial moments of inertia as taught by Konstantinovich. This would ensure the system is able to adapt to the state of the object and plan an effective trajectory taking into account the each of these properties.
Claim(s) 48 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Nakatani and in further view of Neville et al. (US 20200306964 A1), hereinafter Neville.
Regarding claim 48, where all the limitations of claim 28 are discussed above, Sun further teaches:
48. (New) The method of claim 28, further comprising:
… while the object is in motion along the trajectory,
wherein estimating the one or more mass characteristics of the object (Paragraph 0050, "In the example shown, one or more of robotic arm 202, end effector 204, and conveyor 208 are operated in coordination by control computer 212. In some implementations, control computer 212 is configured to control a plurality of robotic arms operating at one or more workstations. In various embodiments, a robotic singulation as disclosed herein may include one or more sensors from which an environment of the workspace is modeled. In the example shown in FIG. 2A, system 200 includes image sensors, including in this example 3D cameras 214 and 216. In various embodiments, other types of sensors may be used (individually or in combination) in a singulation system as disclosed herein, including a camera, an infrared sensor array, a laser array, a scale, a gyroscope, a current sensor, a voltage sensor, a power sensor, a force sensor, a pressure sensor, a weight sensor, and the like. In various embodiments, control computer 212 includes a workspace environment state system such as a vision system used to discern individual items, debris on the workspace, and each item's orientation based on sensor data such as image data provided by image sensors, including in this example 3D cameras 214 and 216. The workspace environment state system in some embodiments includes sensors in the robotic arm to detect a weight of an item (e.g., a grasped item) or to detect information from which an estimated weight is determined. For example, information pertaining to an amount of current, voltage, and/or power used by one or more motors driving movement of the robotic arm can be used to determine the weight (or an estimated weight) of the item. As another example, the chute includes a weight sensor, and the weight of the item is determined based on a difference of the weight on the chute as measured by the weight sensor before the item is picked up and after the item is picked up. As another example, information pertaining to an output from one or more sensor arrays can be used to determine a location of the item in the workspace, a location of the item while the item is grasped and/or being moved by the robotic arm, and/or a location of the robotic arm (e.g., based on a determination of an output from a subset of sensors of the one or more sensor arrays compared to another subset of sensors of the one or more sensor arrays). As another example, information pertaining to an output from one or more sensor arrays can be used to determine a dimension or size of an item to be singulated and/or another item or object within the workspace.") …
Sun does not specifically disclose the sensors sensing wrench or using that information to determine the mass characteristics. However, Neville, in the same field of endeavor of robotics, teaches:
… sensing, using one or more sensors of the robot, a wrench applied to the object (Paragraph 0063, “In some examples, such as FIG. 1C, the sensor system 170 includes one or more sensors 172 mounted on or coupled to the wrist joint J.sub.A3 of the arm 150. For instance, a wrist sensor 172 is a six-axis force/torque sensor 172. Here, the sensor 172 senses the force(s) on the box 20 that the robot 100 is holding. The palletizer 200 uses this force on the box 20 that the robot 100 is holding to indirectly detect interaction between the box 20 that the robot 100 is holding and one or more other boxes 20. In some implementations, the palletizer 200 determines (or receives from another system of the robot 100) a velocity for the box 20 that the robot 100 is holding based on the joints J of the robot 100. In other words, with the sensor system 170, the robot 100 may determine the velocity corresponding to each joint J and the drive wheel(s) 130. Based on these velocities, the robot 100 determines how fast the end-effector 160 is moving with the box 20 and uses this to derive the velocity of the box 20.”) … is further based, at least in part, on the sensed wrench. (Paragraph 0077, “In some implementations, the sensors 172 of the robot 100 determine the weight of each previously placed box 20 and use the determined weight to set the threshold contact force F.sub.thresh. In some examples, the palletizer 200 uses a combination of the impedance control and the force feedback control to actively limit the forces exerted by the robot 100 on the boxes 20 (i.e., to set the threshold contact force F.sub.thresh).”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Sun with the ability to utilize a six axis force/torque sensor to determine characteristics of the payload object as taught by Neville. This would allow the system to make informed decisions on operation in order to most effectively complete work objectives and ensure the object is not damaged.
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights