Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This is a non-final Office Action on the merits in response to communications filed by Applicant on June 19, 2025. Claims 1-49 are currently pending and examined below.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on is/are being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 7, 9, 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
The term “substantially” in claims 1, 7, 9, 11 is a relative term which renders the claim indefinite. The term “substantially” and “transiently coupled” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-49 are rejected under 35 U.S.C. 102(a)(1) and/or 102(a)(2) as being anticipated by Moreno et al. US2022/0297958 (“Moreno”).
Regarding claim(s) 1. Moreno discloses a robotic package handling system (fig. 1), comprising:
an end effector assembly configured to transfer one or more packages from an input assembly to an output assembly ([0058] In the example shown, robotic arm 102 is equipped with a suction-type end effector 108. End effector 108 has a plurality of suction cups 110. Robotic arm 102 is used to position the suction cups 110 of end effector 108 over an item to be picked up, as shown, and a vacuum source provides suction to grasp the item, lift it from conveyor 104, and place it at a destination location on pallet 106.);
a first imaging device positioned and oriented to capture image information pertaining to the one or more packages ([0067]System 100 may obtain attribute information pertaining to one or more items to be palletized/de-palletized. The attribute information may comprise one or more of an orientation of the item, a material (e.g., a packaging type), a size, a weight (or expected weight), or a center of gravity, etc. [0072] At 210, planning (or re-planning) is performed to generate a plan to pick/place items based on the high level objective received at 205 and available sensor information. For example, in the example shown in FIG. 1, 3D or other image data generated by one or more of cameras 112, 114, and 116 may be used, along with sensor data from sensors not shown in FIG. 1 (e.g., weight sensors) to identify and determine a plan to pick, place, and stack on receptacle 106 items arriving via conveyor 104.);
a first computing system operatively coupled to the end effector assembly and the first imaging device, and configured to receive the image information from the first imaging device and command movements of the end effector assembly based at least in part upon the image information ([0059] In various embodiments, one or more of 3D or other camera 112 mounted on end effector 108 and cameras 114, 116 mounted in a space in which robotic system 100 is deployed are used to generate image data used to identify items on conveyor 104 and/or determine a plan to grasp, pick/place, and stack the items on pallet 106 (or place the item in the buffer or staging area, as applicable));
wherein the input assembly is operatively coupled to the first computing system and configured to be operated by the first computing system based at least in part upon the image information to mechanically process a plurality of incoming packages from a substantially disordered mechanical organization to provide a supply of packages to be transferred to the input assembly that is substantially singulated (Fig. 2b, 4b, illustrating an embodiment of a robotic system to singulate items. [0099, 0171, 0216 ] [0099] At 245, the item is singulated. In some embodiments, the item is singulated in response to the plan or strategy for singulating the item being determined. For example, a robotic arm is operated to pick one or more items from the workspace and place each item singly in a corresponding location in a singulation conveyor. The singulation of the item comprises picking the item from the workspace (e.g., from the source pile/flow) and singly placing the item on the conveyor. The robot system singulates the item based at least in part on the plan or strategy for singulating the item.), and
to prune away certain packages which do not become substantially singulated as a result of the mechanical process ([0066] system 100 may simulate removal of one or more items from the pallet and select an order from removing items from the pallet that optimizes the stability of the state of the pallet (e.g., the stack). System 100 may use the model to determine a next item to remove from the pallet. For example, system 100 may select an item as a next item to remove from the pallet based at least in part on a determination that an expected stability of the stack during and/or after removal of the item exceeds a threshold stability.); and
wherein the first computing system is configured to operate the end effector assembly to move a targeted package of the one or more packages from the input assembly based at least in part upon the image information, and release the targeted package to be at least transiently coupled with the output assembly with a position and orientation based at least in part upon the image information (0140, [0227] The system may generate the model of the state of the pallet in connection with determining whether to place an item on the pallet (e.g., on the stack), and selecting a plan for placing the item on the pallet, including a destination location at which the item is to be placed, a trajectory along which the item is to be moved from a source location (e.g., a current destination such as a conveyor) to the destination location. The system may also use the model in connection with determining a strategy for releasing the item, or otherwise placing the item on the pallet (e.g., applying a force to the item to snug the item on the stack).).
Regarding claim(s) 2. Moreno discloses wherein the end effector assembly comprises a first suction cup assembly coupled to a controllably activated vacuum load operatively coupled to the first computing system, the first suction cup assembly configured such that operating the end effector assembly to move a targeted package comprises engaging the targeted package and controllably activating the vacuum load ([0058] In the example shown, robotic arm 102 is equipped with a suction-type end effector 108. End effector 108 has a plurality of suction cups 110. Robotic arm 102 is used to position the suction cups 110 of end effector 108 over an item to be picked up, as shown, and a vacuum source provides suction to grasp the item, lift it from conveyor 104, and place it at a destination location on pallet 106.).
Regarding claim(s) 3. Moreno discloses wherein the end effector assembly further comprises a tray member configured to be at least partially positioned below the targeted package when operating the end effector assembly to move a targeted package ([0103] At 240, a plan or strategy to move one or more items in the workspace is determined. In some embodiments, a robotic system determines the plan or strategy to pick at least one item from a shelf (e.g., a shelf of a shelf machine) in the workspace and to place the at least one item in a receptacle such as a box, tray, tote, bag, or other receptacle. The receptacle may be on a conveyor. The plan or strategy to singulate the one or more items may be determined in various embodiments on a robot by robot basis such that if the robotic system includes a plurality of robots, each robot operates independent of the other robot(s).).
Regarding claim(s) 4. Moreno discloses wherein the first imaging device comprises a camera ([0060] In the example shown, camera 112 is mounted on the side of the body of end effector 108, but in some embodiments camera 112 and/or additional cameras may be mounted in other locations, such as on the underside of the body of end effector 108, e.g., pointed downward from a position between suction cups 110, or on segments or other structures of robotic arm 102, or other locations. In various embodiments, cameras such as 112, 114, and 116 may be used to read text, logos, photos, drawings, images, markings, barcodes, QR codes, or other encoded and/or graphical information or content visible on and/or comprising items on conveyor 104.).
Regarding claim(s) 5. Moreno discloses wherein the first imaging device comprises a stereoscopic camera assembly ([0060] In the example shown, camera 112 is mounted on the side of the body of end effector 108, but in some embodiments camera 112 and/or additional cameras may be mounted in other locations, such as on the underside of the body of end effector 108, e.g., pointed downward from a position between suction cups 110, or on segments or other structures of robotic arm 102, or other locations. In various embodiments, cameras such as 112, 114, and 116 may be used to read text, logos, photos, drawings, images, markings, barcodes, QR codes, or other encoded and/or graphical information or content visible on and/or comprising items on conveyor 104.).
Regarding claim(s) 6. Moreno discloses wherein the first imaging device comprises a depth camera ([0059] In various embodiments, one or more of 3D or other camera 112 mounted on end effector 108 and cameras 114, 116 mounted in a space in which robotic system 100 is deployed are used to generate image data used to identify items on conveyor 104).
Regarding claim(s) 7. Moreno discloses wherein the input assembly is configured to be operated by the first computing system to control the supply of packages based at least in part upon a number of the one or more packages transiently coupled to output assembly (0073] In some embodiments, the system may make a plan based at least in part on a manifest (e.g., invoice or other list corresponding to an order, etc.). A plan to stack items may be generated based on item size, weight, density, weight distribution, rigidity, capacity of the item and/or its box or other packaging to support weight stacked on top of the item, etc. The control computer in some embodiments controls the order in which items arrive at the loading location, such as via conveyor 104 (or via one or more conveyors)).
Regarding claim(s) 8. Moreno discloses wherein the input assembly is configured to be operated by the first computing system to control the supply of packages based at least in part upon the image information pertaining to the one or more packages at the input assembly ([0059] In various embodiments, one or more of 3D or other camera 112 mounted on end effector 108 and cameras 114, 116 mounted in a space in which robotic system 100 is deployed are used to generate image data used to identify items on conveyor 104 and/or determine a plan to grasp, pick/place, and stack the items on pallet 106 (or place the item in the buffer or staging area, as applicable).).
Regarding claim(s) 9. Moreno discloses wherein the input assembly comprises one or more mechanical singulation elements configured to mechanically process and direct the substantially singulated supply of packages toward the output assembly ([0093] FIG. 2B is a flow chart illustrating an embodiment of a process to singulate an item. In some embodiments, process 225 is implemented by a robot system operating to singulate one or more items within a workspace. The robot system includes one or more processors which operate, including by performing the process 225, to cause a robotic structure (e.g., a robotic arm) to pick and place items for sorting.).
Regarding claim(s) 10. Moreno discloses wherein the one or more mechanical singulation elements are selected from the group consisting of: a ramp sequence; a vibratory actuator; a belt; a coordinated plurality of belts; a ball sorter conveyor; a stepsequence; a chute with one or more 90-degree turns; a mechanical diverter; a vertical mechanical filter; and a horizontal mechanical filter ([0036] As used herein, singulation of an item includes picking an item from a source pile/flow and placing the item on a conveyance structure (e.g., a segmented conveyor or similar conveyance). Optionally, singulation may include sortation of the various items on the conveyance structure such as via singly placing the items from the source pile/flow into a slot or tray on the conveyor.).
Regarding claim(s) 11. Moreno discloses wherein the input assembly is configured to be operated by the first computing system to prune away certain packages which do not become substantially singulated as a result of the mechanical process using a diversion element configured to selectably divert one or more targeted packages ([0066] Conversely, in the context of de-palletizing one or more items from a pallet (e.g., a stack on the pallet), system 100 may generate the model of the state of the pallet in connection with determining whether to remove an item on the pallet (e.g., on the stack), and selecting a plan for removing the item from the pallet. The model of the state of the pallet may be used in connection with determining an order in which items are removed from the pallet. For example, system 100 may use the model to determine whether removal of an item is expected to cause stability of the state of the pallet (e.g., the stack) to drop below a threshold stability.).
Regarding claim(s) 12. Moreno discloses a second image capture device operatively coupled to the first computing system and configured to capture information pertaining to the one or more packages on the input assembly ([0158] In various embodiments, one or more sensors (e.g., camera 480) is used to capture information pertaining to items associated with kitting shelf system 102. For example, camera 480 may capture an image of one or more items on shelf 456, shelf 458, and/or shelf 460. As another example, if an item on a shelf 456, shelf 458, and/or shelf 460 is a tray or other receptacle, camera 480 may capture information pertaining to objects within the tray.).
Regarding claim(s) 13. Moreno discloses a second image capture device operatively coupled to the first computing system and configured to capture information pertaining to the one or more packages on the output assembly ([0158] In various embodiments, one or more sensors (e.g., camera 480) is used to capture information pertaining to items associated with kitting shelf system 102. For example, camera 480 may capture an image of one or more items on shelf 456, shelf 458, and/or shelf 460. As another example, if an item on a shelf 456, shelf 458, and/or shelf 460 is a tray or other receptacle, camera 480 may capture information pertaining to objects within the tray.).
Regarding claim(s) 14. Moreno discloses wherein the first image capture device is positioned and oriented to capture information pertaining to packages being moved from the input assembly to the output assembly ([0051] The robotic system may use the information obtained by the one or more sensors or sensor arrays to determine an orientation of a pallet in a corresponding predefined zone, to determine whether an inserted pallet is properly oriented, or otherwise determine to calibrate the robotic system with respect to the pallet inserted into corresponding predefined zone. As an example, in response to determining that the robotic system is to be calibrated with the respect to the pallet, the robotic system may provide a notification to a terminal that the pallet is to be reoriented (e.g., a notification that requests/alerts a human operator to reorient the pallet). As another example, in response to determining that the robotic system is to be calibrated with the respect to the pallet, the robotic system may determine an offset of a current orientation of the pallet (e.g., with respect to a proper orientation of the pallet), and determine a plan or strategy for placing items on the pallet based at least in part on the offset of the current orientation.).
Regarding claim(s) 15. Moreno discloses wherein the input assembly comprises an electromechanical conveyance ([0134] In this example, items are fed into chute 429 from an intake end 430. For example, one or more human and/or robotic workers may feed items into intake end 430 of chute 429, either directly or via a conveyor or other electro-mechanical structure configured to feed items into chute 429.).
Regarding claim(s) 16. Moreno discloses wherein at least a portion of the conveyance comprises multi-axis electromechanical conveyance ([0134] In this example, items are fed into chute 429 from an intake end 430. For example, one or more human and/or robotic workers may feed items into intake end 430 of chute 429, either directly or via a conveyor or other electro-mechanical structure configured to feed items into chute 429.).
Regarding claim(s) 17. Moreno discloses wherein the output assembly is configured to controllably release the targeted package to an output container ([0065] In the context of palletizing one or more items, system 100 may generate the model of the state of the pallet in connection with determining whether to place an item on the pallet (e.g., on the stack), and selecting a plan for placing the item on the pallet, including a destination location at which the item is to be placed, a trajectory along which the item is to be moved from a source location (e.g., a current destination such as a conveyor) to the destination location. System 100 may also use the model in connection with determining a strategy for releasing the item, or otherwise placing the item on the pallet (e.g., applying a force to the item to snug the item on the stack). The modelling of the state of the pallet may include simulating placement of the item at different destination locations on the pallet (e.g., on the stack) and determining corresponding different expected fits and/or expected stability (e.g., a stability metric) that is expected to result from placement of the item at the different locations.).
Regarding claim(s) 18. Moreno discloses wherein the output container is selected from the group consisting of: a bin, a sack, a tote, a gaylord container, a chute, a conveyance ([0139] As illustrated in FIG. 4B, path/trajectory 434 starts from a location from which the item is to be picked, the path/trajectory along which the item is moved to conveyor 427, and a location and time at which the item is to be released to coincide in space and time with a slot, tray, or other destination selected as the location at which the item is to be placed on the destination conveyance structure, i.e., slot/tray 432 of conveyor 427 in this example.).
Regarding claim(s) 19. Moreno discloses wherein the first computing system is operatively coupled to the output assembly ([0139] As illustrated in FIG. 4B, path/trajectory 434 starts from a location from which the item is to be picked, the path/trajectory along which the item is moved to conveyor 427, and a location and time at which the item is to be released to coincide in space and time with a slot, tray, or other destination selected as the location at which the item is to be placed on the destination conveyance structure, i.e., slot/tray 432 of conveyor 427 in this example.).
Regarding claim(s) 20. Moreno discloses wherein the output assembly is configured to controllably release the targeted package to a distribution module, the distribution module being operatively coupled to the first computing system and configured to controllably release the targeted package to an output container ([0139] As illustrated in FIG. 4B, path/trajectory 434 starts from a location from which the item is to be picked, the path/trajectory along which the item is moved to conveyor 427, and a location and time at which the item is to be released to coincide in space and time with a slot, tray, or other destination selected as the location at which the item is to be placed on the destination conveyance structure, i.e., slot/tray 432 of conveyor 427 in this example.).
Regarding claim(s) 21. Moreno discloses wherein the output container is selected from the group consisting of: a bin, a sack, a tote, a gaylord container, a chute, a conveyance ([0139] As illustrated in FIG. 4B, path/trajectory 434 starts from a location from which the item is to be picked, the path/trajectory along which the item is moved to conveyor 427, and a location and time at which the item is to be released to coincide in space and time with a slot, tray, or other destination selected as the location at which the item is to be placed on the destination conveyance structure, i.e., slot/tray 432 of conveyor 427 in this example.).
Regarding claim(s) 22. Moreno discloses wherein the distribution module comprises a tray or bin movably coupled to a transport assembly coupled between the distribution module and the output assembly ([0139] As illustrated in FIG. 4B, path/trajectory 434 starts from a location from which the item is to be picked, the path/trajectory along which the item is moved to conveyor 427, and a location and time at which the item is to be released to coincide in space and time with a slot, tray, or other destination selected as the location at which the item is to be placed on the destination conveyance structure, i.e., slot/tray 432 of conveyor 427 in this example.).
Regarding claim(s) 23. Moreno discloses wherein the distribution module is movably coupled to the transport assembly using a pan-tilt actuation assembly operatively coupled to the first computing system ([0152] For example, the items shown in FIG. 4C as being supplied via kitting shelf system 454 may be supplied in some embodiments via a stationary ramp down which the items are rolled. In some embodiments, a kitting shelf system may comprise any one of a plurality of structures and mechanisms to supply items to an associated pick zone, including without limitation a gravity type conveyor having a plurality of adjacent rollers, a ramp, a conveyor belt, a set of revolving bins, etc.).
Regarding claim(s) 24. Moreno discloses wherein the transport assembly comprises an elevated gantry assembly configured to have physical access to an array of available output containers ([0152] For example, the items shown in FIG. 4C as being supplied via kitting shelf system 454 may be supplied in some embodiments via a stationary ramp down which the items are rolled. In some embodiments, a kitting shelf system may comprise any one of a plurality of structures and mechanisms to supply items to an associated pick zone, including without limitation a gravity type conveyor having a plurality of adjacent rollers, a ramp, a conveyor belt, a set of revolving bins, etc.).
Regarding claim(s) 25. Moreno discloses wherein the transport assembly comprises a rail assembly configured to have physical access to an array of available output containers ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 26. Moreno discloses wherein a plurality of output containers comprising the array of available output containers may be coupled together as an assembly to be transferred away from the output assembly together ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 27. Moreno discloses wherein a plurality of output containers comprising the array of available output containers may be removably coupled together as an assembly to be transferred away from the output assembly together ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 28. Moreno discloses wherein the distribution module is configured to be able to controllably release the targeted package to a targeted output container selected from an array of available output containers ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 29. Moreno discloses wherein the first computing system is configured to operate the distribution module to sequence packages into one a plurality of targeted output containers selected from the array of available output containers based upon a multi-pass sorting algorithm ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 30. Moreno discloses wherein the multi-pass sorting algorithm comprises a hash-sort configuration selected to facilitate efficient sequenced unloading of the plurality of targeted output containers ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 31. Moreno discloses wherein the first computing system is configured to automatically controllably release the targeted package to a targeted output container selected from an array of available output containers based at least in part upon image information from the first image capture device ([0250] In the example shown, at 1010 image data is received and processed. For example, image data from two or more cameras may be received and merged to generate a composite set of points in three dimensional space. [0251] At 1020, image data received and processed at 1010 is fitted to a model. For example, the composite set of points may be compared to corresponding data comprising a library of item models and a “best fit” or “best match” model may be determined. In some embodiments, the library of item models is dynamically updated based at least in part on a machine learning process that uses information from historical models and results of implementation of such models to improve or update. [0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item.).
Regarding claim(s) 32. Moreno discloses wherein the first computing system is configured to automatically controllably release the targeted package to a targeted output container selected from an array of available output containers based at least in part upon image information from the first image capture device which may be utilized by the first computing system to estimate a factor selected from the group consisting of: package geometry; package compliance; estimated package bounding box geometry;package relative mass; relative rigidity of package material; package mechanical stability; and package moment of inertia ([0067] According to various embodiments, a machine learning process is implemented in connection with improving grasping strategies (e.g., strategies for grasping an item). System 100 may obtain attribute information pertaining to one or more items to be palletized/de-palletized. The attribute information may comprise one or more of an orientation of the item, a material (e.g., a packaging type), a size, a weight (or expected weight), or a center of gravity, etc.).
Regarding claim(s) 33. Moreno discloses further comprising a second image capture device fixedly coupled to the distribution module and configured to provide image information pertaining to movement of packages between the distribution module and the array of available output containers ([0252] At 1030, gaps in the image data are filled, e.g., using data from the best fit model, and a bounding container (e.g., a bounding “box” for an item that is rectangular or nearly so in all dimensions) is determined for the item. [0253] Optionally, at 1040, a graphical depiction of the bounding container is superimposed on the composite and/or raw image of the item (e.g., raw video) to provide a composite display, e.g., to a human user monitoring the operation. [0254] At 1050, the model determined at 1020 and the bounding container determined at 830 are used to determine a destination location at which to place the item (e.g., on a pallet). In some embodiments, the bounding container may be displayed in the determined placement location, e.g., in a visual display and/or representation of the operation to pick and place the item.).
Regarding claim(s) 34. Moreno discloses wherein the second image capture device is further configured to provide image information pertaining to interior portions defined within output containers comprising the array of available output containers ([0249] FIG. 10 is a flow chart illustrating an embodiment of a process to place items on a pallet. In various embodiments, the process 1000 is performed by a control computer, such as control computer 118 of FIG. 1 and/or control computer of system 400 of FIG. 4A, based on sensor data, such as image data generated by sensors in the workspace such as cameras 112, 114, and 116. In various embodiments, the process 1000 is used to determine item attributes and placement dynamically, e.g., based on item size and shape as determined based at least in part on image data.).
Regarding claim(s) 35. Moreno discloses wherein the second image capture device is configured to provide image information pertaining to interior portion factors selected from the group consisting of: shape of interiors of output containers, geometry of items within output containers, and fill level within output containers (([0064] the model may be generated based at least in part on input from (e.g., information obtained from) one or more sensors in system 100 such as one or more sensors or sensor arrays within workspace of robot 102. The model may be generated based at least in part on a geometry of the stack, a vision response (e.g. information obtained by one or more sensors in the workspace), and the machine learning processes, etc. System 100 may use the model in connection with determining an efficient (e.g., maximizing/optimizing an efficiency) manner for palletizing/de-palletizing one or more items, and the manner for palletizing/de-palletizing may be bounded by a minimum threshold stability value.).).
Regarding claim(s) 36. Moreno discloses wherein the first computing system is configured to operate the distribution module to place the targeted package into the targeted output container with a position and orientation automatically selected to optimize geometric fit of the targeted package within the targeted output container ([0051] As an example, in response to determining that the robotic system is to be calibrated with the respect to the pallet, the robotic system may provide a notification to a terminal that the pallet is to be reoriented (e.g., a notification that requests/alerts a human operator to reorient the pallet). As another example, in response to determining that the robotic system is to be calibrated with the respect to the pallet, the robotic system may determine an offset of a current orientation of the pallet (e.g., with respect to a proper orientation of the pallet), and determine a plan or strategy for placing items on the pallet based at least in part on the offset of the current orientation.)
Regarding claim(s) 37. Moreno discloses wherein the first computing system is configured to operate the distribution module to sequence packages into one a plurality of targeted output containers selected from the array of available output containers based upon a known pattern regarding a planned unloading of the plurality of targeted output containers ([0084] In some embodiments, a pallet may be deemed to correspond to a robotic arm if the pallet is currently in a predefined zone for the robotic arm and/or planned to be inserted into a predefined zone at a future time when a predefined zone is available. In such as case, the robotic system may determine that the item is to be placed on such a to-be inserted pallet, and the robotic system may determine the plan for picking and placing the item to comprise picking the item from the conveyor and placing the item in a buffer or staging area until such time that the corresponding pallet is inserted into the predefined zone and the item is determined to be placed on such pallet.).
Regarding claim(s) 38. Moreno discloses wherein the first computing system is configured to operate the distribution module to sequence packages into one a plurality of targeted output containers selected from the array of available output containers based upon a known pattern regarding a planned relative positioning of the plurality of targeted output containers within a delivery vehicle ([0050] In some embodiments, the robotic system determines that place an item in a buffer or predefined staging area. The robotic system may determine that a fit of the item on a stack on one or more pallets associated with the manifest and/or order to which the item corresponds is not currently ideal (e.g., that a threshold fit or threshold stability would not be achieved if the item were to be placed on the stack)).
Regarding claim(s) 39. Moreno discloses wherein the first computing system is configured to operate the distribution module to sequence packages into one a plurality of targeted output containers selected from the array of available output containers based upon a known delivery route pertaining to a planned unloading of the plurality of targeted output containers ([0132] In the example of system 400 in FIG. 4A, if the item is determined to be moved to pallet 402, then the system may determine that the cost of moving the item from conveyor 408 is lower than if the item were to be moved from conveyor 410. Accordingly, the system may control for the item to be routed to conveyor 408 (e.g., by controlling a gating structure, by picking and placing the item from an original source location onto conveyor 408, etc.).).
Regarding claim(s) 40. Moreno discloses wherein the input assembly comprises a herringbone conveyance operatively coupled to the first computing system and configured to automatically position the one or more packages in a central position within reach of the end effector assembly ([0049] In various embodiments, a robotic system determines a high level plan to pick an item from a conveyance structure (hereinafter “conveyor”) of one or more conveyors, and to place the item on a pallet of one or more pallets disposed within range of a robotic arm. The robotic system determines the pallet on which to place the item. For example, the robotic system determines the pallet on which to place the item based at least in part on an order and/or manifest associated with the pallet.).
Regarding claim(s) 41. Moreno discloses wherein the first computing system is configured to operate the herringbone conveyance based at least in part upon the image information from the first image capture device ([0036] As used herein, singulation of an item includes picking an item from a source pile/flow and placing the item on a conveyance structure (e.g., a segmented conveyor or similar conveyance). Optionally, singulation may include sortation of the various items on the conveyance structure such as via singly placing the items from the source pile/flow into a slot or tray on the conveyor.).
Regarding claim(s) 42. Moreno discloses wherein the input assembly comprises an input conveyance and a plurality of movable members configured to extend above the input conveyance, the plurality of movable members and input conveyance being operatively coupled to the first computing system and configured to automatically reposition the one or more packages when on the input conveyance ([0062] In the example shown, control computer 118 is connected to an “on demand” teleoperation device 122. In some embodiments, if control computer 118 cannot proceed in a fully automated mode, for example, a strategy to grasp, move, and place an item cannot be determined and/or fails in a manner such that control computer 118 does not have a strategy to complete picking and placing the item in a fully automated mode, then control computer 118 prompts a human user 124 to intervene.)
Regarding claim(s) 43. Moreno discloses wherein the first computing system is configured to operate the plurality of movable members and input conveyance based at least in part upon the image information from the first image capture device ([0249] FIG. 10 is a flow chart illustrating an embodiment of a process to place items on a pallet. In various embodiments, the process 1000 is performed by a control computer, such as control computer 118 of FIG. 1 and/or control computer of system 400 of FIG. 4A, based on sensor data, such as image data generated by sensors in the workspace such as cameras 112, 114, and 116. In various embodiments, the process 1000 is used to determine item attributes and placement dynamically, e.g., based on item size and shape as determined based at least in part on image data.).
Regarding claim(s) 44. Moreno discloses wherein the first image capture device is configured to capture image information pertaining to the one or more packages as they are positioned upon the input assembly ([0059] In various embodiments, one or more of 3D or other camera 112 mounted on end effector 108 and cameras 114, 116 mounted in a space in which robotic system 100 is deployed are used to generate image data used to identify items on conveyor 104 and/or determine a plan to grasp, pick/place, and stack the items on pallet 106 (or place the item in the buffer or staging area, as applicable).).
Regarding claim(s) 45. Moreno discloses wherein the first computing system is configured to establish an identity for each of the one or more packages based at least in part upon the image information captured from the input assembly ([0059] In various embodiments, one or more of 3D or other camera 112 mounted on end effector 108 and cameras 114, 116 mounted in a space in which robotic system 100 is deployed are used to generate image data used to identify items on conveyor 104 and/or determine a plan to grasp, pick/place, and stack the items on pallet 106 (or place the item in the buffer or staging area, as applicable).).
Regarding claim(s) 46. Moreno discloses wherein the first computing system is configured to utilize the established identity in releasing the targeted package to the output assembly ([0093] FIG. 2B is a flow chart illustrating an embodiment of a process to singulate an item. In some embodiments, process 225 is implemented by a robot system operating to singulate one or more items within a workspace. The robot system includes one or more processors which operate, including by performing the process 225, to cause a robotic structure (e.g., a robotic arm) to pick and place items for sorting.).
Regarding claim(s) 47. Moreno discloses wherein the first computing system is configured to utilize the established identity in releasing the targeted package to the output assembly based at least in part upon characteristics of the targeted package which may be associated with the established identity ([0093] FIG. 2B is a flow chart illustrating an embodiment of a process to singulate an item. In some embodiments, process 225 is implemented by a robot system operating to singulate one or more items within a workspace. The robot system includes one or more processors which operate, including by performing the process 225, to cause a robotic structure (e.g., a robotic arm) to pick and place items for sorting.).
Regarding claim(s) 48. Moreno discloses wherein the first computing system may be configured to associated other predetermined information pertaining to the one or more packages from another operatively coupled computing system based at least in part upon the image information captured from the input assembly (0067] According to various embodiments, a machine learning process is implemented in connection with improving grasping strategies (e.g., strategies for grasping an item). System 100 may obtain attribute information pertaining to one or more items to be palletized/de-palletized. The attribute information may comprise one or more of an orientation of the item, a material (e.g., a packaging type), a size, a weight (or expected weight),).
Regarding claim(s) 49. Moreno discloses wherein the input assembly comprises a primary item processing / input system ([0067] System 100 may also obtain a source location (e.g., information pertaining to the input conveyor from which the item is to be picked), and may obtain information pertaining to a pallet on which the item is to be placed (or set of pallets from which the destination pallet is to be determined such as a set of pallets corresponding to the order for which the item is being stacked). In connection with determining a plan for picking and placing the item,).
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRUC M DO whose telephone number is (571)270-5962. The examiner can normally be reached on 9AM-6PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramón Mercado, Ph.D. can be reached on (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRUC M DO/Primary Examiner, Art Unit 3658
Prosecution Insights