DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims: claims 1-13 are pending below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on April 3rd, 2024 and November 13,2025 was filed and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-7 and 10-13 are rejected under 35 U.S.C. 103 as being unpatentable over Cole et al (US 10,929,811) in view of BUIBAS et al (US 2020/0019921).
Claim 1:
Cole et al (US 10,929,811) teaches the following subject matter:
An operation assist apparatus comprising:
a plurality of storage boxes configured to store an article (figure 1 and column 3 lines 20-40 detail mapping location of pallets 1 and containers 90 to store cartons/articles 3);
a three-dimensional sensor (column 14 lines 55-67 detail sensor 8 that is 3D scanner for plurality of data points, range/distance, angles and surface data such as length, width, height or shape of carton 30) configured to detect a position of at least a part of a first surface of a first object that exists in a plurality of first detection regions respectively provided to the plurality of storage boxes and generate a distance data that indicates a distance to the at least a part of the detected first surface (column 15 lines 1-40, specifically lines 20-40 detail dimension and surface data of carton/object with pair points and scale sensor for position/provide in pallet 1 or container 90); and
a processor configured to judge, based on the position of the at least a part of the first surface, whether or not the first object takes out the article stored in the plurality of storage boxes, or whether or not the first object stores the article into the plurality of storage boxes (column 17 lines 45 – column 18 line 10 detail using exterior surface of carton/article to determine available/unavailable volume/space/position in pallet space 5 or container space 96),
Cole et al teaches all the subject matter above, but not the following:
wherein the processor is further configured to judge, when a number of points that represent the at least a part of the first surface in the distance image-data is greater than a first threshold value, that the first object is picking the article or storing the article into the plurality of storage boxes.
BUIBAS et al (US 2020/0019921) teaches the following subject matter:
wherein the processor is further configured to judge, when a number of points that represent the at least a part of the first surface in the distance data is greater than a first threshold value, that the first object is picking the article or storing the article into the plurality of storage boxes (figure 6b-figure 7A-B and 0166 detail comparison to specific threshold to surface area (3D spatial data as well as distance data) in 3D space for placement in position 701).
Cole et al and BUIBAS et al are both in the field of image analysis, especially the use of scanned 3D space and dimension/shape and volume of article/carton for placement in storage boxes/pallet/container such that the combine outcome is predictable.
Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify Cole et al by BUIBAS et al regarding using points and distance data provides calculating the 3D field for reflecting differences in the uncertainties for tracking the different landmarks for different shapes and sizes as disclosed by BUIBAS et al in 0166.
Claim 2:
Cole et al teaches:
The operation assist apparatus according to claim 1, wherein the processor is further configured to output a distance display image, representing a part of the distance data, that represents an object included in an inputted range of distance, wherein the distance display image is viewed from the three-dimensional sensor (figure 1 and column 3 lines 20-40 detail mapping of location of pallets, contain, cartons, all physical object and their shape, dimension, symmetric and isolating items from the exterior space and handling (range and distance between all physical object)).
Claim 3:
BUIBAS et al teach:
The operation assist apparatus according to claim 1, wherein the processor is further configured to output the number of the points in real time (0011 detail tracking in real time to provide robustness and efficiency or tracking process).
Claim 6:
BUIBAS et al teach:
The operation assist apparatus according to claim 1, wherein the longer the distance from the three-dimensional sensor to a corresponding first detection region of the plurality of first detection regions is, the smaller the first threshold value is (0018 detail consideration of less than threshold in 3D field regarding item, storage space and person utilizing the environment).
Claim 7:
BUIBAS et al teach:
The operation assist apparatus according to claim 1, wherein the processor is further configured to judge, when a number of points that represent the at least a part of the first surface is smaller than a second threshold value in the distance data, that the first object is picking the article or storing the article into the plurality of storage boxes (0018 detail spatial probability regarding item and storage space in terms of less than or equal to threshold as well as threshold probability; 0165-0166 detail specified threshold to a threshold (first and second thresholds)).
Claim 10:
BUIBAS et al teach:
The operation assist apparatus according to claim 1,
wherein the three-dimensional sensor is further configured to detect an operator who picks the article or stores the article (figure 1 part 142 with time where operator 103 with object 111), and
wherein the processor is further configured to: distinguish the operator based on information of the detected operator; and identify the operator who corresponds to the first object detected in the plurality of first detection regions (0145 detail monitoring space, items and person, where figure 1 part 142 with time where operator 103 with object 111 over time)
Claim 11:
Cole et al teaches:
The operation assist apparatus according to claim 1,
wherein the three-dimensional sensor is further configured to detect a position of at least a part of a first article surface of an article that exists in a fullness detection region provided to an opening of the plurality of storage boxes and generate the distance data that indicates a distance to the at least a part of the detected second article surface, and wherein the processor is further configured to judge, when a number of points that represent the at least a part of the first article surface in the distance data is greater than a fullness threshold value, that a storage box of the plurality of storage boxes that corresponds to the fullness detection region is full (column 7 lines 35-50 detail determination the area is unavailable on the pallet space 5 or the container space 96; column 17 line 20 – column 18 line 10; figure 1-6 and column 18 lines 10-40).
Claim 12:
Cole et al teaches:
The operation assist apparatus according to claim 1,
to wherein the three-dimensional sensor is further configured to detect a position of at least a part of a second article surface of an article that exists in an emptiness detection region provided at a bottom of the plurality of storage boxes and generate the distance article surface is smaller than an emptiness threshold value, that a storage box of the plurality of storage boxes that corresponds to the emptiness detection region is empty (column 7 lines 1-17 detail determination the area is available on the pallet space 5 or the container space 96; column 7 lines 35-50; column 17 lines 5-17; column 19 lines 55-65).
Claim 13:
Cole et al (US 10,929,811) teaches the following subject matter:
A non-transitory tangible storage medium storing a control program that makes a processor execute (column 2 lines 5-25 detail memory and processor oof the system):
calculating, in a distance data that indicates a distance from a three- dimensional sensor to at least a part of a first surface of a first object that exists in a plurality of first detection regions respectively provided to a plurality of storage boxes configured to store an article, a number of points that represent the at least a part of the first surface (column 14 lines 55-67 detail sensor 8 that is 3D scanner for plurality of data points, range/distance, angles and surface data such as length, width, height or shape of carton 30; column 15 lines 1-40, specifically lines 20-40 detail dimension and surface data of carton/object with pair points and scale sensor for position/provide in pallet 1 or container 90).
Cole et al teaches all the subject matter above, but not the following:
determining, when the calculated number of points is greater than a first threshold value, that the first object takes out the article or stores the article in the plurality of storage boxes.
BUIBAS et al (US 2020/0019921) teaches the following subject matter:
determining, when the calculated number of points is greater than a first threshold value, that the first object takes out the article or stores the article in the plurality of storage boxes (figure 6b-Figure 7A-B and 0166 detail comparison to specific threshold to surface area (3D spatial data as well as distance data) in 3D space for placement in position 701).
Cole et al and BUIBAS et al are both in the field of image analysis, especially the use of scanned 3D space and dimension/shape and volume of article/carton for placement in storage boxes/pallet/container such that the combine outcome is predictable.
Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify Cole et al by BUIBAS et al regarding using points and distance data provides calculating the 3D field for reflecting differences in the uncertainties for tracking the different landmarks for different shapes and sizes as disclosed by BUIBAS et al in 0166.
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Cole et al (US 10,929,811) and BUIBAS et al (US 2020/0019921) as applied to claim 1 above, and further in view of Kumar (US 11,173,391).
Claim 4:
Cole et al and BUIBAS et al teaches all the subject matter above, but not the following which is taught by Kumar (US 11,173,391):
The operation assist apparatus according to claim 1, wherein the processor is further configured to output a cross-sectional distance image that represents a position that the three-dimensional sensor has detected in a cross-section perpendicular to a surface represented by the distance display image (column 67 lines 30-55 detail sorting to group and boxes, where claim 17 detail scanner data used to generate 3D model or environment and object, where surface of subject and cross-section data are used for placement any 3D space which includes perpendicular).
Cole et al and BUIBAS et al and Kumar are all in the field of image analysis, especially the use of scanned 3D space and dimension/shape and volume of article/carton for placement in storage boxes/pallet/container such that the combine outcome is predictable.
Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify Cole et al and BUIBAS et al by Kumar such would provide sorting process may be adapted to changes in the work flow as disclosed by Kumar in column 67 lines 60-65.
Claim 5:
Kumar teaches:
The operation assist apparatus according to claim 4, wherein the processor is further configured to output an image that represents a cross-section position selector configured to set a position of the cross-section represented in the cross-sectional distance image (column 14 lines 30-60 detail consideration of cross-section and range of field for the edges and x,y coordinates, where claim 17 detail output/display of the 3D model of object and environment to determine location and space to the user’s environment).
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. At the time of examination unable to find prior art and/or concept regarding “wherein at least a part of a shape of one first detection region of the plurality of first detection regions is configured such that a first cross-sectional area of the one first detection region monotonically decreases in a first direction in which the first object advances when the first object takes out the article or when the first object stores the article, the first cross-sectional area being perpendicular to the first direction.” Closest prior art Murphy et al (US 10,471,597) teaches stacking shelves/storage using 3D data of environment and objects but not the rest of the claim limitations.
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. At the time of examination unable to find prior art and/or concept regarding “an article shelf configured to store the plurality of storage boxes to be drawable in forward, wherein the three-dimensional sensor is further configured to detect a position of at least a part of a second surface of a second object that exists in a plurality of second detection regions provided in forward of the plurality of storage boxes, and wherein the processor is further configured to set, when a first storage box of the plurality of storage boxes is drawn in forward, a detection region of the plurality of first detection regions above the first storage box the first object stores the article, the first cross-sectional area being perpendicular to the first direction.” Closest prior art Murphy et al (US 10,471,597) teaches stacking shelves/storage using 3D data of environment and objects but not the rest of the claim limitations.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Murphy et al (US 10,471,597) teaches Adaptive Perception For Industrial Robotic Systems - used by a three-dimensional vision system to generate a three-dimensional model of the item, in other embodiments time-of-flight cameras (e.g., cameras that measure the time-of-flight of a light signal between the camera and a surface of the item) could be used to generated the three-dimensional model of the item. In various other embodiments, a Light Detection and Ranging (LIDAR) system, a structured lighting system (e.g., in conjunction with stereo cameras and a three-dimensional vision system), ultrasound arrays, x-ray imaging systems, millimeter wave scanners, and so on could be used to generate the three-dimensional model of the item (e.g., for determining the item's position and pose, for use in controlling movement of the robotic picking arm 210). More generally, any systems capable of determining physical attributes of an item for use in controlling a robotic device can be used, consistent with the functionality described herein (figure 2 and column 9-10).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSUNG YIN TSAI/Primary Examiner, Art Unit 2656
Prosecution Insights