DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 as originally filed are pending and have been considered as follows.
Priority
1. Acknowledgement is made that this application is a continuation of U.S. Patent Application No. 17/537,798 filed on 11/30/2021 which claim priority to U.S. Provisional Application No. 63/253,867 filed on 10/08/2021 and U.S. Provisional Application No. 63/119,533 filed on 11/30/2020.
Information Disclosure Statement
2. The information disclosure statements (IDS) filed on 12/13/2024, 02/21/2025, 04/22/2025, and 06/24/2025 are being considered by the examiner.
Claim Objections
3. Claims 4 and 14 are objected to because of the following informalities:
Claim 4 recites the limitation “each first arm”. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, claim 4 is dependent on claim 3.
Claim 14 recites the limitation “each first arm”. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, claim 4 is dependent on claim 13.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
4. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
5. Claims 12-13, 15-16, and 18-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Specification (citation to US pub. No. 20240359327) is utilized for the description citations below.
Regarding claim 12, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic to" as "the robotic control system logic”.
Regarding claim 13, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
Regarding claim 15, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
Regarding claim 16, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
Regarding claim 18, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
Regarding claim 19, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
Regarding claim 20, the phrase “logic” is unclear. It is unclear if this is referring to the same “robotic control system logic” introduced earlier or new a new logic. For examination purposes, examiner has interpreted "logic" as "the robotic control system logic”.
In the art rejection above, the claims have been treated as best understood by the examiner. Any claim not explicitly rejected under this heading is rejected as being dependent on an indefinite claim.
Double Patenting
6. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-5, 9-13, and 15-24 of U.S. Patent No. 12064880. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims cover substantially the same scope.
Refer to the table below to see claim mapping for double patenting:
Instant Application -18/767,784
U.S. Patent No. 12064880
1. A method comprising:
associating each of a plurality of object categories for objects in an environment with corresponding containers situated in the environment;
1. A method comprising:
associating each of a plurality of object categories for objects in an environment with corresponding containers situated in the environment;
activating a robot at a base station;
activating a robot at a base station;
navigating the robot around the environment using cameras to map a type, a size, and a location of the objects in the environment;
navigating the robot around the environment using cameras to map a type, a size, and a location of the objects;
for each object category:
for each object category:
choosing one or more of the objects to pick up in the category;
choosing one or more of the objects to pick up in the category;
performing path planning to the one or more of the objects to pick up;
performing path planning from a current location of the robot to one or more of the objects to pick up;
navigating to points adjacent to each of the one or more of the objects to pick up;
navigating to an adjacent point of one or more of the objects to pick up;
actuating manipulators coupled at both sides of a front end of a bucket at a front end of the robot,
wherein the manipulators are coupled to open and close across the front end of the bucket to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket;
actuating manipulators to open and close across a front of a bucket at a front end of the robot to move obstacles out of the way and manipulate the one or more objects onto the bucket,
wherein the manipulators include a first manipulator member coupled to a first manipulator pivot point adjacent to a first front side of the bucket and a second manipulator member coupled to a second manipulator pivot point adjacent to a second front side of the bucket, and
wherein the first manipulator pivot point allows the first manipulator member to open and close across the first front side of the bucket and the second manipulator pivot point allows the second manipulator member to open and close across the second front side of the bucket;
one or both of tilting and raising the bucket, and actuating the manipulators to retain the objects in the bucket;
one or both of tilting or raising the bucket and actuating the manipulators to retain the objects in the bucket;
navigating the robot adjacent to the corresponding container for the category;
navigating the robot adjacent to the corresponding container for the category;
aligning a back end of the robot with a side of the corresponding container; and
aligning a back end of the robot with a side of the corresponding container; and
one or both of tilting and raising the bucket along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot to deposit retained objects in the corresponding container.
raising the bucket over the robot and toward the back end of the robot to deposit retained objects in the corresponding container.
2. The method of claim 1, further comprising:
operating the robot to organize the objects in the environment into clusters, where each cluster comprises only objects from one of the categories.
2. The method of claim 1, further comprising:
operating the robot to organize the objects in the environment into clusters, where each cluster comprises only objects from one of the categories.
3. The method of claim 1, further comprising:
operating at least one first arm to actuate the manipulators of the robot to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket; and
operating at least one second arm to tilt or raise the bucket.
3. The method of claim 1, further comprising:
operating at least one first arm to actuate the manipulators of the robot to move obstacles out of the way and manipulate the one or more objects onto the bucket; and
operating at least one second arm to tilt or raise the bucket along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot.
4. The method of claim 1, where each first arm is paired with a corresponding second arm, and further comprising:
operating each pairing of first arm and second arm from a common originating pivot point.
4. The method of claim 3, where each first arm is paired with a corresponding second arm, and further comprising:
operating each pairing of first arm and second arm from a common originating pivot point.
5. The method of claim 1, wherein actuating the manipulators of the robot to move obstacles out of the way comprises actuating the manipulators to form a wedge in front of the bucket, and
wherein the wedge formed in front of the bucket is configurable to form an opening sized to allow passage of a target object between the manipulators while preventing collection of unwanted objects.
5. The method of claim 1, wherein actuating the manipulators of the robot to move obstacles out of the way comprises actuating the manipulators to form a wedge in front of the bucket, and
wherein the wedge formed in front of the bucket is configurable to form an opening sized to allow passage of a target object between the manipulators while preventing collection of unwanted objects.
6. The method of claim 5, further comprising:
actuating the manipulators to form the wedge having the opening sized to allow passage of the target object;
driving forward so that the target object passes between the first manipulator and the second manipulator, the opening size preventing the collection of the unwanted objects.
7. The method of claim 6, further comprising:
actuating the first manipulator and the second manipulator to push the target object onto the bucket.
8. The method of claim 6, further comprising:
actuating the manipulators to form the wedge having the opening sized to allow passage of a next target object;
driving forward so that the next target object passes between the first manipulator and the second manipulator, the opening size preventing the collection of the unwanted objects.
6. The method of claim 1, wherein actuating the manipulators to retain the objects in the bucket comprises actuating the manipulators to form a barrier in front of the bucket.
9. The method of claim 1, wherein actuating the manipulators to retain the objects in the bucket comprises actuating the manipulators to form a barrier in front of the bucket.
7. The method of claim 1, further comprising:
operating a neural network to determine the type, size and location of the objects from images from the cameras.
10. The method of claim 1, further comprising:
operating a neural network to determine the type, size and location of the objects from images from the cameras.
8. The method of claim 1, further comprising:
generating scale invariant keypoints within a decluttering area of the environment based on input from a left camera and a right camera;
11. The method of claim 1, further comprising:
generating scale invariant keypoints within a decluttering area of the environment based on input from a left camera and a right camera;
detecting locations of the objects in the decluttering area based on the input from the left camera and the right camera, thereby defining starting locations;
detecting locations of the objects in the decluttering area based on the input from the left camera and the right camera, thereby defining starting locations;
classifying the objects into the categories;
classifying the objects into the categories;
generating re-identification fingerprints for the objects, wherein the re-identification fingerprints are used to determine visual similarity between the objects;
generating re-identification fingerprints for the objects, wherein the re-identification fingerprints are used to determine visual similarity between the objects;
localizing the robot within the decluttering area based on input from at least one of the left camera, the right camera, light detecting and ranging (LIDAR) sensors, and inertial measurement unit (IMU) sensors, to determine a robot location;
localizing the robot within the decluttering area based on input from at least one of the left camera, the right camera, light detecting and ranging (LIDAR) sensors, and inertial measurement unit (IMU) sensors, to determine a robot location;
mapping the decluttering area to create a global area map including the scale invariant keypoints, the objects, and the starting locations; and
mapping the decluttering area to create a global area map including the scale invariant keypoints, the objects, and the starting locations; and
re-identifying the objects based on at least one of the starting locations, the categories, and the re-identification fingerprints.
re-identifying the objects based on at least one of the starting locations, the categories, and the re-identification fingerprints.
9. The method of claim 8, further comprising:
assigning persistent unique identifiers to the objects;
12. The method of claim 11, further comprising:
assigning persistent unique identifiers to the objects;
receiving a camera frame from an augmented reality robotic interface installed as an application on a mobile device;
receiving a camera frame from an augmented reality robotic interface installed as an application on a mobile device;
updating the global area map with the starting locations and the scale invariant keypoints using a camera frame to global area map transform based on the camera frame; and
updating the global area map with the starting locations and the scale invariant keypoints using a camera frame to global area map transform based on the camera frame; and
generating indicators for the objects, wherein the indicators include one or more of next target, target order, dangerous, too big, breakable, messy, and blocking travel path.
generating indicators for the objects, wherein indicators include one or more of next target, target order, dangerous, too big, breakable, messy, and blocking travel path.
10. The method of claim 9, further comprising:
transmitting the global area map and object details to the mobile device, wherein the object details include at least one of visual snapshots, the categories, the starting locations, the persistent unique identifiers, and the indicators of the objects;
13. The method of claim 12, further comprising:
transmitting the global area map and object details to the mobile device, wherein the object details include at least one of visual snapshots, the categories, the starting locations, the persistent unique identifiers, and the indicators of the objects;
displaying the updated global area map, the objects, the starting locations, the scale invariant keypoints, and the object details on the mobile device using the augmented reality robotic interface;
displaying the updated global area map, the objects, the starting locations, the scale invariant keypoints, and the object details on the mobile device using the augmented reality robotic interface;
accepting inputs to the augmented reality robotic interface, wherein the inputs indicate object property overrides including change object category, put away next, don't put away, and modify user indicator;
accepting inputs to the augmented reality robotic interface, wherein the inputs indicate object property overrides including change object category, put away next, don't put away, and modify user indicator;
transmitting the object property overrides from the mobile device to the robot; and
transmitting the object property overrides from the mobile device to the robot; and
updating the global area map, the indicators, and the object details based on the object property overrides.
updating the global area map, the indicators, and the object details based on the object property overrides.
14. The method of claim 1, wherein the manipulators further include:
a first manipulator arm coupled to a first side of the robot at a first arm pivot point; and
a second manipulator arm coupled to a second side of the robot at a second arm pivot point;
wherein the first manipulator arm is coupled to the first manipulator member at the first manipulator pivot point and the second manipulator arm is coupled to the second manipulator member at the second manipulator pivot point.
11. A robotic system comprising:
a robot;
15. A robotic system comprising:
a robot;
a base station;
a base station;
a plurality of corresponding containers each associated with one or more object categories for objects in an environment;
a plurality of containers each associated with one or more object categories;
a mobile application; and
a mobile application; and
robotic control system logic configured to:
robotic control system logic configured to:
navigate the robot around the environment using cameras to map a type, a size, and a location of the objects in the environment;
navigate the robot around an environment comprising a plurality of objects to map a type, a size, and a location of objects;
for each object category:
for each of the categories:
choose one or more of the objects to pick up in the category;
choose one or more of the objects to pick up in the category;
perform path planning to the one or more of the objects to pick up;
perform path planning to the objects to pick up;
navigate to points adjacent to each of the one or more of the objects to pick up;
navigate to points adjacent to each of the objects to pick up;
actuate manipulators coupled at both sides of a front end of a bucket at a front end of the robot,
wherein the manipulators are coupled to open and close across the front end of the bucket to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket;
actuate manipulators to open and close across a front of a bucket at a front end of the robot to move obstacles out of the way and manipulate the one or more objects onto the bucket;
wherein the manipulators include a first manipulator member coupled to a first manipulator pivot point adjacent to a first front side of the bucket and a second manipulator member coupled to a second manipulator pivot point adjacent to a second front side of the bucket, and
wherein the first manipulator pivot point allows the first manipulator member to open and close across the first front side of the bucket and the second manipulator pivot point allows the second manipulator member to open and close across the second front side of the bucket;
one or both of tilt and raise the bucket, and actuate the manipulators to retain the one or more of the objects to pick up in the bucket;
one or both of tilt and raise the bucket, and actuate the manipulators to retain the objects to pick up in the bucket;
navigate the robot adjacent to the corresponding container for the category;
navigate the robot adjacent to the corresponding container for the category;
align a back end of the robot with a side of the corresponding container; and
align a back end of the robot with a side of the corresponding container; and
one or both of tilt and raise the bucket along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot to deposit retained objects in the corresponding container.
raise the bucket over the robot and toward the back end of the robot along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot to deposit retained objects in the corresponding container.
12. The system of claim 11, further comprising logic to operate the robot to organize the objects in the environment into clusters, where each cluster comprises only objects from one of the categories.
16. The system of claim 15, the robotic control system logic further configured to operate the robot to organize the objects in the environment into clusters, where each cluster comprises only objects from one of the categories.
13. The system of claim 11, wherein the robot comprises at least one first arm and at least one second arm, the system further comprising:
logic to:
17. The system of claim 15, wherein the robot comprises at least one first arm and at least one second arm, the robotic control system logic further configured to:
operate the at least one first arm to actuate the manipulators of the robot to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket; and
operate the at least one first arm to actuate the manipulators of the robot to move obstacles out of the way and push the one or more objects onto the bucket; and
operate the at least one second arm to tilt or raise the bucket.
operate the at least one second arm to tilt or raise the bucket.
14. The system of claim 11, where each first arm is paired with a corresponding second arm, and each pairing of first arm and second arm have a common originating pivot point.
18. The system of claim 17, where each first arm is paired with a corresponding second arm, and each pairing of first arm and second arm have a common originating pivot point.
15. The system of claim 11, further comprising logic to actuate the manipulators of the robot to form a wedge in front of the bucket,
wherein the wedge formed in front of the bucket is configurable to form an opening sized to allow passage of a target object between the manipulators while preventing collection of unwanted objects.
19. The system of claim 15, the robotic control system logic further configured to actuate the manipulators of the robot to form a wedge in front of the bucket.
16. The system of claim 11, further comprising logic to actuate the manipulators to form a closed barrier in front of the bucket.
20. The system of claim 15, the robotic control system logic further configured to actuate the manipulators to form a closed barrier in front of the bucket.
17. The system of claim 11, further comprising:
a neural network configured to determine the type, size and location of the objects from images from the cameras.
21. The system of claim 15, further comprising:
a neural network configured to determine the type, size and location of the objects from images from the cameras.
18. The system of claim 11, further comprising logic to:
generate scale invariant keypoints within a decluttering area of the environment based on input from a left camera and a right camera;
22. The system of claim 15, the robotic control system logic further configured to:
generate scale invariant keypoints within a decluttering area of the environment based on input from a left camera and a right camera;
detect locations of the objects in the decluttering area based on the input from the left camera and the right camera, thereby defining starting locations;
detect locations of the objects in the decluttering area based on the input from the left camera and the right camera, thereby defining starting locations;
classify the objects into the categories;
classify the objects into the categories;
generate re-identification fingerprints for the objects, wherein the re-identification fingerprints are used to determine visual similarity between the objects;
generate re-identification fingerprints for the objects, wherein the re-identification fingerprints are used to determine visual similarity between the objects;
localize the robot within the decluttering area to determine a robot location; and
localize the robot within the decluttering area to determine a robot location; and
map the decluttering area to create a global area map including the scale invariant keypoints, the objects, and the starting locations.
map the decluttering area to create a global area map including the scale invariant keypoints, the objects, and the starting locations.
19. The system of claim 18, further comprising logic to:
re-identify the objects based on at least one of the starting locations, the categories, and the re-identification fingerprints.
23. The system of claim 22, the robotic control system logic further configured to:
re-identify the objects based on at least one of the starting locations, the categories, and the re-identification fingerprints.
20. The system of claim 19, further comprising logic to:
classify the objects as one or more of dangerous, too big, breakable, and messy.
24. The system of claim 23, the robotic control system logic further configured to:
classify the objects as one or more of dangerous, too big, breakable, and messy.
25. The system of claim 15, wherein the manipulators further include:
a first manipulator arm coupled to a first side of the robot at a first arm pivot point; and
a second manipulator arm coupled to a second side of the robot at a second arm pivot point;
wherein the first manipulator arm is coupled to the first manipulator member at the first manipulator pivot point and the second manipulator arm is coupled to the second manipulator member at the second manipulator pivot point.
Allowable Subject Matter
7. Claims 1-20 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112, objections, and nonstatutory double patenting rejection set forth in this Office action.
8. The following is a statement of reasons for the indication of allowable subject matter:
The available prior art fails to teach or suggest actuating manipulators coupled at both sides of a front end of a bucket at a front end of the robot, wherein the manipulators are coupled to open and close across the front end of the bucket to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket; one or both of tilting and raising the bucket, and actuating the manipulators to retain the objects in the bucket; navigating the robot adjacent to the corresponding container for the category; aligning a back end of the robot with a side of the corresponding container; and one or both of tilting and raising the bucket along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot to deposit retained objects in the corresponding container, in combination with the further limitations of the independent claims.
Gilbertson et al. teaches an apparatus and method for a robot that navigates to a drop-off location and raises a shovel over the robot and tilts the shovel tool forward to cause objects to slide off the shovel tool into a garbage can. Bahl et al. (CuBi: Room Decluttering Robot) teaches a method and apparatus for a room decluttering robot that comprises manipulators that open and close across a front side of the bucket to gather toys and objects. Koga et al. teaches a system and method for a refuse collection vehicle that detect and collect objects on one or more sides of the refuse vehicle.
The combination of Gilbertson, Bahl, and Koga fails to teach actuating manipulators coupled at both sides of a front end of a bucket at a front end of the robot, wherein the manipulators are coupled to open and close across the front end of the bucket to move obstacles out of the way and manipulate the one or more of the objects to pick up onto the bucket; one or both of tilting and raising the bucket, and actuating the manipulators to retain the objects in the bucket; navigating the robot adjacent to the corresponding container for the category; aligning a back end of the robot with a side of the corresponding container; and one or both of tilting and raising the bucket along a path that is an arc from the front end of the robot to the back end of the robot above a chassis of the robot to deposit retained objects in the corresponding container.
Therefore, the combination of features is considered to be allowable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIEN MINH LE whose telephone number is (571)272-3903. The examiner can normally be reached Monday to Friday (8:30am-5:30pm eastern time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached on (571)272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M.L./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656