Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1-5 and 8-20 are rejected under 35 U.S.C. 103 as being unpatentable over Murray (US 20230286156, disclosed in IDS submitted on 08/20/2025) in view of Song (US 12145270).
For claim 1, Murray teaches: An autonomous inspection system for detecting and avoiding threats during operation of a robotic system comprising a robotic device, the robotic device configured to perform a series of robotic operations each including a path and a trajectory (abstract, disclosing observing robot workspace to prevent collisions and moving robot along its path when safe. [0044], disclosing robot moves along a trajectory. Trajectory itself contains a path), the autonomous inspection system comprising:
a memory device for storing data ([0051], disclosing robot control system 200 may comprise one or more processor(s) 222, and one or more associated nontransitory computer- or processor-readable storage media for example system memory); and
a processor in communication with the memory device and for processing the data stored by the memory device ([0051], disclosing robot control system 200 may comprise one or more processor(s) 222, and one or more associated nontransitory computer- or processor-readable storage media for example system memory), the processor configured to execute:
vision-guided motion (VGM) software tool configured to iteratively guide the robotic device during an approach phase preceding contact with the target object toward a target pose suitable for engaging the target object
and at least one of:
an autonomous sensing of unexpected obstacles (ASUO) software tool ([0072], disclosing motion planner 204a may optionally include an environment converter 263 that converts output (e.g., digitized representations of the environment) from optional sensors 262 (e.g., digital cameras) into representations of obstacles);
a model-based collision avoidance (MBCA) software tool ([0013], disclosing robots rely on kinematic models of robots and models of shared workspace. [0075], disclosing motion planning based on geometric model. [0102], disclosing processor-based system generates a motion planning graph for a robot based on the respective robot kinematic model. [0055], disclosing motion planning takes collision detection into account);
a worksite surveyor (WS) software tool ([0038], disclosing perception subsystem 124 may include one or more processors, which may execute one or more machine-readable instructions that cause the perception subsystem 124 to generate a respective discretization of a representation of an environment in which the robots 102 will operate).
Murray teaches of iteratively guiding the robot based on image data ([0072], disclosing The motion planner 204a may optionally include an environment converter 263 that converts output (e.g., digitized representations of the environment) from optional sensors 262 (e.g., digital cameras) into representations of obstacles). Representation of environment necessarily has information regarding pose of robot, obstacles and object. However, Murray does not explicitly disclose guiding the robot based on pose estimates determined from image data.
Song teaches guiding robot to a target object based on pose estimate determined from image data (column 3, disclosing using a camera to obtain pose information of target object. converting coordinates of the object into a base coordinate system of the mechanical arm, and controlling the mechanical arm to move to a position near the target object. The process if performed multiple times)
Song and Murray are analogous arts as they are in same field of endeavor i.e., object manipulation. It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to modify art of Murray to guiding the robot based on pose estimates determined from image data as taught by Song to select the optimal grabbing pose of the target object
Method of claim 19 recites limitations similar in scope to claim 1, hence is similarly rejected.
For claim 2, modified Murray teaches: The system of claim 1, wherein the robotic device is a robotic manipulator ([0043] and figure 1, disclosing robotic manipulators 102).
Claim 20 recites limitations similar in scope to claim 2, hence is similarly rejected.
For claim 3, modified Murray teaches: The system of claim 1, wherein the ASUO software tool is configured to identify discrepancies between a modelled world of an environment of the robotic device and a real world of the robotic device ([0085], disclosing motion planning to avoid static and dynamic obstacles. [0113], disclosing generating representation of environment through sensors. The representation can represent static or persistent objects in the environment and/or can represent dynamic or transient objects in the environment. [0115], disclosing representing transient obstacles. A dynamic or transient obstacle obstacle/object is necessarily at a different position compared to previously sampled/stored environment model).
For claim 4, modified Murray teaches: The system of claim 1, wherein the MBCA software tool is configured to check a planned path of the robotic device for potential collisions ([0055], disclosing performing collision assessment of planned motion).
For claim 5, modified Murray teaches: The system of claim 1, wherein the WS software tool is configured to perform automated visual inspection of a worksite in which the robotic device is operating using images taken by one or more worksite cameras prior to proximity operations ([0037], disclosing perception data of work environment is taken by cameras 122a and 122b).
For claim 8, modified Murray teaches: The system of claim 2, wherein the robotic device further includes an end effector mounted on a free end of the robotic manipulator, and wherein the end effector performs a grappling operation on a payload (figure 1 and [0089], disclosing end effectors used to grip an object or workpiece).
For claim 9, modified Murray teaches: The system of claim 8, wherein the end effector further provides an auxiliary service to the payload through the end effector while the payload is grappled ([0042], disclosing multiple tasks i.e., services, any one of those can be an auxiliary service).
For claim 10, modified Murray teaches: The system of claim 8, wherein the processor is configured to execute the ASUO software tool and the MBCA software tool, and wherein the MBCA software tool is executed autonomously for a next planned trajectory of the robotic device only if an output of the ASUO software tool indicates no obstacle was detected in the path of the robotic device ([0055], disclosing motion planning takes collision detection into account. Abstract, disclosing monitoring the other robots and moving the robot toward a goal in response to the path becoming unblocked or cleared. The staging pose can be identified using various heuristics to efficiently position or configure the robot to complete its task one its path becomes unblocked or cleared).
For claim 11, modified Murray teaches: The system of claim 10, wherein the ASUO software tool is configured to register observation data comprising lidar data and optical data to as-built models of a worksite in which the robotic device is operating ([0113], disclosing processor-based system optionally receives or generates a representation of the environment, for example a representation of the environment as sensed by one or more sensors (e.g., cameras, LIDAR). The representation can represent static or persistent objects in the environment and/or can represent dynamic or transient objects in the environment. [0047], disclosing vision sensors to determine what portion of workspace is blocked/unblocked. [0013], disclosing sensor data is used to model workspace), compare the registered data to one or more collision models that have been configured based on scene context, and determine whether any obstacles are present ([0113], disclosing processor-based system optionally receives or generates a representation of the environment, for example a representation of the environment as sensed by one or more sensors (e.g., cameras, LIDAR). The representation can represent static or persistent objects in the environment and/or can represent dynamic or transient objects in the environment. [0047], disclosing vision sensors to determine what portion of workspace is blocked/unblocked).
For claim 12, modified Murray teaches: The system of claim 11, wherein the MBCA software tool is configured to:
use input configuration information and collision models for each element present in the scene, place the collision models in appropriate locations in space and determine which collision checks should be performed ([0010], disclosing taking input models of robots and representations of objects. Representations of objects are interpreted as models. [0013], disclosing using representations of geometric models of workspace. ); and
identify a potential collision situation by determining whether a collision model of the robotic device comes within a clearance threshold of other bodies' collision models in the scene ([0093], disclosing determining is probability of collision with an obstacle is below a threshold or not. Probability threshold is clearance threshold).
For claim 13, modified Murray teaches: The system of claim 12, wherein a control device controlling the robotic device performs the next planned trajectory of the robotic device if an output of the MBCA tool does not identify a potential collision situation (PCS) ([0009], disclosing collision free robot movement, hence robotic device is controlled for next planned trajectory when no potential collision is identified).
l. For claim 14, modified Murray teaches: The system of claim 12, wherein the processor is further configured to execute the WS software tool autonomously once the robotic device has reached a designated position, the WS software tool configured to compare 3D models of robotic interfaces to capture images of a robotic interface to be operated on by the robotic device and determine if there are any anomalies ([0038], disclosing perception subsystem 124 may include one or more processors, which may execute one or more machine-readable instructions that cause the perception subsystem 124 to generate a respective discretization of a representation of an environment in which the robots 102 will operate. [0011], disclosing staging robots and determining if path is blocked and moving robot to goal when path is unblocked or cleared. Path being blocked is an anomaly. [0013], disclosing avoiding dynamic obstacles. Dynamic obstacles are ones that are not present at same position in environment map, hence an anomaly).
m. For claim 15, modified Murray teaches: The system of claim 14, wherein the processor is further configured to execute the VGM software tool autonomously upon determination by the WS software tool that there are no anomalies, the VGM software tool configured to estimate a pose of the robotic interface corresponding to a machine vision target near the robotic interface relative to the robotic device using a pose estimation algorithm, an input image of the machine vision target, and a target ID of the machine vision target ([0011], disclosing staging robots and determining if path is blocked and moving robot to goal when path is unblocked or cleared. Path being blocked is an anomaly. [0013], disclosing avoiding dynamic obstacles. Dynamic obstacles are ones that are not present at same position in environment map, hence an anomaly. [0072], disclosing motion planner 204a may optionally include an environment converter 263 that converts output (e.g., digitized representations of the environment) from optional sensors 262 (e.g., digital cameras) into representations of obstacles. [0091], disclosing motion planner 110a, 110b, 110c (FIG. 1), 204 (FIG. 2) or a portion thereof (e.g., collision detector 252, FIG. 2) determines or assesses a likelihood or probability that a pose (represented by a node) and/or motion or transition (represented by an edge) will result in a collision with an obstacle. [0091], disclosing determines or assesses a likelihood or probability that a pose (represented by a node) and/or motion or transition (represented by an edge) will result in a collision with an obstacle).
n. For claim 16, modified Murray teaches: The system of claim 12, wherein the WS tool is executed before the robotic device performs an operation on the robotic interface and after the robotic device performs an operation on the robotic interface ([0008], disclosing motion planning is performer in real time, hence at each processing cycle i.e., before and after robot is actuated).
o. For claim 17, modified Murray teaches: The system of claim 16, wherein the robotic interface is a grapple fixture mounted to a payload and the grapple fixture is grappled and rigidized by the end effector ([0049], disclosing robot having a fixed or movable base. robot 202 may optionally include a set of links, joints, end-of-arm tools or end effectors, and/or actuators 218a, 218b, 218c (three, shown, collectively 218) operable to move the links about the joints. The set of links, joints, end-of-arm tools or end effectors typically comprise one or more appendages of the robot, which robotic appendages can be moveably coupled to the base of the robot).
p. For claim 18, modified Murray teaches: The system of claim 17, wherein the end effector and the grapple fixture have complementary coupling elements that interlock to limit or prevent radial movement of the grapple fixture relative to the end effector while the grapple fixture is rigidized ([0049], disclosing robot having a fixed base).
q. For claim 21, modified Murray teaches: The system of claim 1, wherein the processor is configured to execute the ASUO software tool, and wherein the ASUO software tool is configured to detect obstacles and modify operation of the robotic device to avoid the obstacles ([0039], disclosing detecting obstacles. [0013], disclosing avoiding dynamic obstacles. Avoiding dynamic obstacle necessitates modifying operation of the robotic device. [0065], disclosing revising motion path).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Murray in view of Song and Sivich (US 20160288331)
For claim 7, modified Murray teaches: The system of claim 1,
Murray does not teach: wherein the processor is further configured to execute a checkout and inspection recommender (CIR) software tool configured to recommend insertion of checkouts and inspections into a task performed by the robotic device.
Sivich teaches of executing a checkout and inspection into a task performed by a robotic device (Abstract, disclosing a robot. [0050], disclosing validating whether a task was performed correctly).
Murray and Sivich are analogous arts as they are in same field of endeavor i.e., robot control. It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to modify art of Murray to wherein the processor is further configured to execute a checkout and inspection recommender (CIR) software tool configured to recommend insertion of checkouts and inspections into a task performed by the robotic device as taught by Sivich to ensure quality and performance control.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARSLAN AZHAR whose telephone number is (571)270-1703. The examiner can normally be reached Mon-Fri 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARSLAN AZHAR/Examiner, Art Unit 3656
Prosecution Insights