DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office action is in response to the amendments filed on June 24, 2026. Claims 1, 3-7, and 12-23, are currently pending with Claim 1 being amended, and Claim 31 being canceled.
Response to Amendments
In response to Applicant’s amendments, filed June 24, 2026, the Examiner withdraws the previous 35 U.S.C. 103 rejections.
Response to Arguments
Applicant’s arguments, filed March 19, 2026, with respect to the rejections of Claims 1, 3-7, and 12-23 under Otto, in view of Theos, Ahlbom, Zhang, Jacobus, Yagyu, Deyle, and Sachs, have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Hoffman, in view of Ahlbom, Trevino, Theos, Zhang, Jacobus, Yagyu, Deyle, and Sachs.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 12, and 21 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Publication No. 2017/0217021 A1, to Hoffman, et al (hereinafter referred to as Hoffman; newly of record), in view of U.S. Patent No. 4,279,328, to Ahlbom (hereinafter referred to as Ahlbom; previously of record), in view of U.S. Patent Publication No. 2011/0216199 A1, to Trevino, et al (hereinafter referred to as Trevino; newly of record), and further in view of U.S. Patent Publication No. 2021/0261392 A1, to Theos, et al (hereinafter referred to as Theos; previously of record).
As per Claim 1, Hoffman discloses the features of a teleoperated material handling system (e.g. Paragraphs [0055]; where the robot may be teleoperated by a remote user, and the robot may transport materials), comprising:
a teleoperation terminal configured for collecting mechanical inputs from a teleoperator and converting to operation commands (e.g. Paragraphs [0056], [0074], [0077]; where the user may user a remote operator control unit (OCU, 10) to teleoperate a remote robot (200) from a distance, and the OCU (100) may send commands and issue status and/or navigation information to the robot controller (400), and the user (10) may select which robot (200) to control and manipulate),
the teleoperation terminal including a display (e.g. Paragraphs [0076]-[0078]; where the OCU (100) includes a display (110), which provides a user interface of the teleoperation software application that is rendered on the display and allows a user or operator (10) to control the robot (200));
a material handling vehicle with a load engaging device including one or more load engaging portions for engaging an object (e.g. Paragraphs [0082]-[0083]; where the robot (200) includes an end effector (270) (e.g., a gripper) for achieving a task, such as picking up an object), wherein
the material handling vehicle is operable through the teleoperation terminal (e.g. Paragraphs [0059], [0076]; where the remote user (10) may access the received sensor data for review, analysis, and control of the robot (200) by sending control signals form the OCU (100) to control the robot (200)),
the teleoperation terminal being separate from the material handling vehicle such that the teleoperator controls the material handling vehicle through the teleoperation terminal from a location that is remote from the material handling vehicle (e.g. Paragraphs [0056], [0074], [0077]; where the user may user a remote operator control unit (OCU, 10) to teleoperate a remote robot (200) from a distance, and the OCU (100) may send commands and issue status and/or navigation information to the robot controller (400), and the user (10) may select which robot (200) to control and manipulate);
a visual capturing module for capturing video imagery in front of the material handling vehicle (e.g. Paragraph [0065]; where the robot (200) may include one or more cameras (218, 219), which may be disposed on the leading end (210c) of the main body (210), and may be positioned to have a field of view directed forward and/or upward, and which may capture images and/or video of the robot environment for navigating and/or performing specialized tasks);
a communication module for establishing a communication link with the teleoperation terminal for transmitting the video imagery to the teleoperation terminal and receiving the operation commands from the teleoperation terminal (e.g. Paragraphs [0074], [0078], [0082]; where the controller (400) of the robot may include a communication system (482), which communicates with a remote OCU (100) to receive commands and issues status and/or navigation information; and where the display (110) of the OCU (100) is configured to show a remote view (120) such as a video feed of the robot’s environment); and
a control module for controlling operations of the material handling vehicle and the load engaging device according to the operation commands (e.g. Paragraphs [0059], [0074]; where the robot (200) includes a controller (400), which may issue drive commands to the drive system based on the received processed data set); wherein
the system comprises an assistive module configured for providing one or more assistive indicators to the teleoperator through the teleoperation terminal (e.g. Paragraph [0081]; where the hand-held drive unit may comprise a touch screen display or other type of displaying for displaying image data representing objects within the field of view of the camera; and a ground-based operator can use the image data as an aide to using the hand-held drive unit to control various functions of the materials handling vehicle), wherein
the system comprises an assistive module configured for providing one or more assistive indicators to the teleoperator through the teleoperation terminal, the one or more assistive indicators are dynamic with respect to changes in the operation commands (e.g. Paragraphs [0056], [0076], [0078], [0081], [0088]; where a user may use a remote operator control unit (OCU, 100) to teleoperate a remote robot (200) from a distance, where the OCU (100) may include a display, and provides a user interface of the teleoperation software that is rendered on the display (110), where the display may show a video feed of the robot’s environment, and an augmented image (121) is overlain on the video feed; and where the robot (200) is configured to provide dynamic feedback of its motions, and may execute display of the movement of the robot (200) in real-time as the user (10) is manipulating a portion of the robot representation (130)),
the one or more assistive indicators provide visual guidance through the teleoperation terminal for the teleoperator to manoeuvre the load engaging device (e.g. Paragraphs [0081]-[0083]; where the user (10) can send commands to the robot (200) to move its gripper (270) and provides dynamic feedback of its motions to the user (10)) ‘…’, wherein
the one or more assistive indicators include at least one augmented indicator superimposed over the video imagery on the display of the teleoperation terminal (e.g. Paragraphs [0019], [0078], [0100]; where the display (110) of the OCU (100) may provide an augmented image (121) overlain on the video feed (120)) ‘…’.
Ahlbom more explicitly teaches the features of a teleoperated material handling system. Ahlbom, in a similar field of endeavor, teaches a device for orienting a lifting means where the truck is provided with a pallet scanning device, which comprises an illuminating device and an electro-optical camera, which are mounted on vehicle to scan for a pallet; where the truck receives via its remote control system information concerning which pallet is to be picked up and the height at which the pallet rests, where the truck raises the fork using the headlights and cameras to reach the lower limit for the scanning range, and the fork is lowered until it no longer “sees” the pallet and the computer lowers the fork the additional distance required to compensate for the difference in height between the height where the optical axis of the camera hits the pallet pocket and where the fork tips point into the pocket (e.g. Col. 4 line 56- Col. 5 line 10; Col. 5 lines 25-45; Figures 3a-b, 4a-c).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the method for remotely operating a robot in the system of Hoffman, with the feature of operating a material handling system in the system of Ahlbom, in order to finely adjust the fork in relation to a loading pallet (see at least Col. 1 lines 16-25 of Ahlbom).
Hoffman fails to disclose every feature of provide visual guidance through ‘…’ to manoeuvre the load engaging device so as to facilitate an alignment with the bottom of the object, and at least one capturable indicator formed by projecting one or more laser beams from the load engaging device to form one or more markings on a side surface of the object, wherein the at least capturable indicator is capturable and visible in the video imagery, wherein the one or more laser beams are projected from one or more laser markers mounted on the one or more load engaging portions of the load engaging device such that the one or more laser markers are substantially levelled with the one or more load engaging portions, wherein the one or more laser markers further projects one or more longitudinal lines downward on the ground aligning with the one or more load engaging portions.
Trevino, in a similar field of endeavor, teaches the features of at least one capturable indicator formed by projecting one or more laser beams from the load engaging device to form one or more markings on a side surface of the object.
Trevino teaches a towing vehicle guidance system, where a light laser beam is located on the towing vehicle so that it projects a light beam onto the target to indicate visibly the relative positions of the hitch portions (i.e., load engaging device) that must be connected (e.g. Paragraphs [0013], [0062]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, with the feature of projecting a laser beam from the load engaging device onto another object in the system of Trevino, in order to enable the operator to observe the image on the target and steer the towing vehicle in response (see at least Paragraph [0014] of Trevino).
Trevino further teaches the features of wherein the at least capturable indicator is capturable and visible in the video imagery.
Trevino teaches a towing vehicle guidance system, where the laser or other light beams (68, 92) from the guidance beam projectors (66, 90) are directed toward the hitch, and the video camera (72) is turned on, and its view is displayed as an image on the monitor screen (74) of the towing vehicle; and the video monitor screen displays the image of the trailer as seen by the camera, and the projected line of the beam may be visible on the monitor screen once the towing vehicle is within several feed from the required position for connection of the trailer hitch (e.g. Paragraphs [0088], [0109], [0117], [0120]; Figures 12, 27).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, with the feature of capturing the projected beam in the video feed in the system of Trevino, in order to enable the operator to observe the image on the target and steer the towing vehicle in response (see at least Paragraph [0014] of Trevino).
Trevino further teaches the features of wherein the one or more laser beams are projected from one or more laser markers mounted on the one or more load engaging portions of the load engaging device such that the one or more laser markers are substantially levelled with the one or more load engaging portions.
Trevino teaches a towing vehicle guidance system, where a light laser beam is located on the towing vehicle so that it projects a light beam onto the target to indicate visibly the relative positions of the hitch portions that must be connected; where the light beam projector (66) is mounted at the rear end of the base portion (46) and is arranged to project a beam onto the target to indicate visibly the relative positions of the hitch portions that must be connected; and where the hitch alignment facilitating system can operate the tongue of the trailer to a height at which the body is kept level with the beam of light projected from the light projector (e.g. Paragraphs [0013], [0062], [0063]-[0064], [0070], [0119]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, with the feature of leveling the load engaging device in the system of Trevino, in order to ensure the towing vehicle module is correctly aligned (see at least Paragraph [0066] of Trevino).
Theos further teaches the features of wherein the one or more laser markers further projects one or more longitudinal lines downward on the ground aligning with the one or more load engaging portions.
Theos teaches a positioning assistance method for a materials handling vehicle, where the light controller (1202) may comprise visible lasers, light bars, projectors, etc., which may project visible indicia on the floor adjacent to, on the side of, in front of, or behind the vehicle; and where the one or more light sources (204) may designate a limited operation area, using first, second, third, etc. indicia, where each indicia is illuminated based on a distance to an object, and the indicia can be lines or arrows indicating the alignment of the vehicle (e.g. Paragraphs [0066], [0068], [0099]; Figures 4D, 6E).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom and Trevino, with the feature of providing a overlaid indicators to the user in the system of Theos, in order to indicate to a user a distance from the vehicle to an object, or a heading of the vehicle with respect to an object (see at least Paragraph [0004] of Theos).
As per Claim 12, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, and Theos further teaches the features of wherein the system comprises a beacon device for providing a tracking functionality for the material handling vehicle.
Theos teaches a positioning assistance method for a materials handling vehicle, where the vehicle (10) may include an antenna (30) that receives control signals form a corresponding wireless remote control device (32), and may include a light at the top, and may be used to identify position information of the vehicle; and where the vehicle (10) may be controlled by the controller (103) to travel an intended path or maintain an intended heading (e.g. Paragraphs [0043], [0055], [0065], [0070]; Figure 1).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom and Trevino, with the feature of providing a beacon signal to indicate a traveling path of the vehicle in the system of Theos, in order to travel an intended path or maintain an intended heading (see at least Paragraph [0095] of Theos).
As per Claim 21, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, and Hoffman further teaches the features of wherein the communication link utilizes wireless communication protocols according to Wi-Fi standards (e.g. Paragraphs [0057], [0059]; where the robot (200) is capable of communicating with the OCU (100) via a wireless telecommunication network, to include the used of Wi-Fi).
Claims 3 and 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, and Theos, as applied to Claim 1 above, and further in view of U.S. Patent Publication No. 2017/0322560 A1, to Zhang, et al (hereinafter referred to as Zhang; previously of record).
As per Claim 3, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the one or more assistive indicators comprise one or more trajectory lines representing an anticipated trajectory of movement of the material handling vehicle.
However, Zhang, in a similar field of endeavor, teaches a vehicle guidance system, where trajectory overlays are displayed by the vehicle guidance system (e.g. Figure 4).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of providing trajectory lines in the system of Zhang, in order to provide improved viewing to assist a driver in maneuvering (see at Paragraphs [0004] and [0050] of Zhang).
As per Claim 5, Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, teaches the features of Claim 3, and Zhang further teaches the features of wherein the one or more trajectory lines change with respect to a change in a steering angle of at least one steerable wheel on the material handling vehicle.
Zhang teaches a vehicle guidance system, where trajectory overlays are displayed by the vehicle guidance system; and where the trajectory overlay (70) is dynamic and may change upon a change of the image due to, for example, movement of the vehicle or upon the driver changing an angular position of the wheels (26) (e.g. Paragraph [0061]; Figure 4).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of changing visual trajectory lines in the system of Zhang, in order to provide improved viewing to assist a driver in maneuvering (see at Paragraphs [0004] and [0050] of Zhang).
As per Claim 6, Hoffman, in view of Ahlbom, Trevino, Theos and Zhang, teaches the features of Claim 5, and Zhang further teaches the features of wherein the one or more trajectory lines change with respect to a change in traveling speed of the material handling vehicle.
Zhang teaches a vehicle guidance system, where trajectory overlays are displayed by the vehicle guidance system; and where the trajectory overlay (70) is dynamic and may change upon a change of the image due to, for example, movement of the vehicle (i.e. speed) or upon the driver changing an angular position of the wheels (26); and where the vehicle utilizes a speed sensor (58) to determine vehicle information parameters, and align the guidance overlay in conjunction with the speed or other parameter (e.g. Paragraph [0061]; Figure 4).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of changing visual trajectory lines in the system of Zhang, in order to provide improved viewing to assist a driver in maneuvering (see at Paragraphs [0004] and [0050] of Zhang).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, as applied to Claim 3 above, and further in view of U.S. Patent No. 8,965,561 B2, to Jacobus, et al (hereinafter referred to as Jacobus; previously of record).
As per Claim 4, Hoffman, in view of Ahlbom and Trevino, Theos, and Zhang, teaches the features of Claim 3, but the combination of Otto, in view of Theos, Ahlbom, and Zhang, fails to teach every feature wherein the one or more trajectory lines change with respect to a change in steering input by the teleoperator according to Ackerman steering geometry.
However, Jacobus, in a similar field of endeavor, teaches an automated warehousing system which uses robotic forklifts, where turning can also be estimated from this data using differential odometry applied to well-known Ackerman or Skid Steer formulas (Column 11, Lines 36-38).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, with the feature using Ackerman steering geometry in the system of Jacobus, in order to reduce sensor error drifts and provide feedback to compensate for the relative sensor drift process (see at Col. 12 lines 17-35 of Jacobus).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, as applied to Claim 3 above, and further in view of U.S. Patent Publication No. 2022/0107201 A1, to Yagyu, et al (hereinafter referred to as Yagyu; previously of record).
As per Claim 7, Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, teaches the features of Claim 3, but the combination of Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, fails to teach every feature of wherein perspective correction is performed for the one or more trajectory lines according to a perspective angle inherent in the video imagery.
However, Yagyu, in a similar field of endeavor, teaches a display control device for a vehicle, where the (HCU, 100) includes a plurality of functional units for controlling superimposed displayed contents of (HUD, 20) by causing the processing unit (11) to execute a display control program stored in the storage unit (13), where the viewpoint position identification unit (71) identifies a position of the eye point (EP) of a driver, and generates three-dimensional coordinates indicating apposition of an eye point (EP); and where the display generation unit (76) corrects a rendering position and a rendering shape of the original image according to the respective positions of an eye point (EP) and a superimposition target (e.g. Paragraphs [0102]-[0103], [0107]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, Theos, and Zhang, with the feature of correcting perspectives in images in the system of Yagyu, in order to provide improved viewing to establish a virtual camera position for the display to inform the user (see at Paragraph [0110] of Yagyu).
Claims 15-16, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, and Theos, as applied to Claim 14 above, and further in view of U.S. Patent Publication No. 2020/0061839 A1, to Deyle, et al (hereinafter referred to as Deyle; previously of record).
As per Claim 15, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 14, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the material handling vehicle provides a front tracking mode such that the material handling vehicle travels along the path of travel with the material handling vehicle trailing the onsite operator carrying the beacon device.
However, Deyle, in a similar field of endeavor, teaches an inventory management robot, where the robot (100) can also follow detected individuals, for instance closely (within a threshold distance) or from a distance (greater than a threshold distance), and where the location of individuals or objects detected by robots, security cameras, or other location-tracking mechanisms (such as GPS tracking devices, mobile phones, or RFID readers) can be updated as the individuals or objects move in real-time; and where the robot can track or follow an individual (e.g. Paragraphs [0122], [0141], [0188]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of having the robot follow the operator in the system of Deyle, in order to help the user by having the robot retrieve or store inventory items as the operator is walking (see at Paragraph [0347] of Deyle).
As per Claim 16, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 14, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the material handling vehicle provides a rear tracking mode such that the material handling vehicle travels along the path of travel in front of the onsite operator carrying the beacon device.
However, Deyle, in a similar field of endeavor, teaches an inventory management robot, where the robot (100) can provide instructions to a user (e.g., “follow me”), such as to escort the user to a new location (e.g. Paragraphs [0140]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of having the robot follow the operator in the system of Deyle, in order to escort users or operators to spaces where they are allowed to be (see at Paragraph [0140] of Deyle).
As per Claim 18, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, but Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the material handling vehicle is switchable between different operation modes including manual operation mode, teleoperation mode, and autonomous operation mode.
However, Deyle, in a similar field of endeavor, teaches an inventory management robot, where the robot can operate autonomously (without explicit instructions from an operator), controlled by a human operator (i.e. manual mode), or controlled by the human operator using teleoperation (e.g. Paragraphs [0121], [0354]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of having the robot operate in different modes in the system of Deyle, in order to determine if the robot can perform a task, and allocate the task to the robot or the user to perform (see at Paragraph [0006] of Deyle).
As per Claim 19, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 16, and Deyle further teaches the features of wherein the material handling vehicle is switchable between different operation modes including manual operation mode, teleoperation mode, autonomous operation mode, front tracking mode and a rear tracking mode.
Deyle teaches an inventory management robot, where the robot can operate autonomously (without explicit instructions from an operator), controlled by a human operator (i.e. manual mode), or controlled by the human operator using teleoperation; and where the robot (100) can provide instructions to a user (e.g., “follow me”), such as to escort the user to a new location; and where the robot (100) can also follow detected individuals, for instance closely (within a threshold distance) or from a distance (greater than a threshold distance) (e.g. Paragraphs [0121], [0140]-[0141], [0354]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of having the robot operate in different modes in the system of Deyle, in order to determine if the robot can perform a task, and allocate the task to the robot or the user to perform (see at Paragraph [0006] of Deyle).
Claims 13-14, 17, 20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, and Theos, as applied to Claims 12, 14, and 1, respectively, above, and further in view of U.S. Patent Publication No. 2020/0259896 A1, to Sachs, et al (hereinafter referred to as Sachs; previously of record).
As per Claim 13, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 12, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the beacon device comprises a wearable carrier adapted to be worn by an onsite operator.
However, Sachs, in a similar field of endeavor, teaches an industrial automation method, where the first device (110) may include a wearable computer for actuating remote control of a vehicle (e.g. Paragraphs [0692], [1793], [1796]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of using a wearable device in the system of Sachs, in order to track assets and personnel working close to a forklift (see at Paragraph [0383] of Sachs).
As per Claim 14, Hoffman, in view of Ahlbom, Trevino, Theos and Sachs, teaches the features of Claim 13, and Theos further teaches the features of wherein the beacon device emits a beacon signal receivable by the control module on the material handling vehicle, the control module is adapted for determining a path of travel for the material handling vehicle with respect to the beacon signal.
Theos teaches a positioning assistance method for a materials handling vehicle, where the vehicle (10) may include an antenna (30) that receives control signals form a corresponding wireless remote control device (32), and may include a light at the top, and may be used to identify position information of the vehicle; and where the vehicle (10) may be controlled by the controller (103) to travel an intended path or maintain an intended heading (e.g. Paragraphs [0043], [0055], [0065], [0070]; Figure 1).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the materials handling vehicle and goods retrieval system of Otto, with the feature of providing a beacon signal to indicate a traveling path of the vehicle in the system of Theos, in order to travel an intended path or maintain an intended heading (see at least Paragraph [0095] of Theos).
As per Claim 17, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 14, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the system utilizes ultrawide band signal positioning to locate the beacon device for determining the path of travel.
However, Sachs, in a similar field of endeavor, teaches an industrial automation method, where remote robot control over a modelled wireless link is used in a manufacturing facility; and where the positioning system used indoors in an industrial or factory floor can include Wi-Fi, radio-frequency identification (RFID), Bluetooth, ultra-wide band (UWB) (e.g. Paragraph [0369]; Figure 13).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of using ultra-wide band communications in the system of Sachs, in order to enable precise positioning of a robot (see at Paragraph [0422] of Sachs).
As per Claim 20, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the communication link utilizes wireless communication protocols according to 5G mobile communication standards.
However, Sachs, in a similar field of endeavor, teaches an industrial automation method, where remote robot control over a modelled wireless link is used in a manufacturing facility; and where 4G/5G connections are used for industrial robotic applications (e.g. Paragraphs [0345], [0352]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of using 5G communications in the system of Sachs, in order to decrease latency in receiving the signals (see at Paragraph [0349] of Sachs).
As per Claim 22, Hoffman, in view of Ahlbom, Trevino, and Theos, teaches the features of Claim 1, but the combination of Hoffman, in view of Ahlbom, Trevino, and Theos, fails to teach every feature of wherein the material handling vehicle is configured for self-navigating based on Simultaneous Localization and Mapping (SLAM).
However, Sachs, in a similar field of endeavor, teaches an industrial automation method, where a simultaneous localization and mapping method may be used to determine position information (e.g. Paragraph [0376]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, and Theos, with the feature of using SLAM functions in the system of Sachs, in order to increase accuracy (see at Paragraph [0376] of Sachs).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Hoffman, in view of Ahlbom, Trevino, Theos, and Sachs, as applied to Claim 22 above, and further in view of U.S. Patent Publication No. 2022/0107201 A1, to Yagyu, et al (hereinafter referred to as Yagyu; previously of record).
As per Claim 23, Hoffman, in view of Ahlbom, Trevino, Theos, and Sachs, teaches the features of Claim 22, and Hoffman further teaches the features of wherein the material handling vehicle is provided with a plurality of sensors comprising light detection and ranging (LiDAR), an inertial navigation system (INS), Global Positioning System (GPS), and ‘...’ maps ‘...’ (e.g. Paragraphs [0073], [0129], [0140]; where the robot (200) may include camera, radar, lidar, an inertial measurement unit, and can obtain global positioning coordinates).
The combination of Hoffman, in view of Ahlbom, Trevino, Theos, and Sachs, fails to teach every feature of wherein the material handling vehicle is provided with a plurality of sensors comprising light detection and ranging (LiDAR), an inertial navigation system (INS), Global Positioning System (GPS), and high-definition maps (HD Map).
However, Yagyu, in a similar field of endeavor, teaches a display control device for a vehicle, where the locator (40) generates accurate positional information of the vehicle based on a combination of acquired or sensed information, to include a GNSS (Global Navigation Satellite System) receiver (41), an inertia sensor (42), a high-precision map database, and a locator ECU (44); and where the periphery monitoring sensor (30) includes a lidar or sonar; and where the GNSS receiver (41) receives positioning signals from systems including GPS, GLONASS, etc. (e.g. Paragraphs [0070]-[0072])
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the method for remotely operating a robot in the system of Hoffman, in view of Ahlbom, Trevino, Theos, and Sachs, with the feature of utilizing different sensors in the system of Yagyu, in order to provide increased accuracy of position information of the vehicle (see at Paragraph [0071] of Yagyu).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kamiya, et al (WO 2020/049897 A1), which teaches a method for operating a remote control system for a forklift, and displaying the trajectory of the vehicle.
Mori, et al (JP 2006096457 A), which teaches a method for superimposing vehicle trajectory lines on to the display to indicate the alignment of the vehicle relative to a pallet.
Sugie (JP H11278799 A), which teaches a method for aligning an unmanned forklift by irradiating laser beams on to the pallet from the forklift.
Zevenbergen, et al (U.S. 2020/0254925 A1), which teaches a method for projecting on to pallets in a work area.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MERRITT LEVY whose telephone number is (571)270-5595. The examiner can normally be reached Mon-Fri 0630-1600.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached at (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MERRITT LEVY/Examiner, Art Unit 3663
/KYLE J KINGSLAND/Primary Examiner, Art Unit 3663
Prosecution Insights