DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 5, 2026, has been entered.
Status of Claims
This Office action is in response to the amendments filed on March 03, 2026. Claims 1-18 are currently pending, with Claims 1 and 10 being amended.
Response to Amendments
Regarding Applicant’s amendments, filed March 03, 2026, the Examiner maintains the previous claim interpretation, and withdraws the previous 35 U.S.C. 103 rejections.
Response to Arguments
Applicant’s arguments, filed March 03, 2026, with respect to the rejections of Claims 1-18 under Tiwari, in view of Ghorbanian-Matloob and Diamond, have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Tiwari, in view of Lang, and Tamagawa.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on March 03, 2026, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the Examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a display configured to display …” in Claims 1, 4, and 8.
“one or more actuator commands configured to cause …” in Claims 1 and 10.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Regarding the limitation of “a display configured to display …”, the instant Drawings at Figure 3C, and at the instant Specification at Paragraph [0091], shows a monitor for displaying road information. As such, the Examiner is interpreting the display as a monitor with capable of receiving inputs from sensors or cameras and providing a visual representation to the user.
Regarding the limitation of “one or more actuator commands configured to cause …”, the instant specification at Paragraphs [0063], [0106], and [0109] at least states that “the computing device 130 may function as a controller for causing one or more functions of the vehicle … “ and where “the vehicle may comprise a control module to receive the transmitted remote trajectory command and/or one or more driving actions … and the vehicle may comprise a control module which may receive the one or more driving actions to cause the vehicle to perform …”. As such, the Examiner is interpreting the one or more actuator commands as software or hardware components capable of receiving signals from a processor or computer equivalent for executing vehicle functions.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2018/0154899 A1, to Tiwari, et al (hereinafter referred to as Tiwari; previously of record), in view of U.S. Patent Publication No. 2017/0272664 A1, to Lang, et al (hereinafter referred to as Lang; newly of record), and further in view of Japanese Patent Publication No. 2023032982 A, to Tamagawa, et al (hereinafter referred to as Tamagawa; newly of record).
As per Claim 1, Tiwari discloses the features of a remote station system for remotely controlling a vehicle (e.g. Paragraphs [0014], [0053]; where the system (100) functions to control a vehicle during operation and is operable between several operating modes including an autonomous, semi-autonomous, and a teleoperation mode, where the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system), comprising:
a transmitter (e.g. Paragraphs [0053], [0082]; where when the system (100) is in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem)
(The Examiner will note that is has been held that a claim is anticipated if each element of the claim is found, either expressly described or under principles of inherency, in a single prior art reference, or that the claimed invention was previously known or embodied in a single prior art device or practice. Kalman v. Kimberly-Clark Corp., 218 USPQ 789. A “transmitter” in this limitation is inherent in the applied prior art, as transmitting and receiving instructions requires a transmitter to serve as a conduit for sharing data, commands, or signals) configured to receive:
one or more data points generated by one or more sensors coupled to a vehicle (e.g. Paragraphs [0053], [0082], [0090]-[0091]; where the system (100) is when in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem; and where the communication module (170) functions to communicatively couple the vehicle control system to a remote computing system and can receive transmissions sent to the vehicle and data to be transmitted away from the vehicle (e.g. sensor data)); and
from the vehicle, a steering position of the vehicle (e.g. Paragraphs [0059], [0079], [0088], [0096]; where the decision-making block receives inputs from the sensor subsystem, inputs by a teleoperator, inputs by a local operator (e.g., the driver turning the steering wheel); and where the inputs can be received at the remote teleoperation interface, such as receiving driver input from the remote vehicle operator using a connected replica steering wheel (i.e. steering wheel input to include position and angle));
a display configured to display the one or more data points generated by the one or more sensors (e.g. Paragraph [0050]; where the route plan can include graphical indicators that can be provided at a teleoperation interface (171) (e.g., rendered at a heads-up display of a remote operator); and the remote teleoperation interface can include a display and a set of inputs, including sensor data (1100) transmitted to the remote teleoperation interface (171) and rendered at the display to a remote vehicle operator),
wherein the one or more data point comprise a camera feed of the field of view of the video (e.g. Paragraph [0038]; where the sensor includes trajectories of vehicles in the field of view (FOV) of the sensors);
a computing device comprising a processor and a memory, wherein the memory is configured to store instructions, that when executed by the processor (e.g. Paragraphs [0014], [0057]; where the vehicle includes an onboard and/or remote computing system for transmitting control instructions to the vehicle; and where the system can be implemented as a machine configured to receive a computer-readable medium storing computer-readable instructions, which are executed by a processor and/or controller, and can be stored in computer-readable media), are configured to cause the processor to:
using one or more remote actuation controls, manually or automatically generate one or more driving actions (e.g. Paragraphs [0026], [0057], [0059], [0071]; where the teleoperator can transmit control instructions to the system; where the system (100) includes a behavior planning module (130) which generates control commands, and the control module (150) receives control commands and controls the actuation subsystem (153); and where the computing system may automatically select a task-block based on the data, where the associated task can be automatically performed),
switch from automatically generating the one or more driving actions to manually generating the one or more driving actions when a remote trajectory demand is present (e.g. Paragraphs [0057], [0106], [0109]-[0110]; where the teleoperator can manually select a task block when the vehicle action can be presented to the teleoperator, who can select or approve the action, and the teleoperator can transmit a signal to the vehicle to transition out of the autonomous mode after the teleoperator has assumed control of the vehicle; and where the instructions can include transferring planning authority to the teleoperator, and the directive can be received from (e.g., manually triggered by) a teleoperator (i.e. a remote trajectory demand is present, and the teleoperator performs manual controls by selecting vehicle actions)); wherein:
the one or more remote actuation controls are remote from the vehicle (e.g. Paragraphs [0014], [0057], [0085], [0105]; where the teleoperator can select vehicle actions based on the processed data, and where vehicle mechanisms can be operated by a remote operator), and
wherein the one or more driving actions correlate to one or more actuator commands configured to cause the vehicle to perform the one or more driving actions (e.g. Paragraphs [0014], [0071], [0075]; where the control outputs can be used to control the actuation subsystem, where the task block can generate control instructions/ signals for control of the actuation subsystem to directly control the control elements of the vehicle (e.g., throttle, steering)),
display, using the display, a first trajectory and a second trajectory of the vehicle (e.g. Paragraphs [0035], [0064]; where the trajectory generator (132) determines a set of potential trajectories that can be used to traverse the physical space surrounding the vehicle; and where trajectories of the vehicle, other vehicles and objects are displayed, and the trajectory generator generates one or more trajectories and provides the one or more trajectories to the decision making block) ‘…’, wherein:
the first trajectory is automatically generated based on vehicle route information (e.g. Paragraphs [0055], [0062]-[0064], [0101]; where the system can automatically generate a planner primitive based on the received sensor data; and where the prediction block predicts trajectories, positions, speeds, etc., of any detected objects in the area surrounding the vehicle and the trajectory generator (132) determines the desired trajectory of the vehicle, and generates trajectory planner primitives based on traffic data, the cost map, and directives from the remote operator), and ‘…’
using the transmitter, transmit the one or more driving actions to the vehicle (e.g. Paragraphs [0053], [0057], [0090], [0110]; where the teleoperator can select vehicle actions based on the processed data and send it to the vehicle to be performed by the onboard vehicle system; and where the selected vehicle action can be transmitted to the vehicle, where the teleoperator performs the action (e.g. using the actuation subsystem of the vehicle); where when the system (100) is in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem)
(The Examiner will note that is has been held that a claim is anticipated if each element of the claim is found, either expressly described or under principles of inherency, in a single prior art reference, or that the claimed invention was previously known or embodied in a single prior art device or practice. Kalman v. Kimberly-Clark Corp., 218 USPQ 789. A “transmitter” in this limitation is inherent in the applied prior art, as transmitting and receiving instructions requires a transmitter to serve as a conduit for sharing data, commands, or signals);
cause the vehicle to perform the one or more actions (e.g. Paragraphs [0014], [0057], [0109]; where the teleoperation mode includes a remote operator transmitting control instructions to the system and controlling the vehicle based on the control instructions).
Tiwari fails to disclose every feature of display, using the display, a first trajectory and a second trajectory of the vehicle over the camera feed; the second trajectory is based on the steering position of the vehicle and is configured to show a reaction of the vehicle to the steering position of the vehicle.
However, Lang, in a similar field of endeavor, teaches the features of display, using the display, a first trajectory and second trajectory of the vehicle over the camera feed.
Lang teaches a method for providing a visual display system for a vehicle, where the visual capturing system may be a camera or an image sensor, where the trajectory information may be superimposed over the image/ feed on the image display unit; and where the trajectory (10) illustrates the track/ driving path of the front wheels, and may illustrate the actual track/driving path, and the trajectory (10’) illustrates the track/driving path of the rear wheels, dependent on a steering angle of the vehicle (i.e., first and second trajectories are displayed) (e.g. Paragraphs [0050], [0060]-[0061]; Figures 3, 11).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the vehicle control system of Tiwari, with the feature of overlaying a trajectory on to a display in the system of Lang, in order to improve the display of a future track of the vehicle (see at least Paragraph [0032] of Lang).
Tamagawa, in a similar field of endeavor, further teaches the features of the second trajectory is based on the steering position of the vehicle and is configured to show a reaction of the vehicle to the steering position of the vehicle.
Tamagawa teaches a method for remotely operating a vehicle, where a trajectory line is superimposed onto the display representing the predicted trajectory of the vehicle (12) on the display unit (22) of the remote cockpit (20), and the display unit (22) displays the trajectory line linked to the steering amount of the steering wheel (36) on the remote cockpit (20) side and the steering amount of the steering wheel on the vehicle (12) side, and the trajectory line is configured to be displayed so that its display changes when the steering wheel on the vehicle (12) side is positioned at or near neutral (e.g. Paragraphs [0048]-[0050]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the vehicle control system of Tiwari, in view of Lang, with the feature of overlaying a steering trajectory on to a display in the system of Tamagawa, in order to notify the driver of the vehicle and the remote operator of the steering wheel position of the vehicle state (see at least Paragraphs [0037]-[0038] of Tamagawa).
As per Claim 10, Tiwari discloses the features of a method for remotely controlling a vehicle (e.g. Paragraphs [0014], [0053]; where the system (100) functions to control a vehicle during operation and is operable between several operating modes including an autonomous, semi-autonomous, and a teleoperation mode, where the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system), comprising:
using a remote station system (e.g. Paragraphs [0014], [0053]; where the system (100) functions includes a teleoperation mode which includes a remote operator transmitting control instructions to the vehicle system);
a transmitter (e.g. Paragraphs [0053], [0082]; where when the system (100) is in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem)
(The Examiner will note that is has been held that a claim is anticipated if each element of the claim is found, either expressly described or under principles of inherency, in a single prior art reference, or that the claimed invention was previously known or embodied in a single prior art device or practice. Kalman v. Kimberly-Clark Corp., 218 USPQ 789. A “transmitter” in this limitation is inherent in the applied prior art, as transmitting and receiving instructions requires a transmitter to serve as a conduit for sharing data, commands, or signals),
a display (e.g. Paragraph [0050]; where the route plan can include graphical indicators that can be provided at a teleoperation interface (171) (e.g., rendered at a heads-up display of a remote operator); and the remote teleoperation interface can include a display and a set of inputs, including sensor data (1100) transmitted to the remote teleoperation interface (171) and rendered at the display to a remote vehicle operator), and
a computing device comprising a processor and a memory (e.g. Paragraphs [0014], [0057]; where the vehicle includes an onboard and/or remote computing system for transmitting control instructions to the vehicle; and where the system can be implemented as a machine configured to receive a computer-readable medium storing computer-readable instructions, which are executed by a processor and/or controller, and can be stored in computer-readable media);
receiving: one or more data points generated by one or more sensors coupled to a vehicle (e.g. Paragraphs [0053], [0082], [0090]-[0091]; where the system (100) is when in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem; and where the communication module (170) functions to communicatively couple the vehicle control system to a remote computing system and can receive transmissions sent to the vehicle and data to be transmitted away from the vehicle (e.g. sensor data)); and
from the vehicle, a steering position of the vehicle (e.g. Paragraphs [0059], [0079], [0088], [0096]; where the decision-making block receives inputs from the sensor subsystem, inputs by a teleoperator, inputs by a local operator (e.g., the driver turning the steering wheel); and where the inputs can be received at the remote teleoperation interface, such as receiving driver input from the remote vehicle operator using a connected replica steering wheel (i.e. steering wheel input to include position and angle));
displaying, using the display: the one or more data points generated by the one or more sensors (e.g. Paragraph [0050]; where the route plan can include graphical indicators that can be provided at a teleoperation interface (171) (e.g., rendered at a heads-up display of a remote operator); and the remote teleoperation interface can include a display and a set of inputs, including sensor data (1100) transmitted to the remote teleoperation interface (171) and rendered at the display to a remote vehicle operator),
wherein the one or more data point comprise a camera feed of the field of view of the video (e.g. Paragraph [0038]; where the sensor includes trajectories of vehicles in the field of view (FOV) of the sensors); and
a first trajectory and a second trajectory of the vehicle (e.g. Paragraph [0064]; where the trajectory generator (132) determines a set of potential trajectories that can be used to traverse the physical space surrounding the vehicle) ‘…’, wherein:
the first trajectory is automatically generated based on vehicle route information (e.g. Paragraphs [0055], [0062]-[0064], [0101]; where the system can automatically generate a planner primitive based on the received sensor data; and where the prediction block predicts trajectories, positions, speeds, etc., of any detected objects in the area surrounding the vehicle and the trajectory generator (132) determines the desired trajectory of the vehicle, and generates trajectory planner primitives based on traffic data, the cost map, and directives from the remote operator), and ‘…’
using one or more remote actuation controls, manually or automatically generate one or more driving actions (e.g. Paragraphs [0026], [0057], [0059], [0071]; where the teleoperator can transmit control instructions to the system; where the system (100) includes a behavior planning module (130) which generates control commands, and the control module (150) receives control commands and controls the actuation subsystem (153); and where the computing system may automatically select a task-block based on the data, where the associated task can be automatically performed),
switching from automatically generating the one or more driving actions to manually generating the one or more driving actions when a remote trajectory demand is present (e.g. Paragraphs [0057], [0106], [0109]-[0110]; where the teleoperator can manually select a task block when the vehicle action can be presented to the teleoperator, who can select or approve the action, and the teleoperator can transmit a signal to the vehicle to transition out of the autonomous mode after the teleoperator has assumed control of the vehicle; and where the instructions can include transferring planning authority to the teleoperator, and the directive can be received from (e.g., manually triggered by) a teleoperator (i.e. a remote trajectory demand is present, and the teleoperator performs manual controls by selecting vehicle actions)); wherein:
the one or more remote actuation controls are remote from the vehicle (e.g. Paragraphs [0014], [0057], [0085], [0105]; where the teleoperator can select vehicle actions based on the processed data, and where vehicle mechanisms can be operated by a remote operator), and
the one or more driving actions correlate to one or more actuator commands configured to cause the vehicle to perform the one or more driving actions (e.g. Paragraph [0071], [0075]]; where the control outputs can be used to control the actuation subsystem, where the task block can generate control instructions/ signals for control of the actuation subsystem to directly control the control elements of the vehicle (e.g., throttle, steering));
transmitting, using the transmitter, the one or more driving actions to the vehicle (e.g. Paragraphs [0053], [0057], [0090], [0110]; where the teleoperator can select vehicle actions based on the processed data and send it to the vehicle to be performed by the onboard vehicle system; and where the selected vehicle action can be transmitted to the vehicle, where the teleoperator performs the action (e.g. using the actuation subsystem of the vehicle); where when the system (100) is in the teleoperation mode includes a remote operator transmitting control instructions to the vehicle system; and where operator inputs are received from the operator and can include a transmission from a teleoperator to actuate a portion of the actuation subsystem); and
causing the vehicle to perform the one or more actions (e.g. Paragraphs [0014], [0057], [0109]; where the teleoperation mode includes a remote operator transmitting control instructions to the system and controlling the vehicle based on the control instructions).
Tiwari fails to disclose every feature of a first trajectory and a second trajectory of the vehicle over the camera feed; and the second trajectory is based on the steering position of the vehicle and is configured to show a reaction of the vehicle to the steering position of the vehicle.
However, Lang, in a similar field of endeavor, teaches the features of display, using the display, a first trajectory and second trajectory of the vehicle over the camera feed.
Lang teaches a method for providing a visual display system for a vehicle, where the visual capturing system may be a camera or an image sensor, where the trajectory information may be superimposed over the image/ feed on the image display unit; and where the trajectory (10) illustrates the track/ driving path of the front wheels, and may illustrate the actual track/driving path, and the trajectory (10’) illustrates the track/driving path of the rear wheels, dependent on a steering angle of the vehicle (i.e., first and second trajectories are displayed) (e.g. Paragraphs [0050], [0060]-[0061]; Figures 3, 11).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to modify the vehicle control system of Tiwari, with the feature of overlaying a trajectory on to a display in the system of Lang, in order to improve the display of a future track of the vehicle (see at least Paragraph [0032] of Lang).
Tamagawa, in a similar field of endeavor, further teaches the features of the second trajectory is based on the steering position of the vehicle and is configured to show a reaction of the vehicle to the steering position of the vehicle.
Tamagawa teaches a method for remotely operating a vehicle, where a trajectory line is superimposed onto the display representing the predicted trajectory of the vehicle (12) on the display unit (22) of the remote cockpit (20), and the display unit (22) displays the trajectory line linked to the steering amount of the steering wheel (36) on the remote cockpit (20) side and the steering amount of the steering wheel on the vehicle (12) side, and the trajectory line is configured to be displayed so that its display changes when the steering wheel on the vehicle (12) side is positioned at or near neutral (e.g. Paragraphs [0048]-[0050]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the vehicle control system of Tiwari, in view of Lang, with the feature of overlaying a steering trajectory on to a display in the system of Tamagawa, in order to notify the driver of the vehicle and the remote operator of the steering wheel position of the vehicle state (see at least Paragraphs [0037]-[0038] of Tamagawa).
As per Claim 2, and similarly for Claim 11, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 1 and 10, respectively, and Tiwari further discloses the features of wherein the transmitter is configured to receive label data from the vehicle (e.g. Paragraphs [0066], [0096]-[0097], [0114]; Figure 6; where the teleoperator can label a nearby vehicle as an “erratic” vehicle and transmit the label to the behavior planning module (e.g., in the form of a directive to avoid the vehicle); and where the sampling sensor data continuously samples data that includes an image stream, a localization signal, and operational data, where the operational data includes the current vehicle speed, current vehicle steering angle, current throttle status, accelerometer data, etc., and the sensor data is transmitted off the vehicle to a remote entity).
As per Claim 3, and similarly for Claim 12, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 2 and 11, respectively, and Tiwari further discloses the features of wherein the label data comprises changes in one or more of the following: a speed of the vehicle; and an acceleration of the vehicle (e.g. Paragraphs [0057], [0096]-[0097], [0114]; Figure 6; where the sampling sensor data continuously samples data that includes an image stream, a localization signal, and operational data, where the operational data includes the current vehicle speed, current vehicle steering angle, current throttle status, accelerometer data, etc., and the sensor data is transmitted off the vehicle to a remote entity; and where low-level data is streamed to the remote computing system, including data indicative of a force applied to the accelerator pedal (i.e. indicative of a change in acceleration or speed)).
As per Claim 4, and similarly for Claim 13, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 2 and 11, respectively, and Tiwari further discloses the features of wherein the display is configured to display the label data as one or more visual cues (e.g. Paragraph [0091]; where the remote teleoperation interface (171) can display transmitted sensor data (1100), where the display includes different visual indicators).
As per Claim 5, and similarly for Claim 14, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 4 and 13, respectively, and Tamagawa further teaches the features of wherein the one or more visual cues comprise one or more of the following: one or more color indicators; blinking text; and one or more flashes.
Tamagawa teaches a method for remotely operating a vehicle, where a trajectory line is superimposed onto the display representing the predicted trajectory of the vehicle (12) on the display unit (22) of the remote cockpit (20); and where the notification unit provides an optical flow, in which at least the left and right sides of the display screen of the display unit (22) are arranged in a line according to the speed of the vehicle (12) and the steering amount of the steering wheel on the vehicle (12) side, and the optical flow is superimposed on the display so that a plurality of light points arranged in a row move in order, and the color depth, color type, color gradation, line thickness, line type, etc., are displayed and changed on the display (e.g. Paragraphs [0045]-[0048], [0050]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the Applicant’s invention, with a reasonable expectation for success, to further modify the vehicle control system of Tiwari, in view of Lang, with the feature of displaying visual cues trajectory on to a display in the system of Tamagawa, in order to notify the driver of the vehicle and the remote operator of the steering wheel position of the vehicle state (see at least Paragraphs [0037]-[0038] of Tamagawa).
As per Claim 6, and similarly for Claim 15, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 1 and 10, respectively, and Tiwari further discloses the features of further comprising
a processor configured to perform remote station system control (e.g. Paragraph [0057]; where the decision-making block (135) functions to determine a task block to execute based on inputs to select a vehicle action from a set of available vehicle actions, which can be executed at a remote computing system), wherein: the instructions, when executed by the processor, are further configured to cause the processor to
perform remote station system control (e.g. Paragraphs [0014], [0071], [0075]; where the teleoperator can transmit control instructions to the system; where the control outputs can be used to control the actuation subsystem, where the task block can generate control instructions/ signals for control of the actuation subsystem to directly control the control elements of the vehicle (e.g., throttle, steering)), and the performing the remote station system control comprises:
receiving, via a control module, the one or more driving actions (e.g. Paragraph [0057]; where the decision-making block (135) functions to determine a task block to execute based on inputs to select a vehicle action from a set of available vehicle actions, which can be executed at a remote computing system; and where the teleoperator can select vehicle actions based on the processed data, send the selected vehicle action to the vehicle, which is then performed by the system); and
causing the vehicle, via the one or more actuation controls, to perform the one or more driving actions (e.g. Paragraphs [0057], [0071], [0075], [0081]; where the decision-making block (135) functions to determine a task block to execute based on inputs to select a vehicle action from a set of available vehicle actions, which can be executed at a remote computing system; and where the teleoperator can select vehicle actions based on the processed data, send the selected vehicle action to the vehicle, which is then performed by the system; where the vehicle actions can include a predefined task used to control the actuation subsystem, such as commands for braking, turning, unlocking the vehicle, actuating a vehicle actuator such as for speed control, etc.).
As per Claim 7, and similarly for Claim 16, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 1 and 10, respectively, and Tiwari further discloses the features of further comprising the one or more sensors (e.g. Paragraph [0036]; where the sensor subsystem includes at least one mapping sensor and at least one monitoring sensor), wherein:
the one or more sensors comprise: a Light Detection and Ranging (LiDAR) sensor (e.g. Paragraphs [0036]-[0037], [0039]; where mapping sensors of the sensor system gathers image data, range data (e.g. LIDAR, radar, TOF, etc.), and can include radar, LIDAR, cameras, navigation sensors, etc.); and
a camera (e.g. Paragraphs [0036], [0039]; where mapping sensors of the sensor system gathers image data, range data (e.g. LIDAR, radar, TOF, etc.), and can include radar, LIDAR, cameras, navigation sensors, etc.), and
the one or more data points comprise: a LiDAR point cloud generated by the LIDAR sensor (e.g. Paragraphs [0036], [0113]; where the sensor subsystem gathers the sensor data, such as range data (e.g. LIDAR data, point cloud data); and where the sensor data includes image data collected from one or more camera, as well as cloud point data from one or more rangefinding sensors of the vehicle); and
an image captured by the camera (e.g. Paragraphs [0036], [0038], [0113]; where the sensor system gathers sensor data, such as image data (e.g., still images, video streams, compressed image sequence, etc.) from camera data; and where the sensor data includes image data collected from one or more camera, as well as cloud point data from one or more rangefinding sensors of the vehicle).
As per Claim 8, and similarly for Claim 17, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 1 and 10, respectively, and Tiwari further discloses the features of further comprising the one or more sensors, wherein: the one or more sensors comprise one or more cameras configured to generate one or more images of the environment of the vehicle, and the display is configured to display the one or more images of the environment of the vehicle (e.g. Paragraphs [0031], [0042]; Claim 2; where the perception module (110) of the system (100) functions to perceive environment surrounding the vehicle and output sensor data indicative of the features of the environment to directly perceive the environment (e.g., to sample imaging sensors, rangefinding sensors, etc.); and where the display device may be configured to receive stereoscopic video data transmitted from the movable object (i.e. a vehicle) and display an first person view of the environment).
As per Claim 9, and similarly for Claim 18, Tiwari, in view of Lang and Tamagawa, teaches the features of Claims 1 and 10, respectively, and Tiwari further discloses the features of wherein the one or more actuation controls comprise one or more of: a brake pedal; an acceleration pedal; a gear shift control; and a steering wheel (e.g. Paragraphs [0081]-[0082]; where the actuation subsystem (153) functions to actuate the control interferences of the vehicle, to include a throttle actuation interface, a brake actuation interface, a steering actuation assembly, and a pedal actuation mechanism).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Chaveau (WO 2018/172886 A1), which teaches a method for superimposing vehicle trajectory based on steering angle and vehicle speed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MERRITT LEVY whose telephone number is (571)270-5595. The examiner can normally be reached Mon-Fri 0630-1600.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached at (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MERRITT LEVY/Examiner, Art Unit 3663
/ABBY J FLYNN/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights