DETAILED ACTION
Status of Claims
This action is in response to the application No. 18/612456 filed on 3/21/2024. Claims 1-20 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a remote manual control device,
an operator input device,
a switching information acquisition unit, and
a switching instruction unit in claim 10;
an information acquisition unit in claim 14;
a display unit; and
a display control unit in claim 18;
an area setting unit in claim 20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 6, 10, and 14-16 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Altman US 12,197,209 (“Altman”).
As to claims 1 and 10, Altman discloses a remote operation device that remotely controls an operation of a moving object, comprising:
a remote manual control device (see at least Fig 2, element 255, related text; Tele-Operator Terminal/Human-Machine Interface); and
an operator input device that is connected to the remote manual control device (col 34, lines 25-41: The HMI units of the Tele-Operator Terminal/HMI 255 may include, for example, touch screen, screen, joystick, touch-pad, computer mouse, steering wheel, pedals, gear shift device, microphone, speakers, Augmented Reality (AR) or Virtual Reality (VR) equipment (e.g., wearable device, helmet, headgear, glasses, googles, gloves, or the like), haptic elements, tactile elements, gloves, other wearable elements, lights, alarms, or the like), wherein the moving object has
a manned driving mode that a driver having got on the moving object operates a driver input device provided to the moving object, and thereby determines a running condition that defines a running operation of the moving object to run (see at least col 5, lines 53-55: force the vehicle into being controlled manually and exclusively by a human driver or a human passenger located within the vehicle), and
a remote unmanned driving mode that the moving object receives the running condition of the moving object from an outside of the moving object, and thereby runs without the driver's getting on the moving object (see at least col 34, lines 25-51: A Tele-Operator Terminal/Human-Machine Interface (HMI) 255 for the human tele-operator may also be in communication (directly, or indirectly) with one or more of the vehicles, such as with the primary vehicle 210; and may enable a human tele-operator or a machine-based tele-operator to remotely interact, intervene, operate or drive or control the primary vehicle 210 or otherwise affect its operation via remote commands or remote suggestions or remotely-obtained data or remotely-generated commands),
the remote unmanned driving mode includes
a remote manual driving mode that the moving object receives the running condition generated by an operator by operating an operator input device provided at a place different from a place of the moving object, and thereby runs without the driver's getting on the moving object (see at least col 34, lines 25-51: A Tele-Operator Terminal/Human-Machine Interface (HMI) 255 for the human tele-operator may also be in communication (directly, or indirectly) with one or more of the vehicles, such as with the primary vehicle 210; and may enable a human tele-operator), and
a remote autonomous driving mode that the moving object receives the running condition generated by a remote automatic control device provided at a place different from the place of the moving object, and thereby runs without an operation of any one of the driver input device and the operator input device (see at least col 39, lines 22-37: a vehicular autonomous driving unit, configured to autonomously operate said vehicle based on both (I) inputs generated locally within said vehicle by said AI unit, and (II) said vehicular operating command received from said remote tele-driving processor),
the remote manual control device includes
a switching information acquisition unit that acquires at least one of (2a) abnormality information indicating that a state of the moving object has become an abnormal state, (2b) intrusion information indicating that a moving object has intruded a surrounding area set in advance to surroundings of the moving object, and (2c) failure information indicating that a failure has occurred during running in the remote autonomous driving mode (see at least col 7, lines 31-40: This scenario, in which the AI self-driving module is stopping to function as the dynamic driver, may be referred to as “disengagement” of the self-driving module or system. In some situations, such disengagement implies a partial or full failure of the local AI module(s) to perform the actual dynamic self-driving driving operations, momentary or during a prolonged time period or even permanently; col 23, lines 14-19: In case the communications conditions improve, or the critical obstacle that prevented the local or remote AI from handling case is removed or is traversed by the remote human tele-operator, then the situation may be handed over to the remote AI or to the in-vehicle AI), and
a switching instruction unit that, when a predetermined first switching condition is satisfied, transmits to the moving object a first switching instruction for switching a driving mode of the moving object from the remote autonomous driving mode to the remote manual driving mode (see at least col 4, lines 7-39: In case the communications conditions improve, or the critical obstacle that prevented the local or remote AI from handling case is removed or is traversed by the remote human tele-operator, then the situation may be handed over to the remote AI or to the in-vehicle AI), and
the first switching condition includes as a condition a case where, during the running in the remote autonomous driving mode, the moving object falls in at least one state of (1a) a first state where the state of the moving object has become the abnormal state, (lb) a second state where the moving object has intruded the surrounding area, (1c) a third state that is different from the first state and in which the failure has occurred during the running in the remote autonomous driving mode (see at least col 7, lines 31-40: This scenario, in which the AI self-driving module is stopping to function as the dynamic driver, may be referred to as “disengagement” of the self-driving module or system. In some situations, such disengagement implies a partial or full failure of the local AI module(s) to perform the actual dynamic self-driving driving operations, momentary or during a prolonged time period or even permanently; col 23, lines 14-19: In case the communications conditions improve, or the critical obstacle that prevented the local or remote AI from handling case is removed or is traversed by the remote human tele-operator, then the situation may be handed over to the remote AI or to the in-vehicle AI).
As to claims 2 and 14, Altman discloses wherein the remote manual control device further includes an information acquisition unit that, when the driving mode is switched from the remote autonomous driving mode to the remote manual driving mode, acquires at least one of first sensor information acquired by a first sensor mounted on the moving object and detection information acquired by a detection device provided at a place different from the place of the moving object (see at least col 18, lines 5-22: The system thus uses at least one communication connection to transmit relevant presentation of the relevant sensory data to minimize the latency at which such teleoperator may take over the vehicle. Instead of the vehicle AI component starting to transmit the relevant representation when the required moment of disengagement comes, it transmits a potentially different set of representations of potentially different or same sensory of predictive information before the moment of disengagement and of remote intervention comes. For example, it may continuously assign a relatively high priority, but not top priority, to a sparse or diluted or partial subset of the information or their vectors and therefore continuously transmit it over other less prioritized items. The remote component shall be able to display this sparse representation in a way sufficient for the operator to be able to take over the dynamic driving of the vehicle at any chosen level of certainty or probability or clarity),
the first sensor information includes at least one of moving object position information indicating a position of the moving object, and surroundings information indicating a state of the surrounding area (see at least col 30, lines 5-30: Such vehicles may be referred to herein as “autonomous vehicle”, and may utilize a variety of sensors and detectors in order to sense, detect and/or identify their surroundings and/or other data (e.g., traffic lights, traffic signs, lanes, road blocks, road obstacles, navigation paths, pedestrians); for example, cameras, image camera, video cameras, acoustic microphones, audio capturing devices, image and video capturing devices, radar sensors or devices, LIDAR sensors or devices, laser-based sensors or devices, magnetic or magnet-based sensors and devices, ultrasonic sensors, night-vision modules, Global Positioning System (GPS) units…), and
the detection information includes information that is acquired from the outside of the moving object, and is at least one of the moving object position information indicating the position of the moving object, and the surroundings information indicating the state of the surrounding area (see at least col 30, lines 5-30: Such vehicles may be referred to herein as “autonomous vehicle”, and may utilize a variety of sensors and detectors in order to sense, detect and/or identify their surroundings and/or other data (e.g., traffic lights, traffic signs, lanes, road blocks, road obstacles, navigation paths, pedestrians); for example, cameras, image camera, video cameras, acoustic microphones, audio capturing devices, image and video capturing devices, radar sensors or devices, LIDAR sensors or devices, laser-based sensors or devices, magnetic or magnet-based sensors and devices, ultrasonic sensors, night-vision modules, Global Positioning System (GPS) units…).
As to claims 3 and 15, Altman discloses wherein the first sensor is a sensor that can detect at least the state of the surrounding area (see at least col 30, lines 5-30: Such vehicles may be referred to herein as “autonomous vehicle”, and may utilize a variety of sensors and detectors in order to sense, detect and/or identify their surroundings and/or other data (e.g., traffic lights, traffic signs, lanes, road blocks, road obstacles, navigation paths, pedestrians); for example, cameras, image camera, video cameras, acoustic microphones, audio capturing devices, image and video capturing devices, radar sensors or devices, LIDAR sensors or devices, laser-based sensors or devices, magnetic or magnet-based sensors and devices, ultrasonic sensors, night-vision modules, Global Positioning System (GPS) units…).
As to claims 4 and 16, Altman discloses wherein the first sensor is at least one of a camera, a LiDAR, and a radar (see at least col 30, lines 5-30: Such vehicles may be referred to herein as “autonomous vehicle”, and may utilize a variety of sensors and detectors in order to sense, detect and/or identify their surroundings and/or other data (e.g., traffic lights, traffic signs, lanes, road blocks, road obstacles, navigation paths, pedestrians); for example, cameras, image camera, video cameras, acoustic microphones, audio capturing devices, image and video capturing devices, radar sensors or devices, LIDAR sensors or devices, laser-based sensors or devices, magnetic or magnet-based sensors and devices, ultrasonic sensors, night-vision modules, Global Positioning System (GPS) units…).
As to claim 6, Altman discloses when a predetermined second switching condition is satisfied, the switching execution unit switches the driving mode from the remote manual driving mode to the remote autonomous driving mode, and the second switching condition includes as a condition a case that does not correspond to any one of the first state, the second state, and the third state (see at least col 21, lines 20-65: a remote AI module may take over from the human tele-operator, or vice versa, or the two may interact wherein the remote AI advises the teleoperator about the alternatives or recommends options to it, or the human operator alleviates potential risks or limitations so that the remote or in-vehicle AI unit(s) can then continue with the route planning alternatives or recommendation or actual driving).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Altman in view of Akaba et al. US 11,046,332 (“Akaba”).
As to claim 5, Altman fails to explicitly disclose further comprising a broadcasting control unit that, when the switching execution unit switches the driving mode from the remote autonomous driving mode to the remote manual driving mode, broadcasts to the outside that the moving object is running in the remote manual driving mode. However, Akaba teaches further comprising a broadcasting control unit that, when the switching execution unit switches the driving mode from the remote autonomous driving mode to the remote manual driving mode, broadcasts to the outside that the moving object is running in the remote manual driving mode (see at least Fig 8, related text).
Thus, Altman discloses a system and method for autonomous tele-operation of vehicles and Akaba teaches a similar tele-operation system which further includes a display that broadcasts on the outside of the moving vehicle whether or not the vehicle is being operated remotely.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclsoed by Altman, with the broadcast display taught by Akaba, with reasonable expectation of success, because it would alert people about the vehicle that it is being remotely operated and not operated by a driver onboard the vehicle.
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Altman in view of Krunic et al. US 2018/0053415 (“Krunic”).
As to claim 7, Altman fails to explicitly disclose wherein the third state is a case where a position of the moving object cannot be specified using imaging information. However, Krunic teaches wherein the third state is a case where a position of the moving object cannot be specified using imaging information (see at least [0045]: various responses at the vehicle may be automatically triggered based on detection of certain data. In some examples, based on a detected level of glare encountered along a route, the glare detection system 260 may command the vehicle to automatically darken the windshield, switch at least one aspect of vehicle operation from automated to manual if the glare level impairs the ability of the automated control system, switch at least one aspect of vehicle operation from manual to automated control if the glare level impairs the ability of the driver to operate the vehicle).
Thus, Altman discloses a system for tele-operating a vehicle autonomously or manually and Krunic teaches to switch from an autonomous driving mode to a manual driving mode in the event that glare is too high that the vehicle sensors cannot properly detect the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclosed by Altman, with the glare detection taught by Krunic, with reasonable expectation of success, because it would provide safe operation of the vehicle in the event of high glare situations.
As to claim 8, Altman fails to explicitly disclose wherein the third state is specified on a basis of at least one of a weather condition and a time zone. However, Krunic teaches wherein the third state is specified on a basis of at least one of a weather condition and a time zone (see at least [0045]: various responses at the vehicle may be automatically triggered based on detection of certain data. In some examples, based on a detected level of glare encountered along a route, the glare detection system 260 may command the vehicle to automatically darken the windshield, switch at least one aspect of vehicle operation from automated to manual if the glare level impairs the ability of the automated control system, switch at least one aspect of vehicle operation from manual to automated control if the glare level impairs the ability of the driver to operate the vehicle).
Thus, Altman discloses a system for tele-operating a vehicle autonomously or manually and Krunic teaches to switch from an autonomous driving mode to a manual driving mode in the event that glare is too high that the vehicle sensors cannot properly detect the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclosed by Altman, with the glare detection taught by Krunic, with reasonable expectation of success, because it would provide safe operation of the vehicle in the event of high glare situations.
As to claim 9, Altman fails to explicitly disclose wherein the third state is specified on a basis of a shadow production situation caused by glaring of light. However, Krunic teaches wherein the third state is specified on a basis of a shadow production situation caused by glaring of light (see at least [0045]: various responses at the vehicle may be automatically triggered based on detection of certain data. In some examples, based on a detected level of glare encountered along a route, the glare detection system 260 may command the vehicle to automatically darken the windshield, switch at least one aspect of vehicle operation from automated to manual if the glare level impairs the ability of the automated control system, switch at least one aspect of vehicle operation from manual to automated control if the glare level impairs the ability of the driver to operate the vehicle).
Thus, Altman discloses a system for tele-operating a vehicle autonomously or manually and Krunic teaches to switch from an autonomous driving mode to a manual driving mode in the event that glare is too high that the vehicle sensors cannot properly detect the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclosed by Altman, with the glare detection taught by Krunic, with reasonable expectation of success, because it would provide safe operation of the vehicle in the event of high glare situations.
Claim(s) 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Altman in view of Iwai et al. US 2023/0133873 (“Iwai”).
As to claim 17, Altman fails to explicitly disclose further comprising a display device that includes a display unit that conveys at least one of the first sensor information and the detection information to the operator, wherein the remote manual control device further includes a display control unit that controls display contents to display on the display unit.
However, Iwai teaches further comprising a display device that includes a display unit that conveys at least one of the first sensor information and the detection information to the operator, wherein the remote manual control device further includes a display control unit that controls display contents to display on the display unit (see at least Fig 6, Fig 8, related text; [0052]: The monitoring screen display unit (image display means) 114 displays the image received by the image reception unit 111. The monitoring screen display unit 114 displays, for example, images of the areas on the front, rear, right, and left sides of the vehicle, which are taken by the cameras 300 (refer to FIG. 4), on a display screen. An observer, by monitoring the display screen, monitors whether or not there is a hindrance to traveling of the vehicle 200.).
Thus, Altman discloses a system and method for tele-operating a vehicle and Iwai teaches a similar vehicle operation device that further includes a display to monitor the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclsoed by Altman, with the display taught by Iwai, with reasonable expectation of success, because it would provide a direct monitoring ability for an operator of the vehicle.
As to claim 18, Altman fails to explicitly disclose wherein the display control unit causes the display unit to display an information designation screen for allowing the operator to designate the first sensor information or the detection information to display on the display unit.
However, Iwai teaches wherein the display control unit causes the display unit to display an information designation screen for allowing the operator to designate the first sensor information or the detection information to display on the display unit (see at least Fig 6, Fig 8, related text; [0052]: The monitoring screen display unit (image display means) 114 displays the image received by the image reception unit 111. The monitoring screen display unit 114 displays, for example, images of the areas on the front, rear, right, and left sides of the vehicle, which are taken by the cameras 300 (refer to FIG. 4), on a display screen. An observer, by monitoring the display screen, monitors whether or not there is a hindrance to traveling of the vehicle 200.).
Thus, Altman discloses a system and method for tele-operating a vehicle and Iwai teaches a similar vehicle operation device that further includes a display to monitor the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclsoed by Altman, with the display taught by Iwai, with reasonable expectation of success, because it would provide a direct monitoring ability for an operator of the vehicle.
As to claim 19, Altman fails to explicitly disclose wherein at least one of the first sensor and the detection device is a camera, the detection information may include imaging information imaged by the camera, and intrusion position information indicating a position of the moving object that has intruded the surrounding area, and the display control unit causes the display unit to display information obtained by superimposing the intrusion position information on the imaging information.
However, Iwai teaches wherein at least one of the first sensor and the detection device is a camera, the detection information may include imaging information imaged by the camera, and intrusion position information indicating a position of the moving object that has intruded the surrounding area, and the display control unit causes the display unit to display information obtained by superimposing the intrusion position information on the imaging information (see at least Fig 6, Fig 8, related text; [0052]: The monitoring screen display unit (image display means) 114 displays the image received by the image reception unit 111. The monitoring screen display unit 114 displays, for example, images of the areas on the front, rear, right, and left sides of the vehicle, which are taken by the cameras 300 (refer to FIG. 4), on a display screen. An observer, by monitoring the display screen, monitors whether or not there is a hindrance to traveling of the vehicle 200.).
Thus, Altman discloses a system and method for tele-operating a vehicle and Iwai teaches a similar vehicle operation device that further includes a display to monitor the vehicle environment.
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system disclsoed by Altman, with the display taught by Iwai, with reasonable expectation of success, because it would provide a direct monitoring ability for an operator of the vehicle.
Allowable Subject Matter
Claims 11-13 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS P INGRAM whose telephone number is (571)272-7864. The examiner can normally be reached M-F 10-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fadey Jabr can be reached at 571-272-1516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Thomas Ingram/ Primary Examiner, Art Unit 3668