DETAILED ACTION
1. Claims 1-11 of application 19/136,901, filed on 9-June-2025, are presented for examination. The IDS received on the same date has been considered.
The present application, filed on or after 16-March-2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
2. Claim 1 is objected to under 37 CFR 1.75(d)(1), wherein --
The claim or claims must conform to the invention as set forth in the remainder of the specification and the terms and phrases used in the claims must find clear support or antecedent basis in the description so that the meaning of the terms in the claims may be ascertainable by reference to the description. (See 1.58(a))
The following is a list of the specific objections:
The phrase “autonomous operated” [ln. 2] is understood as and should be changed to
“autonomously operated”.
Claim Rejections under 35 U.S.C. § 102(a)(1)
3.1 The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
3.2 Claims 1-11 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Katz, USP Publication 2022/0203996.
3.3 Katz discloses:
Claim 1: A remote operation assistance apparatus temporarily operating an autonomous
operation system which is autonomous operated from a remote location [¶0032; 0076; 0079 (a drone that is controlled/piloted by a remote driver, an autonomous flying vehicle, and any other machine that may be driven, piloted, or controlled by a human user. In some embodiments, vehicle may include a self-driving vehicle, autonomous vehicle, semi-autonomous vehicle, vehicle traveling on the ground, including but not limited to cars, buses, tracks, train, army-related vehicles, a flying a vehicle, including but not limited to airplanes, helicopters, drones, flying “cars”/taxis, semi-autonomous flying vehicles, or the like); 0108 (the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction.)], comprising:
state detection circuitry detecting surrounding environment information of the autonomous operation system and information of the autonomous operation system [0052 (environmental conditions); 0070 (Current or future driving circumstances, for example, may include one or more road-related parameters or environmental conditions); 0091 (Sensors may include, for example, a CCD image sensor, a CMOS image sensor, a camera, a light sensor, an IR sensor, an ultrasonic sensor, a proximity sensor, a shortwave infrared (SWIR) image sensor, a reflectivity sensor, or any other device that is capable of sensing visual characteristics of an environment); 0093 (Data associated with the environment may include the data related to internal or external parameters of the environment in which the user is located.); 0184 (various environmental conditions)];
an operation interface receiving input from a remote operator including an intervention operation of remotely controlling the autonomous operation system and outputting the input as operation information [0062 (Based on the determined level of control of the driver over the vehicle…); 0104 (when the processor detects the user moving the user's hand toward a multimedia system in a car, the processor may be able to predict that the user's gaze will follow the user's finger rather than stay on the road when the user's fingers move near the display or touch-display of the multimedia system.); 0134 (Gestures may be any motion of one or more part of the user's body, whether the motion of that one or more part is performed mindfully or not, as an action with a purpose to activate something (such as turn on/off the air-condition) or as a way of expression (such as when people are talking and moving their hands simultaneously, or nodding with their head while listening).)];
state presentation circuitry obtaining the surrounding environment information and the information of the autonomous operation system from the state detection circuitry and presenting the surrounding environment information and the information of the autonomous operation system to the remote operator [0052; 0070; 0093; 0108 (the processor may be configured to alert the driver that the driver should not operate the vehicle. In other embodiments, the processor may be configured to control an operation mode of the vehicle based on the detected or predicted change in gaze direction. For example, the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction)];
remote operator state recognition circuitry obtaining state information of the remote operator and recognizing a state of the remote operator using the state information and the operation information obtained from the operation interface [0035 (a driver monitoring system (DMS) may be configured to monitor driver behavior. DMS may comprise a system that tracks the driver and acts accordingly to the driver's detected state, physical condition, emotional condition, cognitive load, actions, behaviors, driving performance, attentiveness, alertness, drowsiness.); 0041-0051 (various detectable states of the driver); 0085; 0091; 0206];
display setting dynamic changing circuitry dynamically determining setting of information presented to the remote operator in accordance with the state of the remote operator recognized in the remote operator state recognition circuitry and outputting the setting as display setting [0032 (activating an activatable icon, selecting a menu item, moving a pointer on a display, manipulating a touch free mouse, an activatable icon on a display, altering information on a display.); 0058 (various indications displayed on a display screen); 0122-0124; 0140 (the location of the object associated with predefined object 24 in the image information may be used to determine a corresponding location on display 4 (including, for example, a virtual location on display 4 that is outside the boundaries of display 4), and the corresponding location on display 4 may be used as the detected location of the gesture in the image information.); 0515; 0517]; and
failure response supplemental information presentation circuitry displaying supplemental information supplementing determination of the remote operator on a display device based on the display setting obtained from the display setting dynamic changing circuitry [0168 (the at least one processor is also configured to cause an action associated with the detected gesture, the detected gesture location, and a relationship between the detected gesture location and the control boundary…a message or execution of a command associated with the gesture…an indication to a user that some event has occurred…an icon presented on a display, change of an icon presented on a display, a change in color of an icon presented on a display, an indication light, an indicator moving on a display, a directional vibration indication, or an air tactile indication. Moreover, for example, the indicator may appear on top of all other images appearing on the display.); 0548-0550; 0561-0562].
Claim 2: wherein the remote operator state recognition circuitry estimates a learning level of the remote operator on an operation, busyness of the remote operator, and a fatigue degree of the remote operator, and recognizes the learning level, the busyness, and the fatigue degree [0035] as the state of the remote operator [0041-0051 (various detectable states of the driver); 0085; 0091; 0206].
Claim 3: wherein the display setting dynamic changing circuitry dynamically determines setting of form of the information and an amount of the information presented to the remote operator, and the failure response supplemental information presentation circuitry adjusts the form of the information and an amount of information obtained from the display setting dynamic changing circuitry and displays the information on the display device [0052; 0070; 0093; 0108 (the processor may be configured to alert the driver that the driver should not operate the vehicle. In other embodiments, the processor may be configured to control an operation mode of the vehicle based on the detected or predicted change in gaze direction. For example, the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction)].
Claim 4: further comprising remote operator state presentation circuitry transmitting the state of the remote operator recognized in the remote operator state recognition circuitry to the remote operator [0035 (a driver monitoring system (DMS) may be configured to monitor driver behavior. DMS may comprise a system that tracks the driver and acts accordingly to the driver's detected state, physical condition, emotional condition, cognitive load, actions, behaviors, driving performance, attentiveness, alertness, drowsiness.); 0041-0051 (various detectable states of the driver); 0085; 0091; 0206] and transmitting a purpose of dynamically changing the display setting to the remote operator [0052; 0070; 0093; 0108 (the processor may be configured to alert the driver that the driver should not operate the vehicle. In other embodiments, the processor may be configured to control an operation mode of the vehicle based on the detected or predicted change in gaze direction. For example, the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction)].
Claim 5: further comprising the display setting dynamic changing circuitry dynamically determines the display setting using the information of the autonomous operation system obtained from the state detection circuitry [0052; 0070; 0093; 0108 (the processor may be configured to alert the driver that the driver should not operate the vehicle. In other embodiments, the processor may be configured to control an operation mode of the vehicle based on the detected or predicted change in gaze direction. For example, the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction); 0032 (activating an activatable icon, selecting a menu item, moving a pointer on a display, manipulating a touch free mouse, an activatable icon on a display, altering information on a display.); 0058 (various indications displayed on a display screen); 0122-0124; 0140 (the location of the object associated with predefined object 24 in the image information may be used to determine a corresponding location on display 4 (including, for example, a virtual location on display 4 that is outside the boundaries of display 4), and the corresponding location on display 4 may be used as the detected location of the gesture in the image information.); 0515; 0517].
Claim 6: wherein the remote operator state recognition circuitry obtains viewpoint information of the remote operator [0035; 0041-0051; 0085; 0091; 0206], and recognizes where the remote operator sees in a screen of the display device [0032; 0058 (various indications displayed on a display screen); 0122-0124; 0140; 0515; 0517], and the display setting dynamic changing circuitry dynamically reflects screen information of the screen which the remote operator recognized in the remote operator state recognition circuitry sees on the display setting to make the supplemental information [0168; 0548-0550; 0561-0562].
Claim 7: wherein the remote operator state recognition circuitry obtains viewpoint information of the remote operator [0035; 0041-0051; 0085; 0091; 0206], and recognizes where the remote operator sees in a screen of the display device [0032; 0058 (various indications displayed on a display screen); 0122-0124; 0140; 0515; 0517] and recognizes screen information of the screen on which the remote operator focuses in the intervention operation as attention information [0108 (the processor may be configured to alert the driver that the driver should not operate the vehicle. In other embodiments, the processor may be configured to control an operation mode of the vehicle based on the detected or predicted change in gaze direction. For example, the processor may be configured to change the operation mode of the vehicle from a manual driving mode to an autonomous driving mode based on the detected or predicted change in gaze direction)], and
the display setting dynamic changing circuitry dynamically reflects the attention information of the remote operator recognized in the remote operator state recognition circuitry on the display setting to make the supplemental information [0168; 0548-0550; 0561-0562].
Claim 8: wherein the remote operator state recognition circuitry calculates the learning level, the busyness, and the fatigue degree of the remote operator [0035; 0041-0051 (various detectable states of the driver); 0085; 0091; 0206] by mechanical learning by an artificial intelligence using the state information and the operation information as input information [0064 (machine learning-based determination of the driver's level of control and response time to act in an event of an emergency may be performed offline by training or “teaching” a CNN (convolution neural network) a driver's different levels of control using a database of images and videos of different historical data associated with the driver…For example, an inability or low ability to respond may be associated with damage to the vehicle due to the driver's slow response time or insufficient control of the steering wheel. Conversely, a high ability to respond may be associated with no damage to the vehicle or other harm. The adequacy of the driver's ability to respond may vary depending on the particular driving event or conditions.); 0066-0068].
Claim 9: wherein the display setting dynamic changing circuitry dynamically determines the setting of the information presented to the remote operator by the mechanical learning by the artificial intelligence [0064; 0066-0068] using the learning level, the busyness, and the fatigue degree of the remote operator calculated in the remote operator state recognition circuitry as input information [0035; 0041-0051 (various detectable states of the driver); 0085; 0091; 0206].
Claim 10: wherein the remote operator state recognition circuitry estimates at least one of the busyness of the remote operator and the fatigue degree of the remote operator and recognizes the at least one of the busyness and the fatigue degree as the state of the remote operator [0035; 0041-0051 (various detectable states of the driver); 0085; 0091; 0206].
Claim 11: comprising remote operator state presentation circuitry transmitting notification of the state of the remote operator recognized in the remote operator state recognition circuitry to the remote operator [0035; 0041-0051 (various detectable states of the driver); 0085; 0091; 0206] and notifying the remote operator of a purpose of change including a reason of change in dynamically changing the display setting [0052; 0070; 0093; 0108].
Prior Art
4. The following prior art, discovered in an updated search and herein made of record, is considered pertinent to Applicant’s disclosure, and consists of documents A-G on the attached PTO-892 Notice of References Cited:
Document A defines a document of particular relevance, wherein the claimed invention cannot be considered novel or cannot be considered to involve an inventive step when the document is taken alone.
Document B defines a document of particular relevance, wherein the claimed invention cannot be considered to involve an inventive step when the document is combined with one or more other such documents, such combination being obvious to a person skilled in the art.
Documents C-G define the general state of the art which is not considered to be of particular relevance.
Prior Art of Record
5. The Examiner has cited particular paragraphs or columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. The prompt development of a clear issue requires that the replies of the Applicant meet the objections to and rejections of the claims. Applicant should also specifically point out the support for any amendments made to the disclosure (see MPEP §2163.06). Applicant is reminded that the Examiner is entitled to give the Broadest Reasonable Interpretation (BRI) of the language of the claims. Furthermore, the Examiner is not limited to Applicant’s definition which is not specifically set forth in the claims. [SEE MPEP 2141.02 [R-07.2015] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123].
In addition, disclosures in a reference must be evaluated for what they would fairly teach one of ordinary skill in the art [See In re Snow, 471 F.2d 1400, 176 USPQ 328 (CCPA 1973) and In re Boe, 355 F.2d 961, 148 USPQ 507 (CCPA 1966)]. Specifically, in considering the teachings of a reference, it is proper to take into account not only the specific teachings of the reference, but also the inferences that one skilled in the art would reasonably have been expected to draw from the reference [See In re Freda, 401 F.2d 825, 159 USPQ 342 (CCPA 1968) and In re Shepard, 319 F.2d 194, 138 USPQ 148 (CCPA 1963)]. Likewise, it is proper to take into consideration not only the teachings of the prior art, but also the level of ordinary skill in the art [See In re Luck, 476 F.2d 650, 177 USPQ 523 (CCPA 1973)]. Specifically, those of ordinary skill in the art are presumed to have some knowledge of the art apart from what is expressly disclosed in the references [See In re Jacoby, 309 F.2d 513, 135 USPQ 317 (CCPA 1962)].
Response Guidelines
6.1 A shortened statutory period for response to this non-final action is set to expire 3 (three) months and 0 (zero) days from the date of this letter. Unless the applicant is notified in writing that a reply is required in less than six months (see the shortened response period previously noted), a maximum period of six months is allowed, if a petition for an extension of time and the fee set in § 1.17(a) are filed [see MPEP 710 and 35 U.S.C. 133]. Failure to respond within the required period for response will cause the application to become abandoned [see MPEP 710.02, 710.02(b)].
6.2 Any response to the Examiner in regard to this non-final action should be
directed to: Russell Frejd, telephone number (571) 272-3779, Monday-Friday from 0730 to
1600 ET. If attempts to reach the examiner by telephone are unsuccessful,
please contact the examiner’s supervisor, Peter Nolan, who can be reached at
(571) 270-7016.
mailed to: Commissioner of Patents and Trademarks
P.O. Box 1450, Alexandria, VA 22313-1450
faxed to: (571) 273-8300
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Hand-delivered responses should be brought to the Customer Service Window, Randolph Building, 401 Dulany Street, Alexandria, VA, 22314.
/RUSSELL FREJD/
Primary Examiner AU 3661
Prosecution Insights