DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
This first non-final action is in response to applicant's filing filed on Oct. 07, 2024. Claims 1-9 are pending and have been considered as follows.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1 and 7-9 are rejected under 35 U.S.C. 103 as being obvious over by Lockwood (US 10564638 B1)
Regarding claim 9, Lockwood teaches a control system for an autonomous work machine (construction and agricultural vehicles are disclosed in col. 4, I. 15-21, an automobile having four wheels 128 and respective tires for each of the wheels 128. Other types and configurations of vehicles are contemplated, such as, for example, vans, sport utility vehicles, cross-over vehicles, trucks, buses, agricultural vehicles, construction vehicles), the control system comprising:
a robotic controller (Figs. 2 and 3);
a position detection system coupled to the robotic controller (Figs. 2 and 3; col. 7, I. 20-25. The external sources may include global satellites for facilitating operation of a global positioning system (GPS); see "object data calculator" 216 in Fig. 2 and col 8, I. 13-26) while performing a mission with the autonomous work machine);
a sensor coupled to the robotic controller and configured to provide a sensor signal (Fig. 2, 8, receive sensor data from the driverless vehicle 904 in Fig. 8)
a controlled system coupled to the robotic controller to receive control signals from the robotic controller (“object classifier" 218 in Fig. 2 and col. 8, I.27-59, "collision predictor" 220 in Fig. 2 and col. 9, I. 13-28; Figs. 2 and 3);
wherein the robotic controller is configured to generate an event relative to an object in an environment around the autonomous work machine or a machine health/job quality issue, document the event, and store the documented event ("object data calculator" 216 in Fig. 2 and col 8, I. 13-26, collision predictor" 220 in Fig. 2 and col. 9, I. 13-28, ); and
wherein the robotic controller is configured to selectively generate a communication containing at least some information relative to the documented event to a supervisor (see modules "kinematics calculator" 222, "planner" 214 and "operation state engine" 234 in Fig. 2 respectively described col. 9, I. 29-60, col. 10, I. 32-43 and col. 10, I. 44-50; according to confidence level further described col. 29, I. 15-33; claims 5 and 8; blocs 808A and 808B in fig. 8; col. 9, I. 62 - col. 10, I. 7 and col. 29, I. 33-34, col. 31, I. 5-25, At 908, the example process 900 may include selecting a 40 teleoperation option from among a plurality of teleoperation options to present via the user interface according to any of the techniques discussed herein; to calculate a confidence level associated with a probability that the vehicle 102 will be able to successfully maneuver past the object 624, for example, without assistance from the tele-operations system 148. In some examples, the vehicle 102 20 can record confidence levels over time and determine a rate of change of the confidence levels) and to receive user input from the supervisor and take responsive action based on the user input (Fig. 5B-5C and corresponding paragraphs, an option to activate a horn of the vehicle 102 if the operation state data of the vehicle 102 indicates “near occlusion” as an event at the vehicle 102).
Lockwood does not explicitly teacha machine health/job quality issue, but it is well known that events including vehicles drive in work/construction zone and having low confidence levels require vehicle perform reliable job.
Regarding claim 1, Lockwood teaches a computer-implemented method of handling a machine health/job quality issue (the vehicle 102 may operate autonomously until the vehicle 102 encounters an event along the route 106 for which it may request assistance from, a teleoperations system 148 located remotely from the vehicle 102) in an autonomous work machine ( example vehicle 102 shown in FIG. 1 is an automobile having four wheels 128 and respective tires for each of the wheels 128. Other types and configurations of vehicles are contemplated, such as, for example, vans, sport utility vehicles, cross-over vehicles, trucks, buses, agricultural vehicles, two-wheeled vehicles, and/or construction vehicles), the method comprising:
detecting a machine health/job quality issue while performing a mission with the autonomous work machine(vehicles have sensor to detect safety related events while traveling);
determining, with a robotic controller of the autonomous work machine, whether the detected machine health/job quality issue has an exception routine for the detected machine health/job quality issue (As the confidence level drops below a threshold minimum confidence level (e.g., confidence level is less than or equal to 79%) and/or if the vehicle detects that a rate of change of the confidence levels over time indicates a negative rate of change for a threshold duration of time (e.g., −2% per second for at least 5 seconds) and/or a rate of change having a threshold magnitude (e.g., −9% per second) (and/or any other qualities of various forms of rates of change such as average rate of change over a moving window), the vehicle 102 may slow its speed or stop (exception routine for low confidence events), and use its network interface 234 ( FIG. 2) to send communication signals to the teleoperations system 148 providing sensor data and/or operation state data with a request for guidance from the teleoperations system 148 );
performing the exception routine for the detected machine health/job quality issue (FIG. 6A shows an example vehicle 102 in a first example event scenario in which the example static object 624 is in the road 106 . In some examples, as the vehicle 102 approaches the object 624, the sensors 204 ( FIG. 2) associated with the vehicle 102 may detect the object 624 . Once detected, one or more of the planner 214, the object data calculator 216, the object classifier 218, the collision predictor system 220, and the kinematics calculator 222 ( FIG. 2) may be used to determine the location of the object 624, classify the object 624, determine whether the object 624 is static or dynamic, and if the object is dynamic, predict a possible track (e.g., velocity, acceleration, and/or heading) of the object 624 . As the vehicle 102 approaches the object 624, one or more of these systems may be used to calculate a confidence level associated with a probability that the vehicle 102 will be able to successfully maneuver past the object 624, for example, without assistance from the teleoperations system 148 . In some examples, the vehicle 102 can record confidence levels over time and determine a rate of change of the confidence levels. In some examples, any portion of these determinations or representations thereof may be amalgamated into operation state data; [0188] the vehicle 102 may use any additional or alternate portion of the sensor and/or operation state data to determine whether to transmit a request for assistance; [0197]);
determining whether the exception routine resolved the machine health/job quality issue ([0193], [0197] As the vehicle 102 approaches the construction zone 702, the vehicle 102 may calculate a confidence level associated with a probability that the vehicle 102 will be able to successfully maneuver past the constructions zone, for example, without assistance from the teleoperations system 148 . As the confidence level drops below a threshold minimum confidence level and/or if the vehicle detects that a rate of change of the confidence levels over time indicates a negative rate of change for a threshold duration of time (e.g., −2% per second for at least 5 seconds) and/or a rate of change having a threshold magnitude (e.g., −9% per second), the vehicle 102 may slow its speed or stop, and use its network interface 234 ( FIG. 2) to send communication signals to the teleoperations system 148 providing sensor data and/or operation state data with a request for guidance from the teleoperations system 148 . Alternately, or additionally, the vehicle 102 may identify that it is approaching a region where one or more other requests have been sent within a time period (e.g., last day, last half hour), a high-risk area (e.g., train crossing, school zone), and/or that maneuvers determined by the vehicle 102 as being ones that avoid a collision are of a type that is high-risk and is therefore associated with a low confidence (e.g., crossing double yellow line), [0200]-[0203]);
based on whether the determination of whether exception routine resolved the machine health/job quality issue, selectively generating a communication to a supervisor, wherein the communication requests a supervisor decision (the teleoperator may be able to manipulate the time representation and/or select confidence levels to view sensor and/or operation data at a corresponding time. In some examples, upon receipt of a request for assistance, the teleoperations interface 154 may automatically present a replay to apprise the teleoperator 150 of the situation); and
receiving user input indicative of a supervisor decision and taking responsive action (a teleoperator 150 is enabled to identify what caused confidence levels to drop; the teleoperator 150 may be able to identify that a camera has been knocked out of position and/or that the confidence dropped because the field of view of the camera was insufficient to identify the dynamic object 706, either of which may be conditions that may cause a decrease in confidence levels because the sensor and/or operation data seems to conflict. In some examples, the overall system confidence and/or any of the displayed data may be replayed (e.g., the system confidence shown in the alert bar 636 may additionally or alternatively reflect the drop in confidence levels at 712 ). Fig. 5B-5C and corresponding paragraphs).
Regarding claim 7, Lockwood teaches wherein generating the communication to the supervisor includes selecting, with the robotic controller, the supervisor among a plurality of supervisors based on a categorization of the machine health/job quality issue (a teleoperator elevation requirement can be associated with the event and/or combination. For example, a teleoperator elevation requirement can specify a “high elevation” that indicates that only teleoperations devices having elevated teleoperator credentials associated therewith are permitted to handle the request (e.g., the elevated teleoperator credentials could indicate a senior teleoperator or a more advance or specially trained artificial intelligence) a teleoperator elevation requirement can specify that a teleoperator response to the request for assistance must be confirmed by one or more additional teleoperation device and/or that the request is to be disseminated to more than one teleoperations device (e.g., because of a highly complex event or the involvement of multiple vehicles requiring guidance in concert)).
Regarding claim 8, Lockwood teaches wherein generating the communication to the supervisor includes generating the communication to a supervisor at a remote operation center (the teleoperations system 148 may include one or more teleoperators 150, which may be human teleoperators, located at a teleoperations center 152).
Claim 2 is rejected under 35 U.S.C. 103 as being obvious over by Lockwood (US 10564638 B1) in view of Fang (US 20210192867A1)
Regarding claim 2, While Lockwood teaches pulling the vehicle 102 over, Lockwood does not explicitly teach but Fang teaches responsively shut down the automated work machine based on the user input ([0314] [0333] the remote-control manager shuts down the vehicle and disables remote control support in response to the battery charge falling).
It would have been obvious to one of ordinary skill in the art before the effective date of the present invention to modify, teleoperator situational awareness, as taught by Lockwood, shutting down the automated work machine, as taught by Fang, as Lockwood and Fang are directed to vehicle control (same field of endeavor), and one of ordinary skill in the art would have recognized the established utility using shutting down the automated work machine ([0033] Fang) and predictably applied it to save vehicle energy.
Claim 3 is rejected under 35 U.S.C. 103 as being obvious over by Lockwood (US 10564638 B1) in view of Konrardy (US11016504 B)
Regarding claim 3, while Lockwood teaches pulling the vehicle 102 over, Lockwood does not explicitly teach but Konrardy teaches wherein the responsive action includes generating a path to a service location and moving the autonomous work machine to the service location (the method 600 may include then locating the nearest repair facility with the necessary. the controller 204 or the server 140 may send a request for assistance (block 434). autonomous vehicle caravan method for causing a malfunctioning or damaged autonomous or semi-autonomous vehicle to follow an autonomous tow/repair vehicle to enhance the functionality of the autonomous or semi-autonomous vehicle 600. the AV or SAV vehicle controller may select the nearest repair facility that is qualified to repair the AV or SAV damage, include determining the best route to the repair facility based upon the AV or SAV current condition; 4) if the AV or SAV remains serviceable, locating a nearest repair facility having the necessary electronic components in stock and technical expertise to repair the autonomous feature or sensor that is malfunctioning (such as via wireless communication or data transmission over one or more radio links or wireless communication channels); The method 600 may include causing the AV or SAV to mimic ARV maneuvers until reaching the repair facility).
It would have been obvious to one of ordinary skill in the art before the effective date of the present invention to modify, teleoperator situational awareness, as taught by Lockwood, moving the autonomous work machine to the service location, as taught by Konrardy, as Lockwood and Konrardy are directed to vehicle control (same field of endeavor), and one of ordinary skill in the art would have recognized the established utility using moving the autonomous work machine to the service location and predictably applied it to Lockwood’s teaching for repairing a malfunctioning autonomous vehicle timely.
Claim 4 is rejected under 35 U.S.C. 103 as being obvious over by Lockwood (US 10564638 B1) in view of Konrardy (US11016504 B) in view of Fang (US 20210192867A1)
Regarding claim 4, While Lockwood as modified by Konrardy teaches pulling the vehicle 102 over, Lockwood as modified by Konrardy does not explicitly teach but Fang teaches shutting down the autonomous work machine at the service location ([0314] [0333] the remote-control manager shuts down the vehicle and disables remote control support in response to the battery charge falling).
It would have been obvious to one of ordinary skill in the art before the effective date of the present invention to modify, teleoperator situational awareness, as taught by Lockwood as modified by Konrardy, shutting down the automated work machine, as taught by Fang, as Lockwood, Konrardy and Fang are directed to vehicle control (same field of endeavor), and one of ordinary skill in the art would have recognized the established utility using shutting down the automated work machine ([0033] Fang) and predictably applied it to save vehicle energy (battery).
Claim 5-6 are rejected under 35 U.S.C. 103 as being obvious over by Lockwood (US 10564638 B1) in view of Schwie (US 20200103239 A)
Regarding claim 5, while Lockwood teaches “The remote-control request message 230 requests the remote controller 208 to take some control of the autonomous vehicle 102. The remote controller 208 may then grant or deny the remote-control request message 230 via a remote-control response message 240 while the autonomous vehicle 102 perform missions”, Lockwood does not explicitly teach but Schwie teaches wherein the user input is indicative of a decision to ignore the machine health/job quality issue and the responsive action is continuing to perform the mission ( the self-driving vehicle 2 may enter an emergency mode when the vehicle management system 65 is unable to communicate with the remote computing device 12 (ignored). It should be appreciated that when the self-driving vehicle 2 is in the emergency mode, the vehicle management system 65 sends a series of second wireless communications 136 to the remote computing device 12. In other words, in the emergency mode, the vehicle management system 65 sends multiple requests to the remote computing device 12 (cannot be ignored)).
It would have been obvious to one of ordinary skill in the art before the effective date of the present invention to modify, teleoperator situational awareness, as taught by Lockwood, ignoring the machine health/job quality issue, as taught by Schwie, as Lockwood and Schwie are directed to vehicle control (same field of endeavor), and one of ordinary skill in the art would have recognized the established utility ignoring the machine health/job quality issue to prioritize the response based on vehicles’ confidence levels.
Regarding claim 6, while Lockwood teaches “the SAE 308 could receive a request from a vehicle 102 at a construction site and determine that the request is related to a request that was previously received (e.g., geographic location matches, event label matches, and/or second request received within time limit). The SAE 308 may display data from the prior request and provide an input option corresponding to the guidance that was transmitted to the vehicle that sent the prior request”, Lockwood does not explicitly teach but Schwie teaches determining with the robotic controller that the machine health/job quality issue cannot be ignored and generating a second communication to the supervisor indicating that the machine health/job quality issue cannot be ignored ( the self-driving vehicle 2 may enter an emergency mode when the vehicle management system 65 is unable to communicate with the remote computing device 12 (ignored). It should be appreciated that when the self-driving vehicle 2 is in the emergency mode, the vehicle management system 65 sends a series of second wireless communications 136 to the remote computing device 12. In other words, in the emergency mode, the vehicle management system 65 sends multiple requests to the remote computing device 12 (cannot be ignored))
It would have been obvious to one of ordinary skill in the art before the effective date of the present invention to modify, teleoperator situational awareness, as taught by Lockwood, ignoring the machine health/job quality issue, as taught by Schwie, as Lockwood and Schwie are directed to vehicle control (same field of endeavor), and one of ordinary skill in the art would have recognized the established utility ignoring the machine health/job quality issue to prioritize the response based on vehicles’ confidence levels.
Conclusion
Please refer to form 892 for cited references.
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33,216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006,1009, 158 USPQ 275,277 (CCPA 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JINGLI WANG whose telephone number is (571)272-8040. The examiner can normally be reached on Mon-Fri 9 am-5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Anne Antonucci can be reached on (313)446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-100.
/J.W./Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666