DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice for all US Patent Applications filed on or after March 16, 2013
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of the Claims
This communication is in response to communications received on 6/24/26. Claim(s) 1, 5, 7, and 15 is/are amended, claim(s) 2-4, 6, 21, and 22 is/are cancelled, claim(s) 23-28 is/are new, and applicant states support can be found at instant specification [0014, 0020, 0028, 0029, 0055, 0067, 0071]. Therefore, Claims 1, 5, 7, 9-17, 19, 20, and 23-28 is/are pending and have been addressed below.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 6/24/26 was/were considered by the examiner.
Response to Arguments
Applicant’s arguments, see applicant’s remarks, filed 6/24/25, with respect to rejections under 35 USC 101 for claim(s) 1-7, 9-17, and 19-22 have been fully considered but they are not persuasive as far as they apply to the amended 101 rejection(s) below.
Applicant’s arguments, see applicant’s remarks, filed 6/24/25, with respect to rejections under 35 USC 103 for claim(s) 1-7, 9-17, and 19-22 have been fully considered but they are not persuasive as far as they apply to the amended 103 rejection(s) below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1, 5, 7, 9-17, 19, 20, and 23-28 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter as noted below.
The limitation(s) below for representative claim(s) 1, 5, and 15 that, under its broadest reasonable interpretation, is directed to remote (human) workers queueing to perform teleoperations for autonomous vehicles.
Step 1: The claim(s) as drafted, is/are a process (claim(s) 5, 7, and 9-14 recites a series of steps) and system (claim(s) 1, 15-17, 19-20, 23-26, and 28 recites a series of components).
Step 2A – Prong 1: The claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) (emphasis added):
Claim 5: receiving, from a vehicle, a request to provide remote operator assistance;
associating the request with a queue;
determining, among a set of available remote operators, a status set by a remote operator to resolve the request;
determining criteria associated with the remote operator;
monitoring a response time of the remote operator and updating, based at least in part on the response time meeting or exceeding a threshold, the status;
determining, based at least in part on the request, an updated status, and the criteria, to receive remote operator assistance from the remote operator;
sending information associated with the request to the remote operator; and
presenting, via a display of a device associated with the remote operator, a set of controls configured to receive an input from the remote operator;
determining, based at least in part on the input, the remote operator assistance;
transmitting the remote operator assistance to the vehicle,
wherein the vehicle is controlled based at least in part on the remote operator assistance.
Claim(s) 1 and 15: same analysis as claim(s) 5.
Claim 1 additionally: receive, from an autonomous vehicle, a request for remote operator assistance, the request including information indicative of one or more of:
an event type, a mission type associated with the autonomous vehicle, sensor data associated with the autonomous vehicle, a location of the autonomous vehicle, a heading of the autonomous vehicle, or a speed of the autonomous vehicle;
determine a status of the remote operator, the status comprising a designation of whether the remote operator is on a break;
establish, based at least in part on determining to send the request to the remote operator, a direct communication channel between the remote operator and the autonomous vehicle, the direct communication channel configured to bypass a queue interface of the remote operations system;
send the request to a device of the remote operator to cause display of the request and a set of controls to the remote operator, the set of controls configured to receive an input from the remote operator;
determine, based at least in part on the input, assistance provided by the remote operator via the set of controls, the assistance associated with an at least partially defined path for the autonomous vehicle; and
transmit, to the autonomous vehicle via the direct communication channel, the assistance, wherein the autonomous vehicle is configured to collaborate with the remote operator via the direct communication channel to plan a trajectory based at least in part on the assistance.
Claim 15 additionally: determining, among a set of available remote operators, a status of a remote operator to resolve the request, the status comprising a designation of whether the remote operator is in training;
determining a workload balance of the set of available remote operators;
determining, based at least in part on the request, the status, the workload balance, and the criteria, to request the remote operator assistance from the remote operator; sending information associated with the request to the remote operator;
transmitting, as processor-executable instructions and via a network interface device associated with the remote operator, the remote operator assistance to the vehicle.
Dependent claims 7, 9-14, 16-17, 19, 20, and 23-28 recite the same or similar abstract idea(s) as independent claim(s) 1, 5, and 15 with merely a further narrowing of the abstract idea(s): .
The identified limitations of the independent and dependent claims above fall well-within the groupings of subject matter identified by the courts as being abstract concepts of:
a method of organizing human activity (commercial or legal interactions including advertising, marketing or sales activities or behaviors, or business relations) because the invention is directed to economic and/or business relationships as they are associated with remote (human) workers queueing to perform teleoperations for autonomous vehicles.
Step 2A – Prong 2: This judicial exception is not integrated into a practical application because:
The additional elements unencompassed by the abstract idea include vehicle (claim(s) 5, 15), a remote operations system for a fleet of autonomous vehicles, comprising: at least one processor; and at least one non-transitory memory, device, channel (claim(s) 1), a computer system comprising: one or more processors; and a non-transitory computer-readable storage medium storing instructions executable by the one or more processors one or more non-transitory computer-readable media, one or more processors, channel (claim(s) 15), device (claim(s) 3), vehicle (claim(s) 7, 8, 9, 11-12, 16-17, 23), terminal (claim(s) 26.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements as described above with respect to Step 2A Prong 2 fails to describe:
Improvements to the functioning of a computer, or to any other technology or technical field - see MPEP 2106.05(a)
Applying or using a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition – see Vanda Memo
Applying the judicial exception with, or by use of, a particular machine – see MPEP 2106.05(b)
Effecting a transformation or reduction of a particular article to a different state or thing - see MPEP 2106.05(c)
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP 2106.05(e) and Vanda Memo.
Thus the additional elements as described above with respect to Step 2A Prong 2 merely amount to (as additionally noted by instant specification [0025]) invoked as a tool and/or general purpose computer to apply instructions of an abstract idea in a particular technological environment, and/or mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do not integrate an abstract idea into a practical application (MPEP 2106.05(f)&(h)).
Step 2B: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus the additional elements as described above with respect to Step 2A Prong 2 merely amount to (as additionally noted by instant specification [0025]) invoked as a tool and/or a general purpose computer to apply instructions of an abstract idea in a particular technological environment, and/or mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do not integrate an abstract idea into a practical application and thus similarly the combination and arrangement of the above identified additional elements when analyzed under Step 2B also fails to necessitate a conclusion that the claims amount to significantly more than the abstract idea for the same reasons as set forth above (MPEP 2106.05(f)&(h)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
It has been held that a prior art reference must either be in the field of applicant’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the applicant was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992).
Claim(s) 1, 23-24, and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen et al. (US 2020/0310417 A1) in view of Hambridge et al. (US 2018/0330304 A1), Mertens (US 12,158,756 B1), and Weslosky et al. (US 2023/0244226 A1).
Regarding claim 1 (currently amended), Pedersen teaches a remote operations system for a fleet of autonomous vehicles, comprising:
at least one processor; and at least one non-transitory memory having stored thereon processor-executable instructions that, when executed by the at least one processor, configure the remote operations system to [see at least [0053, 0056, 0068] operations center 2400, vehicles 2100 (340-370), server 2500;
[0068] fleet of vehicles]:
receive, from an autonomous vehicle, a request for remote operator assistance, the request including information indicative of one or more of:
an event type, a mission type associated with the autonomous vehicle, sensor data associated with the autonomous vehicle, a location of the autonomous vehicle, a heading of the autonomous vehicle, or a speed of the autonomous vehicle;
associate, based at least in part on the information, the request with a queue of remote operator requests amonq remote operator requests, where the queue is associated with remote operators [for the limitations above, see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”;
[0128-0130] “The request can include information (e.g., state data) that can be useful to a tele-operator in responding to the request. …
The state data can include, but is not limited to, data that indicates the state or condition of the vehicle that is executing the technique 500, including any of kinetic state data relating to any of the velocity and acceleration of a vehicle, location data, including the geographical location of a vehicle (e.g., the latitude and longitude of the vehicle) or the location of the vehicle with respect to another object, vehicle position, including the orientation and inclination (e.g., slope of the vehicle on an incline) of the vehicle, the operational state of the vehicle, including the electrical state or mechanical state of the vehicle (e.g., health of the electrical vehicle systems, mechanical vehicle systems, tire pressure, etc.), maintenance data related to maintenance of the vehicle, energy source data including an amount of fuel remaining or an amount of battery charge remaining, sensor data based on outputs from sensors including, optical sensors, audio sensors, an motion sensors, internal state data, including a temperature and humidity inside the passenger cabin of the vehicle, and a current task (e.g., pick up a passenger) of the vehicle.
The transmitted information can also include state information of other road users that are maintained in a world model module, such as the world model module 402 of FIG. 4. For example, state of an object that is, or is a part of, the obstruction, and/or the state of other objects proximal to the vehicle can be included in the transmitted information. For example, zero or more of a velocity, a pose, a geometry (such as width, height, and depth), a classification (e.g., bicycle, large truck, pedestrian, road sign, etc.), a location, other information, or a subset thereof can be transmitted from the vehicle so that the tele-operator can have sufficient information to determine a response.”];
determine, among the set of remote operators, an availability of a remote operator;
determine a status of the remote operator, the status [the limitation is interpreted as determine if an operator is available, this interpretation is based on available meaning the operator is unoccupied (a status which is not defined in the specification) as suggested by the instant application specification [0079],
then see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determine criteria associated with the remote operator [see at least [0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determine, based at least in part on the information, the availability, the status, and the criteria, to send the request to the remote operator [see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0027] “Based on the risk, the AV can perform a response whereby the AV can, if the risk is less than a risk threshold, (a) autonomously go around the obstruction in normal manner; or if the risk is not less than the risk threshold, (b) contact a tele-operator for assistance. For ease of reference, the responses (a)-(b) can be referred to, respectively, as a normal response, and an assistance response. As such, the possible responses of the AV to an obstruction can include a wait response, a normal response, and an assistance response.”;
[0028] “In the case of a wait response, the AV comes to a complete stop and can re-evaluate the situation over time. Depending on how the situation changes over time, the AV can either continue to wait, autonomy go around the obstruction, or contact a tele-operator. In an example, if the obstruction situation does not resolve itself within a predefined period of time, the AV can perform an assistance response”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
send the request to a device of the remote operator to cause display of the request and a set of controls to the remote operator, the set of controls configured to receive an input from the remote operator [see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”;
[0063] “The operations center 2400 may include one or more computing devices, which are able to exchange (send or receive) data from: vehicles such as the vehicle 2100;”;
[0131] “FIG. 7 is an illustration of user interfaces 700 of a tele-operator according to implementations of this disclosure. The user interfaces 700 are mere illustrative examples of visual information that a tele-operator may need in order to respond to the request. One or more of the user interfaces 700 can be displayed (or available for display) to a tele-operator. The user interfaces 700 include a map view 702, an augmented view 704, and a dashcam view 706. However, the user interfaces 700 can include more, fewer, or other user interfaces and/or information that can aid the tele-operator in responding to the received request.”;
[0136] one or more action buttons (controls) “FIG. 8 is an illustration of a ticket 800 according to implementations of this disclosure. The ticket 800 illustrates an example of what may be presented to a tele-operator at a tele-operations center in response to an AV initiating a request for tele-operator assistance. The ticket 800 includes an exception description 802, occupancy information 804, destination information 806, feature information 808, and an action button 810. In some implementations, more, fewer, other information or actions, or a combination thereof can be displayed in the ticket.”;
Figs. 8-9 and [0140-0141, 0145] functions of actions buttons “A tele-operator can select (e.g., click, activate, press, etc.) the action button 810 when the tele-operator is ready to resolve the ticket. When the action button 810 is selected, the tele-operator can evaluate the obstruction situation, such as by observing, in real time (or near real time) the augmented view 704, the dashcam view 706, or any other views. The tele-operator can also evaluate the trajectory indicator 738. The tele-operator can be presented with a menu, as described with respect to FIG. 9, for selecting a response.
By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.”];
determine, based at least in part on the input, assistance provided by the remote operator via the set of controls, the assistance associated with an at least partially defined path for the autonomous vehicle;
transmit, to the autonomous vehicle via communication channel, the assistance, wherein the autonomous vehicle is configured to collaborate with the remote operator via the communication channel to plan a trajectory based at least in part on the assistance [for the limitations above, see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”
[0145-0146] “By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.
Referring again to FIG. 5, at 512, the technique 500 receives a response (i.e., a received response) from the tele-operator. As already mentioned, the response can be one of a first response to wait, a second response to proceed, or a third response that includes a trajectory.”;
Fig. 2 and [0055, 0060] “In this way, the controller apparatus 2410 may tele-operate the vehicles or external objects from a remote location.” via “The access point 2330, which may include a computing device, may be configured to communicate with the vehicle 2100, with the electronic communication network 2300, with the operations center 2400, or with a combination thereof via wired or wireless communication links 2310/2340.”].
Pedersen doesn’t/don’t explicitly teach but Hambridge discloses
determine a status of the remote operator, the status comprising a designation of whether the remote operator is on a break [see at least [0032-0033, 0046] determine candidate status such as at lunch;
[0032] “For example, if the location information indicates the candidate is located in his/her office, it can be inferred that the candidate presently is working, and thus presently available. If the location information indicates the candidate is located in a break room, it can be inferred that the candidate presently is on break, and thus not presently available. If the location information indicates the candidate is located in a conference room, it can be inferred that the candidate presently is in a conference, and thus not presently available.”;
[0033] “Thus, the sensor data 175 can indicate, for each candidate in the candidate pool 160, whether the candidate is at home, in the office, at a movie theater, in a restaurant, in a break room, in a conference room, in the candidate's office, etc.”;
[0046] “The on-call management application 150 can determine the value for the weighted score 230 based on historical data. In illustration, the on-call management application 150 can determine how long the candidate has been present at the candidate's current location and, based on that determination and historical data, determine when the candidate will become available to respond to issues. For example, if the user profile 165 of the candidate indicates that the candidate usually takes one hour for lunch, the candidate is at a restaurant during lunchtime, and the candidate has been in the restaurant for forty-five minutes, the on-call management application 150 can infer that the candidate soon will be heading back to the office and be available in the office within the next fifteen minutes.”;
[0030, 0047] status set by candidate such as at a conference (training) “If a candidate currently is at a location and/or attending an event for which the candidate's user profile 165 indicates that the candidate chooses not to participate in the resolution of issues when the candidate is in the location and/or attending the event,”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen with Hambridge to include the limitation(s) above as disclosed by Hambridge. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer such as based on the operator’s skills [see at least Hambridge [0002, 0010, 0028] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen and b) Hambridge and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Pedersen in view of Hambridge doesn’t/don’t explicitly teach but Mertens discloses
associate, based at least in part on the information, the request with a queue of operator requests from amonq a plurality of queues of operator requests, where the queue is associated with a set of operators [see at least [col 4-5 respectively ln 59-67 and 1-10] “The remote operation system 148 includes a request queue 158 and operators 160. The request queue 158 includes a series of requests for remote operations from vehicles in a fleet of autonomous vehicles. The requests are organized and ordered in the request queue based on priority for each of the requests. … The request queue 158 may be ordered based on priority for each of the requests, the priority based on one or more factors described herein.”;
[col 1-2 respectively ln 65-67 and 1-15] “The remote operator requests may be prioritized based on safety considerations, such as a vehicle operating speed (e.g., highway operation versus city street operation), occupancy status of the vehicle (occupied or vacant, number of occupants, etc.), length of ride, traffic volume, or other factors.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen in view of Hambridge with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen in view of Hambridge’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen in view of Hambridge and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Pedersen in view of Hambridge and Mertens doesn’t/don’t explicitly teach but Weslosky discloses
establish, based at least in part on determining to send the request to the remote operator, a direct communication channel between the remote operator and the autonomous vehicle, the direct communication channel configured to bypass a queue interface of the remote operations system;
the direct communication channel, the direct communication channel [see at least [0067] “A teleoperations monitoring and operations platform may, in one embodiment, enable a teleoperations monitor arrangement to substantially directly assign control of a vehicle to a teleoperations operator arrangement. That is, a platform may effectively bypass the use of a teleoperations queue arrangement.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen in view of Hambridge and Mertens with Weslosky to include the limitation(s) above as disclosed by Weslosky. Doing so would further define Pedersen in view of Hambridge and Mertens’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining when assistance a human operator is needed [see at least Weslosky [0002-0003, 0045] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen in view of Hambridge and Mertens and b) Weslosky and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 23, modified Pedersen teaches the remote operations system of claim 1, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein the plurality of queues of remote operator requests are associated with at least one of:
a geographic location of an individual autonomous vehicle of the fleet of autonomous vehicles, a vehicle type of the individual autonomous vehicle of the fleet of autonomous vehicles, or a request type associated with the request [see at least [col 14 ln 35-53] “In some examples, the queue interface 308 may maintain multiple different or separate queues that may operate in parallel to one another. In such examples, the separate parallel queues may correspond to different geographic locations (of the vehicles), different vehicle types, different request types (e.g., requesting guidance assistance versus a request from a passenger of the vehicle system), and other queues related to different filters for the queues may also be applied. The multiple parallel queues may each be accessible by distinct subsets of remote operators, for example with a queue for a first vehicle type only accessible to remote operators qualified to provide guidance or assistance to that class of vehicle. In some examples, the remote operators may be available to handle requests from one or more of the parallel queues.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 24, modified Pedersen teaches the remote operations system of claim 1, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein the plurality of queues of remote operator requests are configured to operate in parallel, and wherein individual remote operators of the set of remote operators are available to handle requests from one or more of the plurality of queues [see at least [col 14 ln 35-53] “In some examples, the queue interface 308 may maintain multiple different or separate queues that may operate in parallel to one another. In such examples, the separate parallel queues may correspond to different geographic locations (of the vehicles), different vehicle types, different request types (e.g., requesting guidance assistance versus a request from a passenger of the vehicle system), and other queues related to different filters for the queues may also be applied. The multiple parallel queues may each be accessible by distinct subsets of remote operators, for example with a queue for a first vehicle type only accessible to remote operators qualified to provide guidance or assistance to that class of vehicle. In some examples, the remote operators may be available to handle requests from one or more of the parallel queues.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 26, modified Pedersen teaches the remote operations system of claim 1, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein determining to send the request to the remote operator is further based at least in part on a latency associated with a terminal of the remote operator [see at least [col 5-6 ln 45-67 and 1-30 respectively] “In some examples, the vehicles 102(a), 102(b), and 102(c) may encounter different events that result in a request for remote operator input at or around the same time. In such examples, the vehicles 102(a), 102(b), 102(c) may convey a remote operations request to the remote operation system 148. … The remote operators 150 may be filtered based on experience levels, specialized training (e.g., emergency technician, medical training, nursing training, etc.), experience with particular environments, situations, vehicle systems, network connection speed or latency of a particular remote operator; or other such experience or factors. …
The remote operations requests received by the remote operation system 148 are ordered within the queue and subsequently matched to remote operators for processing.”;
Fig. 1 and [col 5 ln 5-30] “In some examples, for example as shown in FIG. 1 , the remote operation system 148 may include one or more remote operators 150, which may be human remote operators, located at a remote operations center 152.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge and Hambridge et al. (US 2018/0330304 A1).
Regarding claim 5 (currently amended), Pedersen teaches a method, comprising [see at least [0053, 0056, 0068] operations center 2400, vehicles 2100 (340-370), server 2500;
[0068] fleet of vehicles]:
receiving, from a vehicle, a request to provide remote operator assistance;
associating the request with a queue [for the limitations above, see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”;
[0128-0130] “The request can include information (e.g., state data) that can be useful to a tele-operator in responding to the request. …
The state data can include, but is not limited to, data that indicates the state or condition of the vehicle that is executing the technique 500, including any of kinetic state data relating to any of the velocity and acceleration of a vehicle, location data, including the geographical location of a vehicle (e.g., the latitude and longitude of the vehicle) or the location of the vehicle with respect to another object, vehicle position, including the orientation and inclination (e.g., slope of the vehicle on an incline) of the vehicle, the operational state of the vehicle, including the electrical state or mechanical state of the vehicle (e.g., health of the electrical vehicle systems, mechanical vehicle systems, tire pressure, etc.), maintenance data related to maintenance of the vehicle, energy source data including an amount of fuel remaining or an amount of battery charge remaining, sensor data based on outputs from sensors including, optical sensors, audio sensors, an motion sensors, internal state data, including a temperature and humidity inside the passenger cabin of the vehicle, and a current task (e.g., pick up a passenger) of the vehicle.
The transmitted information can also include state information of other road users that are maintained in a world model module, such as the world model module 402 of FIG. 4. For example, state of an object that is, or is a part of, the obstruction, and/or the state of other objects proximal to the vehicle can be included in the transmitted information. For example, zero or more of a velocity, a pose, a geometry (such as width, height, and depth), a classification (e.g., bicycle, large truck, pedestrian, road sign, etc.), a location, other information, or a subset thereof can be transmitted from the vehicle so that the tele-operator can have sufficient information to determine a response.”];
determining, among a set of available remote operators, a status set by a remote operator to resolve the request [the limitation is interpreted as determine if an operator is available, this interpretation is based on available meaning the operator is unoccupied (a status which is not defined in the specification) as suggested by the instant application specification [0079],
then see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determining criteria associated with the remote operator;
the remote operator [for the limitations above, see at least [0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determining, based at least in part on the request, an updated status, and the criteria, to receive remote operator assistance from the remote operator [see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0027] “Based on the risk, the AV can perform a response whereby the AV can, if the risk is less than a risk threshold, (a) autonomously go around the obstruction in normal manner; or if the risk is not less than the risk threshold, (b) contact a tele-operator for assistance. For ease of reference, the responses (a)-(b) can be referred to, respectively, as a normal response, and an assistance response. As such, the possible responses of the AV to an obstruction can include a wait response, a normal response, and an assistance response.”;
[0028] “In the case of a wait response, the AV comes to a complete stop and can re-evaluate the situation over time. Depending on how the situation changes over time, the AV can either continue to wait, autonomy go around the obstruction, or contact a tele-operator. In an example, if the obstruction situation does not resolve itself within a predefined period of time, the AV can perform an assistance response”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
sending information associated with the request to the remote operator;
presenting, via a display of a device associated with the remote operator, a set of controls configured to receive an input from the remote operator;
determining, based at least in part on the input, the remote operator assistance [see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”;
[0063] “The operations center 2400 may include one or more computing devices, which are able to exchange (send or receive) data from: vehicles such as the vehicle 2100;”;
[0131] “FIG. 7 is an illustration of user interfaces 700 of a tele-operator according to implementations of this disclosure. The user interfaces 700 are mere illustrative examples of visual information that a tele-operator may need in order to respond to the request. One or more of the user interfaces 700 can be displayed (or available for display) to a tele-operator. The user interfaces 700 include a map view 702, an augmented view 704, and a dashcam view 706. However, the user interfaces 700 can include more, fewer, or other user interfaces and/or information that can aid the tele-operator in responding to the received request.”;
[0136] one or more action buttons (controls) “FIG. 8 is an illustration of a ticket 800 according to implementations of this disclosure. The ticket 800 illustrates an example of what may be presented to a tele-operator at a tele-operations center in response to an AV initiating a request for tele-operator assistance. The ticket 800 includes an exception description 802, occupancy information 804, destination information 806, feature information 808, and an action button 810. In some implementations, more, fewer, other information or actions, or a combination thereof can be displayed in the ticket.”;
Figs. 8-9 and [0140-0141, 0145] functions of actions buttons “A tele-operator can select (e.g., click, activate, press, etc.) the action button 810 when the tele-operator is ready to resolve the ticket. When the action button 810 is selected, the tele-operator can evaluate the obstruction situation, such as by observing, in real time (or near real time) the augmented view 704, the dashcam view 706, or any other views. The tele-operator can also evaluate the trajectory indicator 738. The tele-operator can be presented with a menu, as described with respect to FIG. 9, for selecting a response.
By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.”]; and
transmitting the remote operator assistance to the vehicle,
wherein the vehicle is controlled based at least in part on the remote operator assistance [for the limitations above, see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”
[0145-0146] “By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.
Referring again to FIG. 5, at 512, the technique 500 receives a response (i.e., a received response) from the tele-operator. As already mentioned, the response can be one of a first response to wait, a second response to proceed, or a third response that includes a trajectory.”].
Pedersen doesn’t/don’t explicitly teach but Hambridge discloses
determining, among a set of available remote operators, a status set by a remote operator to resolve the request [see at least [0032-0033, 0046] determine candidate status such as at lunch;
[0032] “For example, if the location information indicates the candidate is located in his/her office, it can be inferred that the candidate presently is working, and thus presently available. If the location information indicates the candidate is located in a break room, it can be inferred that the candidate presently is on break, and thus not presently available. If the location information indicates the candidate is located in a conference room, it can be inferred that the candidate presently is in a conference, and thus not presently available.”;
[0033] “Thus, the sensor data 175 can indicate, for each candidate in the candidate pool 160, whether the candidate is at home, in the office, at a movie theater, in a restaurant, in a break room, in a conference room, in the candidate's office, etc.”;
[0046] “The on-call management application 150 can determine the value for the weighted score 230 based on historical data. In illustration, the on-call management application 150 can determine how long the candidate has been present at the candidate's current location and, based on that determination and historical data, determine when the candidate will become available to respond to issues. For example, if the user profile 165 of the candidate indicates that the candidate usually takes one hour for lunch, the candidate is at a restaurant during lunchtime, and the candidate has been in the restaurant for forty-five minutes, the on-call management application 150 can infer that the candidate soon will be heading back to the office and be available in the office within the next fifteen minutes.”;
[0030, 0047] status set by candidate such as at a conference (training) “If a candidate currently is at a location and/or attending an event for which the candidate's user profile 165 indicates that the candidate chooses not to participate in the resolution of issues when the candidate is in the location and/or attending the event,”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen with Hambridge to include the limitation(s) above as disclosed by Hambridge. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer such as based on the operator’s skills [see at least Mertens [0002, 0010, 0028] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen and b) Hambridge and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Pedersen in view of Hambridge doesn’t/don’t explicitly teach but Nate discloses
monitorinq a response time of the operator and updating, based at least in part on the response time meeting or exceedinq a threshold, the status [see at least [0028] “The task recommendation module 130 then recommends or presents those task bundles to specific workers. In the case that workers receiving task or task bundle recommendations do not accept the recommended tasks within some period time (e.g., a task acceptance round), the task recommendation module will then recommend some or all of those tasks to alternate workers in subsequent rounds.”;
[0029] “The worker model generation module 180 also update the learned worker models 135 over time when additional observations (e.g., task completions, worker history, etc.) become available.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen in view of Hambridge with Nate to include the limitation(s) above as disclosed by Nate. Doing so would further define Pedersen in view of Hambridge’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer such as based on the operator’s status learned from observations [see at least Nate [0028-0029] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen in view of Hambridge and b) Nate and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 7 (currently amended), modified Pedersen teaches the method of claim 5,
and Pedersen teaches wherein the request comprises information indicative of a mission type associated with at least one of:
whether the vehicle is transporting a passenger, whether the vehicle is traveling to pick up an additional passengers, whether the vehicle is charging, or whether the vehicle is performing training; wherein the status indicates whether the remote operator is occupied to receive requests for providing the remote operator assistance [see at least [0128-0130] “The request can include information (e.g., state data) that can be useful to a tele-operator in responding to the request. …
The state data can include, but is not limited to, data that indicates the state or condition of the vehicle that is executing the technique 500, including … a current task (e.g., pick up a passenger) of the vehicle.”].
Claim(s) 9-10, 13-14, and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge as applied to claim(s) 5 above and further in view of Mertens (US 12,158,756 B1).
Regarding claim 9, modified Pedersen teaches the method of claim 5, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein the criteria indicates at least one of:
a geographical area in which the remote operator is knowledgeable for providing a response to the request,
a vehicle type that the remote operator is knowledgeable for providing the response,
a type of the remote operator assistance that the remote operator provides, or
a mission type associated with autonomous vehicles [see at least [col 17-18 respectively ln 40-67 and 1-15] “At operation 408, the process 400 includes identifying a suitable remote operator. The suitable remote operator may be identified based on filters including experience level, certification, number of requests processed, experience with a situation, and other such filters described herein. … In some examples, the operation 408 includes identifying one or more remote operators having credentials or experience matching or meeting one or more requirements of the remote operation request. The credentials of the remote operators may be accessed to filter the remote operators to identify remote operators capable of handling the request. In some examples, the filters may include filters for availability, e.g., not presently handling a request or otherwise unavailable, filters for vehicle types, filters for experience level, filters for experience with particular locations or environments, and other such filters. By applying such filters, one or more remote operators may be identified to handle one or more requests from the queue. The filters may include mission status information, for example for demonstration purposes, especially valuable cargo or personnel, weather conditions, and other such filters. In some examples, the filters applied to the queue may be focused on the vehicle and vehicle conditions while other filters for remote operators may also be applied in other examples including tiers of remote operators, experience levels, number of requests processed by remote operators, and other user-defined filters applied by each remote operator. In some examples, user-defined filters may be customizable for input or selection by the remote operators, for example to only view remote operation requests for selection based on their customized preferences.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 10, modified Pedersen teaches the method of claim 9, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
further comprising:
determining, for a plurality of remote operators, an availability of individual remote operators; and
determining, based at least in part on the availability of the individual remote operators, the set of available remote operators [see at least [col 1-2 respectively ln 65-67 and 1-15] “In some examples, the requests and available remote operators may be matched by filtering available remote operators based on details of the request such as a vehicle type, environment location, passenger status (e.g., whether passengers are in the vehicle or not), and other such information.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define modified Pedersen’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 11, modified Pedersen teaches the method of claim 5, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein the status comprises:
a designation of whether the remote operator is providing remote operator assistance to another vehicle [see at least [col 1-2 respectively ln 65-67 and 1-15] “In some examples, the requests and available remote operators may be matched by filtering available remote operators based on details of the request such as a vehicle type, environment location, passenger status (e.g., whether passengers are in the vehicle or not), and other such information.”;
[col 17-18 respectively ln 40-67 and 1-15] “At operation 408, the process 400 includes identifying a suitable remote operator. In some examples, the filters may include filters for availability, e.g., not presently handling a request or otherwise unavailable”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim(s) 13, the claim(s) recite(s) analogous limitations to claim(s) 19 below and is/are therefore rejected on the same premise.
Regarding claim(s) 14, the claim(s) recite(s) analogous limitations to claim(s) 20 below and is/are therefore rejected on the same premise.
Regarding claim 27, modified Pedersen teaches the method of claim 5, as well as
wherein updating the status comprises updating the status (determining, based at least in part on the request, an updated status, and the criteria, to receive remote operator assistance from the remote operator - claim 5).
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
status to indicate the remote operator is unavailable to receive additional requests [see at least [col 17-18 respectively ln 40-67 and 1-15] “At operation 408, the process 400 includes identifying a suitable remote operator. In some examples, the filters may include filters for availability, e.g., not presently handling a request or otherwise unavailable”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define modified Pedersen’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge as applied to claim(s) 5 above and further in view of Sasaki (US 2022/0217237 A1).
Regarding claim(s) 12, the claim(s) recite(s) analogous limitations to claim(s) 16 below and is/are therefore rejected on the same premise.
Claim(s) 15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen et al. (US 2020/0310417 A1) in view of Hambridge et al. (US 2018/0330304 A1) and Weslosky et al. (US 2023/0244226 A1).
Regarding claim 15 (currently amended), Pedersen teaches one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform actions comprising [see at least [0053, 0056, 0068] operations center 2400, vehicles 2100 (340-370), server 2500;
[0068] fleet of vehicles]:
receiving, from a vehicle, a request to provide remote operator assistance;
associating the request with a queue [for the limitations above, see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”;
[0128-0130] “The request can include information (e.g., state data) that can be useful to a tele-operator in responding to the request. …
The state data can include, but is not limited to, data that indicates the state or condition of the vehicle that is executing the technique 500, including any of kinetic state data relating to any of the velocity and acceleration of a vehicle, location data, including the geographical location of a vehicle (e.g., the latitude and longitude of the vehicle) or the location of the vehicle with respect to another object, vehicle position, including the orientation and inclination (e.g., slope of the vehicle on an incline) of the vehicle, the operational state of the vehicle, including the electrical state or mechanical state of the vehicle (e.g., health of the electrical vehicle systems, mechanical vehicle systems, tire pressure, etc.), maintenance data related to maintenance of the vehicle, energy source data including an amount of fuel remaining or an amount of battery charge remaining, sensor data based on outputs from sensors including, optical sensors, audio sensors, an motion sensors, internal state data, including a temperature and humidity inside the passenger cabin of the vehicle, and a current task (e.g., pick up a passenger) of the vehicle.
The transmitted information can also include state information of other road users that are maintained in a world model module, such as the world model module 402 of FIG. 4. For example, state of an object that is, or is a part of, the obstruction, and/or the state of other objects proximal to the vehicle can be included in the transmitted information. For example, zero or more of a velocity, a pose, a geometry (such as width, height, and depth), a classification (e.g., bicycle, large truck, pedestrian, road sign, etc.), a location, other information, or a subset thereof can be transmitted from the vehicle so that the tele-operator can have sufficient information to determine a response.”];
determining, among a set of available remote operators, a status of a remote operator to resolve the request, the status [the limitation is interpreted as determine if an operator is available, this interpretation is based on available meaning the operator is unoccupied (a status which is not defined in the specification) as suggested by the instant application specification [0079],
then see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determining criteria associated with the remote operator [see at least [0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”];
determining, based at least in part on the request, the status, other data, and the criteria, to receive remote operator assistance from the remote operator [see at least [0020] “When the AV encounters an exception situation, the AV can stop and request assistance from a tele-operator. For example, when the AV encounters an obstruction (e.g., a construction site, a stopped vehicle, etc.) in a roadway, the AV might not go around the obstruction if doing so means that the AV will travel through an area that is physically safe but is restricted by traffic regulations. Accordingly, a tele-operator (e.g., a human operator, a vehicle manager) can be tasked with assisting the AV in negotiating its problematic situation by, for example, mapping a path (i.e., a trajectory) for the AV around the obstruction. The tele-operator may be one of many tele-operators that are available at a tele-operation center (i.e., a remote vehicle assistance center) where each tele-operator can monitor the state or condition of one or more AVs.”;
[0027] “Based on the risk, the AV can perform a response whereby the AV can, if the risk is less than a risk threshold, (a) autonomously go around the obstruction in normal manner; or if the risk is not less than the risk threshold, (b) contact a tele-operator for assistance. For ease of reference, the responses (a)-(b) can be referred to, respectively, as a normal response, and an assistance response. As such, the possible responses of the AV to an obstruction can include a wait response, a normal response, and an assistance response.”;
[0028] “In the case of a wait response, the AV comes to a complete stop and can re-evaluate the situation over time. Depending on how the situation changes over time, the AV can either continue to wait, autonomy go around the obstruction, or contact a tele-operator. In an example, if the obstruction situation does not resolve itself within a predefined period of time, the AV can perform an assistance response”;
[0032] “In an implementation, the assistance response can be in the form of an issuance by the AV of a ticket. The ticket can be assigned to a tele-operator at the tele-operation center. The ticket can be assigned to a specific tele-operator who may be selected based on expertise of the tele-operator, a geographical location of the AV, state information of the AV, information regarding the obstruction situation, or some other criteria. In another implementation, the ticket may be placed in a first-in-first-out queue and is assigned to a next available tele-operator.”]; and
sending information associated with the request to the remote operator [see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”;
[0063] “The operations center 2400 may include one or more computing devices, which are able to exchange (send or receive) data from: vehicles such as the vehicle 2100;”;
[0131] “FIG. 7 is an illustration of user interfaces 700 of a tele-operator according to implementations of this disclosure. The user interfaces 700 are mere illustrative examples of visual information that a tele-operator may need in order to respond to the request. One or more of the user interfaces 700 can be displayed (or available for display) to a tele-operator. The user interfaces 700 include a map view 702, an augmented view 704, and a dashcam view 706. However, the user interfaces 700 can include more, fewer, or other user interfaces and/or information that can aid the tele-operator in responding to the received request.”;
[0136] “FIG. 8 is an illustration of a ticket 800 according to implementations of this disclosure. The ticket 800 illustrates an example of what may be presented to a tele-operator at a tele-operations center in response to an AV initiating a request for tele-operator assistance. The ticket 800 includes an exception description 802, occupancy information 804, destination information 806, feature information 808, and an action button 810. In some implementations, more, fewer, other information or actions, or a combination thereof can be displayed in the ticket.”]; and
transmitting the remote operator assistance to the vehicle,
wherein the vehicle is controlled based at least in part on the remote operator assistance [for the limitations above, see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”
[0145-0146] “By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.
Referring again to FIG. 5, at 512, the technique 500 receives a response (i.e., a received response) from the tele-operator. As already mentioned, the response can be one of a first response to wait, a second response to proceed, or a third response that includes a trajectory.”;
[0053-0055] “For example, the vehicle 2100 or the external object 2110 may receive information, such as information representing the transportation network 2200, from the operations center 2400 via the electronic communication network 2300.
The operations center 2400 includes a controller apparatus 2410 which includes some or all of the features of the controller 1300 shown in FIG. 1. The controller apparatus 2410 can monitor and coordinate the movement of vehicles, including autonomous vehicles. …
Further, the controller apparatus 2410 can establish remote control over one or more vehicles, such as the vehicle 2100, or external objects, such as the external object 2110. In this way, the controller apparatus 2410 may tele-operate the vehicles or external objects from a remote location.”;
Fig. 7 and [0131] “FIG. 7 is an illustration of user interfaces 700 of a tele-operator according to implementations of this disclosure. The user interfaces 700 are mere illustrative examples of visual information that a tele-operator may need in order to respond to the request.”].
Pedersen doesn’t/don’t explicitly teach but Hambridge discloses
determining, among a set of available remote operators, a status of a remote operator to resolve the request, the status comprising a designation of whether the remote operator is in training [see at least [0032-0033, 0046] determine candidate status such as at lunch;
[0032] “For example, if the location information indicates the candidate is located in his/her office, it can be inferred that the candidate presently is working, and thus presently available. If the location information indicates the candidate is located in a break room, it can be inferred that the candidate presently is on break, and thus not presently available. If the location information indicates the candidate is located in a conference room, it can be inferred that the candidate presently is in a conference, and thus not presently available.”;
[0033] “Thus, the sensor data 175 can indicate, for each candidate in the candidate pool 160, whether the candidate is at home, in the office, at a movie theater, in a restaurant, in a break room, in a conference room, in the candidate's office, etc.”;
[0046] “The on-call management application 150 can determine the value for the weighted score 230 based on historical data. In illustration, the on-call management application 150 can determine how long the candidate has been present at the candidate's current location and, based on that determination and historical data, determine when the candidate will become available to respond to issues. For example, if the user profile 165 of the candidate indicates that the candidate usually takes one hour for lunch, the candidate is at a restaurant during lunchtime, and the candidate has been in the restaurant for forty-five minutes, the on-call management application 150 can infer that the candidate soon will be heading back to the office and be available in the office within the next fifteen minutes.”;
[0030, 0047] status set by candidate such as at a conference (training) “If a candidate currently is at a location and/or attending an event for which the candidate's user profile 165 indicates that the candidate chooses not to participate in the resolution of issues when the candidate is in the location and/or attending the event,”];
determining a workload balance of the set of available remote operators;
determining, based at least in part on the request, the status, the workload balance, and the criteria, to receive remote operator assistance from the remote operator [see at least [0055] “In one arrangement, the on-call management application 150 can include a frequency selection limiter 250. The on-call management application 150 can use the frequency selection limiter 250 to limit a frequency at which candidates are chosen as selected candidates 210. In illustration, it can be cumbersome for certain candidates to be selected too frequently. The frequency selection limiter 250 can be used to balance the response workloads across the candidate pool 160, for example based on a frequency at which the candidate has been requested to respond to previous issues and/or a last time the candidate was requested to respond to a previous issue.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen with Hambridge to include the limitation(s) above as disclosed by Hambridge. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer such as based on the operator’s skills [see at least Mertens [0002, 0010, 0028] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen and b) Hambridge and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Pedersen in view of Hambridge doesn’t/don’t explicitly teach but Weslosky discloses
establishing a communication channel between the remote operator and the vehicle, the communication channel configured to bypass a queue interface [see at least [0067] “A teleoperations monitoring and operations platform may, in one embodiment, enable a teleoperations monitor arrangement to substantially directly assign control of a vehicle to a teleoperations operator arrangement. That is, a platform may effectively bypass the use of a teleoperations queue arrangement.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen in view of Hambridge and Chan with Weslosky to include the limitation(s) above as disclosed by Weslosky. Doing so would further define Pedersen in view of Hambridge and Chan’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining when assistance a human operator is needed [see at least Weslosky [0002-0003, 0045] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen in view of Hambridge and b) Weslosky and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 17, modified Pedersen teaches the one or more non-transitory computer-readable media of claim 15,
and Pedersen teaches the actions further comprising:
receiving, from the remote operator, the remote operator assistance; and
transmitting, to the vehicle, the remote operator assistance,
wherein the vehicle is configured to be controlled based at least in part on the remote operator assistance [for the limitations above, see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”
[0145-0146] “By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.
Referring again to FIG. 5, at 512, the technique 500 receives a response (i.e., a received response) from the tele-operator. As already mentioned, the response can be one of a first response to wait, a second response to proceed, or a third response that includes a trajectory.”].
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge and Weslosky as applied to claim(s) 15 above and further in view of Sasaki (US 2022/0217237 A1).
Regarding claim 16, modified Pedersen teaches the one or more non-transitory computer-readable media of claim 15, the actions further comprising:
and Pedersen teaches the method further comprising:
determining that the remote operator accepted the request; and
the remote operator is while providing the remote operator assistance to the vehicle [for the limitations above, see at least [0106, 0153-0154] “At 1008, the technique 1008 determines a risk associated with the first trajectory. The risk can be determined by a risk evaluation module, such as risk evaluation module 408 of FIG. 4. If the risk exceeds a risk threshold, the technique 1000 proceeds to 1012; otherwise, the technique 1000 can proceed to 1018.
At 1012, the technique 1000 initiates a request to a tele-operator. Initiating the request can be as described with respect to 508 of FIG. 5. As such, the request can include the first trajectory. At 1014, the technique 1000 waits for a response from the tele-operator. That is, the technique 1000 can halt, or cause the AV to be halted, while waiting for a response from the tele-operator.”;
[0145-0146] “By selecting a menu option 902 (i.e., “Authorize”), the tele-operator authorizes to AV to proceed along the second trajectory that is proposed by the AV. That is, the technique 500 receives a response to proceed. By selecting a menu option 904 (i.e., “Reject”), the tele-operator instructs the AV to continue to wait (e.g., halt in place to await further instructions). That is, the technique 500 receives a response to wait. By selecting a menu option 906, the tele-operator can select (e.g., draw, define, etc.) a new route (i.e., a third trajectory) to be followed by the AV.
Referring again to FIG. 5, at 512, the technique 500 receives a response (i.e., a received response) from the tele-operator. As already mentioned, the response can be one of a first response to wait, a second response to proceed, or a third response that includes a trajectory.”].
Modified Pedersen doesn’t/don’t explicitly teach but Sasaki discloses
updating the status of the remote operator to indicate the remote operator is unavailable to receive additional requests while providing the remote operator assistance to the vehicle [see at least [0032] “When the operator terminal 30 is connected to any of the AD vehicles 10 and when the connection is broken, the operator terminal 30 gives notification thereof to the remote assistance apparatus 20. As a result, an assigning unit 22 of the remote assistance apparatus 20 can ascertain which of the plurality of operator terminals 30 are in operation, which are on standby, that is, which operators are available, and how many operators are available.”;
[0078] “When an operator is available (“Yes” at step S83), the assigning unit 22 assigns the call of which the waiting period exceeds the maximum waiting period UL to the available operator (step S84). When an operator is not available (“No” at step S83), the assigning unit 22 monitors availability state of the operators until an operator becomes available (step S83). When assignment of an operator to a certain call (step S84) is completed, the assigning unit 22 determines whether another call of which the waiting period exceeds the maximum waiting period UL is present (step S85).”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Sasaki to include the limitation(s) above as disclosed by Sasaki. Doing so would further define modified Pedersen’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Sasaki [0003] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Sasaki and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge and Weslosky as applied to claim(s) 15 above and further in view of Mertens (US 12,158,756 B1).
Regarding claim 19, modified Pedersen teaches the one or more non-transitory computer-readable media of claim 15, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein the remote operator is a first remote operator, the status is a first status, the method further comprising:
determining, among the set of available remote operators, a second status of a second remote operator to resolve the request; and
determining, based at least in part on the second status, that the second remote operator is occupied,
wherein sending the request to the first remote operator is further based at least in part on the second remote operator being occupied [for the limitations above, see at least [col 1-2 respectively ln 65-67 and 1-15] “In some examples, the requests and available remote operators may be matched by filtering available remote operators based on details of the request such as a vehicle type, environment location, passenger status (e.g., whether passengers are in the vehicle or not), and other such information.”;
[col 17-18 respectively ln 40-67 and 1-15] “At operation 408, the process 400 includes identifying a suitable remote operator. In some examples, the filters may include filters for availability, e.g., not presently handling a request or otherwise unavailable”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define modified Pedersen’s (Pedersen) [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 20, modified Pedersen teaches the one or more non-transitory computer-readable media of claim of claim 15, .
Modified Pedersen doesn’t/don’t explicitly teach but Mertens discloses
wherein determining the remote operator comprises:
filtering, using a first filter associated with a mission-type, the set of available remote operators to determine a first portion of the set of available remote operators; and
filtering, using a second filter associated with a geographical area, the first portion to determine a second portion of the set of available remote operators,
wherein the remote operator is associated with the second portion [for the limitations above, see at least [col 17-18 respectively ln 40-67 and 1-15] “At operation 408, the process 400 includes identifying a suitable remote operator. The suitable remote operator may be identified based on filters including experience level, certification, number of requests processed, experience with a situation, and other such filters described herein. … In some examples, the operation 408 includes identifying one or more remote operators having credentials or experience matching or meeting one or more requirements of the remote operation request. The credentials of the remote operators may be accessed to filter the remote operators to identify remote operators capable of handling the request. In some examples, the filters may include filters for availability, e.g., not presently handling a request or otherwise unavailable, filters for vehicle types, filters for experience level, filters for experience with particular locations or environments, and other such filters. By applying such filters, one or more remote operators may be identified to handle one or more requests from the queue. The filters may include mission status information, for example for demonstration purposes, especially valuable cargo or personnel, weather conditions, and other such filters. In some examples, the filters applied to the queue may be focused on the vehicle and vehicle conditions while other filters for remote operators may also be applied in other examples including tiers of remote operators, experience levels, number of requests processed by remote operators, and other user-defined filters applied by each remote operator. In some examples, user-defined filters may be customizable for input or selection by the remote operators, for example to only view remote operation requests for selection based on their customized preferences.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pedersen with Mertens to include the limitation(s) above as disclosed by Mertens. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Mertens [col 1 ln 5-15] ].
Furthermore, all of the claimed elements were known in the prior arts of a) Pedersen and b) Mertens and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge, Mertens, and Weslosky as applied to claim(s) 1 above and further in view of Nath et al. (US 2015/0317582 A1).
Regarding claim 25, modified Pedersen teaches the remote operations system of claim 1,
as well as wherein determining to send the request to the remote operator, remote operator.
Modified Pedersen doesn’t/don’t explicitly teach but Nath discloses
wherein determining to send the request to the operator is further based at least in part on the operator being associated with a previously successful resolution to a second request with similar characteristics to the request [see at least [0060] “A task is considered to be successfully completed if it is performed by at least one worker within the deadline of the task.”;
[0071] “In general, worker's preference towards payment and task complexity and how much payment he expects per unit “effort” (e.g., time spent, distance travelled, etc.) varies across workers. The Context-Aware Crowdsourced Task Optimizer automatically learns and models such worker preferences by analyzing the history of individual workers or particular groups or bundles of workers with respect to various tasks acceptances, task completions, task parameters including payment amounts, complexity or difficulty, distance, time, etc. The learned model for each worker is then used by the Context-Aware Crowdsourced Task Optimizer to evaluate the likelihood that a particular worker will successfully complete a future task.”;
[0028] “Once one or more tasks 105 have been received from any of the task publishers (110, 115, 120), a task recommendation module 130 evaluates those tasks, any associated task contexts, learned worker models 135 for one or more human or virtual workers (140, 145, 150, 155) in a worker pool 160, and current and future worker contexts (165 and 170, respectively) to construct task bundles that optimize completion rates and pricing relative to one or more particular workers in the worker pool. The task recommendation module 130 then recommends or presents those task bundles to specific workers.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Nath to include the limitation(s) above as disclosed by Nath. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Nath [0001-0002, 0028] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Nath and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pedersen in view of Hambridge and Weslosky as applied to claim(s) 15 above and further in view of Nath et al. (US 2015/0317582 A1).
Regarding claim 28, modified Pedersen teaches the one or more non-transitory computer-readable media of claim of claim 15, as well as wherein determining the workload balance.
Modified Pedersen doesn’t/don’t explicitly teach but Nath discloses
is based at least in part on one or more of:
a number of requests that individual remote operators of the set of available remote operators have responded to over a predetermined period of time, a historical response time of the individual remote operators, or a number of restrictions associated with the individual remote operators [see at least [0096] “Note that in the event that any of the workers shown as paired with particular tasks or tasks bundles fails to either accept the recommended tasks to fails to complete those tasks within some predefined deadline, the Context-Aware Crowdsourced Task Optimizer operates will recommend those unaccepted or uncompleted tasks to other workers. Further, it should be understood that when those unaccepted or uncompleted tasks are recommended to other workers, the way in which those tasks are bundled is likely to change since those bundles are optimized in each round, relative to any other available tasks and workers, as if that round were the only round.”;
[0044] “In general a worker's current context is inferred or determined from sensors associated with one or more of the worker's mobile computing devices (e.g., GPS, accelerometers, Wi-Fi, etc.), while his future contexts and preferences are dynamically learned from historical data such as frequently visited places, transportation modes (e.g., walking, bicycling, driving, train travel, air travel, etc.), travel routes, completed tasks, preferred times, etc.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify modified Pedersen with Nath to include the limitation(s) above as disclosed by Nath. Doing so would further define Pedersen’s [0002] decision making on when an autonomous vehicle might benefit from assistance of a human operator by via further defining what assistance a human operator can offer [see at least Nath [0001-0002, 0028] ].
Furthermore, all of the claimed elements were known in the prior arts of a) modified Pedersen and b) Nath and c) one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
When responding to the office action, any new claims and/or limitations should be accompanied by a reference as to where the new claims and/or limitations are supported in the original disclosure.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP §706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES WEBB whose telephone number is (313)446-6615. The examiner can normally be reached on M-F 10-3.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O’Connor can be reached on (571) 272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES WEBB/Examiner, Art Unit 3624
Prosecution Insights