DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 06/24/2026 have been fully considered; however, Applicant’s arguments are mooted as the additional subject matter recited in the amendment to the claims requires a new ground of rejection. The arguments largely focus on the lack of disclosure of the amended subject matter (particularly the highlighted item in the image provided via the notification) within the previously relied upon references (U.S. PG Pub. 2021/0011471 to Hurd and U.S. PG Pub. 2017/0311534 to Rusciolelli et al.). Examiner relies on U.S. Pat. 9,098,752 to Huntzicker to teach the newly recited limitation in the rejections proffered below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7-9, 13, 15, 16, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Hurd (US Publication Number 20210011471 A1; hereinafter Hurd) in view of Rusciolelli et al. (US Publication Number 20170311534 A1; hereinafter Rusciolelli) and in further view of Huntzicker (U.S. Pat. 9,098,752; hereinafter Huntzicker).
Regarding Claim 1 (independent), which recites substantially similar subject matter to independent Claims 9/16 but separated for clarity of record, Hurd discloses:
a control and monitoring system for autonomous agricultural systems (See at least Figure 1 and Paragraph 0002), including autonomous tractor systems (Paragraph 0019, "Examples of agricultural vehicles (1) include tractors, trucks, spreaders, sprayers, combines, utility vehicles, or any other vehicle"), comprising:
obtaining, at a device, operational instructions for operation of an autonomous tractor system to perform an agricultural task within an operational environment (Figure 9 and Paragraphs 0058-0067, "At step 214, agricultural operations are selected in the application for use At step 218, executable instructions are generated for the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields for the selected agricultural operations by the agricultural operation and control application");
directing, by the device, the operational instructions to the autonomous tractor system (Figure 9 and Paragraph 0068 describes the autonomous tractor system performing operational instructions ("At step 220, autonomously operating the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields using the application to perform the selected agricultural operations") which reasonably indicates that the operational instructions were directed to the autonomous tractor system);
while the autonomous tractor system is autonomously performing the agricultural task per the operational instructions, obtaining, at the device, a notification that the autonomous tractor system is experiencing an event, the notification generated by the autonomous tractor system and providing an indication of the event (Paragraphs 0036-0037 describes monitoring the autonomous tractor system while performing the agricultural task and requests user input when an event occurs ("The system (100) logs data and is capable of displaying it on a map interface that allows the user to monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands. Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event") which reasonably indicates that a user device obtains a notification indicating the event).
While Hurd further discloses features that allow a user to select from a plurality of actions performable by the autonomous tractor system (Paragraph 0035, the system (100) allows the user to define an objective or plan an operation they desire the agricultural vehicle (1, 2) to perform by selecting from a predefined list of agricultural operations") and that user input may be requested to determine how the autonomous tractor system reacts to an event (Paragraph 0037), Hurd does not explicitly disclose a plurality of responses performable by the autonomous tractor system in response to the event, the autonomous tractor system determining the plurality of responses based on the event and based on one or more of: 2 not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task; in response to the notification, obtaining, based on input from a user, a selection of one of the plurality of responses; and directing, by the device, the selected one of the plurality of responses to the autonomous tractor system, the autonomous tractor system autonomously performing the selected one of the plurality of responses while performing the agricultural task to accommodate the event.
Nevertheless, Rusciolelli teaches an autonomous agricultural vehicle (see at least Abstract) configured to respond to events (Paragraph 0004, "Upon receiving an event condition, the system may revise the mission plan to resolve the event condition while providing an optimization based on current agricultural conditions") comprising:
a plurality of responses performable by the autonomous tractor system in response to the event (Figures 6-7 and Paragraphs 0053-0055 describe a plurality of selectable options ("Multiple mission revisions may be presented as options and/or adjustments may be made before communicating to vehicles 10 for execution, similar to providing a mission plan as described above with respect to FIG. 6") which are in response to event conditions);
the autonomous tractor system determining the plurality of responses based on the event and based on one or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task (Paragraphs 0048-0050 describe the selectable options as being based on the event (" to resolve the event condition") along with agricultural conditions not associated with the event (" also provide an optimization based on current agricultural conditions"), which comprise operating conditions not associated with the event the vehicles 10 may report progress information which may include a position of each vehicle 10 with respect to its assigned path, an amount of agricultural product collected, an amount of agricultural product dispensed, and so forth. The base station receiving the progress information may track the progress information for providing optimizations in subsequent mission revision"));
in response to the notification, obtaining, based on input from a user, a selection of one of the plurality of responses and directing, by the device, the selected one of the plurality of responses to the autonomous tractor system, the autonomous tractor system autonomously performing the selected one of the plurality of responses while performing the agricultural task to accommodate the event (Paragraph 0054 describes the autonomous tractor system being directed to perform a selected option ("the mission revision may be provided and communicated to the vehicles 10 and/or other equipment for execution")).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Rusciolelli by including features that allow the autonomous tractor system to determine a plurality of selectable responses based on an event and conditions not associated with the event. Particularly because Hurd and Rusciolelli relate to requesting user input for an autonomous agricultural vehicle, whereas in Rusciolelli, a plurality of selectable actions are determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing user control during critical situations of the autonomous vehicle.
Hurd also does not appear to disclose the newly amended portions of the claim which recite, “the notification including an image or video feed of the event captured by the autonomous tractor system, the image or video feed including a highlighted item that resulted in the event.”
Nevertheless, Huntzicker in the field of vehicle path assessment, teaches a notification including an image or video feed of the event captured by an imaging system, the image or video feed including a highlighted item that resulted in an event (At least Fig. 2C where a notification, 17 is provided including a highlighted item, the at least one object 15 may be highlighted on the display screen and displayed to the pilot of the vehicle 12 and may be accompanied by audible, physical or other sensory notifications 17 of the same).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Huntzicker by including features that allow the autonomous tractor system to provide an image of an event with a highlighted item that resulted in the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing warnings to an operator when an obstacle is in the vehicle path and showing a visual cue focused on the obstacle to alert the driver to prevent a collision scenario.
Regarding Claim 9 (independent), which recites substantially similar subject matter to Claims 1/16 but separated for clarity of record, Hurd discloses a control and monitoring system for autonomous agricultural systems (See at least Figure 1 and Paragraph 0002), including autonomous tractor systems (Paragraph 0019, "Examples of agricultural vehicles (1) include tractors, trucks, spreaders, sprayers, combines, utility vehicles, or any other vehicle"), comprising:
a user interface configured to present information to a user and to obtain input from the user (Paragraph 0026, "The cloud based server application (13) displays an interactive user interface on a user mobile device (11) such as a smart phone, tablet, laptop or any other device, which allows users to monitor and control the connected first agricultural vehicle (1), second agricultural vehicle (2) and any other vehicle as well as their associated agricultural equipment (102)"; Examiner notes that server application (13) may present information or obtain user input through user interface on mobile device (11));
one or more computer readable media configured to store instructions; and a processor coupled to the computer readable media and the user interface, the processor configured to execute the instructions to cause or direct the device to perform operations (Paragraph 0032, "executable command(s) which is uploaded to the agricultural vehicle's (1, 2) executive controller unit (3, 9) and/or controlling computer. Once the appropriate executable commands are received, the executive controller unit (3, 9) and/or controlling computer actives the controllers and waits for a launch command Once the launch command is received, the executive controller unit (3, 9) and/or controlling computer executes the executable commands to control the agricultural vehicle (1. 2)"; Examiner notes that as ECU (3, 9) is capable of waiting to implement the received executable commands, the ECU necessarily requires capabilities to store the executable commands on computer-readable media), the operations comprising:
obtain, via the user interface, operational instructions for operation of an autonomous tractor system to perform an agricultural task within an operational environment (Figure 9 and Paragraphs 0058-0067, "At step 200, an agricultural operation and control application displayable on an interactive interface At step 214, agricultural operations are selected in the application for use At step 218, executable instructions are generated for the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields for the selected agricultural operations by the agricultural operation and control application"));
direct the operational instructions to the autonomous tractor system over a network (Figure 9 and Paragraph 0068 describes that the autonomous tractor system performs the obtained operational instructions ("At step 220, autonomously operating the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields using the application to perform the selected agricultural operations") which reasonably indicates that the operational instructions were directed to the autonomous tractor system over a network);
while the autonomous tractor system is autonomously performing the agricultural task per the operational instructions, obtaining, via the network, a notification that the autonomous tractor system is experiencing an event, the notification generated by the autonomous tractor system and providing an indication of the event; present the notification via the user interface (Paragraphs 0036-0037 describes monitoring the autonomous tractor system while performing the agricultural task and requests user input when an event occurs (" The system (100) logs data and is capable of displaying it on a map interface that allows the userto monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands. Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event") which reasonably indicates that a user device obtains a notification indicating the event).
While Hurd further discloses features that allow a user to select from a plurality of actions performable by the autonomous tractor system (Paragraph 0035, 'the system (100) allows the user to define an objective or plan an operation they desire the agricultural vehicle (1, 2) to perform by selecting from a predefined list of agricultural operations") and that user input may be requested to determine how the autonomous tractor system reacts to an event (Paragraphs 0036-0037), Hurd does not explicitly disclose a plurality of responses performable by the autonomous tractor system in response to the event, the autonomous tractor system determining the plurality of responses based on the event and based on one or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task; in response to the notification, obtaining, based on input from a user, a selection of one of the plurality of responses; and directing, by the device, the selected one of the plurality of responses to the autonomous tractor system, the autonomous tractor system autonomously performing the selected one of the plurality of responses while performing the agricultural task to accommodate the event.
Nevertheless, Rusciolelli teaches an autonomous agricultural vehicle (see at least Abstract) configured to respond to events (Paragraph 0004, "Upon receiving an event condition, the system may revise the mission plan to resolve the event condition while providing an optimization based on current agricultural conditions") comprising:
a plurality of responses performable by the autonomous tractor system in response to the event (Figures 6-7 and Paragraphs 0053-0055 describes a plurality of selectable options ("Multiple mission revisions may be presented as options and/or adjustments may be made before communicating to vehicles 10 for execution, similar to providing a mission plan as described above with respect to FIG. 6") which are in response to event conditions);
the autonomous tractor system determining the plurality of responses based on the event and based on one or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task (Paragraphs 0048-0050 describe the selectable options as being based on the event (" to resolve the event condition") along with agricultural conditions not associated with the event (" also provide an optimization based on current agricultural conditions"), which comprise operating conditions not associated with the event ("the vehicles 10 may report progress information which may include a position of each vehicle 10 with respect to its assigned path. an amount of agricultural product collected, an amount of agricultural product dispensed, and so forth. The base station receiving the progress information may track the progress information for providing optimizations in subsequent mission revision"));
in response to the notification, obtaining, based on input from a user, a selection of one of the plurality of responses and directing, by the device, the selected one of the plurality of responses to the autonomous tractor system, the autonomous tractor system autonomously performing the selected one of the plurality of responses while performing the agricultural task to accommodate the event (Paragraph 0054 describes the autonomous tractor system being directed to perform a selected option ("the mission revision may be provided and communicated to the vehicles 10 and/or other equipment for execution")).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Rusciolelli by including features that allow the autonomous tractor system to determine a plurality of selectable responses based on an event and conditions not associated with the event. Particularly because Hurd and Rusciolelli relate to requesting user input for an autonomous agricultural vehicle, whereas in Rusciolelli, a plurality of selectable actions is determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing user control during critical situations of the autonomous vehicle.
Hurd also does not appear to disclose the newly amended portions of the claim which recite, “the notification including an image or video feed of the event captured by the autonomous tractor system, the image or video feed including a highlighted item that resulted in the event.”
Nevertheless, Huntzicker in the field of vehicle path assessment, teaches a notification including an image or video feed of the event captured by an imaging system, the image or video feed including a highlighted item that resulted in an event (At least Fig. 2C where a notification, 17 is provided including a highlighted item, the at least one object 15 may be highlighted on the display screen and displayed to the pilot of the vehicle 12 and may be accompanied by audible, physical or other sensory notifications 17 of the same).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Huntzicker by including features that allow the autonomous tractor system to provide an image of an event with a highlighted item that resulted in the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing warnings to an operator when an obstacle is in the vehicle path and showing a visual cue focused on the obstacle to alert the driver to prevent a collision scenario.
Regarding Claim 16 (independent), which recites substantially similar subject matter to Claims 1/9 but separated for clarity of record, Hurd discloses a control and monitoring system for autonomous agricultural systems (see at least Figure 1 and Paragraph 0002), including autonomous tractor systems (Paragraph 0019, "Examples of agricultural vehicles (1) include tractors, trucks, spreaders, sprayers, combines, utility vehicles, or any other vehicle"), comprising:
obtaining, at an autonomous tractor system, operational instructions to autonomously perform an agricultural task within an operational environment (Figure 9 and Paragraphs 0058-0067, "At step 214, agricultural operations are selected in the application for use At step 218, executable instructions are generated for the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields for the selected agricultural operations by the agricultural operation and control application");
autonomously performing, by the autonomous tractor system, the operational instructions to perform the agricultural task (Figure 9 and Paragraph 0068, "At step 220, autonomously operating the selected agricultural vehicle and the selected agricultural equipment in the selected agricultural fields using the application to perform the selected agricultural operations");
while the autonomous tractor system is autonomously performing the agricultural task per the operational instructions, identifying an event; generating a notification that provides an indication of the event (Paragraphs 0036-0037 describes monitoring the autonomous tractor system while performing the agricultural task and requests user input when an event occurs ("The system (100) logs data and is capable of displaying it on a map interface that allows the user to monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands. Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event") which reasonably indicates a notification indicating the event).
While Hurd further discloses features that allow a user to select from a plurality of actions performable by the autonomous tractor system (Paragraph 0035, the system (100) allows the user to define an objective or plan an operation they desire the agricultural vehicle (1, 2) to perform by selecting from a predefined list of agricultural operations") and that user input may be requested to determine how the autonomous tractor system reacts to an event (Paragraphs 0036-0037), Hurd does not explicitly disclose in response to the event, determining, based on the event and based on one or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task, a plurality of responses to the event that are performable by the autonomous tractor system; directing the notification to another device; obtaining an answer to the notification from the other device, the answer including a selected one of the plurality of responses; and autonomously performing, by the autonomous tractor system, the selected on one of the plurality of responses while performing the agricultural task to accommodate the event.
Nevertheless, Rusciolelli teaches an autonomous agricultural vehicle (see at least Abstract) configured to respond to events (Paragraph 0004, "Upon receiving an event condition, the system may revise the mission plan to resolve the event condition while providing an optimization based on current agricultural conditions") comprising: in response to the event, determining, based on the event and based on one or more of:
environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task, a plurality of responses to the event that are performable by the autonomous tractor system (Figures 6-7 and Paragraphs 0053-0055 describes a plurality of selectable options ("Multiple mission revisions may be presented as options and/or adjustments may be made before communicating to vehicles 10 for execution, similar to providing a mission plan as described above with respect to FIG. 6") which are in response to an event condition; Paragraphs 0048-0050 describe the selectable options as being based on the event (" to resolve the event condition") along with agricultural conditions not associated with the event ("...also provide an optimization based on current agricultural conditions"), which comprise operating conditions not associated with the event ("the vehicles 10 may report progress information which may include a position of each vehicle 10 with respect to its assigned path, an amount of agricultural product collected, an amount of agricultural product dispensed, and so forth. The base station receiving the progress information may track the progress information for providing optimizations in subsequent mission revision"));
directing the notification to another device; obtaining an answer to the notification from the other device, the answer including a selected one of the plurality of responses and autonomously performing, by the autonomous tractor system, the selected on one of the plurality of responses while performing the agricultural task to accommodate the event (Paragraph 0054 describes the autonomous tractor system being directed to perform a selected option ("the mission revision may be provided and communicated to the vehicles 10 and/or other equipment for execution")).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Rusciolelli by including features that allow the autonomous tractor system to determine a plurality of selectable responses based on an event and conditions not associated with the event. Particularly because Hurd and Rusciolelli relate to requesting user input for an autonomous agricultural machine, whereas in Rusciolelli, a plurality of selectable actions are determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing user control during critical situations of the autonomous vehicle.
Hurd also does not appear to disclose the newly amended portions of the claim which recite, “the notification including an image or video feed of the event captured by the autonomous tractor system, the image or video feed including a highlighted item that resulted in the event.”
Nevertheless, Huntzicker in the field of vehicle path assessment, teaches a notification including an image or video feed of the event captured by an imaging system, the image or video feed including a highlighted item that resulted in an event (At least Fig. 2C where a notification, 17 is provided including a highlighted item, the at least one object 15 may be highlighted on the display screen and displayed to the pilot of the vehicle 12 and may be accompanied by audible, physical or other sensory notifications 17 of the same).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Hurd invention to incorporate the teachings of Huntzicker by including features that allow the autonomous tractor system to provide an image of an event with a highlighted item that resulted in the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing warnings to an operator when an obstacle is in the vehicle path and showing a visual cue focused on the obstacle to alert the driver to prevent a collision scenario.
Regarding Claims 5, 13, and 20, Hurd as currently modified teaches claims 1/9/16. Hurd further discloses: wherein the notification is obtained [or provided] in real-time while the autonomous tractor system is performing the agricultural task (Paragraph 0037 describes requesting user input while monitoring progress of the autonomous tractor system ("The system (100) logs data and is capable of displaying it on a map interface that allows the user to monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands") which reasonably indicates a real-time notification to a user).
Regarding Claims 7 and 15, Hurd as currently modified teaches claims 1/9. Hurd further discloses: wherein the user is separate from the autonomous tractor system such that the user cannot see or interact directly with the autonomous tractor system (Paragraph 0092 describes remote operation ("User mobile device (11) is formed of any suitable size, shape and design and facilitates the operation of server application (13) on the mobile device (11) remote from the first agricultural vehicle (1)") which reasonably indicates that the user is separate from the autonomous tractor system such that the user is not required to see or interact directly with the autonomous tractor system).
Regarding Claim 8, Hurd as currently modified teaches claim 1. Hurd further discloses: one or more computer readable media configured to store instructions, which when executed, are configured to cause performance of the method of claim 1. (Paragraph 0032, "When the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) and the agricultural field and agricultural operation have been set up and saved the system (100) combines all of the saved data to generate an executable command(s) which is uploaded to the agricultural vehicle's (1, 2) executive controller unit (3, 9) and/or controlling computer. Once the appropriate executable commands are received, the executive controller unit (3, 9) and/or controlling computer actives the controllers and waits for a launch command Once the launch command is received, the executive controller unit (3, 9) and/or controlling computer executes the executable commands to control the agricultural vehicle (1, 2)).
Regarding Claim 19, Hurd as currently modified teaches claim 16. Hurd does not explicitly disclose: wherein one or more of the plurality of responses provided in the notification are selected by the autonomous tractor system based on the agricultural task being performed by the autonomous tractor system.
Nevertheless, Rusciolelli further teaches: wherein one or more of the plurality of responses provided in the notification are selected by the autonomous tractor system based on the agricultural task being performed by the autonomous tractor system (Paragraphs 0048-0050 describes the selectable options ("revised mission plans"), which are based on the event ("to resolve event conditions") and conditions not associated with the event "optimization based on current agricultural conditions such as those reported by the progress information"); Paragraph 0048 describes the progress information related to the agricultural task being performed ("a position of each vehicle 10 with respect to its assigned path, an amount of agricultural product collected, an amount of agricultural product dispensed, and so forth"))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Rusciolelli by including features that allow the autonomous tractor system to determine a plurality of selectable responses based on the agricultural task being performed.
Particularly because Hurd and Rusciolelli relate to requesting user input for an autonomous agricultural vehicle, whereas in Rusciolelli, a plurality of selectable actions are determined based on the event. It would be desirable to incorporate this feature into Hurd for the well-known benefit of providing user control during critical situations of the autonomous vehicle.
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Watanabe et al. (US Publication Number 20180105185 A1; hereinafter Watanabe).
Regarding Claims 2 and 10, Hurd as currently modified teaches claims 1/9. While Hurd as currently modified teaches a plurality of responses, Hurd as currently modified does not explicitly teach: wherein one or more of the plurality of responses provided in the notification are highlighted by the autonomous tractor system based on the event experienced by the autonomous tractor system, the agricultural task, or both the event experienced by the autonomous tractor system and the agricultural task being performed by the autonomous tractor system.
Nevertheless, Watanabe teaches an autonomous vehicle that displays a plurality of selectable options to a user (see at least abstract) comprising: wherein one or more of the plurality of responses provided in the notification are highlighted by the autonomous tractor system based on the event experienced by the autonomous tractor system, the agricultural task, or both the event experienced by the autonomous tractor system and the agricultural task being performed by the autonomous tractor system (Paragraph 0550, "the action candidates to be displayed in the action selection screen are narrowed down based on the current detection information and the previous travel history, whereby the display content of the action selection screen can be simplified. Further, the recommended action candidate in the action candidates displayed in the action selection screen is displayed in a highlighted manner").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Watanabe by including features that allow highlighting one or more selectable options. Particularly because Hurd and Watanbe teach features that allow user input to direct an autonomous vehicle, whereas in Watanabe, user input is provided with a plurality of options that may be highlighted. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of simplifying the decision-making process for a user (Watanabe, Paragraph 0550).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Peake et al. (US Publication Number 20230205195 A1, filed on December 28, 2021; hereinafter Peake).
Regarding Claim 3, Hurd as currently modified teaches claim 1. While Hurd further discloses features that allow a user to select options that are performed sequentially (Paragraph 0035, "the system (100) allows the user to define an objective or plan an operation they desire the agricultural vehicle (1, 2) to perform by selecting from a predefined list of agricultural operations After the vehicle controller (such as executive controller unit (3, 9) and/or a controlling computer) is sent the command or chain of commands it runs a process to sequentially execute them"), Hurd as currently modified does not explicitly teach: wherein in response to the notification, a selection of a set of responses of the plurality of responses is obtained, the set of responses including two or more responses and including the selected one of the plurality of responses, the two or more responses including a first response that is the selected one of the plurality of responses and that is performed first and a second response of the two or more responses that is performed after the first response in response to the first response not resolving the event.
Nevertheless, Peake teaches an autonomous agricultural vehicle configured to select performable responses to an event (see at least Abstract), wherein the selection may be performed by a user (Paragraph 0030, The farming machine 100 includes a control system 130 for controlling operations of system components. The control system 130 can receive information from and/or provide input to the detection mechanism 110, the verification mechanism 150, and the treatment mechanism 120. The control system 130 can be automated or can be operated by a user") comprising: wherein in response to the notification, a selection of a set of responses of the plurality of responses is obtained, the set of responses including two or more responses and including the selected one of the plurality of responses, the two or more responses including a first response that is the selected one of the plurality of responses and that is performed first and a second response of the two or more responses that is performed after the first response in response to the first response not resolving the event (Paragraph 0064 describes two or more responses being performed until an event is resolved ("For each operational failure, the respective solution operations may be structured in a preset sequence of solution operations, attempted in order by the farming machine 100 until the operational failure is resolved") which reasonably indicate a second response is performed if a first response does not resolve the event).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Peake by including features that allow sequential responses to be performed.
Particularly because Hurd and Peake teach features that allow user input to direct an autonomous vehicle, whereas in Peake, a second response is performed if a first response does not resolve an event. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of reducing interruptions to the user from each failed response.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Gerrish (US Publication Number 20170261986 A1; hereinafter Gerrish).
Regarding Claim 4, Hurd as currently modified teaches claim 1. While Hurd as currently modified teaches wherein the autonomous tractor system determines the plurality of responses based on the event and based on [one] or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task, Hurd as currently modified does not explicitly teach that the plurality of responses may be based on two or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task.
Nevertheless, Gerrish teaches an autonomous agricultural robot (see at least Abstract) that provides a plurality of selectable responses (Paragraph 0024, the autonomous agricultural machine 10 may communicate the preferred courses of operation to the central control station 16 for selection by the farmer") wherein: the plurality of responses may be based on two or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task (Paragraph 0023 describes the plurality of responses being based environmental conditions not associated with the event and operating conditions not associated with the event (" determine preferred courses of action based on the weather information received and consideration of operational information)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Gerrish by including features that allow the autonomous tractor system to determine a plurality of selectable responses based at least in part on weather conditions and operating conditions not associated with the event. Particularly because Hurd and Gerrish relate to requesting user input for an autonomous agricultural vehicle, whereas in Gerrish, a plurality of selectable actions are determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of including a variety of selectable responses (Gerrish, Paragraph 0024).
Claims 6 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Kumavat et al. (US Publication Number 20230176573 A1, with priority to December 6, 2021; hereinafter Kumavat).
Regarding Claim 6, Hurd as currently modified teaches while the autonomous tractor system is autonomously performing the agricultural task per the operational instructions, obtaining, at the device, a [notification] that the autonomous tractor system is experiencing [an event], the [notification] generated by the autonomous tractor system and providing an indication of the [event] (Hurd, Paragraphs 0036-0037 describes monitoring the autonomous tractor system while performing the agricultural task and requests user input when an event occurs ("The system (100) logs data and is capable of displaying it on a map interface that allows the user to monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands. Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event") which reasonably indicates that a user device obtains a notification indicating the event) and a plurality of [responses] performable by the autonomous tractor system in response to the [event] (Rusciolelli, Figures 6-7 and Paragraphs 0053-0055 describe a plurality of selectable options ("Multiple mission revisions may be presented as options and/or adjustments may be made before communicating to vehicles 10 for execution, similar to providing a mission plan as described above with respect to FIG. 6") which are in response to event conditions). While Hurd further reasonably discloses performing operations for plurality of events (Paragraph 0037, "events"), Hurd as currently modified does not explicitly recite duplicating/repeating the steps for a [second] event, nor wherein the autonomous tractor system autonomously performs a self-selected response to the second event in response to not receiving a selected one of the plurality of second responses within a threshold amount of time of directing the second notification, including the plurality of second responses, to the device.
Nevertheless, Rusciolelli further teaches repeating processes in response to a: [second] event (see at least Figure 5, wherein steps for adjusting missions may be repeated from Blocks 184-190).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Rusciolelli by repeating steps in response to a second event. It would be desirable to incorporate this feature into Hurd as currently modified for the well-known benefit of allowing continuous operation in the field.
However, Hurd as currently modified still does not explicitly teach: wherein the autonomous tractor system autonomously performs a self-selected response to the second event in response to not receiving a selected one of the plurality of second responses within a threshold amount of time of directing the second notification, including the plurality of second responses, to the device.
Nevertheless, Kumavat teaches an autonomous vehicle configured to request user selection in response to an event (see at least Abstract; Paragraph 0040, " such that a failure response. can be selected") comprising: wherein the autonomous tractor system autonomously performs a self-selected response to the second event in response to not receiving a selected one of the plurality of second responses within a threshold amount of time of directing the second notification, including the plurality of second responses, to the device (Paragraph 0093, "In a specific example involving tiered minimal risk maneuvers, for instance, in response to detecting a warning or failure, the vehicle can initiate (e.g., autonomously initiate) a first minimal risk maneuver (e.g., slowing down) while providing a remote operator request for input (e.g., within a predetermined time frame) before the vehicle initiates a more intense minimal risk maneuver (e.g., coming to a stop) ..."). ").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Kumavat by including features that allow a threshold amount of time for user response. Particularly because Hurd and Kumavat teach features that allow user input to direct an autonomous vehicle, whereas in Kumavat, a threshold amount of time is provided for user selection. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of preventing downtime if a selection is not provided.
Regarding Claim 21, Hurd as currently modified teaches while the autonomous tractor system is autonomously performing the agricultural task per the operational instructions, identifying [an event]; generating a [notification] that provides an indication of the [event] (Hurd, Paragraphs 0036-0037 describes monitoring the autonomous tractor system while performing the agricultural task and requests user input when an event occurs ("The system (100) logs data and is capable of displaying it on a map interface that allows the user to monitor progress of the agricultural vehicle(s) (1, 2) and associated agricultural equipment (102) while the system (100) executes commands. Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event") which reasonably indicates a notification indicating the event) and in response to the [event], determining, based on the [event] a plurality of [responses] to the event that are performable by the autonomous tractor system (Rusciolelli, Figures 6-7 and Paragraphs 0053-0055 describes generating a plurality of selectable options ("Multiple mission revisions may be presented as options and/or adjustments may be made before communicating to vehicles 10 for execution, similar to providing a mission plan as described above with respect to FIG. 6") which are in response to an event condition; Paragraphs 0048-0050 describe the selectable options as being based on the event ("...to resolve the event condition")) and directing the [notification] to the other device (Rusciolelli, Paragraph 0054, "the mission revision may be provided and communicated to the vehicles 10 and/or other equipment for execution")). While Hurd further reasonably discloses performing operations for plurality of events (Paragraph 0037, "events"), Hurd as currently modified does not explicitly recite duplicating/repeating the steps for a [second] event, nor in response to not obtaining an answer within a threshold amount of time of directing the second notification to the other device, autonomously performing a self-selected response to the event.
Nevertheless, Rusciolelli further teaches repeating processes in response to a: [second] event (see at least Figure 5, wherein steps for adjusting missions may be repeated from Blocks 184-190).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Rusciolelli by repeating steps in response to a second event. It would be desirable to incorporate this feature into Hurd as currently modified for the well-known benefit of allowing continuous operation in the field.
However, Hurd as currently modified still does not explicitly teach: in response to not obtaining an answer within a threshold amount of time of directing the second notification to the other device, autonomously performing a self-selected response to the event.
Nevertheless, Kumavat teaches an autonomous vehicle configured to request user selection in response to an event (see at least Abstract; Paragraph 0040, " such that a failure response can be selected") comprising: in response to not obtaining an answer within a threshold amount of time of directing the second notification to the other device, autonomously performing a self-selected response to the event (Paragraph 0093, In a specific example involving tiered minimal risk maneuvers, for instance, in response to detecting a warning or failure, the vehicle can initiate (e.g., autonomously initiate) a first minimal risk maneuver (e.g., slowing down) while providing a remote operator request for input (e.g., within a predetermined time frame) before the vehicle initiates a more intense minimal risk maneuver (e.g., coming to a stop) ").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Kumavat by including features that allow a threshold amount of time for user response. Particularly because Hurd and Kumavat teach features that allow user input to direct an autonomous vehicle, whereas in Kumavat, a threshold amount of time is provided for user selection. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of preventing downtime if a selection is not provided.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Cavender-Bares et al. (US Publication Number 20200125098 A1; hereinafter Cavender-Bares).
Regarding Claim 14, Hurd as currently modified teaches claim 9. While Hurd as currently modified teaches sending a notification of an unexpected situation to a user through a user interface (Paragraphs 0036-0037, "This transmitted data is then received by the system (100) through cloud service provider (14) by cloud based server application (13), analyzed and displayed in real-time or near real time to the user through user mobile device (11) Serving as a monitoring station, the system (100) flags events and asks for user input in situations where clear protocols are not defined such as an unknown obstacle in its path or any other aberration or unanticipated event"), Hurd does not explicitly disclose that the notification may include an image captured by the autonomous tractor system.
Nevertheless, Cavender-Bares teaches an autonomous agricultural management system that can present a plurality of responses to a user during an event (Paragraph 0076, "In the case where autonomous vehicle platform 100 encounters a situation that cannot be resolved autonomously, a message or alert can be routed to an operator for assistance") comprising: wherein the notification includes an image captured by autonomous tractor system that relates to the event (Paragraph 0076, "In the case where autonomous vehicle platform 100 encounters a situation that cannot be resolved autonomously, a message or alert can be routed to an operator for assistance Such a message can include, for example, information that autonomous vehicle platform 100 or system 200 has been stopped for a particular reason, one or more images of a situation that autonomous vehicle platform 100 has encountered, a variety of statistics, such as heading. tractions, engine status, tank status. tilt angle, a video or series of images of the last several seconds of operation before the message was routed, or a combination thereof. Using this information, in one embodiment, the operator or team of operators can remotely resolve the situation. For example, the operator can select one of several preprogrammed commands or options, such as hold position, break through planted crops 106 and proceed to next row, or back up and start again on adjacent row").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Cavender-Bares by including features that allow imagery captured by an agricultural tractor system related to the identified event. Particularly because Hurd and Cavender-Bares both relate to managing agricultural operations, whereas in Cavender-Bares, imagery is provided related to an event. It would be desirable to incorporate this feature into Hurd for the benefit of presenting additional information to a user (Cavender-Bares, Paragraph 0076).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and Kumavat and further in view of Peake (previously cited further above).
Regarding Claim 22, Hurd as currently modified teaches claim 21. While Hurd further discloses features that allow a user to select options that are performed sequentially (Paragraph 0035, "the system (100) allows the user to define an objective or plan an operation they desire the agricultural vehicle (1, 2) to perform by selecting from a predefined list of agricultural operations After the vehicle controller (such as executive controller unit (3, 9) and/or a controlling computer) is sent the command or chain of commands it runs a process to sequentially execute them"), Hurd as currently modified does not explicitly teach: wherein the answer includes a selection of a set of responses of the plurality of responses, the set of responses including two or more responses and including the selected one of the plurality of responses and the selected one of the plurality of responses is a first response in the set of responses, the method further comprising in response to the first response not resolving the event, performing a second response in the set of responses while performing the agricultural task to resolve the event.
Nevertheless, Peake teaches an autonomous agricultural vehicle configured to select performable responses to an event (see at least Abstract), wherein the selection may be performed by a user (Paragraph 0030, The farming machine 100 includes a control system 130 for controlling operations of system components. The control system 130 can receive information from and/or provide input to the detection mechanism 110, the verification mechanism 150, and the treatment mechanism 120. The control system 130 can be automated or can be operated by a user") comprising: wherein the answer includes a selection of a set of responses of the plurality of responses, the set of responses including two or more responses and including the selected one of the plurality of responses and the selected one of the plurality of responses is a first response in the set of responses, the method further comprising in response to the first response not resolving the event, performing a second response in the set of responses while performing the agricultural task to resolve the event (Paragraph 0064 describes two or more responses being performed until an event is resolved ("For each operational failure, the respective solution operations may be structured in a preset sequence of solution operations, attempted in order by the farming machine 100 until the operational failure is resolved") which reasonably indicate a second response is performed if the event is not resolved by the first response).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Peake by including features that allow sequential responses to be performed. Particularly because Hurd and Peake teach features that allow user input to direct an autonomous vehicle, whereas in Peake, a second response is performed if a first response does not resolve an event. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of reducing interruptions to the user for each failed response.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli, Kumavat and Peake and further in view of Gerrish (previously cited further above) and Krog et al. (US Publication Number 20230112003 A1, with priority to October 7, 2021; hereinafter Krog).
Regarding Claim 23, Hurd as currently modified teaches claim 22. While Hurd as currently modified teaches wherein the autonomous tractor system determines the plurality of responses based on the event and based on [one] or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task, Hurd as currently modified does not explicitly teach that the plurality of responses may be based on the event and based on environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task.
Nevertheless, Gerrish teaches an autonomous agricultural robot (see at least Abstract) that provides a plurality of selectable responses (Paragraph 0024, the autonomous agricultural machine 10 may communicate the preferred courses of operation to the central control station 16 for selection by the farmer") comprising: wherein the autonomous tractor system determines the plurality of responses based on the event and based on environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event and [completion status] of the agricultural task (Paragraph 0023 describes the plurality of responses being based environmental conditions not associated with the event and operating conditions not associated with the event (" determine preferred courses of action based on the weather information received and consideration of operational information resource needs (e.g., amount of seed, fertilizer, and/or pesticide on the machine 10)") wherein the operational information further comprises a time remaining for completing an agricultural task ("the time required for the machine 10 to complete its agricultural operations for given row of a current field 18") which is reasonably indicative of a completion status).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Gerrish by including features that allow the autonomous tractor system to determine a plurality of selectable responses based at least in part on weather conditions not associated with the event, operating conditions not associated with the event and a completion status. Particularly because Hurd and Gerrish relate to requesting user input for an autonomous agricultural vehicle, whereas in Gerrish, a plurality of selectable actions are determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of including a variety of selectable responses (Gerrish, Paragraph 0024).
Even though Hurd as currently modified teaches the completion status of the agricultural task, Hurd as currently modified does not explicitly recite that a completion status can be in the form of a percent of completion of the agricultural task.
Nevertheless, Krog teaches an autonomous agricultural vehicle (see at least Abstract and Paragraph 0007) configured to determine: percent of completion of the agricultural task (Figure 15B and Paragraph 0203 describes a percent of completion of the agricultural task ("defining a percentage of completion")).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Krog by including features that allow the completion status of the autonomous agricultural vehicle to be a percentage of completion. Particularly because Hurd as currently modified and Krog relate to monitoring an autonomous agricultural vehicle, whereas in Krog, a completion status is indicated by a percentage. It would be desirable to modify the time to completion to include a percentage of completion for the benefit providing another intuitive indication of a completion status.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Hurd in view of Rusciolelli and further in view of Gerrish (previously cited further above) and Krog (previously cited further above).
Regarding Claim 24, Hurd as currently modified teaches claim 16. While Hurd as currently modified teaches wherein the autonomous tractor system determines the plurality of responses based on the event and based on [one] or more of: environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task, Hurd as currently modified does not explicitly teach that the plurality of responses may be based on the event and based on environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event, and percent of completion of the agricultural task.
Nevertheless, Gerrish teaches an autonomous agricultural robot (see at least Abstract) that provides a plurality of selectable responses (Paragraph 0024, "the autonomous agricultural machine 10 may communicate the preferred courses of operation to the central control station 16 for selection by the farmer") comprising: wherein the autonomous tractor system determines the plurality of responses based on the event and based on environmental conditions not associated with the event, operating conditions of the autonomous tractor system not associated with the event and [completion status] of the agricultural task (Paragraph 0023 describes the plurality of responses being based environmental conditions not associated with the event and operating conditions not associated with the event (" determine preferred courses of action based on the weather information received and consideration of operational information resource needs (e.g., amount of seed, fertilizer, and/or pesticide on the machine 10)") wherein the operational information further comprises a time remaining for completing an agricultural task ("the time required for the machine 10 to complete its agricultural operations for given row of a current field 18") which is reasonably indicative of a completion status).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Gerrish by including features that allow the autonomous tractor system to determine a plurality of selectable responses based at least in part on weather conditions not associated with the event, operating conditions not associated with the event and a completion status. Particularly because Hurd and Gerrish relate to requesting user input for an autonomous agricultural machine, whereas in Gerrish, a plurality of selectable actions are determined based on an identified event and conditions not associated with the event. It would be desirable to incorporate this feature into Hurd as currently modified for the benefit of including a variety of selectable responses (Gerrish, Paragraph 0024).
Even though Hurd as currently modified teaches the completion status of the agricultural task, Hurd as currently modified does not explicitly recite that a completion status can be in the form of a percent of completion of the agricultural task.
Nevertheless, Krog teaches an autonomous agricultural vehicle (see at least Abstract and Paragraph 0007) teaches: percent of completion of the agricultural task (Figure 15B and Paragraph 0203 describes a percent of completion of the agricultural task ("defining a percentage of completion")).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the Hurd invention to incorporate the teachings of Krog by including features that allow the completion status of the autonomous agricultural vehicle to be a percentage of completion. Particularly because Hurd as currently modified and Krog relate to monitoring an autonomous agricultural vehicle, whereas in Krog, a completion status is indicated by a percentage. It would be desirable to modify the time to completion to include a percentage of completion for the benefit providing another intuitive indication of a completion status.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brodie Follman whose telephone number is (571)270-1169. The examiner can normally be reached 8am-4:30pm EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at (571)270-7429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRODIE J FOLLMAN/Primary Patent Examiner, Art Unit 3669
Prosecution Insights