DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
2. This office action is in response to Amendments and Remarks filed on 04/20/2026 for application number 18/921,915 filed on 10/21/2024, in which claims 1-20 were previously presented for examination.
3. Claim(s) 21-22 has/have been added as new, claim(s) 4, and 15 has/have been canceled, and claim(s) 1, 12, and 19-20 has/have been amended. Accordingly, claim(s) 1-3, 5-14, and 16-22 is/are currently pending.
Examiner Notes
4. The Examiner has cited particular paragraphs or columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. The prompt development of a clear issue requires that the replies of the Applicant meet the objections to and rejections of the claims. Applicant should also specifically point out the support for any amendments made to the disclosure (see MPEP §2163.06). Applicant is reminded that the Examiner is entitled to give the Broadest Reasonable Interpretation (BRI) of the language of the claims. Furthermore, the Examiner is not limited to Applicant’s definition which is not specifically set forth in the claims. SEE MPEP 2141.02 [R-07.2015] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
Response to Arguments
5. Applicant's arguments filed 04/20/2026 have been fully considered but they are not persuasive.
6. Applicant’s arguments and amendments have been addressed in the new rejection outlined below.
7. Applicant’s arguments with respect to claim(s) 1, 12, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
8. Applicant argues dependent claim(s) is/are patentable by the virtue of their dependency on the allowable independent claims and the additional features recited in the dependent claims.
9. This argument is unpersuasive as each independent claim and dependent claim has been fully rejected and for the reasons given above.
Claim Rejections - 35 USC § 103
10. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
11. Claim(s) 1, 5-6, 8-12, 16-20, and 22
is/are rejected under 35 U.S.C. 103 as being unpatentable over Dolgov et al. (US-20190019349-A1) in view of Chen et al. (US-20200401135-A1).
In regard to claim 1
, Dolgov discloses a method comprising (Dolgov, in at least [0031], discloses methods and systems for remote assistance):
receiving, at a computing device, sensor data and a request for assistance from a vehicle, wherein the sensor data depicts a trajectory of the vehicle in an environment (Dolgov, in at least Figs. 4A-4D, 6B, [0142 & 0152], discloses the vehicle requests remote assistance in substantially real-time [i.e., receiving a request for assistance from a vehicle], which the computing system [i.e., at a computing device] uses as a means for alerting the human operator. At block 622, the computing system operates by receiving image data from the autonomous vehicle of an environment of the autonomous vehicle [i.e., receiving, at a computing device, sensor data from a vehicle, wherein the sensor data depicts a trajectory of the vehicle in an environment]. Examiner notes, as illustrated by Fig. 4C-4D, the sensor data, such as the image of the environment, depicts the trajectory of the vehicle in the environment);
displaying, by the computing device and based on the sensor data, a representation of the environment showing the trajectory of the vehicle (Dolgov, in at least in at least Figs. 1, 4A-4D, 6B, and [0050 & 0155], discloses navigation/pathing system 142 determines a driving path for vehicle 100. At block 626, the computing system operates by providing at least one image to an operator [i.e., displaying, by the computing device and based on the sensor data, a representation of the environment showing the trajectory of the vehicle] from the memory, wherein at least one image comprises previously-stored image data related to the at least one object of an environment of the autonomous vehicle has a detection confidence below a threshold. Examiner notes, as illustrated by Fig. 4C-4D, the representation of the environment shows the trajectory of the vehicle);
While Dolgov discloses at block 628, the computing system operates by receiving an operator input. The operator provides a correct identification of the object having the low detection confidence. The human operator indicates a natural-language question 420 to identify the object identified as the temporary stop sign 404 (See at least Fig. 4E and [0125-0126]), Dolgov does not explicitly disclose
receiving a first input provided by an operator, the first input selecting an area of the representation of the environment;
receiving a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area; and
providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of objects.
However, Chen teaches receiving a first input provided by an operator, the first input selecting an area of the representation of the environment (Chen, in at least [0033], teaches the interactive object annotation system obtains the data regarding a plurality of objects in the surrounding environment of the vehicle and provides the data to a remote operator [i.e., an operator] by displaying via a graphical user interface. The interactive object annotation system provides for highlighting or otherwise visualizing the object(s) that the autonomous vehicle is stuck on or reacting to in a user interface that presents a rendering of the autonomous vehicle's environment on a display device. If the problem object that the autonomous vehicle is included in a group of objects, the interactive object annotation system identifies the group of objects within the autonomous vehicle environment and allow for selection of one or more objects within the group of objects [i.e., receiving a first input provided by an operator, the first input selecting an area of the representation of the environment] that should have a new/updated classification applied);
receiving a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area (Chen, in at least [0033], teaches an operator selects multiple objects and identify a new/updated classification by providing user input via the user interface [i.e., assigning a bulk annotation to a plurality of objects located in the area]. The interactive object annotation system determines the new/updated classification [i.e., wherein the bulk annotation associates a label with each object of the plurality of objects located in the area] for the problem object(s) based at least in part on such user input [i.e., receiving a second input provided by the operator]);
providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of object (Chen, in at least [0033], teaches the new/updated classification is applied simultaneously to all the objects in the selected group of objects and then provided to the autonomous vehicle [i.e., providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of object] to facilitate the autonomous vehicle implementing maneuvers with regard to the group of objects such that the autonomous vehicle can continue on its route).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov in view of Chen with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – autonomous vehicles – and provide a user interface that allows the user to select and annotate objects in the vicinity of the vehicle and the combination would provide for improving the ability of an autonomous vehicle to effectively provide vehicle services to others and support the various members of the community in which the autonomous vehicle is operating, including persons with reduced mobility and/or persons that are underserved by other transportation options (Chen, see at least [0007]).
In regard to claim 5
, Dolgov, as modified by Chen, teaches the method of claim 1, further comprising:
displaying a selectable option with the representation of the environment, wherein the selectable option enables bulk annotation of the plurality of objects (Dolgov, in at least Fig. 4E, [0125 & 0144-0148], discloses the control menu 418 allows the operator to input guidance to the vehicle in a number of different ways (e.g., selecting from a list of operations, typing in a particular mode of operation, selecting a particular region of focus within an image of the environment, etc.). The user-interface includes various selectable and non-selectable elements for presenting aspects of the at least one image [i.e., displaying a selectable option with the representation of the environment], such as windows, sub-windows, text boxes, and command buttons. The GUI enables the human operator to select an area of interest in the pre-stored data for further analysis. Such an area of interest includes important objects in the environment that the vehicle did not correctly identify or did not attempt to identify, or includes any object for which the human operator believes their feedback is desired [i.e., wherein the selectable option enables bulk annotation of the plurality of objects]. The computing system displays, to the human operator, an image of the pre-stored data that the vehicle may have annotated with the alleged identities of various relevant objects).
In regard to claim 6
, Dolgov, as modified by Chen, teaches the method of claim 1, wherein the representation of the environment shows a construction site positioned along the trajectory of the vehicle (Dolgov, in at least [0091], discloses the vehicle is configured to determine objects based on the context of the data. Street signs related to construction generally have an orange color. Accordingly, the vehicle is configured to detect objects that are orange, and located near the side of roadways as construction-related street signs [i.e., wherein the representation of the environment shows a construction site positioned along the trajectory of the vehicle]), and
wherein the bulk annotation is assigned to a plurality of construction elements positioned proximate the construction site (Dolgov, in at least [0109], discloses if the object at issue is an orange construction cone, the human operator enters via a keyboard, or speak via a microphone, a response including the words “construction cone” [i.e., wherein the bulk annotation is assigned to a plurality of construction elements positioned proximate the construction site]).
In regard to claim 8
, Dolgov, as modified by Chen, teaches the method of claim 1, wherein receiving sensor data and the request for assistance from the vehicle comprises:
receiving image data from one or more cameras coupled to the vehicle (Dolgov, in at least Fig. 1, and [0026 & 0044], discloses the remote assistance process acquires (e.g., via cameras, LIDAR, radar, and/or other sensors) [i.e., receiving image data from one or more cameras coupled to the vehicle] environment data including an object or objects in the vehicle's environment. Camera 130 includes one or more devices (e.g., still camera or video camera) configured to capture images of the environment of vehicle 100); and
wherein displaying the representation of the environment showing the trajectory of the vehicle comprises:
displaying the image data received from the one or more cameras (Dolgov, in at least Fig. 4D, and [0124], discloses Fig. 4D shows a GUI on a remote computing system that is presented to a human operator. The GUI 412 includes separate sub-windows 414 and 416. The first sub-window 414 includes the vehicle's sensor data representation of its environment. The second sub-window 416 includes a video stream of a portion of the environment [i.e., displaying the image data received from the one or more cameras]).
In regard to claim 9
, Dolgov, as modified by Chen, teaches the method of claim 1, wherein receiving sensor data and the request for assistance from the vehicle comprises:
receiving lidar data from one or more lidars coupled to the vehicle (Dolgov, in at least [0159], discloses the computing system receives the sensor data, and detect the tactile event based on the received data. The sensor data come from sensors such as an IMU, accelerometer, RADAR, LIDAR [i.e., receiving lidar data from one or more lidars coupled to the vehicle], impact sensor, etc.); and
generating the representation of the environment based on the lidar data (Dolgov, in at least Fig. 4E, and [0125], discloses a GUI that contains a first sub-window showing the vehicle's sensor data representation of its environment [i.e., generating the representation of the environment based on the lidar data] and a second sub-window showing a video stream of a portion of the vehicle's environment).
In regard to claim 10
, Dolgov, as modified by Chen, teaches the method of claim 1, wherein the computing device is positioned remotely from the vehicle (Dolgov, in at least Fig. 3A, 6A and [0136], discloses a computing system (e.g., remote computing system 302 or server computing system 306) [i.e., wherein the computing device is positioned remotely from the vehicle] operates in a rewind mode as shown by method 600. Examiner notes, as depicted by Fig. 3A, the server and remote computer systems are positioned remotely from the vehicle).
In regard to claim 11
, Dolgov, as modified by Chen, teaches the method of claim 1, wherein the request for assistance indicates that the vehicle is stopped (Dolgov, in at least Fig. 6A, and [0137-0138], discloses block 602 further includes periodically determining if the vehicle is stopped [i.e., wherein the request for assistance indicates that the vehicle is stopped] and/or determining if the vehicle has been stopped for a predetermined threshold period of time. After each minute the vehicle is stopped, the review criterion triggers remote assistance).
In regard to claim 12
, Dolgov discloses a system comprising (Dolgov, in at least [0031], discloses methods and systems for remote assistance):
a vehicle (Dolgov, in at least Fig. 2, and [0036], discloses vehicle 200 [i.e., a vehicle] includes sensor unit 202); and
a computing device positioned remote from the vehicle (Dolgov, in at least Fig. 3A, 6A and [0136], discloses a computing system (e.g., remote computing system 302 or server computing system 306) [i.e., wherein the computing device is positioned remotely from the vehicle] operates in a rewind mode as shown by method 600. Examiner notes, as depicted by Fig. 3A, the server and remote computer systems are positioned remotely from the vehicle),
receive sensor data and a request for assistance from the vehicle, wherein the sensor data depicts a trajectory of the vehicle in an environment (Dolgov, in at least Figs. 4A-4D, 6B, [0142 & 0152], discloses the vehicle requests remote assistance in substantially real-time [i.e., receive a request for assistance from the vehicle], which the computing system uses as a means for alerting the human operator. At block 622, the computing system operates by receiving image data from the autonomous vehicle of an environment of the autonomous vehicle [i.e., receive sensor data, wherein the sensor data depicts a trajectory of the vehicle in an environment]. Examiner notes, as illustrated by Fig. 4C-4D, the sensor data, such as the image of the environment, depicts the trajectory of the vehicle in an environment);
display, based on the sensor data, a representation of the environment showing the trajectory of the vehicle (Dolgov, in at least in at least Figs. 1, 4A-4D, 6B, and [0050 & 0155], discloses avigation/pathing system 142 determines a driving path for vehicle 100. At block 626, the computing system operates by providing at least one image to an operator [i.e., display, based on the sensor data, a representation of the environment showing the trajectory of the vehicle] from the memory, wherein at least one image comprises previously-stored image data related to the at least one object of an environment of the autonomous vehicle has a detection confidence below a threshold. Examiner notes, as illustrated by Fig. 4C-4D, the representation of the environment shows the trajectory of the vehicle);
While Dolgov discloses at block 628, the computing system operates by receiving an operator input. The operator provides a correct identification of the object having the low detection confidence. The human operator indicates a natural-language question 420 to identify the object identified as the temporary stop sign 404 (See at least Fig. 4E and [0125-0126]), Dolgov does not explicitly disclose receive a first input provided by an operator, the first input selecting an area of the representation of the environment;
receive a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area; and
provide, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of objects.
However, Chen teaches receive a first input provided by an operator, the first input selecting an area of the representation of the environment (Chen, in at least [0033], teaches the interactive object annotation system obtains the data regarding a plurality of objects in the surrounding environment of the vehicle and provides the data to a remote operator [i.e., an operator] by displaying via a graphical user interface. The interactive object annotation system provides for highlighting or otherwise visualizing the object(s) that the autonomous vehicle is stuck on or reacting to in a user interface that presents a rendering of the autonomous vehicle's environment on a display device. If the problem object that the autonomous vehicle is included in a group of objects, the interactive object annotation system identifies the group of objects within the autonomous vehicle environment and allow for selection of one or more objects within the group of objects [i.e., receive a first input provided by an operator, the first input selecting an area of the representation of the environment] that should have a new/updated classification applied);
receive a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area (Chen, in at least [0033], teaches an operator selects multiple objects and identify a new/updated classification by providing user input via the user interface [i.e., assigning a bulk annotation to a plurality of objects located in the area]. The interactive object annotation system determines the new/updated classification [i.e., wherein the bulk annotation associates a label with each object of the plurality of objects located in the area] for the problem object(s) based at least in part on such user input [i.e., receive a second input provided by the operator]);
provide, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of objects (Chen, in at least [0033], teaches the new/updated classification is applied simultaneously to all the objects in the selected group of objects and then provided to the autonomous vehicle [i.e., provide, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of object] to facilitate the autonomous vehicle implementing maneuvers with regard to the group of objects such that the autonomous vehicle can continue on its route).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov in view of Chen with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – autonomous vehicles – and provide a user interface that allows the user to select and annotate objects in the vicinity of the vehicle and the combination would provide for improving the ability of an autonomous vehicle to effectively provide vehicle services to others and support the various members of the community in which the autonomous vehicle is operating, including persons with reduced mobility and/or persons that are underserved by other transportation options (Chen, see at least [0007]).
In regard to claim 16
, Dolgov, as modified by Chen, teaches the system of claim 12.
Claim 16 recites a system having substantially the same features of claim 5 above, therefore claim 16 is rejected for the same reasons as claim 5.
In regard to claim 17
, Dolgov, as modified by Chen, teaches the system of claim 12.
Claim 17 recites a system having substantially the same features of claim 6 above, therefore claim 17 is rejected for the same reasons as claim 6.
In regard to claim 18
, Dolgov, as modified by Chen, teaches the system of claim 12, wherein the sensor data is provided by a camera or a lidar coupled to the vehicle (Dolgov, in at least Fig. 1, and [0026 & 0044], discloses the remote assistance process acquires (e.g., via cameras, LIDAR, radar, and/or other sensors) [i.e., wherein the sensor data is provided by a camera or a lidar coupled to the vehicle] environment data including an object or objects in the vehicle's environment. Camera 130 includes one or more devices (e.g., still camera or video camera) configured to capture images of the environment of vehicle 100).
In regard to claim 19
, Dolgov, as modified by Chen, teaches the system of claim 12, wherein the computing device is further configured to:
based on the first input, generate a bounding region around the area (Dolgov, in at least Fig. 4E (reproduced and annotated below for Applicant’s convenience), and [0126], discloses the human operator indicates a natural-language question 420 to identify the object identified as the temporary stop sign 404. Examiner notes, as illustrated by Fig. 4E of Dolgov, a bounding region is generated); and
PNG
media_image1.png
574
1121
media_image1.png
Greyscale
Annotated Fig. 4E of Dolgov - Generating a bounding region
maintain the bounding region around the area after updating the representation of the environment based on additional sensor data (Dolgov, in at least [0126], discloses when an identification is confirmed, the identification is added to a global map [i.e., maintain the bounding region around the area after updating the representation of the environment based on additional sensor data]. When the identification is added to the global map, other vehicles may not have to request an identification of the object in the future).
In regard to claim 20
, Dolgov discloses a non-transitory computer-readable medium storing instructions, the instructions being executable by one or more processors to perform operations comprising (Dolgov, in at least [0164], discloses methods are implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format, or on other non-transitory media or articles of manufacture):
receiving sensor data and a request for assistance from a vehicle, wherein the sensor data depicts a trajectory of the vehicle in an environment (Dolgov, in at least Figs. 4A-4D, 6B, [0142 & 0152], discloses the vehicle requests remote assistance in substantially real-time [i.e., receiving a request for assistance from a vehicle], which the computing system uses as a means for alerting the human operator. At block 622, the computing system operates by receiving image data from the autonomous vehicle of an environment of the autonomous vehicle [i.e., receiving sensor data, wherein the sensor data depicts a trajectory of the vehicle in an environment]. Examiner notes, as illustrated by Fig. 4C-4D, the sensor data, such as the image of the environment, depicts the trajectory of the vehicle in an environment);
displaying, based on the sensor data, a representation of the environment showing the trajectory of the vehicle (Dolgov, in at least in at least Figs. 1, 4A-4D, 6B, and [0050 & 0155], discloses avigation/pathing system 142 determines a driving path for vehicle 100. At block 626, the computing system operates by providing at least one image to an operator [i.e., displaying, based on the sensor data, a representation of the environment showing the trajectory of the vehicle] from the memory, wherein at least one image comprises previously-stored image data related to the at least one object of an environment of the autonomous vehicle has a detection confidence below a threshold. Examiner notes, as illustrated by Fig. 4C-4D, the representation of the environment shows the trajectory of the vehicle);
While Dolgov discloses at block 628, the computing system operates by receiving an operator input. The operator provides a correct identification of the object having the low detection confidence. The human operator indicates a natural-language question 420 to identify the object identified as the temporary stop sign 404 (See at least Fig. 4E and [0125-0126]), Dolgov does not explicitly disclose receiving a first input provided by an operator, the first input selecting an area of the representation of the environment;
receiving a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area; and
providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of objects.
However, Chen teaches receiving a first input provided by an operator, the first input selecting an area of the representation of the environment (Chen, in at least [0033], teaches the interactive object annotation system obtains the data regarding a plurality of objects in the surrounding environment of the vehicle and provides the data to a remote operator [i.e., an operator] by displaying via a graphical user interface. The interactive object annotation system provides for highlighting or otherwise visualizing the object(s) that the autonomous vehicle is stuck on or reacting to in a user interface that presents a rendering of the autonomous vehicle's environment on a display device. If the problem object that the autonomous vehicle is included in a group of objects, the interactive object annotation system identifies the group of objects within the autonomous vehicle environment and allow for selection of one or more objects within the group of objects [i.e., receiving a first input provided by an operator, the first input selecting an area of the representation of the environment] that should have a new/updated classification applied);
receiving a second input provided by the operator, the second input assigning a bulk annotation to a plurality of objects located in the area, wherein the bulk annotation associates a label with each object of the plurality of objects located in the area (Chen, in at least [0033], teaches an operator selects multiple objects and identify a new/updated classification by providing user input via the user interface [i.e., assigning a bulk annotation to a plurality of objects located in the area]. The interactive object annotation system determines the new/updated classification [i.e., wherein the bulk annotation associates a label with each object of the plurality of objects located in the area] for the problem object(s) based at least in part on such user input [i.e., receiving a second input provided by the operator]);
providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of object (Chen, in at least [0033], teaches the new/updated classification is applied simultaneously to all the objects in the selected group of objects and then provided to the autonomous vehicle [i.e., providing, based on the first input and the second input, a response to the vehicle, the response conveying the bulk annotation assigned to the plurality of object] to facilitate the autonomous vehicle implementing maneuvers with regard to the group of objects such that the autonomous vehicle can continue on its route).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov in view of Chen with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – autonomous vehicles – and provide a user interface that allows the user to select and annotate objects in the vicinity of the vehicle and the combination would provide for improving the ability of an autonomous vehicle to effectively provide vehicle services to others and support the various members of the community in which the autonomous vehicle is operating, including persons with reduced mobility and/or persons that are underserved by other transportation options (Chen, see at least [0007]).
In regard to claim 22
, Dolgov, as modified by Chen, teaches the method of claim 1.
Claim 22 recites a method having substantially the same features of claim 19 above, therefore claim 22 is rejected for the same reasons as claim 19.
12. Claim(s) 2, 7 and 13
is/are rejected under 35 U.S.C. 103 as being unpatentable over Dolgov et al. (US-20190019349-A1) in view of Chen et al. (US-20200401135-A1) and further in view of Korjus et al. (US-20210209367-A1).
In regard to claim 2
, Dolgov, as modified by Chen, teaches the method of claim 1, accordingly the rejection of claim 1 is incorporated.
Dolgov, as modified by Chen, is silent on all limitations of the claim.
However, Korjus teaches wherein each object of the plurality of objects is a first type of object, and wherein the bulk annotation assigns a classification to each object of the plurality of objects based on the first type of object (Korjus, in at least [0030 & 0066], teaches the method comprises identifying a type of occlusion present in the preprocessed data [i.e., wherein each object of the plurality of objects is a first type of object]. That is, upon positively identifying occlusion, a further step comprises detecting what the occlusion corresponds to. The method further comprises classifying the detected occlusion [i.e., wherein the bulk annotation assigns a classification to each object of the plurality of objects based on the first type of object]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov, as already modified by Chen, in view of Korjus with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – mobile robots – and identify the object type and classify the detected objects and the combination would provide for increased safety of operations (Korjus, see at least [0001]).
In regard to claim 7
, Dolgov, as modified by Chen, teaches the method of claim 1, accordingly the rejection of claim 1 is incorporated.
Dolgov, as modified by Chen, is silent on all limitations of the claim.
However, Korjus teaches wherein the bulk annotation is assigned to a plurality of vegetation positioned proximate a road associated with the trajectory of the vehicle (Korjus, in at least [0121], teaches the neural network first is trained on images where a certain type of occlusion (such as occlusion of a traffic road via parked cars, vegetation etc.) is annotated, and then applied to the images detected by the mobile robot [i.e., wherein the bulk annotation is assigned to a plurality of vegetation positioned proximate a road associated with the trajectory of the vehicle]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov, as modified by Chen, in view of Korjus with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – mobile robots – and use the method of Korjus to train and annotate the vegetation along the trajectory of the vehicle and the combination would provide for increased safety of operations (Korjus, see at least [0001]).
In regard to claim 13
, Dolgov, as modified by Chen, teaches the system of claim 12.
Claim 13 recites a system having substantially the same features of claim 2 above, therefore claim 13 is rejected for the same reasons as claim 2.
13. Claim(s) 3, and 14
is/are rejected under 35 U.S.C. 103 as being unpatentable over Dolgov et al. (US-20190019349-A1) in view of Chen et al. (US-20200401135-A1) and further in view of Korjus et al. (US-20210209367-A1) and further in view of Kario et al. (US-20240163402-A1).
In regard to claim 3
, Dolgov, as modified by Chen and Korjus, teaches the method of claim 2, accordingly the rejection of claim 2 is incorporated.
Dolgov, as modified by Chen and Korjus, is silent on all limitations of the claim.
However, Kario teaches wherein the response further indicates that the bulk annotation assigned to the plurality of objects is applicable to additional objects that match the first type of object for a predetermined amount of time (Kario, in at least [0059-0061], teaches performing surveillance on the target and/or the area over a predetermined period of time wherein identifying the target and one or more properties of the target based on data gathered at a first point in time in the predetermined period of time. The set of steps comprising identifying the target at a second point in time in the predetermined period of time based on the one or more properties [i.e., wherein the response further indicates that the bulk annotation assigned to the plurality of objects is applicable to additional objects that match the first type of object for a predetermined amount of time]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov, as modified by Chen and Korjus, in view of Kario with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – vehicle systems – and identify or match an object in a predetermined period of time and the combination would provide for detecting objects using a fixed angle and/or known angles (Kario, see at least [0003]).
In regard to claim 14
, Dolgov, as modified by Chen and Korjus, teaches the system of claim 13.
Claim 14 recites a system having substantially the same features of claim 3 above, therefore claim 14 is rejected for the same reasons as claim 3.
14. Claim(s) 21
is/are rejected under 35 U.S.C. 103 as being unpatentable over Dolgov et al. (US-20190019349-A1) in view of Chen et al. (US-20200401135-A1) and further in view of Sachdeva et al. (US-20190197778-A1).
In regard to claim 21
, Dolgov, as modified by Chen, teaches the method of claim 1, accordingly the rejection of claim 1 is incorporated.
Dolgov, as modified by Chen, is silent on all limitations of the claim.
However, Sachdeva teaches further comprising:
refining one or more boundaries of the area for a subsequent image frame of the representation of the environment to annotate one or more objects in the subsequent image frame with the label (Sachdeva, in at least [0193], teaches a user pauses the video at any frame and in any perspective view [i.e., a subsequent image frame of the representation of the environment] to define and/or refine a boundary of a composite object [i.e., refining one or more boundaries of the area]. Accordingly, the boundary of a composite object [i.e., one or more objects] is defined and/or refined across multiple perspective views and/or multiple frames, and respective data indicative thereof is stored in conjunction with the label for the composite object as training data [i.e., the label]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Dolgov, as already modified by Chen, in view of Sachdeva with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – vehicle systems – and refine the boundary of the composite objects for labeling and the combination would provide for increasing the efficiency and accuracy of the labeling itself, thereby increasing both the amount and quality of labeled data used to train the perception component, and ultimately increasing the safety of autonomous operation of a vehicle whose operations and maneuvers are controlled by the trained perception component (Sachdeva, see at least [0007]).
Conclusion
15. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
David et al. (US-20220388543-A1) teaches a vehicle that renders a trust zone on the display to indicate safe trajectories around an object.
Gonzalez et al. (US-20190228262-A1) teaches a vehicle control system that detects a trigger to initiate an annotation prompt associated with an object classified from an image.
16. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
17. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
18. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Preston J Miller whose telephone number is (703)756-1582. The examiner can normally be reached Monday through Friday 7:30 AM - 4:30 PM EST.
19. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
20. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramya P Burgess can be reached at (571) 272-6011. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
21. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.J.M./Examiner, Art Unit 3661
/Tarek Elarabi/Primary Examiner, Art Unit 3661
Prosecution Insights