DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on February 19, 2026 has been entered.
Status of Claims
Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments, see p. 9-15, filed February 2, 2026, with respect to the 35 USC 103 rejection of the claims have been fully considered but are moot because of the new grounds of rejection presented in the 35 USC 103 rejections below. Applicant argues that the previously proposed references do not teach a distinct second processing stage comprising a sub-model that receives only unknown-classified tracked objects from a main understanding model. However, the newly proposed St. Romain II reference teaches a secondary model for determining annotations of the unknown objects. The secondary model executes the model over only the unknown objects to automatically generate annotations (i.e., classifications) (see St. Romain II, Para. [0050]).
Applicant further argues that the references do not teach the sub-model including a shared backbone and a plurality of heads. However, the combination of the Capellier, St. Romain II, Funke and Mao references does teach these limitations. The sub-model of St. Romain II combined with Mao’s teachings of a backbone neural network and an object detection head and trajectory prediction head downstream of the backbone (see Mao, Para. [0058]), along with Funke’s teachings of a perception component to determine an entity classification and a prediction component for generating predicted positions, velocities, trajectories, etc. is sufficient in teaching “the second processing stage comprising a sub-model including: a shared backbone configured to receive and process only sensor data generated from the sensors corresponding to tracked objects provided by the main understanding model as having the unknown object classification” and “a plurality of heads downstream of the shared backbone configured to output respective inferences including one or more road object classifications and one or more road object attributes for only the tracked objects classified as unknown by the main understanding model”. Therefore, the 35 USC 103 rejection of the claims is upheld.
Claim Objections
Claim 1 is objected to because of the following informalities: “the main understanding model is configured to provide only track objects…” should read “the main understanding model is configured to provide only tracked objects…” and “a second processing stage distinct from the fist processing stage…” should read “a second processing stage distinct from the first processing stage…”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4-7, 9, 16-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1).
Regarding claim 1, Capellier teaches a vehicle (Capellier, Col. 3, line 37, a vehicle (such as an autonomous vehicle) comprising:
sensors (Capellier, Col. 7, line 34-36, autonomous system includes a sensor suite that includes one or more devices such as cameras, LiDAR sensors, radar sensors, and microphones);
one or more processors (Capellier, Col. 13, lines 4-14, device performs these processes base4d on a processor executing software instructions stored by a computer-readable medium); and
one or more storage media encoding instructions executable by the one or more processors to implement an understanding part (Capellier, Col. 13, lines 4-14, device performs these processes base4d on a processor executing software instructions stored by a computer-readable medium), wherein the understanding part includes:
a first processing stage comprising a main understanding model (Capellier, Col. 16, lines 7-24, implement at least one machine learning model) configured to classify each tracked object into one of: a plurality of road user classifications and an unknown object classification (Capellier, Col. 21, lines 5-38, the vehicle includes one or more sensors that are configured to detect at least a portion of the objects. The objects include, for example, at least one vehicle, at least one pedestrian, at least one cyclist, at least one structure, and/or the like. Objects can include known objects (previously identified and classified objections) (i.e., road user classifications) and unknown objects that do not match with any previously identified objects).
Although Capellier teaches identifying a new class based on the embeddings (Capellier, Col. 25, line 27-52), Capellier does not explicitly teach “wherein the main understanding model is configured to provide only track objects classified as unknown to a second processing stage distinct from the first processing stage” and “the second processing stage comprising a sub-model only sensor data generated from the sensors corresponding to tracked objects provided by the main understanding model as having the unknown object classification”. However, in an analogous field of endeavor, St. Romain II teaches previously acquired samples having unknown classes are further processed to produce annotations identifying classes for the previously unknown objects. The training module implements a secondary model that is trained according to a separate ontology than that of the ontological detector. In general, the separate ontology may be inclusive of the ontology of the detector, distinct, or share a portion of the same classes. Additionally, the secondary model may be specific to a certain single class (e.g., birds), and, thus, may provide highly accurate determinations in relation to a single class. In such an instance, the training module may execute a plurality of different models that are each specific to a single class when annotating the unknown objects. In any case, the training module may implement one or more secondary models and execute the model(s) over the unknown objects to automatically generate the annotation(s) (St. Romain II, Para. [0050]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Capellier with the teachings of St. Romain II by including a second processing stage (i.e., secondary model) to process only sensor data corresponding to unknown objects. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for accurately and efficiently processing information of objects uncommon to the functioning of the autonomous vehicle, as recognized by St. Romain II.
Although Capellier in view of St. Romain II teaches identifying a new class based on the embeddings (Capellier, Col. 25, line 27-52), they do not explicitly teach “configured to output respective inferences including one or more road object classifications and one or more road object attributes for only the tracked objects classified as unknown by the main understanding model”. However, in an analogous field of endeavor, Funke teaches the perception component can include functionality to perform object detection, segmentation, and/or classification. The perception component can provide processed sensor data that indicates a presence of an entity that is proximate to the vehicle and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, etc.). The prediction component can include functionality to generate predicted information associated with objects in an environment. The prediction component can generate one or more predicted positions, predicted velocities, predicted trajectories, etc. for such target objects based on attributes of the target object (Funke, Col. 21, lines 12-51).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II with the teachings of Funke by including inferences including the classification of the object and one or more attributes of the object. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for autonomous vehicles to safely and comfortably traverse through the environment, as recognized by Funke.
Although Capellier in view of St. Romain II further in view of Funke teaches at least one machine learning model (Capellier, Col. 16, lines 7-24), they do not explicitly teach the sub-model includes “a shared backbone”, “a plurality of heads downstream of the shared backbone” and “the shared backbone being common to and feeding a plurality of heads”. However, in an analogous field of endeavor, Mao teaches the system processes the initial feature representations using a backbone neural network to generate the feature representations (Mao, Para. [0062]) and teaches a backbone neural network 320, an object detection head 350 and a trajectory prediction head 360 (see Fig. 3, a plurality of heads downstream of the backbone, the backbone being common to and feeding the heads) (Mao, Para. [0058]; Fig. 3).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke with the teachings of Mao by including that the sub-model includes a shared backbone common to and feeding a plurality of heads downstream of the backbone. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for autonomous vehicle to perform various prediction tasks, as recognized by Mao. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 4, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, wherein the sub-model further includes:
a shared temporal network coupled to receive an output from the shared backbone and to generate an output to the plurality of heads (Mao, Para [0058]; Fig. 3, the system then processes the sequence 302 using a spatio-temporal-interactive neural network that includes, in the example of FIG. 3, an encoder neural network 310, a backbone neural network 320, a temporal region proposal neural network 330, a spatio-temporal-interactive (STI) feature extractor 340, an object detection head 350, and a trajectory prediction head 360.).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke and Mao references presented in the rejection of Claim 1, apply to Claim 4 and are incorporated herein by reference. Thus, the vehicle recited in Claim 4 is met by Capellier in view of St. Romain II further in view of Funke and Mao.
Regarding claim 5, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, wherein the plurality of heads include:
a road object classification head to output a road object subtype inference (Capellier, Col. 25, line 27-52, identifying a new class based on the embeddings. Col. 24, lines 25-35, A dataset is updated with references to any identified new classes of objects (e.g., previously unknown classes such as 3-wheeled vehicles, trolleys, etc. (i.e., road object subtype inference)).
Regarding claim 6, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 5, and further teaches wherein the road object subtype inference selects between two or more of the following:
debris classification; animal classification; construction object classification; sign classification; and vulnerable road user classification (Funke, Col. 3, lines 2-9, the vehicle may analyze the sensor data to detect and classify various objects in the environment. Objects encountered by an autonomous vehicle may include other dynamic objects that are capable of movement (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and/or static objects (e.g., buildings, road surfaces, trees, signs, barriers, parked or disabled vehicles, debris, etc.)).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke and Mao references presented in the rejection of Claim 1, apply to Claim 6 and are incorporated herein by reference. Thus, the vehicle recited in Claim 6 is met by Capellier in view of St. Romain II further in view of Funke and Mao.
Regarding claim 7, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, wherein the plurality of heads include:
an animal classification head to output an animal subtype inference (Funke, Col. 3, lines 2-9, the vehicle may analyze the sensor data to detect and classify various objects in the environment. Objects encountered by an autonomous vehicle may include other dynamic objects that are capable of movement (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and/or static objects (e.g., buildings, road surfaces, trees, signs, barriers, parked or disabled vehicles, debris, etc.)).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke and Mao references presented in the rejection of Claim 1, apply to Claim 7 and are incorporated herein by reference. Thus, the vehicle recited in Claim 7 is met by Capellier in view of St. Romain II further in view of Funke and Mao.
Regarding claim 9, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, wherein the plurality of heads include:
a first debris attribute head to output a drivability probability (Funke, Col. 3, lines 2-18, the vehicle may classify the object or potential object as an undrivable region that should be avoided by the vehicle while navigating the environment).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke and Mao references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Thus, the vehicle recited in Claim 9 is met by Capellier in view of St. Romain II further in view of Funke and Mao.
Regarding claim 16, Capellier teaches a computer-implemented method for understanding road users and road objects and controlling a vehicle based on the understanding, the method comprising:
determining, by a tracker, tracked objects in an environment of the vehicle (Capellier, Col. 14, lines 29-44, perception system receives data associated with at least one physical object (e.g., data that is used by perception system to detect the at least one physical object) in an environment and classifies the at least one physical object);
classifying, in a first processing stage comprising a main understanding model, each tracked object into one of: a plurality of road users classifications and an unknown object classification (Capellier, Col. 21, lines 5-38, the vehicle includes one or more sensors that are configured to detect at least a portion of the objects. The objects include, for example, at least one vehicle, at least one pedestrian, at least one cyclist, at least one structure, and/or the like. Objects can include known objects (previously identified and classified objections) (i.e., road user classifications) and unknown objects that do not match with any previously identified objects);
planning a trajectory of the vehicle based on tracked objects information from the tracker and predictions from the prediction part (Capellier, Col. 14, lines 45-67, the planning system receives data associated with a destination and generates data associated with at least one route along which a vehicle can travel along toward a destination. In some embodiments, planning system periodicallyor continuously receives data from perception system (e.g., data associated with the classification of physical objects, described above) and planning system updates the at least one trajectory or generates at least one different trajectory based on the data generated by perception system).
Although Capellier teaches identifying a new class based on the embeddings (Capellier, Col. 25, line 27-52), Capellier does not explicitly teach “providing, by the main understanding model, only tracked objects classified as unknown to a second processing stage distinct form the first processing stage, the second processing stage comprising a sub-model” and “processing, in the sub-model, only sensor data corresponding to the tracked objects classified as unknown”. However, in an analogous field of endeavor, St. Romain II teaches previously acquired samples having unknown classes are further processed to produce annotations identifying classes for the previously unknown objects. The training module implements a secondary model that is trained according to a separate ontology than that of the ontological detector. In general, the separate ontology may be inclusive of the ontology of the detector, distinct, or share a portion of the same classes. Additionally, the secondary model may be specific to a certain single class (e.g., birds), and, thus, may provide highly accurate determinations in relation to a single class. In such an instance, the training module may execute a plurality of different models that are each specific to a single class when annotating the unknown objects. In any case, the training module may implement one or more secondary models and execute the model(s) over the unknown objects to automatically generate the annotation(s) (St. Romain II, Para. [0050]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Capellier with the teachings of St. Romain II by including a second processing stage (i.e., secondary model) to process only sensor data corresponding to unknown objects. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for accurately and efficiently processing information of objects uncommon to the functioning of the autonomous vehicle, as recognized by St. Romain II.
Although Capellier in view of St. Romain II teaches identifying a new class based on the embeddings (Capellier, Col. 25, line 27-52), they do not explicitly teach “generating, by the plurality of heads of the sub-model downstream of the shared backbone, respective inferences including one or more road object classifications and one or more road object attributes for only the tracked objects classified as unknown by the main understanding model” and “providing the inferences to a tracker that collects the inferences of the tracked objects and a prediction part that predicts behaviors of the tracked objects”. However, in an analogous field of endeavor, Funke teaches the perception component can include functionality to perform object detection, segmentation, and/or classification. The perception component can provide processed sensor data that indicates a presence of an entity that is proximate to the vehicle and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, etc.). The prediction component can include functionality to generate predicted information associated with objects in an environment. The prediction component can generate one or more predicted positions, predicted velocities, predicted trajectories, etc. for such target objects based on attributes of the target object (Funke, Col. 21, lines 12-51).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II with the teachings of Funke by including inferences including the classification of the object and one or more attributes of the object and predicted behaviors of the tracked objects. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for autonomous vehicles to safely and comfortably traverse through the environment, as recognized by Funke.
Although Capellier in view of St. Romain II further in view of Funke teaches at least one machine learning model (Capellier, Col. 16, lines 7-24), they do not explicitly teach “wherein the sub-model includes a shared backbone and a plurality of heads” and ”the plurality of heads of the sub-model downstream of the shared backbone”. However, in an analogous field of endeavor, Mao teaches the system processes the initial feature representations using a backbone neural network to generate the feature representations (Mao, Para. [0062]) and teaches an object detection head and a trajectory prediction head (Mao, Para. [0058]). Object detection head and trajectory prediction head are downstream of the backbone (Mao, Fig. 3).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke with the teachings of Mao by including a shared backbone and a plurality of heads for performing the process of Capellier, St. Romain II and Funke. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for autonomous vehicle to perform various prediction tasks, as recognized by Mao. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 17, Capellier in view of St. Romain II further in view of Funke and Mao teaches the computer-implemented method of claim 16, and further teaches wherein determining the tracked objects comprises determining bounding boxes of the tracked objects in the environment of the vehicle based on sensor data (Mao, Para. [0097], the output of the object detection head can be the regressed coordinates of a region, e.g., a bounding box, in the feature representations that represents the predicted, location of the possible agent at the end of the time interval).
The proposed combination as well as the motivation for combining the Capellier in view of St. Romain II further in view of Funke and Mao references presented in the rejection of Claim 16, apply to Claim 17 and are incorporated herein by reference. Thus, the method recited in Claim 17 is met by Capellier in view of St. Romain II further in view of Funke and Mao.
Regarding claim 18, Capellier in view of St. Romain II further in view of Funke and Mao teaches the computer-implemented method of claim 16, wherein the main understanding model produces a road user inference that selects between road user classifications and an unknown object classification (Capellier, Col. 22, lines 16-36, the processors can be configured to differentiate between known and unknown objects and to generate embeddings (feature vectors) for the detected unknown objects. The processors can generate a classification of the objects including new classes of objects that correspond to unknown objects with similar embeddings).
Claim 20 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 16. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Capellier, St. Romain II, Funke, and Mao references, presented in rejection of Claim 16, apply to this claim. Finally, the combination of the Capellier, St. Romain II, Funke, and Mao references discloses a computer readable storage medium (Capellier, Col. 13, lines 4-14, a computer-readable medium (e.g., a non-transitory computer readable medium)).
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Qi et al. (US 2022/0388522 A1).
Regarding claim 2, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches processing the feature representations output by the backbone network using a temporal region proposal neural network to generate a plurality of temporal region proposals (Mao, Paras. [0065]-[0068]), they do not explicitly teach “a plurality of temporal networks to process an output from the shared backbone and to generate outputs to respective heads”. However, in an analogous field of endeavor, Qi teaches a temporal feature learning network comprising a plurality of long short-term memory (LSTM) layers (Qi, Para. [0044]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Qi by including a plurality of long short-term memory (LSTM) layers as a plurality of temporal networks. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for outputting an optimal driving policy for a vehicle, as recognized by Qi. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 3, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, wherein the sub-model further includes:
a first temporal network to process an output from the shared backbone and to generate an output to a first head of the plurality of heads (Mao, Para [0058]; Fig. 3, the system then processes the sequence 302 using a spatio-temporal-interactive neural network that includes, in the example of FIG. 3, an encoder neural network 310, a backbone neural network 320, a temporal region proposal neural network 330, a spatio-temporal-interactive (STI) feature extractor 340, an object detection head 350, and a trajectory prediction head 360.).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke, and Mao references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches processing the feature representations output by the backbone network using a temporal region proposal neural network to generate a plurality of temporal region proposals (Mao, Paras. [0065]-[0068]), they do not explicitly teach “a second temporal network to process the output from the shared backbone and to generate an output to a plurality of second heads of the plurality of heads”. However, in an analogous field of endeavor, Qi teaches a temporal feature learning network comprising a plurality of long short-term memory (LSTM) layers (Qi, Para. [0044]).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II Funke, Mao, and Qi references presented in the rejection of Claim 2, apply to Claim 3 and are incorporated herein by reference. Thus, the vehicle recited in Claim 3 is met by Capellier in view of St. Romain II further in view of Funke, Mao and Qi.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Akata et al. (US 2014/0376804 A1).
Regarding claim 8, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 7, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches an animal classification (Funke, Col. 3, lines 2-9), they do not explicitly teach “wherein the animal subtype inference selects between an animal can fly classification and an animal cannot fly classification”. However, in an analogous field of endeavor, Akata teaches an intermediate representation between image descriptors and classes. Attribute class descriptions are an example of such an intermediate representation. They correspond to high level image descriptors that are meaningful for, and shared across, multiple classes. By way of illustrative example, attributes for classifying images of animals could be "has paws", "has wings" (i.e., can fly), "has four legs", "has snout", "is underwater", and so forth (Akata, Para. [0003]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Akata by including a subtype inference of an animal selects if the animal “has wings” (i.e., can fly or cannot fly). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for classifying unknown object types, as recognized by Akata. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Agrawal et al. (US 2024/0278797 A1, filed February 21, 2023).
Regarding claim 10, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches classifying an object as debris (Funke, Col. 3, lines 2-9), they do not explicitly teach “a second debris attribute head to output a rigidity probability”. However, in an analogous field of endeavor, Agrawal teaches a supervised machine learning model may be trained to predict the hardness of an object based on the identification of the object and the obtained driving conditions (Agrawal, Para. [0040]).
Therefore, it would have been obvious to one having ordinary skill in art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Agrawal by including a head to determine the hardness of the object. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for predicting incidents involving a vehicle, as recognized by Agrawal. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Atchley et al. (US 2016/0260161 A1).
Regarding claim 11, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches classifying an object as debris (Funke, Col. 3, lines 2-9), they do not explicitly teach “a third debris attribute head to output an emptiness inference that selects between an empty object classification and a full object classification”. However, in an analogous field of endeavor, Atchley teaches an ultrasonic sensor that can be used, for example, to determine whether there is something inside an item or whether the item is empty (e.g., whether a water bottle has fluid in it) (Atchley, Para. [0221]). The sensor data may be used to determine a state of the item such as one or more of: new, used, full, empty, dirty, damaged, broken, etc. (Atchley, Para. [0269]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Atchley by including and emptiness inference for an object. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for determining a state of an object, as recognized by Atchley. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Kido et al. (US 2023/0004169 A1).
Regarding claim 12, Capellier in view of Funke further in view of Mao and Heck teaches the vehicle of claim 1, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches classifying an object as debris (Funke, Col. 3, lines 2-9), they do not explicitly teach “a fourth debris attribute head to output a material inference”. However, in an analogous field of endeavor, Kido teaches an object attribute estimation unit that estimates the material of the object detected from an image captured by the image-capturing device. The material of the object to be estimated is, for example, metal, stone, tree, or plastic (Kido, Para. [0057]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Kido by including determining a material type inference for an object. One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for estimating attributes of objects, as recognized by Kido. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 13, Capellier in view of St. Romain II further in view of Funke, Mao and Kido teaches the vehicle of claim 12, and further teaches wherein the material inference selects between two or more of the following: cardboard classification; fabric classification; foliage classification; metal classification; paper classification; plastic classification; stone classification; wood classification; and unknown material classification (Kido, Para. [0057], an object attribute estimation unit that estimates the material of the object detected from an image captured by the image-capturing device. The material of the object to be estimated is, for example, metal, stone, tree, or plastic).
The proposed combination as well as the motivation for combining the Capellier, St. Romain II, Funke, Mao, and Kido references presented in the rejection of Claim 12, apply to Claim 13 and are incorporated herein by reference. Thus, the vehicle recited in Claim 13 is met by Capellier in view of St. Romain II further in view of Funke, Mao and Kido.
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Shimon Pertsel (US 11,655,893 B1).
Regarding claim 14, Capellier in view of St. Romain II further in view of Funke and Mao teaches the vehicle of claim 1, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches determining an object is a sign (Funke, Col. 3, lines 2-9), they do not explicitly teach “a traffic sign head to output a traffic sign inference”. However, in an analogous field of endeavor, Pertsel teaches detecting various objects that the vehicle is approaching that may impact the speed and/or drivetrain configuration of the vehicle. In an example, the computer vision may detect speed signs, stop signs, traffic lights, slow traffic ahead, turns that require slowing, etc. (Pertsel, Col. 3, lines 7-17).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the vehicle of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Pertsel by including outputting a traffic sign inference. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for determining driving conditions that may be used to predict a future drivetrain configuration for a vehicle, as recognized by Pertsel. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 15, Capellier in view of St. Romain II further in view of Funke, Mao and Pertsel teaches the vehicle of claim 14, and further teaches wherein the traffic sign inference selects between two or more of the following:
a road closed sign classification; a stop sign classification; a keep left sign classification; a keep right sign classification; a double arrow sign classification; and unknown sign classification (Pertsel, Col. 45, lines 47-67, the CNN module 150 may detect objects such as the vehicles 410a-410b and/or the stop signs (e.g., infrastructure. The CNN module 150 may recognize other objects such as the lane markers, the curb, the sidewalk, the stop line, pedestrians, painted road signs (e.g., turning lane indicators), pedestrian cross-walks, traffic lights, etc. Col. 48, lines 44-59, other types of driving conditions detected that slow down the ego vehicle may be right-angle turns, yield signs, emergency vehicles on the side of the road, inclined roads, declined roads, etc.).
The proposed combination as well as the motivation for combining the Capellier, Funke, St. Romain II, Mao, and Pertsel references presented in the rejection of Claim 14, apply to Claim 15 and are incorporated herein by reference. Thus, the vehicle recited in Claim 15 is met by Capellier in view of St. Romain II further in view of Funke, Mao and Pertsel.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Edouard Capellier (US 12,183,061 B2) in view of Randall J. St. Romain II (US 2021/0390351 A1) further in view of Funke et al. (US 12,208, 819 B1) and Mao et al. (US 2021/0150199 A1), as applied to claims 1, 4-7, 9, 16-18 and 20 above, and further in view of Iventosch et al. (US 2019/0385009 A1).
Regarding claim 19, Capellier in view of St. Romain II further in view of Funke and Mao teaches the computer-implemented method of claim 16, as described above.
Although Capellier in view of St. Romain II further in view of Funke and Mao teaches a bounding box surrounding the object of interest (Mao, Para. [0097]), they do not explicitly teach “wherein the sensor data corresponding to the tracked object having the unknown object classification comprises an image cropped based on a projection of a bounding box corresponding to the tracked object onto a camera image”. However, in an analogous field of endeavor, Iventosch teaches applying a bounding box to an object, capturing an image of the object with an image capturing device, and cropping the image to the bounding box, thereby yielding a cropped image of the object (Iventosch, Para. [0007]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Capellier in view of St. Romain II further in view of Funke and Mao with the teachings of Iventosch by including cropping the sensor data image to a bounding box surrounding the tracked object. One having ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to combine these references because doing so would allow for recognizing objects in a plurality of different classes from camera images, as recognized by Iventosch. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights