Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1 – 19 are pending and examined herein.
Claim 18 is rejected under 35 U.S.C. 112(b).
Claims 1 – 19 are rejected under 35 U.S.C. 101.
Claims 1 – 19 are rejected under 35 U.S.C. 102/103.
Specification
The disclosure is objected to because of the following informalities:
Reference 123 used to refer “network” in [0058]. Fig. 2B,C refers to 123 as “ADAS control unit”.
Reference 134 used to refer “processing system” in [0065]. Fig. 2B,C refers to 134 as “Remote computerized systems”.
Reference 180 used to refer “response software” in [0074]. Fig. 2C refers to “response software” with reference 160 instead and 180 doesn’t exist in figures.
Fig. 3 “multi dimension identifier generator” 584, “match unit” 587-1 are not described in specification.
Reference 610 used to refer “image” in [00135,136]. [00137] refers to 610 as “road” instead.
Fig. 4 reference 50 is not described in specification.
Fig. 5 reference 660 is not described in specification.
Reference 612 used to refer “road” in [00136,137]. [00140] refers to 612 as “second vehicle”.
Appropriate correction is required.
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should not refer to purported merits or speculative applications of the invention and should not compare the invention with the prior art.
If the patent is of a basic nature, the entire technical disclosure may be new in the art, and the abstract should be directed to the entire disclosure. If the patent is in the nature of an improvement in an old apparatus, process, product, or composition, the abstract should include the technical disclosure of the improvement. The abstract should also mention by way of example any preferred modifications or alternatives.
Where applicable, the abstract should include the following: (1) if a machine or apparatus, its organization and operation; (2) if an article, its method of making; (3) if a chemical compound, its identity and use; (4) if a mixture, its ingredients; (5) if a process, the steps.
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
The abstract of the disclosure is objected to because the abstract merely recites claim language without providing a clear and concise summary of the technical disclosure of the invention. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
Claim 10 is objected to because of the following informalities: The limitation starts with phrase “that once executed by a computerized system cause the object computerized system to:”. Then the following limitations recite the acts with “receiving…”, “determining…”, and ”training…”. These should be changed to “receive”, “determine”, and “train” to properly follow the phrase recited above. Appropriate correction is required.
Claims 7, 16 are objected to under 37 CFR 1.75 as being a substantial duplicate of claims 6, 15. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 18 recites the limitation "the scenario information comprises environment" in line 2. It is unclear whether the scenario information comprises environmental information, information about an environment, or some other information. Therefore, the metes and bounds of claim 18 are unclear. For examination purposes, it would be reciting same limitation as claim 9 wherein “the scenario information comprises environment information about an environment located outside the vehicle.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 - 19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP § 2109(III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject matter. The analysis of claims 1 – 19, in accordance with these steps, follows.
Step 1 Analysis:
Step 1 is to determine whether the claim is directed to a statutory category (process, machine, manufacture, or composition of matter.
Claims 1 – 9 are directed to a method, meaning that it is directed to the statutory category of process. Claims 10 – 18 are directed to a non-transitory computer readable medium, which is the statutory category of manufacture. Claim 19 is directed to a system, which can be an article of machine.
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 1, the following claim elements are abstract ideas:
determining, based on the scenario information and the behavior information, a relevancy metric providing an indication of an impact of the road user on a driving of the vehicle; (This is practical to perform in the human mind under its broadest reasonable interpretation aside from the recitation of generic computer components.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A method that is computer implemented for self-learning of relevancy metrics for perception related applications, the method comprising (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
receiving a training dataset that comprises (i) scenario information regarding a scenario faced by a vehicle and involving a road user, and (ii) behavior information indicative of a response of the vehicle to a presence of the road user; (This is mere data gathering, an insignificant extra solution activity, which is a well-understood, routine conventional activity. It does not integrate the judicial exception into a practical application. See MPEP § 2106.05(d). Therefore, this does not amount to significantly more than the judicial exception.)
and training, using self-learning, a machine learning process to infer the relevancy metric according to the scenario. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 2, the rejection of claim 1 is incorporated herein. Further, claim 2 recites the following additional element:
wherein the training is by applying a semi-supervised training process. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 3, the rejection of claim 1 is incorporated herein. Further, claim 3 recites the following additional element:
wherein the training is by applying a self-training process. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 4, the rejection of claim 3 is incorporated herein. Further, claim 4 recites the following abstract idea:
… assigning pseudo labels (This is practical to perform in the human mind under its broadest reasonable interpretation aside from the recitation of generic computer components.)
Claim 4 further recites the following additional element:
wherein the applying of the self-training process comprises iteratively learning a classifier by … (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 5, the rejection of claim 3 is incorporated herein. Further, claim 5 recites the following additional element:
wherein the applying of the self-training process comprises training two classifiers. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 6, the rejection of claim 1 is incorporated herein. Further, claim 6 recites the following additional element:
wherein the training comprises applying a self- supervised training process. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 7, the rejection of claim 1 is incorporated herein. Further, claim 7 recites the following additional element:
wherein the training comprises applying a self- supervised training process. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 8, the rejection of claim 7 is incorporated herein. Further, claim 8 recites the following additional element:
wherein the applying of the self-supervised training process comprises executing a proxy task before inferring the relevancy of the road user. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Regarding claim 9, the rejection of claim 1 is incorporated herein. Further, claim 9 recites the following additional element:
wherein the behavior information comprises kinematics sensor information, whereas the scenario information comprises environment information about an environment located outside the vehicle. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Claims 10 – 18 recite substantially similar subject matter to claims 1 – 9 respectively and are rejected with the same rationale, mutatis mutandis.
Regarding claim 19, the following claim elements are abstract ideas:
and a processing circuit that is configured to: identify the scenario using the received scenario information; (This is practical to perform in the human mind under its broadest reasonable interpretation aside from the recitation of generic computer components.)
determine, based on the identified scenario, a resource operation parameter that conform to the identified scenario and is related to operation of a perception related process; (This is practical to perform in the human mind under its broadest reasonable interpretation aside from the recitation of generic computer components.)
Claim 19 further recites the following additional elements:
a memory unit that is configured to store scenario information about a scenario faced by a vehicle; (This is mere data gathering, an insignificant extra solution activity, which is a well-understood, routine conventional activity. It does not integrate the judicial exception into a practical application. See MPEP § 2106.05(d). Therefore, this does not amount to significantly more than the judicial exception.)
wherein the scenario information comprises environmental information about an environment of the vehicle; (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
and make the resource operation parameter available in the operation of the perception related process. (This falls under mere instructions to apply an exception. See MPEP § 2106.05(f). Therefore, this does not amount to significantly more than the judicial exception.)
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 – 4, 9, 10 – 13, 18 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Li et al. (U.S. Pub. 20220413507).
Regarding Claim 1, Li teach
receiving a training dataset that comprises (i) scenario information regarding a scenario faced by a vehicle and involving a road user, and (ii) behavior information indicative of a response of the vehicle to a presence of the road user; ([0010] of Li states “According to one aspect, a computer-implemented method for object identification may include extracting a first set of visual features from a first image of a scene detected by a first sensor, extracting a second set of visual features from a second image of the scene detected by a second sensor of a different sensor type than the first sensor, concatenating the first set of visual features, the second set of visual features, and a set of bounding box information associated with the first image and the second image, determining a number of object features associated with a corresponding number of objects from the scene and a global feature for the scene, receiving ego-vehicle feature information associated with an ego-vehicle, receiving the number of object features, the global feature, and the ego-vehicle feature information, generating relational features with respect to relationships between each of the number of objects from the scene, and classifying each of the number of objects from the scene based on the number of object features, the relational features, the global feature, the ego-vehicle feature information, and an intention of the ego-vehicle.”)
determining, based on the scenario information and the behavior information, a relevancy metric providing an indication of an impact of the road user on a driving of the vehicle; ([0039] of Li states “An object may be considered ‘relevant’ or ‘important’ if the object is within a scenario, such as a driving scenario, and may or should influence the behavior of an autonomous vehicle operating within the driving scenario. An example of a driving scenario may be a scenario where the autonomous vehicle is navigating through an intersection, and may determine or detect one or more surrounding objects, such as other vehicles or pedestrians, and label or classify these detected objects as relevant or non-relevant (e.g., important or non-important).” [0065] of Li states “which may be fed into the classifier (e.g., multi-layer perceptron) to obtain a corresponding relevance score sj∈[0, 1] (e.g., the probability that the object is relevant). During training phase, Sj may be used to compute loss directly for labeled objects and generate pseudo-labels for unlabeled ones.”)
and training, using self-learning, a machine learning process to infer the relevancy metric according to the scenario. ([0013] of Li states “The method for object identification for the ego-vehicle may include training the object classifier utilizing supervised learning including a labeled dataset. The method for object identification for the ego-vehicle may include training the object classifier utilizing semi-supervised learning including a labeled dataset and an unlabeled dataset, the unlabeled dataset may be annotated with pseudo labels generated from classifying each of the number of objects.”)
Regarding claim 2, the rejection of claim 1 is incorporated herein. Furthermore, the Li teaches that
wherein the training is by applying a semi-supervised training process. ([0013] of Li states “The method for object identification for the ego-vehicle may include training the object classifier utilizing supervised learning including a labeled dataset. The method for object identification for the ego-vehicle may include training the object classifier utilizing semi-supervised learning including a labeled dataset and an unlabeled dataset, the unlabeled dataset may be annotated with pseudo labels generated from classifying each of the number of objects.”)
Regarding claim 3, the rejection of claim 1 is incorporated herein. Furthermore, the Li teaches that
wherein the training is by applying a self-training process. ([0068] of Li states “Pseudo-label generation may be implemented in semi-supervised learning algorithms. In the task, a naive way may be to use the learned relevance classifier at the last iteration directly to assign pseudo labels for the objects in the unlabeled data samples by arg max(1−sj, sj).”)
Regarding claim 4, the rejection of claim 3 is incorporated herein. Furthermore, the Li teaches that
wherein the applying of the self-training process comprises iteratively learning a classifier by assigning pseudo labels. ([0068] of Li states “Pseudo-label generation may be implemented in semi-supervised learning algorithms. In the task, a naive way may be to use the learned relevance classifier at the last iteration directly to assign pseudo labels for the objects in the unlabeled data samples by arg max(1−sj, sj).” [0082] of Li states “The weight γ may be initialized as 0, which implies that the unlabeled dataset is not necessarily used at the beginning of training. It increases to a maximum value over a fixed number of epochs with a linear schedule since the model becomes more accurate and confident thus may generate more reliable pseudo-labels as training goes on.”)
Regarding claim 9, the rejection of claim 1 is incorporated herein. Furthermore, the Li teaches that
wherein the behavior information comprises kinematics sensor information, ([0042] of Li states “The feature extractor 120 may extract object features from frontal-view visual observations and the ego-vehicle state information. In this regard, if the sensor 110 includes the image capture device 112 or image capture sensor, the measurement from the image capture sensor 112 may be an RGB image, for example. According to another aspect, if the sensor includes the LiDAR sensor 114, the measurement from the LiDAR sensor 114 may be a point cloud. Thus, the first sensor may be an image capture sensor 112 and the second sensor may be the LiDAR sensor 114. Additionally, the sensors 110, such as the image capture sensor 112 and the LiDAR sensor 112 may be mounted on the ego-vehicle and may be forward-facing or side-facing, according to one aspect.” [0055] of Li states “EGO-VEHICLE FEATURES: The system for object identification may extract the ego-state features vego from a sequence of state information 220 (e.g., position, velocity, acceleration) with the Ego-State Feature Encoder 222.”)
whereas the scenario information comprises environment information about an environment located outside the vehicle. ([0034] of Li states “Different from which only consider dynamic traffic participants, the system for object identification may also consider traffic lights or signs in the driving scenes to enable semantic reasoning of the environment.” [0054] of Li states “BOUNDING BOX FEATURES: The location and scale of the object bounding box information 206 in the frontal-view images may provide additional indications of the size and relative positions of the objects with respect to the ego-vehicle, which may influence their relevance.”)
Claims 10 – 13, 18 recite substantially similar subject matter as claims 1 – 4, 9 respectively, and are rejected with the same rationale, mutatis mutandis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (U.S. Pub. 20220413507) in view of Wang et al. (U.S. Pub. 20230207105).
Regarding Claim 5, the rejection of claim 3 is incorporated herein. Li does not explicitly teach that
wherein the applying of the self-training process comprises training two classifiers.
However, Wang teaches that
wherein the applying of the self-training process comprises training two classifiers. ([0036] of Wang states “With reference to FIG. 2 , and with continuing reference to FIG. 1 , the at least one electronic processor 20 is configured as described above to perform the method or process 100 for co-training the image classifier 14 and the radiology report classifier 16. The non-transitory storage medium 26 stores instructions which are readable and executable by the at least one electronic processor 20 to perform disclosed operations including performing the method or process 100.” [0019] of Wang states “Improved embodiments disclosed herein leverage the corresponding radiology reports in a co training paradigm. In this approach, an initial image classifier is trained on the labeled training images, and an initial report classifier is trained on the labeled radiology reports. These are used to generate pseudo-labeled images and pseudo-labeled reports, respectively. However, in this co-training paradigm, the feedback for further training of the image classifier relies (at least in part) on the pseudo-labels generated by the report classifier; and vice versa.”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to combine the teachings of Li and Wang. Li teaches training an object relevance classifier for vehicle perception using semi supervised learning and pseudo labels generated for unlabeled data. Wang teaches a known co-training technique in which two classifiers are trained together, with pseudo label feedback from one classifier being used to further train the other classifier, and vice versa. One with ordinary skill in the art would be motivated to incorporate the teachings of Wang into that of Li to improve the reliability of pseudo labels and reduce reliance on a single classifier when training from unlabeled data. The combination would have been predictable as it is merely applying a known co-training technique to pseudo label training process for improved semi supervised classifier training.
Claim 14 recites substantially similar subject matter as claim 5 respectively, and is rejected with the same rationale, mutatis mutandis.
Claims 6 – 8, 15 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (U.S. Pub. 20220413507) in view of Chang et al. (U.S. Pub. 20250128731).
Regarding Claim 6, the rejection of claim 1 is incorporated herein. Li does not explicitly teach that
wherein the training comprises applying a self- supervised training process.
However, Chang teaches that
wherein the training comprises applying a self- supervised training process. ([0016] of Chang states “As also described herein, the backbone machine learned model may be trained in a self-supervised manner to operate on a driving scene constructed as a graph containing nodes and edges and may output a representation (for example, in the form of an embedding) for a node associated with a particular object or feature within the environment. In this disclosure, since the graph containing multiple nodes may represent the environment, the output, which is generated based at least in part on the graph, may alternatively be referred to as a node-level embedding or a node embedding.”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to combine the teachings of Li and Chang. Li teaches training an object relevance classifier for vehicle perception using semi supervised learning and pseudo labels generated for unlabeled data. Chang teaches self supervised training for vehicle perception using a proxy task, including masking a portion of driving scene training data and training the model to reconstruct the missing portion before adapting the model to a downstream task. One with ordinary skill in the art would be motivated to incorporate the teachings of Chang into that of Li to learn useful scene and object representation from unlabeled driving data before performing the relevance inference task. The combination would have been predictable use of known self supervised pretraining to improve a known vehicle perception machine learning model.
Claim 7 recites substantially similar subject matter as claim 6 respectively, and is rejected with the same rationale, mutatis mutandis.
Regarding Claim 8, the rejection of claim 7 is incorporated herein. The combination of Li and Chang teaches that
wherein the applying of the self-supervised training process comprises executing a proxy task before inferring the relevancy of the road user. ([0048] of Chang states “A portion of the training data corresponding to a node 121 of the graph 120 may be masked, so that the training data comprises a masked portion 125 of training data and an unmasked portion of training data. The masking operation may be carried out by setting a portion of training data to a constant. Different ways of masking data are described in more detail in relation to FIG. 3 . Masking one or more nodes is a way of training the SSL model 130 to reconstruct the scene of the environment by predicting what was missing from the input training data (that is, the masked portion) using the unmasked portion of training data, since the unmasked portion may comprise data that suggests there may be another feature of the environment (for example, an additional vehicle) that was not present in the input data.” [0108] of Chang states “The task to be performed and thus defined by task 710 may be at least one of the following: detecting one or more objects within the environment (whether static or dynamic); predicting a trajectory for an object within the environment;)
Claims 15-17 recite substantially similar subject matter as claims 6-8 respectively, and are rejected with the same rationale, mutatis mutandis.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (U.S. Pub. 20220413507) in view of Agrawal (U.S. Pub. 20210099643).
Regarding Claim 19, Li teaches that
a memory unit that is configured to store scenario information about a scenario faced by a vehicle; wherein the scenario information comprises environmental information about an environment of the vehicle; ([0042] of Li states “The feature extractor 120 may extract object features from frontal-view visual observations and the ego-vehicle state information. In this regard, if the sensor 110 includes the image capture device 112 or image capture sensor, the measurement from the image capture sensor 112 may be an RGB image, for example. According to another aspect, if the sensor includes the LiDAR sensor 114, the measurement from the LiDAR sensor 114 may be a point cloud. Thus, the first sensor may be an image capture sensor 112 and the second sensor may be the LiDAR sensor 114. Additionally, the sensors 110, such as the image capture sensor 112 and the LiDAR sensor 112 may be mounted on the ego-vehicle and may be forward-facing or side-facing, according to one aspect.” [0055] of Li states “EGO-VEHICLE FEATURES: The system for object identification may extract the ego-state features vego from a sequence of state information 220 (e.g., position, velocity, acceleration) with the Ego-State Feature Encoder 222.” [0034] of Li states “Different from which only consider dynamic traffic participants, the system for object identification may also consider traffic lights or signs in the driving scenes to enable semantic reasoning of the environment.” [0054] of Li states “BOUNDING BOX FEATURES: The location and scale of the object bounding box information 206 in the frontal-view images may provide additional indications of the size and relative positions of the objects with respect to the ego-vehicle, which may influence their relevance.”)
Li does not explicitly teach
identify the scenario using the received scenario information;
determine, based on the identified scenario, a resource operation parameter that conform to the identified scenario and is related to operation of a perception related process;
and make the resource operation parameter available in the operation of the perception related process.
However, Agrawal teaches that
identify the scenario using the received scenario information; ([0029] of Agrawal states “Vehicle state 225 includes the current position, velocity, and other state(s) of the vehicle. Vehicle intent 230 includes the intent of the vehicle, such as lane change, turning, etc. World map 235 is a high-definition map of the world, which includes semantics and height information.” [0030] of Agrawal states ”In step 305, the perception system (e.g., the perception filter) determines the intent of the autonomous vehicle (e.g., using the planned route thereof) and the current state of the autonomous vehicle. In step 310, the perception system (e.g., the perception filter) crops and filters data received from imagers and sensors (e.g., cameras, LIDARs, and RADARs) according to an ROI determined based on the current state and intent of the autonomous vehicle and provides the cropped and filter data to a perception module of the perception system. In step 315, the perception module uses the cropped and filtered data to perceive the environment of the autonomous vehicle and take appropriate action based on the perceived environment, current state, and intent.” )
determine, based on the identified scenario, a resource operation parameter that conform to the identified scenario and is related to operation of a perception related process; ([0009] of Agrawal states “a perception filter for receiving the images produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle;” [0019] of Agrawal states “Additionally, the perception filter of the perception system may scale the compute resources and resolution for the particular sensor based on the state and intent of the autonomous vehicle. In particular, given the nature of autonomous vehicle driving, many systems often compete for limited resources (e.g., CPU and GPU resources). This resource allocation can be changed at real-time depending upon the relative importance of the resource data as dictated by the ROI(s) based on autonomous vehicle state and intent.” [0031] of Agrawal states “Additionally and/or alternatively, in step 320, the perception system (e.g., the perception filter) determines compute resource priority based on ROI and/or current state and intent of the autonomous vehicle and provides compute resource priority instructions to a compute module of the perception system.” )
and make the resource operation parameter available in the operation of the perception related process. ([0009] of Agrawal states “and a compute module for receiving the compute resource priority instructions from the perception filter and allocating compute resources among the imaging devices in accordance with the compute resource priority instructions.” [0018] of Agrawal states “For example, if the autonomous vehicle is traveling on a highway at high speed, straight ahead at a long range to the horizon is likely the most important area on which to focus, or region of interest (“ROI”). In such a situation, high-resolution crops of the image data comprising the ROI, rather than the entire image from the imager(s), may be provided to a perception module to perceive the autonomous vehicle's surroundings. In contrast, when driving at low speed on in a city, the surroundings all around the autonomous vehicle are important and may comprise the ROI. In such a situation, low resolution images and sensor data from all around the vehicle may be provided to a perception module to perceive the autonomous vehicle's relevant surroundings.” [0031] of Agrawal states “In step 325, the compute module implements the compute resource priority instructions by allocating resources to the imagers and sensors comprising the sensor suite in accordance with the priority instructions.”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to combine the teachings of Li and Agrawal. Li teaches training an object relevance classifier for vehicle perception using semi supervised learning and pseudo labels generated for unlabeled data. Agrawal teaches determining compute resource priority instructions based on vehicle intent and current vehicle state, and allocating compute resources among perception imaging devices according to those instructions. One with ordinary skill in the art would be motivated to incorporate the teachings of Agrawal into that of Li so that perception resources could be allocated based on the identified driving scenario and the relevance of objects in that scenario. The combination would have been predictable to improve the operation of the perception process by directing processing resources toward portions of the environment most relevant to vehicle operation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BYUNGKWON HAN whose telephone number is (571)272-5294. The examiner can normally be reached M-F: 9:00AM-6PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at (571)272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BYUNGKWON HAN/ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights