Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s argument filed 07/01/2025 have been fully considered but they are not persuasive. The amendments of claims 16, 17, and 19 fail to overcome 35 U.S.C. § 112(b) rejection. Claims 16, 17, and 19 fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The claims recite “means to” and it is unclear what corresponding structure is disclosed to perform the claimed function.
Applicant’s Argument: On page 14-17 of Applicant’s response to rejections under 35 U.S.C. 101, applicant states that the amended claims do not recite a judicial exception because the claims as a whole is directed to a technological improvement of controlling an autonomous vehicle based on negatively impacted autonomous vehicle sensor data. Applicant also states that the amended claims recite “control an operation of the autonomous vehicle based on the output data” integrates the judicial exception into a practical application. Further, applicant argues that the amended claims recite significantly more than the abstract idea.
Examiner’s Response: Applicant’s argument is not persuasive. During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement (see MPEP §2106.05(a)). The MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of “improved autonomous vehicle sensor processing technology” is an improvement to the abstract idea of a mental process that can be performed in the human mind.
An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome (see MPEP 2106.05(a)). The amended claims do not provide sufficient details to describe any technological improvement. If the specifications explicitly set forth an improvement but in a conclusory manner (see MPEP 2106.04(d)(1): a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.
Under BRI, the claim element “control an operation of the autonomous vehicle based on the output data” recites the use of a computer or “other machinery” as a tool to perform the abstract idea. Examiner suggest to further amend the claim element with the details from par. 164 of the Specification in hopes to overcome 101 rejections by integrating the judicial exception into a practical application.
Applicant’s Argument: On page 18 of Applicant’s response to rejections under 35 U.S.C. 103, applicant states that the cited references do not teach the amended claims. Applicant states that the claimed invention recites an application of feasibility of correcting a particular latent representation that is relevant to a perception task and evaluates the impact of negative effects in object classification.
Examiner’s Response: Applicant’s argument is not persuasive. Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Radha (par. 27-28) discloses training an object detection model to identify vehicles and pedestrians from images of driving during daylight conditions. The object detection model is tested on images with generated synthetic rain of different levels of intensity. The system determines the performance of the model to classify the object based on the negative effects of rainy driving conditions.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claims 14-20 are interpreted under 112(f) because they meet the three-prong test for invocation.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 16, 17, and 19 are rejected under 35 U.S.C. 112(b) for indefiniteness.
Claim limitations, “means to determine a perception type” in claims 16, 17, and 19, “means to identify a sub negative effect” in claims 16 and 17, and “means to identify a first aspect” in claim 17 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The instant specification does not disclose sufficient structure, materials, or acts for performing the claimed function. Therefore, the claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Subject Matter Eligibility Analysis Step 1:
Claims 1-7 are directed to a machine (“A device comprising a processor configured to”), claims 8-13 are directed to an article of manufacture (“A non-transitory computer-readable medium comprising”), and claims 14-20 are directed to a machine (“A system, comprising”), which each fall within one of the four statutory categories.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“identify a correlated latent representation of a correctable negative effect of the environment within the autonomous vehicle sensor data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“determine that the correlated latent representation corresponds to the correctable negative effect, based on the plurality of pre-identified latent representations and the plurality of ground truth labels” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“estimate an error distribution for the task data based on the identified latent representation, the task data, and the perception task based on a feasibility of correcting the correctable negative effect” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Claim 1 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“a processor configured to” and “wherein the processor is further configured to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"receive autonomous vehicle sensor data representative of an environment of an autonomous vehicle by way of a data connection to a communication network” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“generate task data using the autonomous vehicle sensor data in accordance with a perception task” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“the task data comprising a plurality of features of the environment” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
"receive a domain specific dataset (DSD) comprising a plurality of pre-identified latent representations and a plurality of ground truth labels corresponding to the plurality of pre-identified latent representations” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“generate output data comprising a normalized distribution of the plurality of features based on the estimated error distribution and the task data” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“control an operation of the autonomous vehicle based on the output data” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“a processor configured to” and “wherein the processor is further configured to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"receive autonomous vehicle sensor data representative of an environment of an autonomous vehicle by way of a data connection to a communication network” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“generate task data using the autonomous vehicle sensor data in accordance with a perception task” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“the task data comprising a plurality of features of the environment” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
"receive a domain specific dataset (DSD) comprising a plurality of pre-identified latent representations and a plurality of ground truth labels corresponding to the plurality of pre-identified latent representations” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“generate output data comprising a normalized distribution of the plurality of features based on the estimated error distribution and the task data” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“control an operation of the autonomous vehicle based on the output data” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is subject-matter ineligible.
Regarding Claim 8:
The claim recites an article of manufacture that performs the method as described in claim 1. Therefore, claim 8 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 8 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“A non-transitory computer-readable medium comprising: a memory having computer-readable instructions stored thereon; and a processor operatively coupled to the memory and configured to read and execute the computer-readable instructions to perform or control performance of operations comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 14:
The claim recites a system that performs the method as described in claim 1. Therefore, claim 14 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 14 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“a system… means to” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claims 2, 9, and 15:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“identify a second latent representation of a negative effect of the environment within the second sensor data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“estimate an error distribution for the second task data based on the identified second latent representation, the second task data, and the second perception task” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“generate second task data using the second sensor data in accordance with a second perception task, the second task data comprising a second plurality of features of the environment” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“wherein the perception task comprises a first perception task, the autonomous vehicle sensor data comprises first sensor data, the task data comprises first task data, the plurality of features comprise a first plurality of features, the correlated latent representation comprises a first latent representation, the output data comprises first output data” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
“the processor is further configured to: receive second sensor data representative of the environment” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“generate second output data comprising a normalized distribution of the second plurality of features based on the estimated error distribution for the second task and the second task data” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 3 and 16:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“determining a perception type of the perception task” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“identifying a sub negative effect of the environment using the autonomous vehicle sensor data based on the perception type” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“mapping the sub negative effect to a pre-identified latent representation of a plurality of pre-identified latent representations” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the processor is configured to identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data by: ... wherein the identified latent representation comprises the pre-identified latent representation” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claims 4 and 17:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“determining a perception type of the perception task” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“identifying a first aspect and a second aspect of the perception task” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“identifying a sub negative effect of the environment using the autonomous vehicle sensor data based on the perception type” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“mapping the sub negative effect to a first pre-identified latent representation of a plurality of pre-identified latent representations” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“mapping the sub negative effect to a second pre-identified latent representation of the plurality of pre-identified latent representations” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the processor is configured to identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data by: … wherein the first pre-identified latent representation corresponds to the first aspect; … wherein the second pre-identified latent representation corresponds to the second aspect, wherein the identified latent representation comprises the first pre-identified latent representation and the second pre-identified latent representation” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claims 5, 10, and 18:
The claim element “means to receive a domain specific dataset (DSD) ...” of claim 18 is addressed in claim 1.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“identify the correlated latent representation of the correctable negative effect within the autonomous vehicle sensor data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“train a negative effect latent representation (NELR) module using a machine learning algorithm, the plurality of pre-identified latent representations, and the plurality of ground truth labels, wherein the processor is configured to identify the latent representation of the negative effect within the sensor data using the NELR module” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 6, 11, and 19:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“estimate the error distribution for the task data” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“determining a perception type of the perception task” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“mapping the correlated latent representation to a pre-identified error” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“mapping the correlated latent representation to a feature of the plurality of features based on the perception type” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the processor is configured to estimate the error distribution for the task data based on the correlated latent representation, the task data, and the perception task by: … wherein the pre-identified error corresponds to an algorithm associated with the perception task based on the perception type; … wherein the estimated error distribution is based on the pre-identified error and the feature that the correlated latent representation is mapped to” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claims 7, 13, and 20:
The claim element “means to receive a domain specific dataset (DSD) ...” of claim 20 is addressed in claim 1.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“determine a plurality of pre-identified errors of an algorithm corresponding to the perception task using a loss function, the training task data, and the plurality of ground truth labels” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“generate training task data using the training sensor data in accordance with the perception task” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“train an error estimation (EE) module using a machine learning algorithm, the training task data, the plurality of pre-identified latent representations, and the plurality of pre-identified errors, wherein the processor is configured to estimate the error distribution for the task data using the EE module” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Claim 12
Claim 12 depends from claim 8 and recites a non-transitory, computer-readable medium that corresponds to the limitations of claims 4 and 6, and therefore claim 12 is rejected under the same rationale as outlined above for claims 4, 6, and 8 for being substantially similar, mutatis mutandis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6, 8-9, 11-12, 14-17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over US 20220041185 A1, referenced herein as RADHA, in view of “Context-Aware Sensor Uncertainty Estimation for Autonomous Vehicles”, referenced herein as ALHARBI.
Regarding claim 1, RADHA teaches:
“A device comprising a processor” ([0059], “Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability”).
“configured to: receive autonomous vehicle sensor data representative of an environment of an autonomous vehicle by way of a data connection to a communication network” ([0026, 0044, 0058], Image data is encompassed by the BRI of data which is representative of an environment of a vehicle. YOLO is selected as the object detection framework, which is a real-time object detection framework and can be employed for autonomous systems. The object detection framework may be employed in a different platform and receive the image data from the camera through a network connection).
“generate task data using the autonomous vehicle sensor data in accordance with a perception task” ([0027-0028], The perception task is to explore the impact of rain of different intensities on visual quality of the data captured by the sensors. The test dataset consists of frames having different levels of synthesize rain streaks.)
“the task data comprising a plurality of features of the environment” ([0028, 0047], As outlined above, the task data corresponds to the test streaks augmented with the simulated rain streaks. The levels of rain intensity correspond to the features of the image data. The system can also be adapted to detect other weather conditions).
“receive a domain specific dataset (DSD) comprising a plurality of pre-identified latent representations and a plurality of ground truth labels corresponding to the plurality of pre-identified latent representations” ([0027, 0028], The Udacity annotated data represents the domain of autonomous driving. The dataset is labeled with vehicles and pedestrians and provided as ground truth objects. Simulation is performed on the test frames to synthesize rain streaks. The synthesized rain intensity corresponds to the ground truth label because the object detection model is applied to the synthetic rainy frames to compare the predicted detection with the ground truth label).
“identify a correlated latent representation of a correctable negative effect of the environment within the autonomous vehicle sensor data” ([0032, 0037, 0042, 0044], As outlined above, the sensor data corresponds to the image data and the task data corresponds to the synthesized rainy frames. Paragraph [0056] of the instant specification states "The latent representation may represent different adverse conditions of the environment of the vehicle", but does not appear to explicitly define a latent representation. The assigned rain intensity range corresponds to the latent representation of the image data and is provided to the CNN model as an input. Additionally, deraining algorithm can be employed to remove rain from images to improve the performance of object detection. A CNN framework is used to determine the differences between clear and rainy images and use the difference to isolate the rain from the images).
“wherein the processor is further configured to determine that the correlated latent representation corresponds to the correctable negative effect, based on the plurality of pre-identified latent representations and the plurality of ground truth labels” ([0027-0028, 0032, 0037], Synthetic rain is simulated for the selected test frames with different levels of intensity. The YOLO object detection framework is tested to determine how well it performs on the synthetic rainy frames compared to the original clear frames. The objective is to determine the impact of rain on object detection. The rain streaks identified in the image data corresponds to different levels of rain intensities and the rain can negatively impact the object detection. The predicted detection is compared to the ground-truth detection to determine the performance of the object detection model when the data contains rain streaks of varying intensity).
“estimate an error distribution for the task data based on the identified latent representation, the task data, and the perception task based on a feasibility of correcting the correctable negative effect” ([0029-0030, 0048], The prediction result corresponds to the predicted detection which is generated by a rain intensity specific object classifier, which integrates the identified latent representation, the task data, and the perception task. The Average Precision is encompassed by the BRI of an estimated error distribution. Rain distorts the desired visual details that are required for reliable detection and as rain intensity increases, normalized average precision of pedestrian declines rapidly. Thus, a high level of rain intensity makes it challenging for an object detection model to detect pedestrians in rainy weather).
“and generate [[output]] data comprising a normalized distribution [[of the plurality of features]] based on the estimated error distribution and the task data” ([0042], The normalized mean Average Precision comprises statistics from the estimated error distribution based on the task data).
RADHA does not appear to explicitly teach “and generate output data comprising a normalized distribution of the plurality of features based on the estimated error distribution and the task data” and “control an operation of the autonomous vehicle based on the output data”.
Analogous art ALHARBI provides this additional functionality by teaching:
“and generate output data comprising a normalized distribution of the plurality of features based on the estimated error distribution and the task data” ([pg. 7-9, Section 3.3, par. 1 & 8; pg. 9, Section 4, par. 2-4], The input data comprises of the Ford AV dataset, which includes different environmental and driving conditions. The output is sensor uncertainty estimation based on the plurality of features from the input dataset).
“control an operation of the autonomous vehicle based on the output data” ([pg. 3, Section 2.2, par. 1; pg. 12, par. 2], In one embodiment, the sensor uncertainty output can be used in decision-making of autonomous vehicles for safer navigation. In path planning approaches, routes are computed with the objective of reducing uncertainty. The reference discloses that the uncertainty of each sensor on every road segment could help an autonomous vehicle better maneuver through road segments).
RADHA and ALHARBI are analogous art because they are from the same field of endeavor as the claimed invention. RADHA teaches to receive sensor data representative of an environment of a vehicle, but does not appear to distinctly disclose to generate output data comprising a normalized distribution of the plurality of features based on the estimated error distribution and the task data as taught by ALHARBI. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have improved upon the machine learning system of RADHA with ALHARBI because the performance of autonomous vehicle can be improved by implementing a method of predicting sensor performance in challenging environments (abstract, ALHARBI), as suggested by ALHARBI.
Claim 8
Claim 8, which recites the additional limitation of "a non-transitory, computer-readable medium" ([0059], RADHA: "a computer program stored on a computer readable medium that can be accessed by the computer. Such a computer program may be stored in a tangible computer readable storage medium"), is rejected under the same rationale as claim 1 for being substantially similar, mutatis mutandis.
Claim 14
Claim 14, which recites the additional limitation of "a system… means to" ([0059], RADHA: "The present disclosure also relates to an apparatus for performing the operations herein"; (EN): The BRI of a system includes an apparatus), is rejected under the same rationale as claim 1 for being substantially similar, mutatis mutandis.
Claims 2, 9, and 15
The combination of RADHA and ALHARBI teaches claims 2, 9, and 15 as follows:
RADHA further teaches “wherein the perception task comprises a first perception task” ([0028], “object detection under rainy conditions”).
RADHA further teaches “the autonomous vehicle sensor data comprises first sensor data” ([0007], “image data from the camera”).
RADHA further teaches “the task data comprises first task data” ([0028], As outlined above, the task data corresponds to the augmented sensor data).
RADHA further teaches “the plurality of features comprise a first plurality of features” ([0028], As outlined above, the levels of rain intensities correspond to the recited features).
RADHA further teaches “the correlated latent representation comprises a first latent representation” ([0044], As outlined above, the latent representation corresponds to the assigned rain intensity range showing different rain streaks).
ALHARBI further teaches “the output data comprises first output data” ([pg. 7-9, Section 3.3, par. 1 & 8], “the output is modeled as normal distribution with learnable parameters”).
RADHA further teaches “and the processor is further configured to: receive second sensor data representative of the environment” ([0047], The camera and images correspond to the first sensor and first sensor data. The rain sensor’s provided input is encompassed by the BRI received second sensor data. The current rain status is encompassed by the BRI of data representative of the environment).
RADHA further teaches “generate second task data using the second sensor data in accordance with a second perception task” ([0048], The activated object classifier which is selected based on the input from the sensor data is encompassed by the BRI of second task data. The second perception task corresponds to determining the level, or intensity, of rain present in the environment).
RADHA further teaches “the second task data comprising a second plurality of features of the environment” ([0048], As outlined above, the rain intensities correspond to features of the environment).
RADHA further teaches “identify a second latent representation of a negative effect of the environment within the second sensor data” ([0048], As outlined above, the range of rain intensity assigned to the sensor data (which is associated with the object classifier activated) is encompassed by the BRI of a latent representation of a negative effect of the environment within the sensor data).
RADHA further teaches “estimate an error distribution for the second task data based on the identified second latent representation, the second task data, and the second perception task” ([0051], As outlined above, the second task data corresponds to the activated object classifier, the identified second latent representation corresponds to the rain intensity determined by the rain sensor, and the second perception task corresponds to determining a rain intensity in the environment. Average precision is determined to evaluate the performance of the detection task).
RADHA further teaches “and generate second [[output]] data comprising a normalized distribution [[of the second plurality of features]] based on the estimated error distribution for the second task and the second task data” ([0042], The normalized mean Average Precision comprises statistics from the estimated error distribution based on the task data. As outlined above, the second task data corresponds to the activated classifier. The activated classifier’s performance is measured by the NmAP metric).
ALHARBI further teaches “and generate second output data comprising a normalized distribution of the second plurality of features based on the estimated error distribution for the second task and the second task data” ([pg. 7-9, Section 3.3, par. 1 & 8; pg. 9, Section 4, par. 2-4; pg. 10, Table 1], The Ford AV dataset consists of 18 trajectories captured by a plurality of Ford autonomous vehicles. Therefore, the second task data and second plurality of features are collected from a different vehicle to perform the second task of sensor uncertainty estimation).
Claims 3 and 16
The combination of RADHA and ALHARBI teaches claims 3, and 16 as follows:
RADHA further teaches “wherein the processor is configured to identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data” ([0049], As outlined above, the latent representation of the negative effect of the environment corresponds to the assigned rain intensity range and the sensor data corresponds to the image data from the camera).
RADHA further teaches “by: determining a perception type of the perception task” ([0007], Cameras and their associated data type (images) correspond to the perception type. As outlined above, the object detection under rainy conditions corresponds to the perception task).
RADHA further teaches “identifying a sub negative effect of the environment using the autonomous vehicle sensor data based on the perception type” ([0049], As outlined above, a specific level of rain intensity is encompassed by the BRI of a negative effect. The instant specification does not appear to explicitly define a sub negative effect. As such, the BRI of a sub negative effect encompasses RADHA’s specific level of rain intensity).
RADHA further teaches “and mapping the sub negative effect to a pre-identified latent representation of a plurality of pre-identified latent representations” ([0044], As outlined above, the latent representations correspond to the assigned ranges of rain intensity and the sub negative effects correspond to the specific level of rain intensity. The instant specification does not appear to explicitly define a pre-identified latent representation. RADHA specifies that these ranges (shared across the synthetic rain intensities and the determined rain intensities) were set because “these discrete levels provided the most viable range” ([0028], RADHA). As such, the BRI of a pre-identified latent representation of a plurality of pre-identified latent representations includes RADHA’s assigned range of rain intensity).
RADHA further teaches “wherein the identified latent representation comprises the pre-identified latent representation” ([0044], As discussed above, RADHA’s ranges of rain intensity correspond to the latent representations both with reference to the synthetic rain and the sensor detected rain, which includes both rain detection in images and rain detection via a rain sensor).
Claims 4 and 17
The combination of RADHA and ALHARBI teaches claims 4 and 17 as follows:
RADHA further teaches “wherein the processor is configured to identify the correlated latent representation of the correctable negative effect of the environment within the autonomous vehicle sensor data” ([0049], As outlined above, the latent representation of the negative effect of the environment within the first sensor data corresponds to the range of rain intensity assigned by the determined level of rain intensity).
RADHA further teaches “by: determining a perception type of the perception task” ([0007], Cameras and their associated data type (images) correspond to the perception type. As outlined above, the object detection under rainy conditions corresponds to the perception task).
RADHA further teaches “identifying a first aspect and a second aspect of the perception task” ([0007], Paragraph [0108] of the instant specification states “The NELR may identify a first aspect and a second aspect of the first perception task”, but does not appear to explicitly define a first or second aspect of the perception task. As such, the BRI of the first and second aspect correspond to the camera sensor and the rain sensor).
RADHA further teaches “identifying a sub negative effect of the environment using the autonomous vehicle sensor data based on the perception type” ([0049], As outlined above, the sub negative effect corresponds to the negative effect and to the specific level of rain intensity).
RADHA further teaches “mapping the sub negative effect to a first pre-identified latent representation of a plurality of pre-identified latent representations” ([0044], As outlined above, the pre-identified latent representations correspond to the assigned ranges of rain intensity and the sub negative effects correspond to the specific level of rain intensity).
RADHA further teaches “wherein the first pre-identified latent representation corresponds to the first aspect” ([0049], As outlined above, the first aspect corresponds to the image data and the extractor 75 determines a pre-identified latent representation, or assigned range of rain intensity, from the image data).
RADHA further teaches “and mapping the sub negative effect to a second pre-identified latent representation of the plurality of pre-identified latent representations, wherein the second pre-identified latent representation corresponds to the second aspect” ([0048], As outlined above, the second latent representations correspond to the activated classifier with respect to the second aspect).
RADHA further teaches “wherein the identified latent representation comprises the first pre-identified latent representation and the second pre-identified latent representation” ([0050], Determining a second pre-identified latent representation as the latent representation based on input from the first and second aspects is encompassed by the BRI of the identified latent representation comprises the first pre-identified latent representation and the second pre-identified latent representation).
Claims 6, 11, and 19
The combination of RADHA and ALHARBI teaches claims 6, 11, and 19 as follows:
RADHA further teaches “wherein the processor is configured to estimate the error distribution for the task data based on the correlated latent representation, the task data, and the perception task” ([0029-0030, 0048], The prediction result corresponds to the predicted detection which is generated by a rain intensity specific object classifier, which integrates the identified latent representation, the task data, and the perception task. The Average Precision is encompassed by the BRI of an estimated error distribution).
RADHA further teaches “by: determining a perception type of the perception task” ([0007], As outlined above, cameras and their associated data type (images) correspond to the perception type. As outlined above, the object detection under rainy conditions corresponds to the perception task).
RADHA further teaches “mapping the correlated latent representation to a pre-identified error” ([0029, 0048], Paragraph [0083] of the instant specification states “The set of pre-identified errors 707 may be generated using a loss function, the training task data 705, the ground truth labels 703, or some combination thereof”, but does not appear to explicitly define a pre-identified error. As such, the range of rain intensities associated with the activated classifier is encompassed by the BRI of a pre-identified error. The model calculates the mean average precision to evaluate the performance of the detection having different levels of rain intensities).
RADHA further teaches “wherein the pre-identified error corresponds to an algorithm associated with the perception task based on the perception type” ([0048], As outlined above, the pre-identified error corresponds to the rain intensity range associated with the activated classifier).
RADHA further teaches “and mapping the correlated latent representation to a feature of the plurality of features based on the perception type” ([0044], As outlined above, the features correspond to the level of rain intensity. The level of rain intensity is utilized to determine the appropriate latent representation, which corresponds to the assigned range of rain intensity).
RADHA further teaches “wherein the estimated error distribution is based on the pre-identified error and the feature that the correlated latent representation is mapped to” ([0029-0030, Figure 3], As outlined above, the range of rain intensities associated with the activated classifier corresponds to the pre-identified error and the rain intensity associated with the latent representation corresponds to the feature. As outlined above, the normalized Average Precision is encompassed by the BRI of an estimated error distribution).
Claim 12
Claim 12 depends from claim 8 and recites a non-transitory, computer-readable medium that corresponds to the limitations of claims 4 and 6, and therefore claim 12 is rejected under the same rationale as outlined above for claims 4, 6, and 8 for being substantially similar, mutatis mutandis.
Claims 5, 7, 10, 13, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of RADHA and ALHARBI, in view of US 2017/0357029 A1, referenced herein as LAKSHMANAN.
Claims 5, 10, and 18
The combination of RADHA and ALHARBI teaches claims 5, 10, and 18 as follows:
RADHA further teaches “train a negative effect latent representation (NELR) module using a machine learning algorithm” ([0049], The deraining algorithms correspond to the machine learning algorithm).
RADHA further teaches “and train a [[negative effect latent representation (NELR)]] module using a machine learning algorithm, the plurality of pre-identified latent representations, and the plurality of ground truth labels” ([0043], As outlined above, the level of rain intensity simulated in the provided data corresponds to the ground truth label and the assigned range of rain intensity corresponds to the pre-identified latent representations).
RADHA further teaches “wherein the processor is configured to identify the correlated latent representation of the correctable negative effect within the autonomous vehicle sensor data using the NELR module” ([0049], As outlined above, the sensor data corresponds to the image data and the latent representation corresponds to the assigned range of rain intensity).
The combination of RADHA and ALHARBI does not appear to explicitly disclose “train a negative effect latent representation (NELR) module using a machine learning algorithm, the plurality of pre-identified latent representations, and the plurality of ground truth labels”.
Analogous art LAKSHMANAN provides this additional functionality by teaching “train a negative effect latent representation (NELR) module using a machine learning algorithm, the plurality of pre-identified latent representations, and the plurality of ground truth labels” ([0175], The training rain gauge data corresponds to the ground truth label and the predicted rain level corresponds to the latent representation).
RADHA and LAKSHMANAN are analogous art because they are from the same field of endeavor as the claimed invention, namely prediction leveraging machine learning. The combination of ALHARBI and RADHA does not appear to distinctly disclose to train a negative effect latent representation (NELR) module using a machine learning algorithm, the plurality of pre-identified latent representations, and the plurality of ground truth labels as taught by LAKSHMANAN. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have improved upon the machine learning system of the combination of RADHA and ALHARBI with LAKSHMANAN’s rainfall prediction model because “Unfortunately, rain gauges have many limitations… Due to the shortcomings of rain gauges, meteorologists have turned into [sic] other tools” ([0004]-[0005], LAKSHMANAN), as suggested by LAKSHMANAN.
Claims 7, 13, and 20
The combination of RADHA, ALHARBI, and LAKSHMANAN teaches claims 7, 13, and 20 as follows:
RADHA further teaches “generate training task data using the training sensor data in accordance with the perception task” ([0027], As outlined above, RADHA augments the test frames, which correspond to the training sensor data, to generate the training task data).
RADHA further teaches “determine a plurality of pre-identified errors of an algorithm corresponding to the perception task using a loss function, the training task data, [[and the plurality of ground truth labels]]” ([0049], As discussed above, the pre-identified errors correspond to the latent representations of the assigned ranges of rain intensity. Training a CNN includes a loss function and training data. The image data (comprised of test frames with synthesized rain) corresponds to the training task data).
LAKSHMANAN teaches “and the plurality of ground truth labels” ([0175], The training rain gauge data corresponds to the ground truth label).
RADHA further teaches “and train an [[error estimation (EE)]] module using a machine learning algorithm, the training task data, the plurality of pre-identified latent representations, and the plurality of pre-identified errors” ([0043], As outlined above, the level of rain intensity simulated in the provided data corresponds to the ground truth label).
RADHA further teaches “wherein the processor is configured to estimate the error distribution for the task data using the [[EE]] module” ([0048], When activated, the classifier performs prediction via object detection which includes determining an error distribution for the predictions).
LAKSHMANAN teaches “and train an error estimation (EE) module using a machine learning algorithm, the training task data, the plurality of pre-identified latent representations, and the plurality of pre-identified errors” ([0175], The training rain gauge data corresponds to the ground truth label and the predicted rain level corresponds to the latent representation and error).
LAKSHMANAN further teaches “using the EE module” ([0002], LAKSHMANAN: “to using a computer programmed with a supervised neural network”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Examiner or Mr. GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/ Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights