DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement – Only Partial Compliance Noted
The information disclosure statement (IDS) submitted on 04/10/2025 is only in partial compliance with the provisions of 37 CFR 1.97. Examiner has reviewed all cases cited therein, however, a copy of the cited foreign case of Safeai (AU2021313775A1) was not included in the docket. Rather, a WPO case with the same applicant was copied to the docket instead.
Appropriate correction is required.
Status of Claims
This Office action is in reply to filing by applicant on 04/10/2025.
Claims 1 – 15 are currently pending and have been examined.
THIS ACTION IS MADE NON-FINAL
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Examiner notes that claim 12 (initial claims of 4/10/2025) sets forth:
A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method of claim 1.
The above bulleted claim constitutes structureless pure signals consisting of transitory code / executable instructions (i.e., software) normally designed to be uploaded into a generic computer.
When a claim covers signals per se as above, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter) and Interim Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C. §101, Aug. 24, 2009; p. 2.
Appropriate correction is required.
Independent claims 1, 8 and 10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis of Independent claims 1, 8 and 10:
A method for determining a road surface condition, the method comprising:
obtaining first data, wherein the first data comprises a representation of the road surface and wherein the first data originates from a sensor of a first type,
obtaining second data, wherein the second data comprises a representation of the road surface and wherein the second data originates from a sensor of a second type,
generating third data by applying a feature extraction technique on the first data,
generating fourth data by applying a feature extraction technique on the second data,
generating fifth data by fusing the third data and the fourth data, and
determining the road surface condition by classifying the fifth data in at least one class of a set of predefined classes.
101 Analysis - Step 1: Statutory category – Yes
The subject claims recite a method (process, claim 1), a method (process, claim 8) and a vehicle (machine, claim 10). These claims all fall within one of the four statutory categories. MPEP 2106.03
101 Analysis - Step 2A Prong one evaluation: Judicial Exception – Yes – Mental processes
In Step 2A, Prong one of the 2019 Patent Eligibility Guidance (PEG), a claim is to be analyzed to determine whether it recites subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) mental processes, and/or c) certain methods of organizing human activity.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the limitations can be “performed in the human mind, or by a human using a pen and paper”. See MPEP 2106.04(a)(2)(III).
The claim recites the limitations of:
obtaining first data, … comprises a representation of the road surface … obtaining second data, … comprises a representation of the road surface … generating third data by applying a feature extraction … generating fourth data by applying a feature extraction … generating fifth data by fusing the third data and the fourth data, … determining the road surface condition by classifying the fifth data in at least one class of a set of predefined classes. These limitations, as drafted, and under their broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of being performed using processors and sensors. That is, other than the processor and sensors, nothing in the claim elements preclude the steps from practically being performed in the mind. For example, the claim encompasses a person thinking about one, then another, oncoming, viewable, road surface conditions whilst driving, then classifying the combination of these two surfaces conditions as, for example, dangerous (say the two road surface conditions viewed by driver are rain and snow). The mere nominal recitations of processors and sensors do not take the claim limitations out of the mental process grouping. Thus, the claims recite a mental process.
101 Analysis - Step 2A Prong two evaluation: Practical Application – No
In Step 2A, Prong two of the 2019 PEG, a claim is to be evaluated whether, as a whole, it integrates the recited judicial exception into a practical application. As noted in MPEP 2106.04(d), it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. The courts have indicated that additional elements such as: merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
The Office submits that the foregoing underlined limitations recite additional elements that do not integrate the recited judicial exception into a practical application.
The independent claims 1, 8, and 10 recite additional elements or steps of using a processor / sensors to obtain first data, … comprises a representation of the road surface … obtaining second data, … comprises a representation of the road surface … generating third data by applying a feature extraction … generating fourth data by applying a feature extraction … generating fifth data by fusing the third data and the fourth data, … determining the road surface condition by classifying the fifth data in at least one class of a set of predefined classes. These additional elements (i.e., the processors / sensors) are recited at a high level of generality (i.e. as a general means for viewing something ahead of the vehicle) and amount to mere data gathering, which is a form of insignificant extra-solution activity. Moreover, these limitations merely describe generally “applying” the otherwise mental judgements using a generic or general-purpose processors / sensors, as noted above. The processor / sensors are recited at a high level of generality and they merely automate the several determining, obtaining, obtaining, generating, generating, generating, determining, causing, and determining steps.
Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
101 Analysis - Step 2B evaluation: Inventive concept - No
In Step 2B of the 2019 PEG, a claim is to be evaluated as to whether the claim, as a whole, amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception using generic devices, processors, memories, and/or generic computer-readable media, cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B. Here, the several determining, obtaining, obtaining, generating, generating, generating, determining, causing, and determining steps were considered to be insignificant extra-solution activity in Step 2A, and thus they are re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. There is nothing in the disclosure that recites that the processors / sensors are anything other than a conventional, generic, computer and/or computer controlled components. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere collection or receipt of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). Further, the Federal Circuit in Trading Techs. Int’l v. IBG LLC, 921 F.3d 1084, 1093 (Fed. Cir. 2019), and Intellectual Ventures I LLC v. Erie Indemnity Co., 850 F.3d 1315, 1331 (Fed. Cir. 2017), for example, indicated that the mere displaying of data is a well understood, routine, and conventional function. Accordingly, a conclusion that the above underlined several elements / steps of determining, obtaining, obtaining, generating, generating, generating, determining, causing, and determining amount to well-understood, routine, conventional activity and are supported under Berkheimer.
Thus, independent claims 1, 8, and 10 are ineligible.
Dependent Claims
Dependent claims 2 – 7, 9, 11 - 13 and 15 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of these dependent claims are directed toward additional aspects of the judicial exception. As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer component(s). The same analysis applies here in 2B, i.e., mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Dependent claims 2 – 7, 9, 11 - 13 and 15 are not patent eligible under the same rationale as provided for in the above rejection of independent claims 1, 8, and 10.
Given the above analyses, all claims 1 – 15 are ineligible under 35 USC §101.
Claim Rejections – 35 USC 103
In the event the determination of the status of the application as subject to AIA 35 USC 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office Action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 USC 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
Claims 1 – 15 are rejected pursuant to 35 USC 103 as being unpatentable over Zhao (US20220032946A1) in view of Naseer (US20220083792A1).
Regarding claim 8 (reads on claims 1 and 10):
A method for controlling a vehicle, the method comprising:
Zhao discloses:
determining a road surface condition using a method comprising: obtaining first data, wherein the first data comprises a representation of the road surface and wherein the first data originates from a sensor of a first type, (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LIDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “first type” to detect a road surface may be a camera according to the Specification herein (see “26. [0025]”);
obtaining second data, wherein the second data comprises a representation of the road surface and wherein the second data originates from a sensor of a second type, (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LiDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “second type” to detect a road surface may be a LIDAR sensor according to the Specification herein (see “26. [0025]”);
determining the road surface condition by classifying the fifth data in at least one class of a set of predefined classes, and (“generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [Abstract, published 2/3/2022]);
causing adjustment of a driving parameter and/or causing a warning based on the determined road surface condition. (“In addition, in certain embodiments, control of vehicle movement may also be automatically be implemented and/or adjusted based on the road surface type 126, based on instructions provided by the processor 142.”, [083]).
Zhao does not expressly disclose, but Naseer teaches:
generating third data by applying a feature extraction technique on the first data, (“extracting, with the aid of a first neural network situated in the physical system, at least one defined object from the first and second surroundings sensor data into first extracted data; and extracting, with the aid of a second neural network situated in the physical system, characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.”, [009]);
generating fourth data by applying a feature extraction technique on the second data, (“extracting, with the aid of a first neural network situated in the physical system, at least one defined object from the first and second surroundings sensor data into first extracted data; and extracting, with the aid of a second neural network situated in the physical system, characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.”, [009]);
generating fifth data by fusing the third data and the fourth data, and (“A map representation of the surroundings of the vehicle may be achieved by combining multiple sets of surroundings sensor data that describe various sections of the surroundings, so that a continuous, contiguous map representation of the surroundings, made up of a plurality of combined sets of surroundings sensor data of adjoining sections of the surroundings of the vehicle, may be achieved.”, [033]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Zhao to incorporate the teachings of Naseer because Zhao would be more efficient and versatile should it employ feature extraction as done in Naseer, to wit, … “types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method”, Naseer at [024]).
Regarding independent claim 14:
Zhao discloses:
wherein the first data comprises a representation of a road surface and (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LIDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “first type” to detect a road surface may be a camera according to the Specification herein (see “26. [0025]”);
wherein the first data originates from a sensor of a first type (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LIDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “first type” to detect a road surface may be a camera according to the Specification herein (see “26. [0025]”);
wherein the second data comprises a representation of a road surface and (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LIDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “first type” to detect a road surface may be a camera according to the Specification herein (see “26. [0025]”);
wherein the second data originates from a sensor of a second type, (“In accordance with an exemplary embodiment, a method for controlling a vehicle action based on a condition of a road on which a vehicle is travelling is provided, the method including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [004]) and (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LIDAR, ultrasound, cameras, and/or other types of detection sensors). It will similarly be appreciated that the number and/or placement of the detection sensors 124 may vary in different embodiments.”, [044], examiner notes that a vehicle mounted sensor of the “first type” to detect a road surface may be a camera according to the Specification herein (see “26. [0025]”);
providing a fourth artificial neural network configured for classifying the fifth data in at least one of a set of predefined classes, and (“generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [Abstract, published 2/3/2022]);
training the combination of the first artificial neural network, the second artificial neural network, the third artificial neural network, and the fourth artificial neural network in an end-to-end manner (“With reference back to FIG. 2, in various embodiments, the fused image 222 of step 220 is utilized (in various iterations) in connection with a neural network classifier at step 224. In various embodiments, the processor 142 utilizes a convolutional neural network (CNN) model 154 stored in the memory 144 of FIG. 1 in multiple iterations to classify the road surface depicted in images from multiple detection sensors 124 of FIG. 1. In various embodiments, the CNN model utilizes training data in which predicted truth is compared against the ground truth with respect to a condition of the road. Also in various embodiments, the road conditions are “dry”, “wet”, and “snow” with respect to the road. In certain embodiments, other possible types of road surfaces and/or other conditions may also be utilized. For example, in certain embodiments, the road conditions may also comprise one or more other surface conditions for the road, such as whether the road surface comprises asphalt, concrete, gravel, dirt, and so on, among other possible surface conditions. In addition, in various embodiments, a training dataset is utilized to build a confusion matrix utilizing various values of “ground truth dry”, “ground truth wet”, and “ground truth snow” in comparison with “values of “predicted dry”, “predicted wet”, and “predicted snow” values (and/or similar comparisons for other road surface conditions), for training the CNN model.”, [080]), examiner notes that the above noted training “end to end” includes the meaning that said neural networks are trained together (see Specification herein at 35. [029]
using training data comprising first data annotated with at least one of the predefined classes and Examiner interprets this limitation to include the meaning that “training data” is utilized with classed sensor data, … (“For example, in certain embodiments, the road conditions may also comprise one or more other surface conditions for the road, such as whether the road surface comprises asphalt, concrete, gravel, dirt, and so on, among other possible surface conditions. In addition, in various embodiments, a training dataset is utilized to build a confusion matrix utilizing various values of “ground truth dry”, “ground truth wet”, and “ground truth snow” in comparison with “values of “predicted dry”, “predicted wet”, and “predicted snow” values (and/or similar comparisons for other road surface conditions), for training the CNN model.”, [080]);
comprising second data annotated with at least one of the predefined classes. Examiner interprets this limitation to include the meaning that “training data” is utilized with classed sensor data, … (“For example, in certain embodiments, the road conditions may also comprise one or more other surface conditions for the road, such as whether the road surface comprises asphalt, concrete, gravel, dirt, and so on, among other possible surface conditions. In addition, in various embodiments, a training dataset is utilized to build a confusion matrix utilizing various values of “ground truth dry”, “ground truth wet”, and “ground truth snow” in comparison with “values of “predicted dry”, “predicted wet”, and “predicted snow” values (and/or similar comparisons for other road surface conditions), for training the CNN model.”, [080]).
Zhao does not expressly disclose, but Naseer teaches:
providing a first artificial neural network configured for generating third data by applying a feature extraction technique on first data, (“extracting, with the aid of a first neural network situated in the physical system, at least one defined object from the first and second surroundings sensor data into first extracted data; and extracting, with the aid of a second neural network situated in the physical system, characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.”, [009]);
providing a second artificial neural network configured for generating fourth data by applying a feature extraction technique on second data, (“extracting, with the aid of a first neural network situated in the physical system, at least one defined object from the first and second surroundings sensor data into first extracted data; and extracting, with the aid of a second neural network situated in the physical system, characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.”, [006]);
providing a third artificial neural network configured for generating fifth data by fusing the third data and the fourth data, (“a first neural network that is configured to extract at least one defined object from first and second surroundings sensor data into first extracted data, the surroundings sensor data capturing the surroundings in an at least partially overlapping manner, first surroundings sensor data including three-dimensional information, and second surroundings sensor data including two-dimensional information; and a second neural network that is configured to extract characteristic features including descriptors from the first extracted data into second extracted data, the descriptors being provided for a defined alignment of the second extracted data in a map creation process.”, [012])
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Zhao to incorporate the teachings of Naseer because Zhao would be more efficient and versatile should it employ feature extraction as done in Naseer, to wit, … “types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method”, Naseer at [024]).
Regarding claim 15:
The combination of Zhao and Naseer disclose the limitations of claim 14:
Zhao further discloses:
wherein training the combination of the first artificial neural network, the second artificial neural network, the third artificial neural network, and the fourth artificial neural network in an end-to-end manner comprises back propagation. Examiner notes that training the neural networks in an “end-to-end” manner, per the instant Specification at (36. [0030]), includes the meaning that sensor parameters may be adjusted together during the training of the several neural networks, moreover, “back propagation” (undefined in entire disclosure) also includes the meaning that said sensor parameters can be adjusted in hindsight, … (“In addition, in certain embodiments, control of vehicle movement may also be automatically be implemented and/or adjusted based on the road surface type 126, based on instructions provided by the processor 142. For example, in certain embodiments, automatic braking, automatic steering, and/or one or more other vehicle functions may be implemented based at least in part on the road surface type 126 in accordance with instructions provided by the processor 142. By way of continued example, one or more automatic braking thresholds and/or automatic steering thresholds (e.g., pertaining to a distance to object or time to object, and so on) may be adjusted based on the road surface type 126,”, [083]).
Regarding claim 2:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Naseer further discloses:
wherein the first data comprises image data representing the road surface and/or wherein the second data comprises point cloud data representing the road surface. (“A further advantageous refinement of the method in accordance with the present invention provides that the first neural network is a deep convolutional neural network and the second neural network is a point cloud-based neural network.”, [024]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Zhao to incorporate the teachings of Naseer because Zhao would be more efficient and versatile should it employ feature extraction as done in Naseer, to wit, … “types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method”, Naseer at [024]).
Regarding claim 3:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Naseer further discloses:
wherein generating third data by applying a feature extraction technique comprises using an artificial neural network and/or wherein generating fourth data by applying a feature extraction technique comprises using an artificial neural network. (“A further advantageous refinement of the method in accordance with the present invention provides that the first neural network is a deep convolutional neural network and the second neural network is a point cloud-based neural network. In this way, types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method “, [024]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Zhao to incorporate the teachings of Naseer because Zhao would be more efficient and versatile should it employ feature extraction as done in Naseer, to wit, … “types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method”, Naseer at [024]).
Regarding claim 4:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Zhao further discloses:
wherein generating fifth data by fusing the third data and the fourth data comprises using an artificial neural network and/or wherein determining the road surface condition by classifying the fifth data comprises using an artificial neural network. (“classifying, via a processor using a neural network model, the condition of the road on which the vehicle is travelling, based on the measured parameter and the plurality of road surface channel images; and controlling a vehicle action based on the classification of the condition of the road.”, [Abstract, published 2/3/2022]).
Regarding claim 5:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Naseer further discloses:
limiting the first data to a representation of a sub-section of the road surface and/or further comprising limiting the second data to a representation of a sub-section of the road surface. (“The detected surroundings sensor data represent descriptions of different sections of the surroundings of the vehicle, and are used for creating a map representation of the surroundings. A map representation of the surroundings of the vehicle may be achieved by combining multiple sets of surroundings sensor data that describe various sections of the surroundings, so that a continuous, contiguous map representation of the surroundings, made up of a plurality of combined sets of surroundings sensor data of adjoining sections of the surroundings of the vehicle, may be achieved.”, [033]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of this application to have modified Zhao to incorporate the teachings of Naseer because Zhao would be more efficient and versatile should it employ feature extraction as done in Naseer, to wit, … “types of neural networks that are particularly well suited in each case for the specific tasks of the feature extraction are advantageously used. For extracting the characteristic features with the aid of the second neural network, it is possible to utilize, for example, the above-mentioned method”, Naseer at [024]).
Regarding claim 6:
The combination of Zhao and Naseer disclose the limitations of claim 5:
Zhao further discloses:
wherein the method is executed repeatedly or at least twice in parallel, wherein in each execution of the method the first data and/or the second data is limited to a representation of a different sub-section of the road surface. (“Furthermore, in various embodiments, during step 214, each data cluster is associated with each of the following above-described factors; namely: (i) return energy, (ii) Doppler; (iii) (x,y,z) coordinate; and (iv) sensor index. In addition, in certain embodiments, the energy value, energy distribution, Z value, and Doppler value all show different patterns on different road surface conditions.”, [065]).
Regarding claim 7:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Zhao further discloses:
wherein the predefined classes comprise one or more of a first class relating to a dry road surface, a second class relating to a wet road surface, a third class relating to a slushy road surface, a fourth class relating to a snowy road surface, and a fifth class relating to an icy road surface. (“Also in various embodiments, the road conditions are “dry”, “wet”, and “snow” with respect to the road. In certain embodiments, other possible types of road surfaces and/or other conditions may also be utilized. For example, in certain embodiments, the road conditions may also comprise one or more other surface conditions for the road, such as whether the road surface comprises asphalt, concrete, gravel, dirt, and so on, among other possible surface conditions. In addition, in various embodiments, a training dataset is utilized to build a confusion matrix utilizing various values of “ground truth dry”, “ground truth wet”, and “ground truth snow” in comparison with “values of “predicted dry”, “predicted wet”, and “predicted snow” values (and/or similar comparisons for other road surface conditions), for training the CNN model.”, [080]).
Regarding claim 9:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Zhao further discloses:
A data processing apparatus comprising means for carrying out the method of claim 1. (“Methods and systems are provided for controlling a vehicle action based on a condition of a road on which a vehicle is travelling, including: obtaining first sensor data as to a surface of the road from one or more first sensors onboard the vehicle; obtaining second sensor data from one or more second sensors onboard the vehicle as to a measured parameter pertaining to operation of the vehicle or conditions pertaining thereto; generating a plurality of road surface channel images from the first sensor data, wherein each road surface channel image captures one of a plurality of facets of properties of the first sensor data; classifying, via a processor using a neural network model,”, [Abstract, published 2/3/2022]).
Regarding claim 11:
The combination of Zhao and Naseer disclose the limitations of claim 10:
Zhao further discloses:
wherein one of the sensor of the first type and the sensor of the second type is an optical camera and the respective other one of the sensor of the first type and the sensor of the second type is a lidar sensor. (“However, in various other embodiments, the types of detection sensors 124 may vary, and for example may comprise one or more different types of radar and/or one or more other types of sensors (e.g., which may include sonar, LiDAR, ultrasound, cameras, and/or other types of detection sensors).”, [044]).
Regarding claim 12:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Zhao further discloses:
A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method of claim 1. Note the above express 35 USC 101 rejection of this claim, … (“During operation, the processor 142 executes one or more programs 152 contained within the memory 144 and, as such, controls the general operation of the controller 140 and the computer system of the controller 140, generally in executing the processes described herein,”, [047]).
Regarding claim 13:
The combination of Zhao and Naseer disclose the limitations of claim 1:
Zhao further discloses:
A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 1. (“It will be appreciated that while this exemplary embodiment is described in the context of a fully functioning computer system, those skilled in the art will recognize that the mechanisms of the present disclosure are capable of being distributed as a program product with one or more types of non-transitory computer-readable signal bearing media used to store the program and the instructions thereof and carry out the distribution thereof, such as a non-transitory computer readable medium bearing the program and containing computer instructions stored therein for causing a computer processor (such as the processor 142) to perform and execute the program.”, [052]).
CONCLUSION
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see attached form 892.
Burlina (US20250076486A1) - Vehicle computing systems may receive and fuse long-wave infrared data with radar data to detect and track objects in low-visibility driving environments. In some examples, radar data including position and velocity data may be projected over infrared image data to detect, classify, and track infrared-emitting objects. Machine-learned transformer models with attention also may be trained to output object detections based on combined infrared and radar data. The fusion of infrared and radar data may be used individual and/or may be synchronized with other sensor modalities. In some examples, the fusion and analysis of infrared and radar data may be used in specific low-visibility driving environments, using low-light, fog, and shadowed areas of the environment, to detect and track infrared-emitting and/or moving objects such as pedestrians and animals that may be obscured from detection by other sensor modalities.
Lu (US11788846B2) - Systems, methods, and non-transitory computer-readable media can determine sensor data captured by at least one sensor of a vehicle while navigating a road segment. A plurality of features describing the road segment can be extracted from the sensor data. A map representation of the road segment can be determined based at least in part on the sensor data and the plurality of features extracted from the sensor data, the map representation being determined as the vehicle navigates the road segment. While the map representation of the road segment is being determined, at least one scenario associated with the road segment can be determined based at least in part on the map representation and the plurality of features extracted from the sensor data.
Wang (US20230135234A1 - In various examples, to support training a deep neural network (DNN) to predict a dense representation of a 3D surface structure of interest, a training dataset is generated from real-world data. For example, one or more vehicles may collect image data and LiDAR data while navigating through a real-world environment. To generate input training data, 3D surface structure estimation may be performed on captured image data to generate a sparse representation of a 3D surface structure of interest (e.g., a 3D road surface). To generate corresponding ground truth training data, captured LiDAR data may be smoothed, subject to outlier removal, subject to triangulation to filling missing values, accumulated from multiple LiDAR sensors, aligned with corresponding frames of image data, and/or annotated to identify 3D points on the 3D surface of interest, and the identified 3D points may be projected to generate a dense representation of the 3D surface structure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW COBB whose telephone number is (571) 272-3850. The examiner can normally be reached 9 - 5, M - F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to call examiner Cobb as above, or to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Nolan, can be reached at (571) 270-7016. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at (866) 217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/MATTHEW COBB/Examiner, Art Unit 3661
/PETER D NOLAN/Supervisory Patent Examiner, Art Unit 3661
Prosecution Insights