Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
Providing supporting paragraph(s) for each limitation of amended/new claim(s) in Remarks is strongly requested for clear and definite claim interpretations by Examiner (e.g., to avoid rejections under 35 U.S.C § 112(a) “Lack of written description”)
Applicant can schedule interviews (via Automated Interview Request (AIR)) at any stage of the prosecution (e.g., Non-Final, Final, and After-Final) to discuss any issues related to, for example, rejections under 35 U.S.C § 101 and § 102/103, for moving toward allowance.
If a limitation has bold brackets (i.e. [·]) around claim languages, the bracketed claim languages indicate that they have not been taught yet by the current prior art reference but they will be taught by another prior art reference afterwards.
If a limitation has one or more bold underlines, the one or more bold underlined claim languages indicate that they are taught by the current prior art reference, while the one or more non-underlined claim languages indicate that they have been taught already by one or more previous art references.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No EP21161805.3, filed on 03/10/2021.
Response to Arguments
Applicant's arguments filed on 06/05/2026 have been fully considered but they are not persuasive.
In Remarks, regarding 35 USC § 101, Applicant contends:
“In particular, paragraphs [0013] - [0020] of the published specification describes a technical problem of being unable to properly classify a vehicle based on their driving behavior resulting in a lack of reliably identity management for vehicles which can lead to security concerns. The present applications overcomes this technical problem by classifying vehicles according to the nature of their vehicle drivers in an efficient and reliable classification method. See published specification, paragraphs [0025] and [0026].”
“the technical problems identified in the specification include problems of machine learning-based classification of vehicles based on driver behavior. This problem is solved by training a local prediction model, combining local prediction models to generate a combined prediction model, and classifying the vehicle into a definable vehicle class using the combined prediction model. Claim 1 reflects this improvement by reciting the above amended features.”
“Accordingly, because Applicant believes the claims are novel and non-obvious and recite a combination of features which are unconventional and go beyond what is well-understood, routine, or conventional, the claims recite an inventive concept, this ensures that the claim, as a whole, amounts to significantly more than the alleged abstract idea.”
Examiner’s response:
The examiner understands the applicant’s assertion.
However, it appears that each processing step is just applying the abstract idea to a general field of endeavor with additional elements. In addition, improvements to technology or technical field are not necessarily reflected in the claims. Thus, the claim does not integrate the judicial exception into a practical application, and the claim does not amount to significantly more than the judicial exception.
The examiner understands the applicant’s assertion regarding abstract ideas.
However, as rejected under Claim Rejections - 35 USC § 101, the claims can be interpreted as combinations of abstract ideas and additional elements. The applicant may need to amend the claims with more specificities to provide improvements.
The examiner understands the applicant’s assertion “In particular, paragraphs [0013] - [0020] of the published specification describes a technical problem of being unable to properly classify a vehicle based on their driving behavior resulting in a lack of reliably identity management for vehicles which can lead to security concerns. The present applications overcomes this technical problem by classifying vehicles according to the nature of their vehicle drivers in an efficient and reliable classification method. See published specification, paragraphs [0025] and [0026].”
It appears that inventive concepts to prevent security concerns could be technical improvements as long as the inventive concepts are supported from the specification. However, currently, it is not clear how the claimed invention reflects the asserted improvements. Amending claims to reflect the asserted improvements may help overcome the existing rejections.
The examiner understands the applicant’s assertion “the technical problems identified in the specification include problems of machine learning-based classification of vehicles based on driver behavior. This problem is solved by training a local prediction model, combining local prediction models to generate a combined prediction model, and classifying the vehicle into a definable vehicle class using the combined prediction model. Claim 1 reflects this improvement by reciting the above amended features.”
As explained, the claim 1 has some steps of training, combining, classifying. However, their specificities are not enough yet and the claims are in a high-level yet. For example, it is not clear how to train the local prediction model and it is just using local driving data. In addition, it is not clear how to combine the modes, and it is not clear how to locally classify data. It appears that the claims are described in a high-level without enough specificities.
The examiner understands the applicant’s assertion “Accordingly, because Applicant believes the claims are novel and non-obvious and recite a combination of features which are unconventional and go beyond what is well-understood, routine, or conventional, the claims recite an inventive concept, this ensures that the claim, as a whole, amounts to significantly more than the alleged abstract idea.”
However, the limitations do not clearly show e.g., improvements in computer technology and improvements to other technical fields. Rather, the improvements in Remarks are about just improving the abstract ideas of the independent claims. It doesn’t seem that the specification and/or the independent claims clearly show how the inventive concept of the claims enables improvements and how they are tied together. The applicant may need to amend the claims to show how the claim languages and improvements are tied together.
To find a valid improvement to a technology, MPEP 2106.04(d)(1) says the specification must explain the improvement and that the claim must reflect the disclosed improvement. Furthermore, the improvement should not be merely a consequence of the abstract idea. See MPEP 2106.05(a). An improvement in the abstract idea itself is not an improvement to technology.
For at least these reasons, Applicant's arguments are not convincing.
Applicant’s arguments regarding 35 USC § 102/103 with respect to the independent claims have been considered but are moot because the arguments are directed to amended limitation(s) that has/have not been previously examined.
Claim Objections
Claim(s) 6 is/are objected to because of the following informalities: it appears that “the model” needs to read “a model” or something else. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “local” (claim 1, line 5) is a relative term which renders the claim indefinite. The term “local” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In addition, claim(s) 1 (7 times more including “locally”) 3, 5, 6, 7, 8, 9, 10, 14, 15 (8 times), 16, 17 is/are rejected for the same reason.
Claim(s) 6 recite(s) the limitation “the higher accuracy score” (line 2). There is insufficient antecedent basis for this limitation in the claim. It is not clear what it is referring to. It appears it may need to read “a higher accuracy score”, or something else. For the purposes of examination, “a higher accuracy score” is used.
Claim(s) 9 recite(s) the limitation “their associated confidence estimates” (line 3). There is insufficient antecedent basis for this limitation in the claim. It is not clear what “their” is referring to, since it may indicate any plural forms. It appears it may need to read “confidence estimates associated with classifications”, or something else. For the purposes of examination, “confidence estimates associated with classifications” is used.
Claim(s) 14 recite(s) the limitation “the vehicles” (line 2). There is insufficient antecedent basis for this limitation in the claim. It is not clear what it is referring to, since it may indicate “vehicles” (claim 1, line 1) or “vehicles” (claim 1, line 4), “one or more vehicles” (claim 1, line 6) or something else. It appears it may need to read “vehicles”, or something else. For the purposes of examination, “vehicles” is used.
Claim(s) 1, 3, 5-10, 14-17 each recite(s) limitations that raise issues of indefiniteness as set forth above, and their dependent claims are rejected at least based on their direct and/or indirect dependency from the claims listed above. Appropriate explanation and/or amendment is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“… for classifying vehicles … according to a nature of their vehicle drivers, the method comprising:
…;
…; and
locally classifying at least one of the other vehicles in the predefined local area into a definable vehicle class …”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“by a data processing system”, “using the combined prediction model”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
In particular, the claim recites an additional element(s) (“training a local prediction model by learning a driving policy of a vehicle driving in a predefined local area from driving data of the vehicle collected in a predefined time window, such that the local prediction model is trained to generate a prediction of a definable driver behavior of the vehicle over a definable time horizon”). The additional element is recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
In particular, the claim recites an additional element(s) (“receiving a combined prediction model that was generated by combining the local prediction model of the vehicle with local prediction models of other vehicles in the local area”) – the act of receiving data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of receiving data is recited at a high-level of generality (i.e., as a generic act of receiving performing a generic act function of receiving data) such that it amounts no more than a mere act to apply the exception using a generic act of receiving. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
The additional elements regarding training are recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
As discussed above, the claim recites the additional element(s) of receiving data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 2
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the vehicle class provides information on whether the at least one of the other vehicles is autonomously-driven or human-driven”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 3
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the driving data is collected from at least one sensor or onboard sensor of the vehicle within the predefined local area and/or from at least one road or environment infrastructure sensor”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 4
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the driving data comprises abstract data features and/or synthetized data features”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 5
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“wherein a proprietary implementation of a vehicle or autonomous vehicle is preserved in the training of the local prediction model by learning the driving policy”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim does not recite additional elements. Thus, the claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Regarding claim 6
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“determining an accuracy score of the local prediction model and an accuracy score of the combined prediction model”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the vehicle class is taken from the model having the higher accuracy score”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 7
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“wherein the local classifications of the at least one of the other vehicles are shared with the other vehicles.”) – the act of transmitting data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of transmitting data is recited at a high-level of generality (i.e., as a generic act of performing a generic act function of transmitting data) such that it amounts no more than a mere act to apply the exception using a generic act of transmitting. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of transmitting data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 8
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“determining confidence estimates associated with the local classifications”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“sharing the confidence estimates with the other vehicles”) – the act of transmitting data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of transmitting data is recited at a high-level of generality (i.e., as a generic act of performing a generic act function of transmitting data) such that it amounts no more than a mere act to apply the exception using a generic act of transmitting. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of transmitting data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 9
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
The limitations of
“globally classifying one or more of the other vehicles by combining outputs of the local prediction models of the other vehicles and/or their associated confidence estimates”, as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, nothing in the claim element precludes the step from practically being performed in the mind. For example, the limitations in the context of this claim encompass the user mentally thinking with a physical aid (e.g., pencil and paper).
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim does not recite additional elements. Thus, the claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Regarding claim 10
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“sending to traffic authorities systems the local classification of the at least one of the other vehicles”) – the act of transmitting data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of transmitting data is recited at a high-level of generality (i.e., as a generic act of performing a generic act function of transmitting data) such that it amounts no more than a mere act to apply the exception using a generic act of transmitting. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of transmitting data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 11
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the method is performed on one or more vehicles and/or at one or more external or edge data processing systems”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 12
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the method is performed as a machine learning approach”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 13
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the method is performed with computing servers communicating via direct links, through a cloud backend and/or through a connected, cooperative automated mobility platform (CCAM)”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 14
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“wherein in the method … train a neural network as the local prediction model and update weights …”). The additional element is recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“the vehicles”, “on assigned servers or edge computing servers”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional elements regarding training are recited at such a high level without any details as to how a model is trained such that it amounts to only the idea of a solution or outcome because it fails to recite details of how a solution to a problem is accomplished, and, therefore, represents no more than mere instructions to apply the judicial exception on a computer (see MPEP 2106.05(f)). Accordingly, this additional element does not amount to significantly more than the abstract idea. The claim is directed to an abstract idea.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 15
The claim recites “A system for classifying vehicles by a data processing system according to a nature of their vehicle drivers, the system comprising:” to perform precisely the method of Claim 1. As performance of an abstract idea on generic computer components (see MPEP 2106.05(f)) cannot integrate the abstract idea into a practical application nor provide significantly more than the abstract idea itself, the claim is rejected for reasons set forth in the rejection of Claim 1.
Regarding claim 16
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 7.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“wherein the local classifications are shared with the other vehicles for combining the local classifications”) – the act of transmitting data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of transmitting data is recited at a high-level of generality (i.e., as a generic act of performing a generic act function of transmitting data) such that it amounts no more than a mere act to apply the exception using a generic act of transmitting. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of transmitting data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 17
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 8.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element(s) (“wherein the confidence estimates associated with local classifications are shared with the other vehicles for combining the confidence estimates”) – the act of transmitting data. The claim is adding an insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g). The act of transmitting data is recited at a high-level of generality (i.e., as a generic act of performing a generic act function of transmitting data) such that it amounts no more than a mere act to apply the exception using a generic act of transmitting. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the claim recites the additional element(s) of transmitting data at a high-level of generality and is adding an insignificant extra-solution activity – see MPEP 2106.05(g). However, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood, routine, and conventional. See MPEP 2106.05(d)(II) – “Receiving or transmitting data over a network” or “Storing and retrieving information in memory”. Accordingly, this additional element does not provide an inventive concept and significantly more than the abstract idea. Thus, the claim is not patent eligible.
Regarding claim 18
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 9.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the one or more of the other vehicles are one or more target vehicles”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Regarding claim 19
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 12.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
The claim recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). In particular, the claim recites an additional element(s) (“wherein the machine learning approach is performed in an edge computing network with edge computing servers”) – using a device and/or a model to process data. The device and the model in each step are recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, with respect to integration of the abstract idea into a practical application, the additional elements of using a generic computer component to perform each step amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. MPEP 2106.05(f).
Regarding claim 20
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites the abstract idea identified above regarding claim 13.
Step 2A Prong 2: This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element (“wherein the computing servers comprise edge computing servers”). This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not integrate the abstract idea into a practical application. See MPEP 2106.05(h)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
This is a recitation of a particular type or source of model/data to be used in performing the abstract idea. Limiting the abstract idea to a particular type or source of model/data is an attempt to limit the abstract idea to a particular field of use or technological environment, which does not amount to significantly more than the abstract idea. See MPEP 2106.05(h).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-6, 11-15, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach) in view of Wang et al. (Fed-SCNN: A Federated Shallow-CNN Recognition Framework for Distracted Driving)
Regarding claim 1
Ye teaches
A method for classifying [vehicles] by a data processing system according to a nature of their vehicle drivers, the method comprising:
(Ye [sec(s) I] “We study federated learning in VEC to meet the rapid-growing demands of AI applications in ICV. For federated learning with image classification, a selective model aggregation approach is proposed to reduce the influence from the diversity of image quality and computation capability in vehicular clients. • A geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client is built up to evaluate the image quality in the motion blur level. According to the model, the image quality could be implicitly predicted by observing the instantaneous velocity of each vehicular client. • To tackle the information asymmetry caused by federated learning, the model selection procedure is formulated as a two-dimensional contract theory problem. The problem is successively relaxed and simplified into a tractable problem, and solved by a greedy algorithm. • Using the MNIST and BelgiumTSC datasets, the proposed selective model aggregation approach is shown to outperform the original federated averaging (FedAvg) approach in terms of the accuracy and efficiency of model aggregation. Also, our approach can achieve higher utility at the central server compared with existing baseline approaches.”;)
training a local prediction model by learning a driving policy of a vehicle driving in a predefined local area from driving data of the vehicle collected in a predefined time window, such that the local prediction model is trained to generate a prediction of a definable [driver behavior] of the vehicle over a definable time horizon;
(Ye [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “• Central Server: Central server plays a core role in the procedure of federated learning. It communicates with vehicular clients to collect the updated local DNN models and perform model aggregation. We take image classification as a typical AI application in VEC. DNN-based image classification has been widely used in autopilot and interactive navigation for ICV, as well as object tracking and event detection in ITS [17], [18]. To obtain high accuracy and efficiency of model aggregation, the central server should evaluate the image quality and computation capability of vehicular clients, and select the ‘‘fine’’ models from vehicular clients. • Vehicular Client: Vehicular clients are equipped with a set of built-in sensors, such as cameras, GPS, tachographs, lateral acceleration sensors, and also accommodate storage space, computation and communication resources [18]. The built-in sensors are used to capture images that may be preprocessed for data augment. After that, the preprocessed images are classified and labeled by automatic labeling technology [6], and are cached in vehicular clients. After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.” [sec(s) VI] “The simulation involves the public MNIST dataset [31], and the BelgiumTSC (Belgium Traffic Sign for Classification) vehicular dataset [32]”;)
receiving a combined prediction model that was generated by combining the local prediction model of the vehicle with local prediction models of other vehicles in the local area; and
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”;)
locally classifying at least one of the other [vehicles] in the predefined local area into a definable [vehicle] class using the combined prediction model.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”;)
However, Ye does not appear to explicitly teach:
A method for classifying [vehicles] by a data processing system according to a nature of their vehicle drivers, the method comprising:
generate a prediction of a definable [driver behavior] of the vehicle over a definable time horizon;
locally classifying at least one of the other [vehicles] in the predefined local area into a definable [vehicle] class using the combined prediction model.
Wang teaches
A method for classifying vehicles by a data processing system according to a nature of their vehicle drivers, the method comprising:
(Wang [fig(s) 4] “The example of dataset classification: (a) smoking, (b) drinking, (c) right-hand call, and (d) talking to passenger” [sec(s) 4] “At present, there are few open source datasets for distracted driving, and the annotation quality of the datasets is poor [30]. Therefore, the simulation experiment in this paper first built its own dataset, which mainly includes three key steps: (1) collection of images set related to the recognition of the driver distraction behavior; (2) preprocessing images in the dataset; and (3) classification of data and marking. The experimental data mainly come from open source datasets such as ImageNet [31] and Open Images [32]. To prevent overfitting, we perform preprocessing operations on the data set, such as rotation, translation, and scaling, while the image size is cropped to 224 × 224 to reduce redundant data, which facilitates SCNN analysis. The self-built data, a total of 4233 pictures, are divided into twelve different behavior categories, as shown in Table 1. In general, each image corresponds to only one category. Figure 4 is an example of dataset classification. It should be noted that some sample images may also have multiple labels, for example, the driver makes a phone call with his right hand and leaves the steering wheel with left hand, which belongs to both C3 and C12. At the same time, in order to verify the experiment, the dataset is randomly divided into two parts, the training data occupies 90%, and the rest is used for testing.”;)
generate a prediction of a definable driver behavior of the vehicle over a definable time horizon;
(Wang [fig(s) 4] “The example of dataset classification: (a) smoking, (b) drinking, (c) right-hand call, and (d) talking to passenger” [sec(s) 4] “At present, there are few open source datasets for distracted driving, and the annotation quality of the datasets is poor [30]. Therefore, the simulation experiment in this paper first built its own dataset, which mainly includes three key steps: (1) collection of images set related to the recognition of the driver distraction behavior; (2) preprocessing images in the dataset; and (3) classification of data and marking. The experimental data mainly come from open source datasets such as ImageNet [31] and Open Images [32]. To prevent overfitting, we perform preprocessing operations on the data set, such as rotation, translation, and scaling, while the image size is cropped to 224 × 224 to reduce redundant data, which facilitates SCNN analysis. The self-built data, a total of 4233 pictures, are divided into twelve different behavior categories, as shown in Table 1. In general, each image corresponds to only one category. Figure 4 is an example of dataset classification. It should be noted that some sample images may also have multiple labels, for example, the driver makes a phone call with his right hand and leaves the steering wheel with left hand, which belongs to both C3 and C12. At the same time, in order to verify the experiment, the dataset is randomly divided into two parts, the training data occupies 90%, and the rest is used for testing.”;)
locally classifying at least one of the other vehicles in the predefined local area into a definable vehicle class using the combined prediction model.
(Wang [fig(s) 4] “The example of dataset classification: (a) smoking, (b) drinking, (c) right-hand call, and (d) talking to passenger” [sec(s) 4] “At present, there are few open source datasets for distracted driving, and the annotation quality of the datasets is poor [30]. Therefore, the simulation experiment in this paper first built its own dataset, which mainly includes three key steps: (1) collection of images set related to the recognition of the driver distraction behavior; (2) preprocessing images in the dataset; and (3) classification of data and marking. The experimental data mainly come from open source datasets such as ImageNet [31] and Open Images [32]. To prevent overfitting, we perform preprocessing operations on the data set, such as rotation, translation, and scaling, while the image size is cropped to 224 × 224 to reduce redundant data, which facilitates SCNN analysis. The self-built data, a total of 4233 pictures, are divided into twelve different behavior categories, as shown in Table 1. In general, each image corresponds to only one category. Figure 4 is an example of dataset classification. It should be noted that some sample images may also have multiple labels, for example, the driver makes a phone call with his right hand and leaves the steering wheel with left hand, which belongs to both C3 and C12. At the same time, in order to verify the experiment, the dataset is randomly divided into two parts, the training data occupies 90%, and the rest is used for testing.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Ye with the vehicle classification of Wang.
One of ordinary skill in the art would have been motived to combine in order to improve the recognition accuracy and improve the efficiency of recognition.
(Wang [sec(s) 3] “The former mainly uses distributed data to build a global statistical model through DNN to improve the recognition accuracy, and at the same time, upload the major parameters under the homomorphic encryption condition. Convolutional neural network (CNN) [27], which has the advantage of image feature extraction, is responsible for extracting user-side differentiated features, that is, the personalization of the local model. In order to take into account the loT hardware level, we decided to use SCNN to meet the needs of the current cab, which can improve the efficiency of recognition. The overall framework design is shown in Figure 2, which briefly expresses the process of Fed-SCNN”)
Regarding claim 3
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein the driving data is collected from at least one sensor or onboard sensor of the vehicle within the predefined local area and/or from at least one road or environment infrastructure sensor.
(Ye [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “Vehicular Client: Vehicular clients are equipped with a set of built-in sensors, such as cameras, GPS, tachographs, lateral acceleration sensors, and also accommodate storage space, computation and communication resources [18]. The built-in sensors are used to capture images that may be preprocessed for data augment. After that, the preprocessed images are classified and labeled by automatic labeling technology [6], and are cached in vehicular clients. After receiving a request from a central server, vehicular clients separately train local DNN models with their local images.”;)
Regarding claim 4
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein the driving data comprises abstract data features and/or synthetized data features.
(Ye [sec(s) VI.A] “Because the images in the BelgiumTSC dataset are not all the same size, we just resize the images to a fixed size, i.e., 28×28 pixels. The comparison is divided into two cases. • Blurred Training Image and Unblurred Testing Image (BU): We randomly divide the training images into 10 groups and each group has the same amount of images. We synthesize motion-blurred images by [33]. The motion blur level is divided into 10 levels, i.e., L = 1, 2, . . . , 10. Each group has a motion blur level. Blurred training images and unblurred testing images constitute the training and testing datasets, respectively. • Blurred Training Image and Blurred Testing Image (BB): The training dataset is produced similar to that in BU. The testing images are blurred with level L = 3 to constitute the testing dataset. According to the optimal contract items designed for their own types, each vehicular client picks out a part of training images to train the local DNN model with a convolutional neural network (CNN) in PYTHON. For the MNIST dataset, the local DNN model is executed with iteration round E = 5 and full gradient descent. The CNN consists of two convolutional layers followed by two fully connected layers and then another 10 units activated by soft-max, with totally about 1, 662, 752 parameters. According to [20], the size of the local DNN model φ is about 6.5 MB. For the BelgiumTSC dataset, the local DNN model is executed with iteration round E = 5 and full gradient descent. The CNN consists of two convolutional layers followed by three fully connected layers, with totally about 274, 730 parameters. The size of the local DNN model is about 1 MB.”;)
Regarding claim 5
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein a proprietary implementation of a vehicle or autonomous vehicle is preserved in the training of the local prediction model by learning the driving policy.
(Ye [fig(s) 2] [sec(s) I] “After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”;)
Regarding claim 6
The combination of Ye, Wang teaches claim 1.
Ye further teaches
determining an accuracy score of the local prediction model and an accuracy score of the combined prediction model, wherein the vehicle class is taken from the model having the higher accuracy score.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”; Note that Wang teaches “vehicle class”)
Regarding claim 11
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein the method is performed on one or more vehicles and/or at one or more external or edge data processing systems.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”;)
Regarding claim 12
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein the method is performed as a machine learning approach.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “To improve the accuracy and efficiency of model aggregation, this paper proposes a selective model aggregation approach. First of all, we exploit a geometric model that illustrates the relationship between the object of interest and the camera in each vehicular client. The geometric model is used to evaluate the image quality in the motion blur level by observing the instantaneous velocity of each vehicular client. After that, the computation capability is quantified via a parameter of resource consumption. By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry. To deal with the information asymmetry, the selection procedure of the ‘‘fine’’ local DNN models is formulated as a two-dimensional image-computation-reward contract theory problem. The formulated problem is transformed into a tractable problem through relaxing and simplifying the complicated constraints, and eventually solved by a greedy algorithm.” [sec(s) III.A] “After receiving a request from a central server, vehicular clients separately train local DNN models with their local images. Vehicular clients send updated the local DNN models to the central server for model aggregation.”;)
Regarding claim 13
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein the method is performed with computing servers communicating via direct links, through a cloud backend and/or through a connected, cooperative automated mobility platform (CCAM).
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “Vehicular edge computing (VEC) is a fast-developing vehicular technology, where vehicles and roadside servers at the network edge contribute communication, computation, storage and data resources to close proximity of vehicular users [4]. With the rapid penetration of intelligent connected vehicles (ICV), there is an urgent need to study federated learning in VEC as an important technical framework to meet the ever-increasing demands of AI applications in vehicular networks. In this paper, we consider image classification as a typical AI application in VEC [5]. As we know, the images captured from on-board cameras usually contain sensitive information with individual privacy of the vehicular clients. Using federated learning in VEC is beneficial in exploiting vehicular images for DNN training while protecting their privacy. For example, the vehicular clients use on-board cameras to capture images, which are classified and labeled by automatic labeling technology [6]. After that, the vehicular clients are selected by the central server to participate in federated learning in a supervised fashion and generate global and local DNN model updates. … By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry.”;)
Regarding claim 14
The combination of Ye, Wang teaches claim 1.
Ye further teaches
wherein in the method the vehicles train a neural network as the local prediction model and update weights on assigned servers or edge computing servers.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “Vehicular edge computing (VEC) is a fast-developing vehicular technology, where vehicles and roadside servers at the network edge contribute communication, computation, storage and data resources to close proximity of vehicular users [4]. With the rapid penetration of intelligent connected vehicles (ICV), there is an urgent need to study federated learning in VEC as an important technical framework to meet the ever-increasing demands of AI applications in vehicular networks. In this paper, we consider image classification as a typical AI application in VEC [5]. As we know, the images captured from on-board cameras usually contain sensitive information with individual privacy of the vehicular clients. Using federated learning in VEC is beneficial in exploiting vehicular images for DNN training while protecting their privacy. For example, the vehicular clients use on-board cameras to capture images, which are classified and labeled by automatic labeling technology [6]. After that, the vehicular clients are selected by the central server to participate in federated learning in a supervised fashion and generate global and local DNN model updates. … By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry.”;)
Regarding claim 15
The claim is a system claim corresponding to the method claim 1, and is directed to largely the same subject matter. Thus, it is rejected for the same reasons as given in the rejections of the method claim.
Regarding claim 19
The combination of Ye, Wang teaches claim 12.
Ye further teaches
wherein the machine learning approach is performed in an edge computing network with edge computing servers.
(Ye [sec(s) I] “Vehicular edge computing (VEC) is a fast-developing vehicular technology, where vehicles and roadside servers at the network edge contribute communication, computation, storage and data resources to close proximity of vehicular users [4]. With the rapid penetration of intelligent connected vehicles (ICV), there is an urgent need to study federated learning in VEC as an important technical framework to meet the ever-increasing demands of AI applications in vehicular networks. In this paper, we consider image classification as a typical AI application in VEC [5]. As we know, the images captured from on-board cameras usually contain sensitive information with individual privacy of the vehicular clients. Using federated learning in VEC is beneficial in exploiting vehicular images for DNN training while protecting their privacy. For example, the vehicular clients use on-board cameras to capture images, which are classified and labeled by automatic labeling technology [6]. After that, the vehicular clients are selected by the central server to participate in federated learning in a supervised fashion and generate global and local DNN model updates.”;)
Regarding claim 20
The combination of Ye, Wang teaches claim 13.
Ye further teaches
wherein the computing servers comprise edge computing servers.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “Vehicular edge computing (VEC) is a fast-developing vehicular technology, where vehicles and roadside servers at the network edge contribute communication, computation, storage and data resources to close proximity of vehicular users [4]. With the rapid penetration of intelligent connected vehicles (ICV), there is an urgent need to study federated learning in VEC as an important technical framework to meet the ever-increasing demands of AI applications in vehicular networks. In this paper, we consider image classification as a typical AI application in VEC [5]. As we know, the images captured from on-board cameras usually contain sensitive information with individual privacy of the vehicular clients. Using federated learning in VEC is beneficial in exploiting vehicular images for DNN training while protecting their privacy. For example, the vehicular clients use on-board cameras to capture images, which are classified and labeled by automatic labeling technology [6]. After that, the vehicular clients are selected by the central server to participate in federated learning in a supervised fashion and generate global and local DNN model updates. … By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry.”;)
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach) in view of Wang et al. (Fed-SCNN: A Federated Shallow-CNN Recognition Framework for Distracted Driving) in view of Dougherty et al. (US 2020/0207360 A1)
Regarding claim 2
The combination of Ye, Wang teaches claim 1.
However, the combination of Ye, Wang does not appear to explicitly teach:
wherein the vehicle class provides information on whether the at least one of the other vehicles is autonomously-driven or human-driven.
Dougherty teaches
wherein the vehicle class provides information on whether the at least one of the other vehicles is autonomously-driven or human-driven.
(Dougherty [par(s) 29] “In overview, various embodiments include vehicles equipped with a vehicle autonomous driving system (VADS) that uses information collected from the vehicle's sensors (e.g., camera, radar, LIDAR, etc.) in conjunction with information received via V2V communications from one or more surrounding vehicles to determine an autonomous capability metric (ACM) for one or more nearby vehicles (e.g., car in front, behind, left side, right side, etc.).” [par(s) 54-56] “In some embodiments, the VADS component may be configured detect the level of autonomy of nearby vehicles by implementing or using various observation, monitoring, machine learning and/or prediction techniques. For example, the VADS component may predict that a nearby vehicle has a high level of autonomy based on observing that the nearby vehicle is equipped with a sophisticated LIDAR sensor from a reputable manufacturer. Similarly, the VADS component may predict that a nearby vehicle has a low level of autonomy based on observing that the nearby vehicle is not adequate equipped with communications circuitry suitable for supporting V2V communications, or otherwise lacks particular sensing or communication modules. The VADS component may also predict the level of autonomy based on observed driving behaviors and other similar factors. … In some embodiments, the ACM may identify the level of autonomy in terms of a continuum value (e.g., a spectrum) or values that range from fully manual driving to fully automated driving with zero human intervention. In some embodiments, the ACM may identify the level of autonomy via a set of discrete category values (e.g. L0 to L5). In some embodiments, the ACM may identify the level of autonomy via a vector or matrix of values reflective of different aspects of autonomy and vehicle performance, such as values associated with each of a vehicle's computing capability, sensors, processing algorithms, prediction and control strategies, current autonomy setting (e.g., manual, semi-autonomous, or fully autonomous), etc.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Ye, Wang with the vehicle classification of Dougherty.
One of ordinary skill in the art would have been motived to combine in order to improve traffic flows and vehicle safety by enabling vehicles to share information that may be useful to other vehicles for anti-collision and other safety functions.
(Dougherty [par(s) 35] “In some embodiments, the VADS component may determine that there would be an operational advantage to engaging in a cooperative driving engagement based on the autonomy level of an identified vehicle, such as based on whether the identified vehicle is highly autonomous and includes premium sensors that would enable the autonomous vehicle to operate more safely or with improved performance” [par(s) 43] “V2V systems and technologies hold great promise for improving traffic flows and vehicle safety by enabling vehicles to share information regarding their location, speed, direction of travel, braking, and other factors that may be useful to other vehicles for anti-collision and other safety functions. Vehicles equipped with V2V onboard equipment will frequently (e.g. up to 20 times per second) transmit their vehicle information in packets referred to as Basic Safety Messages (BSM). Autonomous vehicles equipped with an Advanced Driver Assistance System (ADAS) may receive and use such V2V communications to control their speed and position with respect to other vehicles, and form a caravan that allows them to make coordinated maneuvering and navigation decisions.”)
Claim(s) 7-9, 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach) in view of Wang et al. (Fed-SCNN: A Federated Shallow-CNN Recognition Framework for Distracted Driving) in view of Wang et al. (V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction, hereinafter Wang20)
Regarding claim 7
The combination of Ye, Wang teaches claim 1.
However, the combination of Ye, Wang does not appear to explicitly teach:
the local classifications of the at least one of the other vehicles are shared with the other vehicles.
Wang20 further teaches
the local classifications of the at least one of the other vehicles are shared with the other vehicles.
(Wang20 [sec(s) 1] “While a world densely populated with self-driving vehicles (SDVs) might seem futuristic, these vehicles will one day soon be the norm” [sec(s) 3] “An SDV can choose to broadcast three types of information: (i) the raw sensor data, (ii) the intermediate representations of its P&P system, or (iii) the output detections and motion forecast trajectories. While all three message types are valuable for improving performance, we would like to minimize the message sizes while maximizing P&P accuracy gains. Note that small message sizes are critical because we want to leverage cheap, low-bandwidth, decentralized communication devices. While sending raw measurements minimizes information loss, they require more bandwidth. Furthermore, the receiving vehicle would need to process all additional sensor data received, which might prevent it from meeting the real-time inference requirements. On the other hand, transmitting the outputs of the P&P system is very good in terms of bandwidth, as only a few numbers need to be broadcasted. However, we may lose valuable scene context and uncertainty information that could be very important to better fuse the information”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Ye, Wang with the classification information sharing of Wang20.
One of ordinary skill in the art would have been motived to combine in order to improve the perception and motion forecasting performance of self-driving vehicles.
(Wang20 [sec(s) Abs] “In this paper, we explore the use of vehicle-to-vehicle (V2V) communication to improve the perception and motion forecasting performance of self-driving vehicles. By intelligently aggregating the information received from multiple nearby vehicles, we can observe the same scene from different viewpoints.”)
Regarding claim 8
The combination of Ye, Wang teaches claim 1.
However, the combination of Ye, Wang does not appear to explicitly teach:
determining confidence estimates associated with the local classifications, and
sharing the confidence estimates with the other vehicles.
Wang20 teaches
determining confidence estimates associated with the local classifications, and
sharing the confidence estimates with the other vehicles.
(Wang20 [sec(s) 1] “While a world densely populated with self-driving vehicles (SDVs) might seem futuristic, these vehicles will one day soon be the norm” [sec(s) 3] “An SDV can choose to broadcast three types of information: (i) the raw sensor data, (ii) the intermediate representations of its P&P system, or (iii) the output detections and motion forecast trajectories. While all three message types are valuable for improving performance, we would like to minimize the message sizes while maximizing P&P accuracy gains. Note that small message sizes are critical because we want to leverage cheap, low-bandwidth, decentralized communication devices. While sending raw measurements minimizes information loss, they require more bandwidth. Furthermore, the receiving vehicle would need to process all additional sensor data received, which might prevent it from meeting the real-time inference requirements. On the other hand, transmitting the outputs of the P&P system is very good in terms of bandwidth, as only a few numbers need to be broadcasted. However, we may lose valuable scene context and uncertainty information that could be very important to better fuse the information” [sec(s) 5] “For Output Fusion, each vehicle sends post-processed outputs, i.e., bounding boxes with confidence scores, and predicted future trajectories after non-maximum suppression (NMS). At the receiver end, all bounding boxes and future trajectories are first transformed to the ego-vehicle coordinate system and then aggregated across vehicles. NMS is then applied again to produce the final results.”;)
The combination of Ye, Wang is combinable with Wang20 for the same rationale as set forth above with respect to claim 7.
Regarding claim 9
The combination of Ye, Wang teaches claim 1.
However, the combination of Ye, Wang does not appear to explicitly teach:
globally classifying one or more of the other vehicles by combining outputs of the local prediction models of the other vehicles and/or their associated confidence estimates.
Wang20 teaches
globally classifying one or more of the other vehicles by combining outputs of the local prediction models of the other vehicles and/or their associated confidence estimates.
(Wang20 [sec(s) 1] “While a world densely populated with self-driving vehicles (SDVs) might seem futuristic, these vehicles will one day soon be the norm. … In this paper, we consider the vehicle-to-vehicle (V2V) communication setting, where each vehicle can broadcast and receive information to/from nearby vehicles (within a 70m radius). Note that this broadcast range is realistic based on existing communication protocols [21]. We show that to achieve the best compromise of having strong perception and motion forecasting performance while also satisfying existing hardware transmission bandwidth capabilities, we should send compressed intermediate representations of the P&P neural network. Thus, we derive a novel P&P model, called V2VNet, which utilizes a spatially aware graph neural network (GNN) to aggregate the information received from all the nearby SDVs, allowing us to intelligently combine information from different points in time and viewpoints in the scene” [sec(s) 3] “An SDV can choose to broadcast three types of information: (i) the raw sensor data, (ii) the intermediate representations of its P&P system, or (iii) the output detections and motion forecast trajectories. While all three message types are valuable for improving performance, we would like to minimize the message sizes while maximizing P&P accuracy gains.”;)
The combination of Ye, Wang is combinable with Wang20 for the same rationale as set forth above with respect to claim 7.
Regarding claim 16
The combination of Ye, Wang, Wang20 teaches claim 7.
Wang20 teaches
wherein the local classifications are shared with the other vehicles for combining the local classifications.
(Wang20 [sec(s) 1] “While a world densely populated with self-driving vehicles (SDVs) might seem futuristic, these vehicles will one day soon be the norm. … In this paper, we consider the vehicle-to-vehicle (V2V) communication setting, where each vehicle can broadcast and receive information to/from nearby vehicles (within a 70m radius). Note that this broadcast range is realistic based on existing communication protocols [21]. We show that to achieve the best compromise of having strong perception and motion forecasting performance while also satisfying existing hardware transmission bandwidth capabilities, we should send compressed intermediate representations of the P&P neural network. Thus, we derive a novel P&P model, called V2VNet, which utilizes a spatially aware graph neural network (GNN) to aggregate the information received from all the nearby SDVs, allowing us to intelligently combine information from different points in time and viewpoints in the scene” [sec(s) 3] “An SDV can choose to broadcast three types of information: (i) the raw sensor data, (ii) the intermediate representations of its P&P system, or (iii) the output detections and motion forecast trajectories. While all three message types are valuable for improving performance, we would like to minimize the message sizes while maximizing P&P accuracy gains.”;)
The combination of Ye, Wang, Wang20 is combinable with Wang20 for the same rationale as set forth above with respect to claim 7.
Regarding claim 17
The combination of Ye, Wang, Wang20 teaches claim 8.
Wang20 further teaches
wherein the confidence estimates associated with the local classifications are shared with the other vehicles for combining the confidence estimates.
(Wang20 [sec(s) 1] “While a world densely populated with self-driving vehicles (SDVs) might seem futuristic, these vehicles will one day soon be the norm” [sec(s) 3] “An SDV can choose to broadcast three types of information: (i) the raw sensor data, (ii) the intermediate representations of its P&P system, or (iii) the output detections and motion forecast trajectories. While all three message types are valuable for improving performance, we would like to minimize the message sizes while maximizing P&P accuracy gains. Note that small message sizes are critical because we want to leverage cheap, low-bandwidth, decentralized communication devices. While sending raw measurements minimizes information loss, they require more bandwidth. Furthermore, the receiving vehicle would need to process all additional sensor data received, which might prevent it from meeting the real-time inference requirements. On the other hand, transmitting the outputs of the P&P system is very good in terms of bandwidth, as only a few numbers need to be broadcasted. However, we may lose valuable scene context and uncertainty information that could be very important to better fuse the information” [sec(s) 5] “For Output Fusion, each vehicle sends post-processed outputs, i.e., bounding boxes with confidence scores, and predicted future trajectories after non-maximum suppression (NMS). At the receiver end, all bounding boxes and future trajectories are first transformed to the ego-vehicle coordinate system and then aggregated across vehicles. NMS is then applied again to produce the final results.”;)
The combination of Ye, Wang is combinable with Wang20 for the same rationale as set forth above with respect to claim 7.
Regarding claim 18
The combination of Ye, Wang, Wang20 teaches claim 9.
Ye further teaches
wherein the one or more of the other vehicles are one or more target vehicles.
(Ye [fig(s) 1] “A general framework of federated learning in vehicular edge computing”, “Local Model Upload”, “Global Model Download” [fig(s) 2] [sec(s) I] “Vehicular edge computing (VEC) is a fast-developing vehicular technology, where vehicles and roadside servers at the network edge contribute communication, computation, storage and data resources to close proximity of vehicular users [4]. With the rapid penetration of intelligent connected vehicles (ICV), there is an urgent need to study federated learning in VEC as an important technical framework to meet the ever-increasing demands of AI applications in vehicular networks. In this paper, we consider image classification as a typical AI application in VEC [5]. As we know, the images captured from on-board cameras usually contain sensitive information with individual privacy of the vehicular clients. Using federated learning in VEC is beneficial in exploiting vehicular images for DNN training while protecting their privacy. For example, the vehicular clients use on-board cameras to capture images, which are classified and labeled by automatic labeling technology [6]. After that, the vehicular clients are selected by the central server to participate in federated learning in a supervised fashion and generate global and local DNN model updates. … By evaluating local image quality as well as computation capability, the ‘‘fine’’ local DNN models on the ‘‘fine’’ clients are selected and sent to the central server for aggregation. Since federated learning prevents from sending local data, the central server is not aware of the image quality and computation capability of vehicular clients, which is called information asymmetry.”;)
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach) in view of Wang et al. (Fed-SCNN: A Federated Shallow-CNN Recognition Framework for Distracted Driving) in view of Shokravi et al. (A Review on Vehicle Classification and Potential Use of Smart Vehicle-Assisted Techniques)
Regarding claim 10
The combination of Ye, Wang teaches claim 1.
However, the combination of Ye, Wang does not appear to explicitly teach:
sending to traffic authorities systems the local classification of the at least one of the other vehicles.
Shokravi teaches
sending to traffic authorities systems the local classification of the at least one of the other vehicles.
(Shokravi [sec(s) 1] “VANETs were capable of providing global information on vehicles in a real time manner. The provided information could be mobility parameters as well as physical vehicular parameters. The results of the feasibility study show that, in a VANET system, the mobility information—e.g., position, traveling lane, speed, and acceleration and deceleration—as well as the physical characteristic parameters of vehicles—e.g., weight, height and length—are used for a wide variety of applications such as parking management, traffic control, safety, and accident avoidance. Table 3 shows a summary of the literature reviews on VC.” [sec(s) Abs] “Vehicle classification (VC) is an underlying approach in an intelligent transportation system and is widely used in various applications like the monitoring of traffic flow, automated parking systems, and security enforcement.”;)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Ye, Wang with the traffic authorities systems of Shokravi.
One of ordinary skill in the art would have been motived to combine in order to achieve on-road safety and benefits for end-users based on vehicular networks and vehicle classification.
(Shokravi [sec(s) 3] “Embedded sensors, onboard hardware devices, and intelligent antenna systems mounted on a vehicle for transmitting and receiving signals have provided a unique combination of properties, making smart vehicles as an attractive choice for many high-tech applications. VC can greatly benefit from these technologies. The objective and purpose of this research is to study the capability of different vehicle-assisted techniques to extract the kinematic and physical characteristics of vehicles in real time and in a global manner. This information can be used for a wide variety of applications such as parking management, traffic control, safety, and accident avoidance [149]. … The synergistic links between the two worlds of VANETs and smart vehicles are highly promising for achieving further on-road safety and benefits for end-users [176]. Internet of Vehicles (IoV) is a typical application of the Internet of Things (IoT) in the field of transportation that is achieved by expanding the capabilities of VANETs.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEHWAN KIM whose telephone number is (571)270-7409. The examiner can normally be reached Mon - Thu 7:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEHWAN KIM/Examiner, Art Unit 2129
Prosecution Insights