DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
Paragraph 10 has a typo in “SCAFFOLD uses dada size based average” where “dada” is seen as a typo of “data”
Paragraph 60 has a typo in “commutation resources” where “commutation” is seen as a typo of “computation”
Appropriate correction is required.
Claim Objections
Claim 1 objected to because of the following informalities: “train the global machine learning model using the on-bord computer units” is seen as a typo of “train the global machine learning model using the on-board computer units”. Appropriate correction is required.
Claim 3 objected to because of the following informalities: “commutation resources” is seen as a typo of “computation resources”. Appropriate correction is required.
Claim 18 is objected to for containing the same claim objections as analogous claim 1.
Claim 7 objected to because of the following informalities: “as starting point” is seen as a typo of “as a starting point”. Appropriate correction is required.
Claim 11 objected to because of the following informalities: “local training iteration” is seen as a typo of “local training iterations”. Appropriate correction is required.
Claim 12 objected to because of the following informalities: “whereing the local data collected” is seen as a typo of “wherein the local data collected”, “at different location” is seen as a typo of “at different locations”, and “diffenrent time” is seen as a typo of “different time”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
Regarding 112(b):
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-9 and 17-18 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In regard to Claim 4:
A broad range or limitation together with a narrow range or limitation that falls within the broad range or limitation (in the same claim) may be considered indefinite if the resulting claim does not clearly set forth the metes and bounds of the patent protection desired. See MPEP § 2173.05(c). In the present instance, claim 4 recites the broad recitation “the learning server distributes the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the selected vehicle agents via the associated RSUs”, and the claim also recites “wherein the learning server broadcasts the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the RSUs and the RSUs then respectively relay the received set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the associated vehicle agents” which is the narrower statement of the range/limitation. The claim(s) are considered indefinite because there is a question or doubt as to whether the feature introduced by such narrower language is (a) merely exemplary of the remainder of the claim, and therefore not required, or (b) a required feature of the claims. In an alternative wording, the claim(s) are considered indefinite because there is a question or doubt as to whether the feature introduced by such narrower language is a separate limitation, thus is not merely reciting the broad limitation in more steps and thus is required to be performed as well with that of the broad limitation.
Amending the claims to only recite one of the broad or narrow limitations could prevent the 112(b) issue. Amending the claims to more clearly recite whether the broad limitation overwrites the narrow limitation could also help prevent the 112(b) issue.
In regard to claim 5:
Claim 5 recites the limitation " the global training round t" and “the trained local models
x
i
(
t
)
” in “wherein at the global training round t, the learning server aggregates the trained local models
x
i
(
t
)
using a weight…”. There is insufficient antecedent basis for this limitation in the claim.
The idea of variable t representing a particular global training round was introduced in claim 2 (“at global training round t”). However, claim 5 only depends upon claim 1, thus lacks the antecedent basis for “the global training round t”. A possible fix is to use the phrase “a global training round t” or to change the claim that claim 5 depends upon.
The idea of “the trained local models
x
i
(
t
)
” was introduced in claim 2 (“wherein a locally trained model by a vehicle agent i is represented as
x
i
(
t
)
”). However, claim 5 only depends upon claim 1. As a result, how the models are represented by
x
i
(
t
)
is not understood, as i is not defined in the context of claim 5. Introducing all of the required variables or changing what claim that claim 5 depends upon could help fix the 112(b) issue.
In regard to claim 6:
Claim 6 recites the limitation "computed according to
p
i
=
N
i
σ
i
2
∑
j
=
1
n
N
j
σ
j
2
, (i = 1, 2, …, n)". There is insufficient antecedent basis for this limitation in the claim. The meaning of variable j is not given elsewhere in the claim, nor is the meaning of
σ
j
2
. As a result of a lack of indication of what variables in the equation mean, the equation is seen as lacking antecedent basis for elements.
In regard to claim 7:
Claim 7 recites the limitation "upon receiving the set of global model parameters
x
g
l
o
b
a
l
(
t
)
”. There is insufficient antecedent basis for this limitation in the claim. The idea of variable t representing a particular global training round or model parameters was introduced in claim 2 (“the global machine learning model is expressed as a set of global model parameters
x
g
l
o
b
a
l
(
t
)
at global training round t”). However, claim 7 only depends upon claim 1, thus lacks the antecedent basis for the limitation. Defining or properly introducing the required elements could help prevent the 112(b) antecedent issue.
Claim 7 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential steps, such omission amounting to a gap between the steps. See MPEP § 2172.01. The omitted steps are: how parameters of the global model are determined to be in the homogeneous set or the heterogeneous set. The claim limitations do not provide an indication as to how the parameters of the global model are divided into the sets. The specification gives some examples as to what kinds of parameters are divided into what sets ([Current Application 0085]: “To classify model parameters, the parameters including road map and traffic flow are classified into set SHom” and [Current Application 0085]: “The parameters including vehicle trajectories, vehicle destinations and proximal vehicles are classified into set SHet”), but the specification does not provide an indication as to how this step is determined for any kinds of parameters outside of the examples. One of ordinary skill in the art would be left questioning as to how which parameters of a model would be determined to handling what specific types of information to then use the type of information (homogeneous or heterogeneous) to then divide the parameters into sets. While the example of how the consideration could be considered for convolutional networks is given in the specification ([Current Application 0082]: “Considering the structure of NIL model, different layers of a complex model often serve different purposes, take Convolutional Neural Networks (CNNs) in computer vision tasks for instance, it is commonly believed that the lower layers of a CNN serves as a common feature detector which can be kept invariant across different tasks, and the last layers are used to learn specific tasks.”), how such a determination is done is not provided, especially for non-convolutional models. The claim limitations do not limit the models to be convolutional in nature. The description in the specification also presents questions as to whether model parameters are being divided, as the idea of a particular input is noted as being given in a particular format ([Current Application 0085]: “The parameters including vehicle trajectories, vehicle destinations and proximal vehicles are classified into set SHet, where the trajectory is represented (x, y, 1, v, a, w, I)”). That poses the question as to whether the specification is referring more towards the data being provided to the models rather than the parameters/weights of the models. Another aspect that provides confusion is the note of the three modules being used in relation to dividing the parameters ([Current Application 0083]: “To perform the structure-aware model update (structure-aware model training method), a three-module structure is adopted in federated learning with three interacting modules each with unique purposes. Firstly, a graph encoder module encodes map and vehicles nearby the learning vehicle as a directed graph, then a policy header module learns a discrete policy for each vehicle in consideration, the sampled path is decoded into predicted trajectories of learning vehicle by a trajectory decoder module”). The aspects related to the trajectory and the particular modules gives the indication that the global model is made up of multiple models related to the modules, where the whole of a model’s parameters are determined to be related to heterogeneous or homogeneous data (as the example related to trajectories in spec ([Current Application 0085]: “The parameters including vehicle trajectories, vehicle destinations and proximal vehicles are classified into set SHet”) was noted to be heterogeneous and was noted to be related to the trajectory decoder module ([Current Application 0083]: “the sampled path is decoded into predicted trajectories of learning vehicle by a trajectory decoder module”)). This would mean that instead of dividing up one model’s parameters into separate sets as claim 7 appears to read, which is later utilized to use different algorithms on said sets (as noted in claim 8), the claim is deciding whether a model pertains to using homogeneous or heterogeneous data and thus placing a model into the set related to the corresponding data type. The confusion caused by the omission of essential steps prevents an accurate prior art mapping of claim 7 and claim 7’s dependents, as one of ordinary skill in the art could not determine what steps are being claimed in the limitation (as noted by the possible interpretations listed earlier).
For the purpose of helping progress prosecution and a complete office action, an interpretation is given to claim 7 and claim 7’s dependents for prior art mapping. The interpretation is given along with the prior art mapping of the relevant claim.
In regard to claim 9:
Claim 9 recites the limitation “the structure-aware model training method uses a graph encoder module configured to encode the road map, each of the vehicle agents and proximal vehicles into a directed graph, a policy header module…”. This limitation results in the list in the claim being difficult to interpret. One of ordinary skill cannot tell what is connected to what. Examples of interpretations are: is “each of the vehicle agents and proximal vehicles into a directed graph” connected to the “configured to” or the “configured to encode”? Is the “policy header module” in the same list, as that could mean the “configured to encode” also applies to the header module, which the policy header module being encoded doesn’t seem to work with the “policy header module configured to learn a discrete policy”. The policy header module gives the indication this is a list of separate items and not a list of things to be encoded (thus conflicting with an interpretation of the “configured to encode”). Rewriting claim 9 to make apparent whether the elements are separate modules could be a possible method of fixing the 112(b) indefinite issue.
Claim 9 recites the limitation " and a trajectory decoder module configured to predict trajectories of a vehicle agent by decoding sampled paths of the vehicle agent ". There is insufficient antecedent basis for this limitation in the claim. The antecedent issue is caused by “a vehicle agent” in the limitation, as vehicle agents have already been introduced, thus creating the confusion as to whether an existing vehicle agent is being referred to or a new vehicle agent. A possible correction could be to have the “a vehicle agent” be replaced with “one of the vehicle agents”.
In regard to claim 17:
Claim 17 recites the limitation “energy consumption prediction and ADAS/AD parameter calibration”. There is insufficient antecedent basis for this limitation in the claim. ADAS/AD has not been introduced or defined within the claims. The specification indicates ADAS likely refers to [Current Application 0046]: “advanced driver-assistance system (ADAS)”, but no indication as to what AD refers to is given in either claims or specification. As a result, one of ordinary skill in the art is unable to determine what the claim is referring to.
In regard to claim 18:
Claim 18 recites the limitation “wherein the at least one processor continues the selecting…”. There is insufficient antecedent basis for this limitation in the claim. There is no prior recitation of “a processor” in the claim. Appropriate correction is required.
In regard to dependent claims:
Dependent claims of claims rejected under 112(b) are rejected under 112(b) for being dependent upon a 112(b) rejected claim.
In regard to analogous claims:
Analogous claim to rejected claims (such as claims 19 and 20 being analogous to claims 5 and 6) are rejected under the same 112(b) rejections as their analogous claim.
Regarding 112(d):
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 19 and 20 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 19 and 20 depend upon themselves, thus fail to further limit in a dependent claim as no independent claim exists in the hierarchy to further limit. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
For the purpose of prior art examination, claim 19 is treated as being dependent upon claim 18 and claim 20 is treated as being dependent upon claim 19. However, the scope of the claims can change depending on what claims a dependent claim depends upon, thus the interpretation for prior art examination is solely to have a more complete office action and to help further prosecution. The claims must be amended to address to the claim 112(d) issues.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed towards an abstract idea without significantly more.
In regards to Claim 1:
Step 1: Is the claim directed towards a process, machine, manufacture, or composition of matter?
Yes, the claim is directed towards a machine.
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 1 recites the following abstract ideas:
selecting the vehicle agents from on-road vehicles driving on roads associated with a road map with respect to the global machine learning model
This limitation is directed towards the abstract idea of a mental process, or a concept performed in the human mind, including observation, evaluation, judgement or opinion (see MPEP 2106.04(a)(2) subsection 3). Here the limitation is seen as evaluation and judgement.
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 1 recites the following additional elements:
A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network, comprising:
This limitation is directed towards linking or indicating a field of use or technological environment (see MPEP 2106.05(h)).
at least one processor; and a memory having instructions of a vehicular federated learning method stored thereon that cause the at least one processor to perform:
At a high level of generality, this is an activity of using a processor and memory as an “apply it” use (see MPEP 2106.05(f)).
distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the RSUs are associated respectively with the vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
wherein the vehicle agents include on-board computer units and on-board sensors configured to collect local data while the vehicle agents drive on current trajectories of the roads
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
wherein the selected vehicle agents locally train the global machine learning model using the on-bord computer units and the collected local data via a structure-aware model training method, wherein the locally trained models are stored as trained local models
At a high level of generality, this is an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)).
aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method
At a high level of generality, this is an activity of using a variance-based model aggregation method as an “apply it” use (see MPEP 2106.05(f)).
and updating the global machine learning model using the aggregated trained local models
At a high level of generality, this is an activity of updating a model as an “apply it” use (see MPEP 2106.05(f)).
wherein the at least one processor continues the selecting, the distributing, the aggregating and the updating until a global training round reaches a pre-determined number of multi-rounds or learning error stabilizes
This limitation is directed towards the insignificant extra solution activity of repetitive calculations (see MPEP § 2106.05(d)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 1 recites the following additional elements:
A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network, comprising:
This limitation is directed towards linking or indicating a field of use or technological environment (see MPEP 2106.05(h)). Limitations that amount to merely linking/indicating to a field of use or technological environment, such as sending data for machine learning models utilizing roadside units (see MPEP 2106.05(h)(x)), do not amount to significantly more than the exception itself.
at least one processor; and a memory having instructions of a vehicular federated learning method stored thereon that cause the at least one processor to perform:
At a high level of generality, this is an activity of using a processor and memory as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a processor and memory appears to be an implementation of the abstract idea on a computer, so merely using a computer as a tool to perform the abstract idea.
distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the RSUs are associated respectively with the vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
wherein the vehicle agents include on-board computer units and on-board sensors configured to collect local data while the vehicle agents drive on current trajectories of the roads
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of extracting data (see MPEP 2106.05(d) example v in computer functions).
wherein the selected vehicle agents locally train the global machine learning model using the on-bord computer units and the collected local data via a structure-aware model training method, wherein the locally trained models are stored as trained local models
At a high level of generality, this is an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “train the global machine learning model” using on-board computer units, local data, and a structure-aware training method does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method
At a high level of generality, this is an activity of using a variance-based model aggregation method as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “aggregating the trained local models” using a variance-based model aggregation method does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
and updating the global machine learning model using the aggregated trained local models
At a high level of generality, this is an activity of updating a model as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “updating the global machine learning model” using aggregated trained local models does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
wherein the at least one processor continues the selecting, the distributing, the aggregating and the updating until a global training round reaches a pre-determined number of multi-rounds or learning error stabilizes
This limitation is directed towards the insignificant extra solution activity of repetitive calculations (see MPEP § 2106.05(d)). Repetitive calculations are considered a well understood, routine, and conventional activity acknowledged by the courts (see MPEP § 2106.05(d) subsection 2 example 2 for a computer).
In regards to Claim 2:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 2 recites the following additional elements:
wherein the global machine learning model is expressed as a set of global model parameters
x
g
l
o
b
a
l
(
t
)
at global training round t, wherein the set of global model parameters
x
g
l
o
b
a
l
(
t
)
is distributed to the selected vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
for locally training the distributed global machine learning model using local datasets of the vehicle agents, wherein a locally trained model by a vehicle agent i is represented as
x
i
(
t
)
At a high level of generality, this is an activity of updating a model as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 2 recites the following additional elements:
wherein the global machine learning model is expressed as a set of global model parameters
x
g
l
o
b
a
l
(
t
)
at global training round t, wherein the set of global model parameters
x
g
l
o
b
a
l
(
t
)
is distributed to the selected vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
for locally training the distributed global machine learning model using local datasets of the vehicle agents, wherein a locally trained model by a vehicle agent i is represented as
x
i
(
t
)
At a high level of generality, this is an activity of updating a model as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “training the distributed global machine learning model” using local datasets does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
In regards to Claim 3:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 3 recites the following abstract ideas:
the selecting is performed based on one or combination of (1) randomly selecting vehicle agents, (2) selecting vehicle agents being connected to the network longer than a predetermined time period, (3) selecting vehicle agents having better link quality to the associated RSUs, (4) selecting vehicle agents having better performances in previous global training rounds, (5) selecting vehicle agents having larger datasets, (6) selecting vehicle agents based on commutation resources and (7) selecting vehicle agents based on distances to the associated RSUs
This limitation is directed towards the abstract idea of a mental process, or a concept performed in the human mind, including observation, evaluation, judgement or opinion (see MPEP 2106.04(a)(2) subsection 3). Here the limitation is seen as evaluation and judgement.
In regards to Claim 4:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 4 recites the following additional elements:
the learning server distributes the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the selected vehicle agents via the associated RSUs, wherein the learning server broadcasts the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the RSUs and the RSUs then respectively relay the received set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the associated vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 4 recites the following additional elements:
the learning server distributes the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the selected vehicle agents via the associated RSUs, wherein the learning server broadcasts the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the RSUs and the RSUs then respectively relay the received set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the associated vehicle agents
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
In regards to Claim 5:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 5 recites the following abstract ideas:
wherein at the global training round t, the learning server aggregates the trained local models
x
i
(
t
)
using a weight simplex
p
=
(
p
1
,
…
,
p
n
)
as
x
g
l
o
b
a
l
(
t
)
=
∑
i
=
1
n
p
i
x
i
(
t
)
where n is a number of the selected vehicle agents
This limitation is directed towards the abstract idea of a mathematical concept (see MPEP 2106.04(a)(2) subsection 1).
In regards to Claim 6:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 6 recites the following abstract ideas:
wherein while aggregating the trained local models, the learning server applies a variance-based optimal weight simplex
p
=
(
p
1
,
…
,
p
n
)
computed according to
p
i
=
N
i
σ
i
2
∑
j
=
1
n
N
j
σ
j
2
, (i = 1, 2, …, n) where n is a number of the selected vehicle agents,
N
i
is a number of data samples of vehicle agents i and
σ
i
2
is the variance of vehicle agent i.
This limitation is directed towards the abstract idea of a mathematical concept (see MPEP 2106.04(a)(2) subsection 1).
In regards to Claim 7:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 7 recites the following additional elements:
wherein upon receiving the set of global model parameters
x
g
l
o
b
a
l
(
t
)
, the vehicle agents perform the structure-aware model training method by using
x
g
l
o
b
a
l
(
t
)
as starting point, wherein the vehicle agents divide the
x
g
l
o
b
a
l
(
t
)
into homogeneous set
S
H
o
m
and heterogeneous set
S
H
e
t
At a high level of generality, this is an activity of training a model as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 7 recites the following additional elements:
wherein upon receiving the set of global model parameters
x
g
l
o
b
a
l
(
t
)
, the vehicle agents perform the structure-aware model training method by using
x
g
l
o
b
a
l
(
t
)
as starting point, wherein the vehicle agents divide the
x
g
l
o
b
a
l
(
t
)
into homogeneous set
S
H
o
m
and heterogeneous set
S
H
e
t
At a high level of generality, this is an activity of training a model as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “structure-aware model training method” using sets of data does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”. Having sets while performing training does not change the training under BRI, as is still performing training, but now the data just has more labels.
In regards to Claim 8:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 8 recites the following additional elements:
wherein the set of global model parameters in homogeneous set
S
H
o
m
are updated using homogeneous federated learning algorithms such as FedAvg and the set of global model parameters in heterogeneous set
S
H
e
t
are updated using heterogeneous federated learning algorithms such as FedProx
At a high level of generality, this is an activity of training a model as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 8 recites the following additional elements:
wherein the set of global model parameters in homogeneous set
S
H
o
m
are updated using homogeneous federated learning algorithms such as FedAvg and the set of global model parameters in heterogeneous set
S
H
e
t
are updated using heterogeneous federated learning algorithms such as FedProx
At a high level of generality, this is an activity of training a model as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “update” using FedAvg or FedProx does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”. As a result of the current 112(b) rejection on claim 7, claim 8 is interpreted as using an algorithm, such as FedAvg or FedProx, on a model in federated learning to update the global model. Meaning claim 8 is not interpreted as splitting one model in pieces that are not on their own models.
In regards to Claim 9:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 9 recites the following abstract ideas:
and a trajectory decoder module configured to predict trajectories of a vehicle agent by decoding sampled paths of the vehicle agent
This limitation is directed towards the abstract idea of a mental process, or a concept performed in the human mind, including observation, evaluation, judgement or opinion (see MPEP 2106.04(a)(2) subsection 3). Here the limitation is seen as observation and evaluation.
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 9 recites the following additional elements:
wherein the structure-aware model training method uses a graph encoder module configured to encode the road map, each of the vehicle agents and proximal vehicles into a directed graph,
At a high level of generality, this is an activity of using a graph encoder module as an “apply it” use (see MPEP 2106.05(f)).
a policy header module configured to learn a discrete policy for each of the vehicle agents and the proximal vehicles,
At a high level of generality, this is an activity of using a policy header module as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 9 recites the following additional elements:
wherein the structure-aware model training method uses a graph encoder module configured to encode the road map, each of the vehicle agents and proximal vehicles into a directed graph,
At a high level of generality, this is an activity of using a graph encoder module as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “encode… into a directed graph” using a graph encoder module does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
a policy header module configured to learn a discrete policy for each of the vehicle agents and the proximal vehicles,
At a high level of generality, this is an activity of using a policy header module as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “learn a discrete policy” using a policy header module does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
In regards to Claim 10:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 10 recites the following additional elements:
wherein the selected vehicle agents upload the trained local models to the learning server via the RSUs, wherein the selected vehicle agents upload the trained local models to currently connected RSUs, wherein the RSUs relay the received trained local models to the learning server
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim10 recites the following additional elements:
wherein the selected vehicle agents upload the trained local models to the learning server via the RSUs, wherein the selected vehicle agents upload the trained local models to currently connected RSUs, wherein the RSUs relay the received trained local models to the learning server
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
In regards to Claim 11:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 11 recites the following additional elements:
wherein the selected vehicle agents upload the trained local models to the learning server based on one or combination of criteria (1) time specified by the learning server, (2) a predetermined number of local training iteration, (3) local model training error reaching a predetermined threshold and (4) local model training error stabilizing
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 11 recites the following additional elements:
wherein the selected vehicle agents upload the trained local models to the learning server based on one or combination of criteria (1) time specified by the learning server, (2) a predetermined number of local training iteration, (3) local model training error reaching a predetermined threshold and (4) local model training error stabilizing
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
In regards to Claim 12:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 12 recites the following additional elements:
wherein the selected vehicle agents partition local datasets into different clusters such that each cluster is used to train a particular machine learning model, whereing the local data collected at different location and diffenrent time are used to train the corresponding particular learning models.
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 12 recites the following additional elements:
wherein the selected vehicle agents partition local datasets into different clusters such that each cluster is used to train a particular machine learning model, whereing the local data collected at different location and diffenrent time are used to train the corresponding particular learning models.
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)). Noting data for the training is collected at different locations or different times for different models does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
In regards to Claim 13:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 13 recites the following additional elements:
wherein at least two of the selected vehicle agents collect the local data using different types of two sensors respectively equipped on the least two of the selected vehicle agents.
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 13 recites the following additional elements:
wherein at least two of the selected vehicle agents collect the local data using different types of two sensors respectively equipped on the least two of the selected vehicle agents.
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of extracting data (see MPEP 2106.05(d) example v in computer functions).
In regards to Claim 14:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 14 recites the following additional elements:
wherein the two sensors are a high-end GPS and a low-end GPS receiver, wherein the high-end GPS receiver provides more accurate measurements than that of the low-end GPS receiver.
This limitation is directed towards the continuation of an insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 14 recites the following additional elements:
wherein the two sensors are a high-end GPS and a low-end GPS receiver, wherein the high-end GPS receiver provides more accurate measurements than that of the low-end GPS receiver.
This limitation is directed towards the continuation of an insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a continuation of a well understood, routine, conventional activity of extracting data (see MPEP 2106.05(d) example v in computer functions). The sensors for extracting data being a high-end or low-end GPS do not incorporate the application into a practical application.
In regards to Claim 15:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 15 recites the following additional elements:
wherein the global machine learning model is trained by using neural networks with adaptive momentum optimizers
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 15 recites the following additional elements:
wherein the global machine learning model is trained by using neural networks with adaptive momentum optimizers
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)). Noting adaptive momentum optimizers are used does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
In regards to Claim 16:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 16 recites the following additional elements:
wherein training of the global machine learning model is initiated by one or combination of 1) periodic model training, 2) event based model training and 3) feedback based model training
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 16 recites the following additional elements:
wherein training of the global machine learning model is initiated by one or combination of 1) periodic model training, 2) event based model training and 3) feedback based model training
At a high level of generality, this is a continuation of an activity of using on-board computer units, local data, and a structure-aware training method as an “apply it” use (see MPEP 2106.05(f)). Noting when the model performs the training does not integrate the application into a practical application.
In regards to Claim 17:
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 17 recites the following additional elements:
wherein learning server distributes well-trained global machine learning models to all on-road vehicles for their applications,
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)).
wherein the on-road vehicles apply the well-trained global machine learning models to respective tasks of the on-road vehicles such as trajectory prediction, velocity prediction, energy consumption prediction and ADAS/AD parameter calibration
At a high level of generality, this is an activity of applying machine learning models on a task as an “apply it” use (see MPEP 2106.05(f)).
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 17 recites the following additional elements:
wherein learning server distributes well-trained global machine learning models to all on-road vehicles for their applications,
This limitation is directed towards the insignificant extra solution activity of mere data gathering (see MPEP § 2106.05(g)). This is a well understood, routine, conventional activity of transmitting data (see MPEP 2106.05(d) example i in computer functions).
wherein the on-road vehicles apply the well-trained global machine learning models to respective tasks of the on-road vehicles such as trajectory prediction, velocity prediction, energy consumption prediction and ADAS/AD parameter calibration
At a high level of generality, this is an activity of applying machine learning models on a task as an “apply it” use (see MPEP 2106.05(f)). At said high level of generality, a generic recitation of “apply the well-trained global machine learning models” to a respective task does not incorporate the abstract idea into a practical invention and is seen as a variation of the phrase “apply it”.
In regards to Claim 18:
Step 1: Is the claim directed towards a process, machine, manufacture, or composition of matter?
Yes, the claim is directed towards a method, so a process.
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 18 recites the same abstract ideas as analogous claim 1.
Step 2A Prong 2: Does the claim recite additional elements that integrate the exception into a practical application of the exception?
No, the application does not recite any additional elements that would integrate the abstract idea into a practical application.
Claim 18 recites the same additional elements as analogous claim 1.
Step 2B: Does the claim as a whole amount to significantly more than the judicial exception?
No, the claim as a whole does not amount to significantly more than the judicial exception. All elements of the claim, viewed individually or wholistically, do not provide an inventive concept or otherwise significantly more than the abstract idea itself.
Claim 18 recites the same additional elements as analogous claim 1.
In regards to Claim 19:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 19 recites the same abstract ideas as analogous claim 5.
In regards to Claim 20:
Step 2A Prong 1: Does the claim recite a law of nature, a natural phenomenon, or an abstract idea?
Yes, the claim does recite a(n) abstract idea.
Claim 20 recites the same abstract ideas as analogous claim 6.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 5, 7, 8, 10, 11, 13, 14, 18, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document.
Regarding Claim 1:
A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network, comprising: at least one processor; and a memory having instructions of a vehicular federated learning method stored thereon that cause the at least one processor to perform:
[Gao 0005]: “In another aspect, a vehicle is provided that comprises a communication circuit configured to exchange communications with edge servers of a hierarchical federated learning, a memory storing instructions [and a memory having instructions of a vehicular federated learning method stored thereon that cause the at least one processor to perform] network, and one or more processors [at least one processor] communicably coupled to the memory. The one or more processors are configured to execute the instructions to obtain vehicular system conditions of the vehicle [A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network] responsive to joining a hierarchical federated learning network, exchange data with a plurality of edge servers of a hierarchical federated learning network according to a vehicle-to-edge server association protocol that is based on the vehicular system conditions, and identify a machine learning model for the vehicle from a plurality of machine learning models hosted on the plurality of edge servers using data acquired by the vehicle. The one or more processors are further configured to execute the instructions to, at least one of, train the identified machine learning to perform a task using the data acquired by the vehicle to produce a locally trained machine learning model, and apply the data acquired by the vehicle to the identified machine learning model to perform a task.”
via roadside units (RSUs)
via the RSUs, wherein the RSUs are associated respectively with the vehicle agents
[Gao 0044]: “Communication circuit 201 includes either or both a wireless transceiver circuit 202 with an associated antenna 214 and a wired I/O interface 204 with an associated hardwired data port (not illustrated). Communication circuit 201 can provide for vehicle-to-everything (V2X) and/or vehicle-to-vehicle (V2V) communications capabilities, allowing hierarchical federated learning circuit 210 to communicate with edge devices, such as roadside unit/equipment (RSU/RSE) [via roadside units (RSUs)] [via the RSUs, wherein the RSUs are associated respectively with the vehicle agents], network cloud servers and cloud-based databases, and/or other vehicles via network 290. For example, V2X communication capabilities allows hierarchical federated learning circuit 210 to communicate with edge/cloud servers, roadside infrastructure (e.g., such as roadside equipment/roadside unit, which may be a vehicle-to-infrastructure (V21)-enabled street light or cameras, for example), etc. Hierarchical federated learning circuit 210 may also communicate with other connected vehicles over vehicle-to-vehicle (V2V) communications.”
distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the RSUs are associated respectively with the vehicle agents, wherein the vehicle agents include on-board computer units and on-board sensors configured to collect local data while the vehicle agents drive on current trajectories of the roads, wherein the selected vehicle agents locally train the global machine learning model using the on-bord computer units and the collected local data via a structure-aware model training method, wherein the locally trained models are stored as trained local models; aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method; and updating the global machine learning model using the aggregated trained local models, wherein the at least one processor continues the selecting, the distributing, the aggregating and the updating until a global training round reaches a pre-determined number of multi-rounds or learning error stabilizes
[Gao 0016]: “As alluded to above, there are disadvantages to transmitting large quantities of data needed for model training in centralized machine learning platforms. Federated machine learning platforms (or networks) have been leveraged to reduce the this amount of data. In federate learning, machine learning models are trained at the vehicle side rather than at a centralized or edge server. Federated learning is an iterative process in which a vehicle downloads [distributing the global machine learning model to the selected vehicle agents] a machine learning model to train the model locally [wherein the selected vehicle agents locally train the global machine learning model using the on-bord computer units and the collected local data] on vehicle data (e.g., data acquired by the vehicle sensors and subsystems [wherein the vehicle agents include on-board computer units and on-board sensors configured to collect local data while the vehicle agents drive on current trajectories of the roads where the idea of being on a current trajectory of a road is noted from the agents being vehicles on roads given the roadside units]). The vehicle shares the locally trained models [wherein the locally trained models are stored as trained local models] with a server, which aggregates [aggregating the trained local models from the selected vehicle agents] locally trained models from a number of vehicles and shares the global aggregated model for accomplishing vehicle tasks [and updating the global machine learning model using the aggregated trained local models]. The global aggregated model is then a starting point for a subsequent iteration [wherein the at least one processor continues the selecting, the distributing, the aggregating and the updating] of local training at the vehicle side and aggregation at the server side. One major benefit of federated learning is that the vehicles need only transmit model parameters to a server, instead of large quantities of raw sensor data. This significantly reduces communication overhead and also overcomes privacy issues.”
via a variance-based model aggregation method
[Gao 0020]: “In another approach, weighted distance between different classes of users was minimized to reduce the data heterogeneity in the system. However, this approach only considers the variances [via a variance-based model aggregation method] between the collected data (referred to herein as data heterogeneity), with no consideration on system heterogeneity between end-user systems (e.g., variances in the systems conditions that are used to collect data).”
selecting the vehicle agents from on-road vehicles driving on roads associated with a road map with respect to the global machine learning model
[Li et al 3 Federated Optimization: Methods page 3]: “At each outer iteration, a subset of the devices are selected [selecting the vehicle agents from on-road vehicles driving on roads associated with a road map with respect to the global machine learning model where the vehicle agents on roads and a road map are shown by Gao 0005 noting vehicle agents, Gao 0044 noting roadside units, and Gao 0054 noting GPS usage. Meaning Gao teaches the use with vehicles and roads and Li is used here to indicate the selecting of devices, for Gao’s example utilizes server selection (Gao 0021)] and local solvers are used to optimize the local objective functions on each of the selected devices. The devices then communicate their local model updates to the central server, which aggregates them and updates the global model accordingly. The key to allowing flexible performance in this scenario is that each of the local objectives can be solved inexactly. This allows the amount of local computation vs. communication to be tuned based on the number of local iterations that are performed (with additional local iterations corresponding to more exact local solutions).”
or learning error stabilizes
[Li 4.1 Local Dissimilarity page 5]: “For most practical machine learning problems, there is no need to solve the problem to highly accurate stationary solutions [or learning error stabilizes], i.e., e is typically not very small. Indeed, it is wellknown that solving the problem beyond some threshold may even hurt generalization performance due to overfitting (Yao et al., 2007).”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Li. Gao and Li are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Li in order to utilize the random selection (noted in Algorithm 1 of Li) of devices as this enables the use of a subset of all devices rather than all devices thus helping control communication ([Li et al 3 Federated Optimization: Methods page 3]: “The devices then communicate their local model updates to the central server, which aggregates them and updates the global model accordingly. The key to allowing flexible performance in this scenario is that each of the local objectives can be solved inexactly. This allows the amount of local computation vs. communication to be tuned based on the number of local iterations that are performed (with additional local iterations corresponding to more exact local solutions).”). The combination of Gao and Li also enables the utilization of stabilization for model completion or deciding when to stop training a model, which is important for not training an already completed model ([Li 4.1 Local Dissimilarity page 5]: “For most practical machine learning problems, there is no need to solve the problem to highly accurate stationary solutions, i.e., e is typically not very small. Indeed, it is wellknown that solving the problem beyond some threshold may even hurt generalization performance due to overfitting (Yao et al., 2007).”).
via a structure-aware model training method
[Chen 5 Discussion page 22]: “Estimation of the level of heterogeneity. For unsupervised problems where evaluation of a model is difficult, implementation of the dichotomous strategy described in Corollaries 3.1 and 4.1 would require estimating the level of heterogeneity R. Even for supervised problems, estimation of R would be interesting, as it allows one to decide which algorithm to choose [via a structure-aware model training method] without model training.” One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Chen. Gao and Chen are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Chen in order to utilize a structure-ware model training method to better train models using the algorithm better suited ([Chen Introduction page 2-3]: “However, it is an entirely different story in the presence of data heterogeneity. FedAvg has been recognized to give inferior performance when there is a significant departure from complete homogeneity (see, e.g., Bonawitz et al. 2019). To better understand this point, consider the extreme case where the data distributions {Di} are entirely unrelated. This roughly amounts to saying that the model parameters {w*(i)} can be arbitrarily different from each other. In such a “completely heterogeneous” scenario, the objective function (1.2) simply has no clear interpretation, and any single global model—for example, the output of FedAvg—would lead to unbounded risks for most, if not all, clients. As a matter of fact, it is not difficult to see that the optimal training strategy for federated learning in this regime is arguably PureLocalTraining, which lets each client separately run SGD to minimize its own local ERM objective…”).
until a global training round reaches a pre-determined number of multi-rounds
[Berkley 0022]: "and the termination criteria may comprise one of a number of iterations [until a global training round reaches a pre-determined number of multi-rounds], a processing time, a number of measurements, a threshold accuracy for the one or more determinable parameters, a number of digital processor cycles, and a number of quantum processor cycles."
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Berkely. Gao and Berkely are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Berkely in order to help decide when to stop training a model, as a model cannot be trained forever ([Berkley 0022]: "and the termination criteria may comprise one of a number of iterations, a processing time, a number of measurements, a threshold accuracy for the one or more determinable parameters, a number of digital processor cycles, and a number of quantum processor cycles.").
Regarding Claim 2:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein the global machine learning model is expressed as a set of global model parameters
x
g
l
o
b
a
l
(
t
)
at global training round t, wherein the set of global model parameters
x
g
l
o
b
a
l
(
t
)
is distributed to the selected vehicle agents for locally training the distributed global machine learning model using local datasets of the vehicle agents, wherein a locally trained model by a vehicle agent i is represented as
x
i
(
t
)
[Gao 0016]: “As alluded to above, there are disadvantages to transmitting large quantities of data needed for model training in centralized machine learning platforms. Federated machine learning platforms (or networks) have been leveraged to reduce the this amount of data. In federate learning, machine learning models are trained at the vehicle side rather than at a centralized or edge server. Federated learning is an iterative process in which a vehicle downloads a machine learning model to train the model locally on vehicle data [wherein the global machine learning model is expressed as a set of global model parameters
x
g
l
o
b
a
l
(
t
)
at global training round t, wherein the set of global model parameters
x
g
l
o
b
a
l
(
t
)
is distributed to the selected vehicle agents for locally training the distributed global machine learning model using local datasets of the vehicle agents, wherein a locally trained model by a vehicle agent i is represented as
x
i
(
t
)
] (e.g., data acquired by the vehicle sensors and subsystems). The vehicle shares the locally trained models with a server, which aggregates locally trained models from a number of vehicles and shares the global aggregated model for accomplishing vehicle tasks. The global aggregated model is then a starting point for a subsequent iteration of local training at the vehicle side and aggregation at the server side. One major benefit of federated learning is that the vehicles need only transmit model parameters to a server, instead of large quantities of raw sensor data. This significantly reduces communication overhead and also overcomes privacy issues.”
Regarding Claim 5:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein at the global training round t, the learning server aggregates the trained local models
x
i
(
t
)
using a weight simplex
p
=
(
p
1
,
…
,
p
n
)
as
x
g
l
o
b
a
l
(
t
)
=
∑
i
=
1
n
p
i
x
i
(
t
)
where n is a number of the selected vehicle agents
[Chen et al 4.2 Analysis of Baseline Algorithms page 15] notes a vector p akin to “weight simplex p” and shows an equation that mimics the one in the claim limitation:
PNG
media_image1.png
105
641
media_image1.png
Greyscale
“Intuitively speaking, the weight vector p can be regarded as the importance weight on each client and controls ‘how many resources are allocated to each client’.”
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Chen. Gao and Chen are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Chen in order to control how many resources are associated or allocated to each client ([Chen et al 4.2 Analysis of Baseline Algorithms page 15]: “Intuitively speaking, the weight vector p can be regarded as the importance weight on each client and controls ‘how many resources are allocated to each client’.”).
Regarding Claim 7:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein upon receiving the set of global model parameters
x
g
l
o
b
a
l
(
t
)
, the vehicle agents perform the structure-aware model training method by using
x
g
l
o
b
a
l
(
t
)
as starting point,
[Gao 0016]: “As alluded to above, there are disadvantages to transmitting large quantities of data needed for model training in centralized machine learning platforms. Federated machine learning platforms (or networks) have been leveraged to reduce the this amount of data. In federate learning, machine learning models are trained at the vehicle side rather than at a centralized or edge server. Federated learning is an iterative process in which a vehicle downloads a machine learning model to train the model locally on vehicle data (e.g., data acquired by the vehicle sensors and subsystems). The vehicle shares the locally trained models with a server, which aggregates locally trained models from a number of vehicles and shares the global aggregated model for accomplishing vehicle tasks. The global aggregated model is then a starting point [wherein upon receiving the set of global model parameters
x
g
l
o
b
a
l
(
t
)
, the vehicle agents perform the structure-aware model training method by using
x
g
l
o
b
a
l
(
t
)
as starting point] for a subsequent iteration of local training at the vehicle side and aggregation at the server side. One major benefit of federated learning is that the vehicles need only transmit model parameters to a server, instead of large quantities of raw sensor data. This significantly reduces communication overhead and also overcomes privacy issues.”
wherein the vehicle agents divide the
x
g
l
o
b
a
l
(
t
)
into homogeneous set
S
H
o
m
and heterogeneous set
S
H
e
t
As a result of the 112(b) issue with claim 7, claim 7 is interpreted as deciding whether to use homogenous or heterogeneous algorithms. Claim 8 is seen as noting examples of what the algorithms could be.
[Chen 5 Discussion page 22]: “Estimation of the level of heterogeneity. For unsupervised problems where evaluation of a model is difficult, implementation of the dichotomous strategy described in Corollaries 3.1 and 4.1 would require estimating the level of heterogeneity R. Even for supervised problems, estimation of R would be interesting, as it allows one to decide which algorithm [wherein the vehicle agents divide the
x
g
l
o
b
a
l
(
t
)
into homogeneous set
S
H
o
m
and heterogeneous set
S
H
e
t
] to choose without model training.”
The motivation to combine with Chen is the same as the motivation to combine with Chen in claim 1.
Regarding Claim 8:
The learning server of claim 7 is taught by Gao, Li, Chen, Berkley
wherein the set of global model parameters in homogeneous set
S
H
o
m
are updated using homogeneous federated learning algorithms such as FedAvg
As noted in claim 7 mapping, claim 8 is interpreted as a result of the 112(b) as noting what algorithms could be used in the structure aware method of claim 7, thus the art for teaching claim 8 is for showing the algorithms.
[Chen Introduction page 2-3]: “However, it is an entirely different story in the presence of data heterogeneity. FedAvg [wherein the set of global model parameters in homogeneous set
S
H
o
m
are updated using homogeneous federated learning algorithms such as FedAvg] has been recognized to give inferior performance when there is a significant departure from complete homogeneity (see, e.g., Bonawitz et al. 2019). To better understand this point, consider the extreme case where the data distributions {Di} are entirely unrelated. This roughly amounts to saying that the model parameters {w*(i)} can be arbitrarily different from each other. In such a “completely heterogeneous” scenario, the objective function (1.2) simply has no clear interpretation, and any single global model—for example, the output of FedAvg—would lead to unbounded risks for most, if not all, clients. As a matter of fact, it is not difficult to see that the optimal training strategy for federated learning in this regime is arguably PureLocalTraining, which lets each client separately run SGD to minimize its own local ERM objective…”
and the set of global model parameters in heterogeneous set
S
H
e
t
are updated using heterogeneous federated learning algorithms such as FedProx
[Chen 1.2 Related Work page 4]: “In particular, Hanzely et al. [2020] showed that an accelerated variant of FedProx [and the set of global model parameters in heterogeneous set
S
H
e
t
are updated using heterogeneous federated learning algorithms such as FedProx] is optimal in terms of communication complexity and the local oracle complexity.”
The motivation to combine with Chen is the same as the motivation to combine with Chen in claim 7.
Regarding Claim 10:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein the selected vehicle agents upload the trained local models to the learning server via the RSUs, wherein the selected vehicle agents upload the trained local models to currently connected RSUs, wherein the RSUs relay the received trained local models to the learning server.
[Gao 0016]: “As alluded to above, there are disadvantages to transmitting large quantities of data needed for model training in centralized machine learning platforms. Federated machine learning platforms (or networks) have been leveraged to reduce the this amount of data. In federate learning, machine learning models are trained at the vehicle side rather than at a centralized or edge server. Federated learning is an iterative process in which a vehicle downloads a machine learning model to train the model locally on vehicle data (e.g., data acquired by the vehicle sensors and subsystems). The vehicle shares the locally trained models with a server [wherein the selected vehicle agents upload the trained local models to the learning server via the RSUs, wherein the selected vehicle agents upload the trained local models to currently connected RSUs, wherein the RSUs relay the received trained local models to the learning server], which aggregates locally trained models from a number of vehicles and shares the global aggregated model for accomplishing vehicle tasks. The global aggregated model is then a starting point for a subsequent iteration of local training at the vehicle side and aggregation at the server side. One major benefit of federated learning is that the vehicles need only transmit model parameters to a server, instead of large quantities of raw sensor data. This significantly reduces communication overhead and also overcomes privacy issues.”
Where the use of RSUs for the process is noted in [Gao 0044]: “Communication circuit 201 includes either or both a wireless transceiver circuit 202 with an associated antenna 214 and a wired I/O interface 204 with an associated hardwired data port (not illustrated). Communication circuit 201 can provide for vehicle-to-everything (V2X) and/or vehicle-to-vehicle (V2V) communications capabilities, allowing hierarchical federated learning circuit 210 to communicate with edge devices, such as roadside unit/equipment (RSU/RSE) [RSU], network cloud servers and cloud-based databases, and/or other vehicles via network 290. For example, V2X communication capabilities allows hierarchical federated learning circuit 210 to communicate with edge/cloud servers, roadside infrastructure (e.g., such as roadside equipment/roadside unit, which may be a vehicle-to-infrastructure (V21)-enabled street light or cameras, for example), etc. Hierarchical federated learning circuit 210 may also communicate with other connected vehicles over vehicle-to-vehicle (V2V) communications.”
Regarding Claim 11:
The learning server of claim 10 is taught by Gao, Li, Chen, Berkley
wherein the selected vehicle agents upload the trained local models to the learning server based on one or combination of criteria (1) time specified by the learning server, (2) a predetermined number of local training iteration,
[Berkley 0022]: "and the termination criteria may comprise one of a number of iterations [(2) a predetermined number of local training iteration,], a processing time [wherein the selected vehicle agents upload the trained local models to the learning server based on one or combination of criteria (1) time specified by the learning server], a number of measurements, a threshold accuracy for the one or more determinable parameters, a number of digital processor cycles, and a number of quantum processor cycles."
The motivation to combine with Berkley is the same motivation to combine with Berkley from claim 1.
(3) local model training error reaching a predetermined threshold and
(4) local model training error stabilizing
[Li 4.1 Local Dissimilarity page 5]: “For most practical machine learning problems, there is no need to solve the problem to highly accurate stationary solutions [(4) local model training error stabilizing], i.e., e is typically not very small. Indeed, it is wellknown that solving the problem beyond some threshold [(3) local model training error reaching a predetermined threshold] may even hurt generalization performance due to overfitting (Yao et al., 2007).”
The motivation to combine with Li is the same motivation to combine with Li from claim 1.
Regarding Claim 13:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein at least two of the selected vehicle agents collect the local data using different types of two sensors respectively equipped on the least two of the selected vehicle agents.
[Gao 0056]: “The edge server can add the locally generated result parameters to optimize an aggregation of the one or more models. In another example, hierarchical federated learning client 205 can collect the data from sensors 252 and vehicle systems 258 [wherein at least two of the selected vehicle agents collect the local data using different types of two sensors respectively equipped on the least two of the selected vehicle agents] and apply the data to one or more models. The one or more models output parameters that the hierarchical federated learning client 205 can communicate to vehicle systems 258 for controlling operations of the vehicle systems 258 according to the one or more models.”
Support for the types of sensors [Gao 0036]: “Sensors 152 may be included to detect not only vehicle conditions but also to detect external conditions as well. Sensors that might be used to detect external conditions can include, for example, sonar, radar, lidar or other vehicle proximity sensors, and cameras or other image sensors. Image sensors can be used to detect objects in an environment surrounding vehicle 100, for example, traffic signs indicating a current speed limit, road curvature, obstacles, surrounding vehicles, and so on.”
Regarding Claim 14:
The learning server of claim 13 is taught by Gao, Li, Chen, Berkley
wherein the two sensors are a high-end GPS and a low-end GPS receiver, wherein the high-end GPS receiver provides more accurate measurements than that of the low-end GPS receiver.
[Gao 0054]: “A conventional GPS provides positional information that describes a position of a vehicle with an accuracy of plus or minus 10 meters [wherein the two sensors are… a low-end GPS receiver, wherein the high-end GPS receiver provides more accurate measurements than that of the low-end GPS receiver] of the actual position of the conventional GPS unit. By comparison, a DSRC-compliant GPS unit provides GPS data that describes a position of the DSRC-compliant GPS unit with an accuracy of plus or minus 1.5 meters [a high-end GPS] of the actual position of the DSRC-compliant GPS unit. This degree of accuracy is referred to as “lane-level accuracy” since, for example, a lane of a roadway is generally about 3 meters wide, and an accuracy of plus or minus 1.5 meters is sufficient to identify which lane a vehicle is traveling in on a roadway. Some safety or autonomous driving applications provided by an Advanced Driver Assistance System (ADAS) of a modern vehicle require positioning information that describes the location of the vehicle with lane-level accuracy. In addition, the current standard for DSRC requires that the location of the vehicle be described with lane-level accuracy.”
Regarding Claim 18:
Claim 18 is analogous to claim 1.
Regarding Claim 19:
The computer-implemented method of claim 18 is taught by Gao, Li, Chen, Berkley
Claim 19 is analogous to claim 5.
Claims 3 and 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with Imteaj et al (“FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots”), referred to as Imteaj in this document, and further in combination with Sabella et al (US 20210112441 A1), referred to as Sabella in this document, and further in combination with Cho et al (“Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies”), referred to as Cho in this document, and further in combination with Deo et al (“Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals”), referred to as Deo in this document.
Regarding Claim 3:
The learning server of claim 2 is taught by Gao, Li, Chen, Berkley
the selecting is performed based on one or combination of (1) randomly selecting vehicle agents,
[Li Algorithm 1 page 4] notes “selects a subset S of K devices at random”
PNG
media_image2.png
275
460
media_image2.png
Greyscale
The motivation to combine with Li is the same as the motivation to combine with Li in claim 1.
(2) selecting vehicle agents being connected to the network longer than a predetermined time period,
[Imteaj 3 System Description page 4]: “Every participant client in a training round must submit their model within a given time because it is not feasible for the server to wait for a client for an infinite amount of time. The task publisher can set the threshold time for a task.”
(4) selecting vehicle agents having better performances in previous global training rounds,
[Imteaj Section 5 Conclusion page 8]: “Instead of assuming all the FL clients are consistent and resource-efficient, we consider additional steps to check each client’s resources and previous training performance [(4) selecting vehicle agents having better performances in previous global training rounds,]. We enable a feature of assigning a trust score to each robot client, and avoiding the inconsistent and inadequate resource clients from the training process. To further handle the straggler effect, we selected the most proficient and reliable clients for an FL task and applied asynchronous FL to reduce the convergence time.”
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Imteaj. Gao and Imteaj are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Imteaj in order to help decide what vehicles to select and when to stop training a model, as a model cannot be trained forever ([Imteaj 3 System Description page 4]: “Every participant client in a training round must submit their model within a given time because it is not feasible for the server to wait for a client for an infinite amount of time. The task publisher can set the threshold time for a task.”) and selectively choosing agents helps with convergence time ([Imteaj Section 5 Conclusion page 8]: “We enable a feature of assigning a trust score to each robot client, and avoiding the inconsistent and inadequate resource clients from the training process. To further handle the straggler effect, we selected the most proficient and reliable clients for an FL task and applied asynchronous FL to reduce the convergence time”)
(3) selecting vehicle agents having better link quality to the associated RSUs,
[Sabella 0110]: “As shown in FIG. 8B, at stage 0, the PQoS learning model, for issuing joint radio and edge cloud PQoS predictions, is built by training an ML program based on cellular connectivity measurement data [(3) selecting vehicle agents having better link quality to the associated RSUs], such as RSRP/RSRQ measurements coming from all UEs connected to the cellular network (e.g., vehicles and other devices) and MEC resource utilization measurements from nearby MEC hosts (moving and static ones)”
More support given in [Sabella 0109]: “FIGS. 8A-8B illustrate a signaling framework to collaboratively train an automotive federated learning model for QoS predictions. To collaboratively train the PQoS federated learning model, an iterative process is utilized until the model converges to a predetermined level of accuracy. The federated PQoS learning model is called “holistic” because it includes both radio signal and edge cloud resource availability information.”
(6) selecting vehicle agents based on commutation resources and
[Sabella 0110]: “As shown in FIG. 8B, at stage 0, the PQoS learning model, for issuing joint radio and edge cloud PQoS predictions, is built by training an ML program based on cellular connectivity measurement data, such as RSRP/RSRQ measurements coming from all UEs connected to the cellular network (e.g., vehicles and other devices) and MEC resource utilization measurements [(6) selecting vehicle agents based on commutation resources] from nearby MEC hosts (moving and static ones)”
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Sabella. Gao and Sabella are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Sabella in order to be selective with choosing agents to result with better quality of service as communication and resource usage effects service which federates learning could be considered a service ([Sabella 0110]: “As shown in FIG. 8B, at stage 0, the PQoS learning model, for issuing joint radio and edge cloud PQoS predictions, is built by training an ML program based on cellular connectivity measurement data, such as RSRP/RSRQ measurements coming from all UEs connected to the cellular network (e.g., vehicles and other devices) and MEC resource utilization measurements from nearby MEC hosts (moving and static ones)”).
(5) selecting vehicle agents having larger datasets,
[Cho G Additional Experiment Results page 22]: "We are also not necessarily frequently selecting the clients that have the highest data size [(5) selecting vehicle agents having larger datasets supported by Cho noting that data size is a representation of importance at the end of this quote] such as client 26. This aligns well with our main motivation of POWER-OF-CHOICE that weighting the clients’ importance based on their data size does not achieve the best performance, and rather considering their local loss values along with the data size better represents their importance."
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Cho. Gao and Cho are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Cho in order to be selective with choosing agents to select things with more importance ([Cho G Additional Experiment Results page 22]: "We are also not necessarily frequently selecting the clients that have the highest data size such as client 26. This aligns well with our main motivation of POWER-OF-CHOICE that weighting the clients’ importance based on their data size does not achieve the best performance, and rather considering their local loss values along with the data size better represents their importance.").
(7) selecting vehicle agents based on distances to the associated RSUs.
[Deo 4.1 Encoding Scene and Agent Context page 4]: “Drivers co-operate with other drivers and pedestrians to navigate through traffic scenes. Thus, surrounding agents serve as a useful cue for trajectory prediction. Of particular interest are agents that might interact with the target vehicle’s route. We thus update node encodings with nearby agent encodings using scaled dot product attention [4]. We only consider agents within a distance threshold [(7) selecting vehicle agents based on distances to the associated RSUs] of each lane node to update the node encoding. This allows our trajectory decoder (Sec 4.3) to selectively focus on agents that might interact with specific routes that the target vehicle might take.”
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Deo. Gao and Deo are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Deo in order to select more relevant vehicles ([Deo 4.1 Encoding Scene and Agent Context page 4]: “Drivers co-operate with other drivers and pedestrians to navigate through traffic scenes. Thus, surrounding agents serve as a useful cue for trajectory prediction. Of particular interest are agents that might interact with the target vehicle’s route. We thus update node encodings with nearby agent encodings using scaled dot product attention [4]. We only consider agents within a distance threshold of each lane node to update the node encoding. This allows our trajectory decoder (Sec 4.3) to selectively focus on agents that might interact with specific routes that the target vehicle might take.”).
Regarding Claim 4:
The learning server of claim 3 is taught by Gao, Li, Chen, Berkley, Imteaj, Sabella, Cho, Deo
the learning server distributes the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the selected vehicle agents via the associated RSUs, wherein the learning server broadcasts the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the RSUs and the RSUs then respectively relay the received set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the associated vehicle agents.
[Gao 0016]: “As alluded to above, there are disadvantages to transmitting large quantities of data needed for model training in centralized machine learning platforms. Federated machine learning platforms (or networks) have been leveraged to reduce the this amount of data. In federate learning, machine learning models are trained at the vehicle side rather than at a centralized or edge server. Federated learning is an iterative process in which a vehicle downloads [the learning server distributes the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the selected vehicle agents via the associated RSUs, wherein the learning server broadcasts the set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the RSUs and the RSUs then respectively relay the received set of global model parameters
x
g
l
o
b
a
l
(
t
)
to the associated vehicle agents] a machine learning model to train the model locally on vehicle data (e.g., data acquired by the vehicle sensors and subsystems). The vehicle shares the locally trained models with a server, which aggregates locally trained models from a number of vehicles and shares the global aggregated model for accomplishing vehicle tasks. The global aggregated model is then a starting point for a subsequent iteration of local training at the vehicle side and aggregation at the server side. One major benefit of federated learning is that the vehicles need only transmit model parameters to a server, instead of large quantities of raw sensor data. This significantly reduces communication overhead and also overcomes privacy issues.”
Where the use of RSUs for the process is noted in [Gao 0044]: “Communication circuit 201 includes either or both a wireless transceiver circuit 202 with an associated antenna 214 and a wired I/O interface 204 with an associated hardwired data port (not illustrated). Communication circuit 201 can provide for vehicle-to-everything (V2X) and/or vehicle-to-vehicle (V2V) communications capabilities, allowing hierarchical federated learning circuit 210 to communicate with edge devices, such as roadside unit/equipment (RSU/RSE) [RSU], network cloud servers and cloud-based databases, and/or other vehicles via network 290. For example, V2X communication capabilities allows hierarchical federated learning circuit 210 to communicate with edge/cloud servers, roadside infrastructure (e.g., such as roadside equipment/roadside unit, which may be a vehicle-to-infrastructure (V21)-enabled street light or cameras, for example), etc. Hierarchical federated learning circuit 210 may also communicate with other connected vehicles over vehicle-to-vehicle (V2V) communications.”
Claims 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with Deo et al (“Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals”), referred to as Deo in this document.
Regarding Claim 9:
The learning server of claim 7 is taught by Gao, Li, Chen, Berkley
wherein the structure-aware model training method uses a graph encoder module configured to encode the road map, each of the vehicle agents and proximal vehicles into a directed graph, a policy header module configured to learn a discrete policy for each of the vehicle agents and the proximal vehicles, and a trajectory decoder module configured to predict trajectories of a vehicle agent by decoding sampled paths of the vehicle agent
[Deo 3.3 Output Representation page 4] notes in the description and pictures of figure 2 that a graph encoder module, policy header module, and trajectory decoder module exist to perform the steps of the limitations of encode into a directed graph, learn a discrete policy, and predict trajectory. Agents and proximal vehicles for the policy are noted in the figure by the figure showing both the vehicle of interest and surrounding vehicles.
PNG
media_image3.png
646
801
media_image3.png
Greyscale
One of ordinary skill in the art, prior to the effective filing, would have been motivated to combine Gao and Deo. Gao and Deo are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Deo to utilize the modules in Deo in order to be able to predict the movement of vehicles (Deo Figure 2 “Predicted trajectory” in the “Trajectory decoder”).
Claims 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with Mayyuri et al (US 20230385651 A1), referred to as Mayyuri in this document.
Regarding Claim 12:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein the selected vehicle agents partition local datasets into different clusters such that each cluster is used to train a particular machine learning model, whereing the local data collected at different location and diffenrent time are used to train the corresponding particular learning models
[Mayyuri 0030]: “In some aspects, the device periodically checks its zone membership and tabulates its training data [wherein the selected vehicle agents partition local datasets into different clusters such that each cluster is used to train a particular machine learning model] based on the zone membership.”
[Mayyuri 0031]: “In other aspects, the device stores training data along with parameters that are used by the zone membership function. For example, if trying to train a Human-activity-recognition (HAR) model using sensor data, and if the zone-partition keeper provides a zone determination function that accepts GPS coordinates as a parameter, then the device may store the raw sensor data along with GPS information in a sequential or timestamped manner [whereing the local data collected at different location and diffenrent time are used to train the corresponding particular learning models]. When the device is ready to perform local training, the device may use the GPS data included in the data samples to determine for which zone this data will be used.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Mayyuri. Gao and Mayyuri are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Mayyuri in order to make a more relevant model by utilizing data related to the time or location ([Mayyuri 0031]: “When the device is ready to perform local training, the device may use the GPS data included in the data samples to determine for which zone this data will be used”).
Claims 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with Chakraborty et al (US 20230316090 A1), referred to as Chakraborty in this document.
Regarding Claim 15:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein the global machine learning model is trained by using neural networks with adaptive momentum optimizers.
[Chakraborty 0033]: “…enables more advanced updating schemes, such as adaptive momentum [wherein the global machine learning model is trained by using neural networks with adaptive momentum optimizers] (e.g., the Adam algorithm). However, in other aspects, federated learning may be implemented using one or more other suitable algorithms.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Chakraborty. Gao and Chakraborty are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Chakraborty in order to enable or create more advanced updating schemes for improving a model ([Chakraborty 0033]: “…enables more advanced updating schemes, such as adaptive momentum (e.g., the Adam algorithm). However, in other aspects, federated learning may be implemented using one or more other suitable algorithms.”).
Claims 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with U et al (US 20210266284 A1), referred to as U in this document, and further in combination with Budhraja et al (US 20230325427 A1), referred to as Budhraja in this document.
Regarding Claim 16:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein training of the global machine learning model is initiated by one or combination of 1) periodic model training,
[U 0026]: “According to examples of the present disclosure, the training trigger event may be expiry of a timer [wherein training of the global machine learning model is initiated by one or combination of 1) periodic model training] (for example periodic training on a daily, weekly, monthly basis) or some other trigger event. For example, updating of the model may be triggered when a training data set of a threshold size has been assembled.”
2) event based model training and
[U 0025]: “According to examples of the present disclosure, the method may further comprise checking for a training trigger event [2) event based model training and] and on detection of the training trigger event, updating the model using the assembled training data set.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and U. Gao and U are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and U in order to have triggers to train a model to keep the model up to date or relevant by updating ([U 0025]: “According to examples of the present disclosure, the method may further comprise checking for a training trigger event and on detection of the training trigger event, updating the model using the assembled training data set.”)
3) feedback based model training
[Budhraja 0100]: “In FIG. 9 a user has given feedback regarding the domain-specific tags by approving two of the four tags as accurate. This feedback is added to existing databases, e.g. the module of FIG. 7, where this feedback [3) feedback based model training] is used to trigger re-training of the AI agents.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Budhraja. Gao and Budhraja are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Budhraja in order to have triggers to train a model to keep the model up to date or relevant by updating ([Budhraja 0100]: “In FIG. 9 a user has given feedback regarding the domain-specific tags by approving two of the four tags as accurate. This feedback is added to existing databases, e.g. the module of FIG. 7, where this feedback is used to trigger re-training of the AI agents.”)
Claims 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al (US 20240265296 A1), referred to as Gao in this document, and further in combination with Li et al (“Federated Optimization In Heterogeneous Networks”), referred to as Li in this document, and further in combination with Chen et al (“A Theorem of the Alternative for Personalized Federated Learning”), referred to as Chen in this document, and further in combination with Berkley et al (US 20240028938 A1), referred to as Berkley in this document, and further in combination with Butler et al (US 20230063986 A1), referred to as Butler in this document.
Regarding Claim 17:
The learning server of claim 1 is taught by Gao, Li, Chen, Berkley
wherein learning server distributes well-trained global machine learning models to all on-road vehicles for their applications, wherein the on-road vehicles apply the well-trained global machine learning models to respective tasks of the on-road vehicles such as trajectory prediction,
[Gao 0002]: “Machine learning approaches train mathematical models to perform tasks such as making predictions based on input features. Autonomous vehicles and intelligent driving-assistance systems increasingly rely on machine-learning approaches to accomplish their tasks. For example, a machine learning model can be trained to recognize objects from RGB images or to predict the trajectory [wherein learning server distributes well-trained global machine learning models to all on-road vehicles for their applications, wherein the on-road vehicles apply the well-trained global machine learning models to respective tasks of the on-road vehicles such as trajectory prediction where more information on the distribution of the global model to vehicles is given in Gao 0016] of external road agents (e.g., vehicles, cyclists, pedestrians, etc.) based on various types of input sensor data.”
velocity prediction,
[Gao 0056]: “For example, autonomous or semi-autonomous driving systems 280 (or other sensors 252), can determine position and velocity of the vehicle [velocity prediction], and/or location of obstacles or other environmental features including traffic signs, trees, shrubs, neighboring vehicles, pedestrians, etc.”
and ADAS/AD parameter calibration
[Gao 0054]: “A conventional GPS provides positional information that describes a position of a vehicle with an accuracy of plus or minus 10 meters of the actual position of the conventional GPS unit. By comparison, a DSRC-compliant GPS unit provides GPS data that describes a position of the DSRC-compliant GPS unit with an accuracy of plus or minus 1.5 meters of the actual position of the DSRC-compliant GPS unit. This degree of accuracy is referred to as “lane-level accuracy” since, for example, a lane of a roadway is generally about 3 meters wide, and an accuracy of plus or minus 1.5 meters is sufficient to identify which lane a vehicle is traveling in on a roadway. Some safety or autonomous driving applications provided by an Advanced Driver Assistance System (ADAS) of a modern vehicle require positioning information [and ADAS/AD parameter calibration] that describes the location of the vehicle with lane-level accuracy. In addition, the current standard for DSRC requires that the location of the vehicle be described with lane-level accuracy.”
energy consumption prediction
[Butler 0047]: “A machine learning model to predict metered energy consumption is developed using the combined data, aggregated by meter periods, as inputs to train development of a machine learning model to predict the targeted metered energy consumption [energy consumption prediction] for each meter period for each residence, as shown in FIG. 11. Separate models are developed for each fuel type.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Gao and Butler. Gao and Butler are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Gao and Butler in order to have a model predict energy consumption to help understand energy saved or wasted ([Butler 0048]: “The trained models are then used to predict energy consumption savings from energy efficient investments and thermostat changes as shown in FIG. 12. The developed machine learning models are applied to data inputs including prior thermostat setpoint schedule and weather data for a new meter period to predict the targeted energy consumption that would be realized had the residence not experienced any changes.”)
Allowable Subject Matter
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Claims 6 and 20 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, and 35 U.S.C. 101 set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 6:
The learning server of claim 5 is taught by Gao, Li, Chen, Berkley
wherein while aggregating the trained local models, the learning server applies a variance-based optimal weight simplex
p
=
(
p
1
,
…
,
p
n
)
computed according to
p
i
=
N
i
σ
i
2
∑
j
=
1
n
N
j
σ
j
2
, (i = 1, 2, …, n) where n is a number of the selected vehicle agents,
N
i
is a number of data samples of vehicle agents i and
σ
i
2
is the variance of vehicle agent i.
The prior art of record does not teach:
wherein while aggregating the trained local models, the learning server applies a variance-based optimal weight simplex
p
=
(
p
1
,
…
,
p
n
)
computed according to
p
i
=
N
i
σ
i
2
∑
j
=
1
n
N
j
σ
j
2
, (i = 1, 2, …, n) where n is a number of the selected vehicle agents,
N
i
is a number of data samples of vehicle agents i and
σ
i
2
is the variance of vehicle agent i.
The closest prior art of record teaches:
Chen notes the use of a weight simplex weight simplex
p
=
(
p
1
,
…
,
p
n
)
as shown by the teachings for claim 5 [Chen et al 4.2 Analysis of Baseline Algorithms page 15]. However, Chen does not teach a variation of p that matches the equation given in claim 6. Chen notes some examples of equation such as pi = 1/m and pi=ni/N ([Chen et al 4.2 Analysis of Baseline Algorithms page 15]: “For example, setting pi = 1/m enforces ‘fair allocation’, so that each client is treated uniformly, regardless of sample sizes. As another example, setting pi=ni/N means that the central server pays more attention to clients with larger sample sizes, which, to a certain extend, incentivize the clients to contribute more data”). Gao in claim 1 notes the use of variance ([Gao 0020]), but does not provide equations.
Research into the subject gives the indication the equation in the current application is related inverse variance weighting, which is supported by [Current Application 0071]: “The theorem states that in order to minimize generalization error, the optimal aggregation weight is proportional to the local dataset size and inversely proportional to the variance of the local dataset”. The equation in claim 6 appears even more complex than the indication given in paragraph 71, as
N
i
σ
i
2
indicates a use of global variance as Ni seems to represent global data while
σ
i
2
is the variance of said global data. Then
∑
j
=
1
n
N
j
σ
j
2
indicates a use of a local variance as Nj where the
σ
j
2
here indicates a variance of the local data.
Zhang et al (“Proportional Fairness in Federated Learning”) indicates many of the relevant elements in table 2 on page 19, but does not appear to ever combine them in the way given in claim 6.
[Zhang et al (“Proportional Fairness in Federated Learning”) table 2 page 19 but highlighted for emphasis]
PNG
media_image4.png
902
676
media_image4.png
Greyscale
As a result of no prior art of record containing elements of the equation for variance-based optimal weight simplex as shown in claim 6, claim 6 is determined to have allowable subject matter and thus not rejected under 103.
Regarding Claim 20:
This claim is analogous to claim 6., thus reasoning for allowable subject matter is the same as claim 6.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Soltani et al (“A Survey on participant Selection for Federated Learning in Mobile Networks”) is relevant art that discusses methods of selected participants in federated learning, which is relevant to aspects of the current disclosure, such as claim 3, discussing methods of selecting participants of vehicle agents.
Karimireddy et al (“SCAFFOLD: Stochastic Controlled Averaging for Federated Learning”) is relevant art that discusses an algorithm noted in the spec as an alternative to FedProx called Scaffold that is used for aggregating in systems with heterogeneous data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D DEVORE whose telephone number is (703)756-1234. The examiner can normally be reached Monday-Friday 7:30 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.D.D./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129