DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of prior-filed a U.S. National Stage Application filed under 35 U.S.C. §371, International Patent Application No. PCT/JP2020/035622, filed on 18 September 2020,, which is acknowledged.
Drawings
The drawings were received on 10/24/2019. These drawings are acceptable.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on the following date(s): 07/23/2024 and 03/14/2023 have been considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 17, the claim recites the limitation “provide, to the plurality of clients, the global Neural Graphical Model” twice that renders the claim indefinite because the intended scope of the claimed invention is unclear. Specifically, is the repeated language a typo or a prompt for repeating claimed limitation? The applicant should consider re-writing the second recitation such that the limitation is further limiting and/or clarifies the intended scope of the claimed invention.
Regarding the claims that depend from claim 17 fail to resolve the noted deficiencies and thus appropriately rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding claims 17-20 of the claimed invention are directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claims fail to recite a hardware component. Specifically, the claim recites a device including a processor which includes virtual processors and other data objects and the recited memory includes signals where the recited elements are not directed to a hardware elements but rather data processing objects. Products that do not have a physical or tangible form, such as information (often referred to as "data per se") and thus deemed as claims that do not fall within at least one of the four categories of patent eligible subject matter.
Applicant should amend claims to directed the claim to a physical or tangible form in light of support provided in the original specification.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Claim 1: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
(Considered directed to a mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I); Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
The broadest reasonable interpretation (BRI) includes a mathematical relationship. Also see specification paragraph 0022.)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
receiving, from a plurality of clients, a plurality of feature dependency graphs, … (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
wherein each client provides a feature dependency graph created using data of each client; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
and providing, to the plurality of clients, the global dependency graph. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the global dependency graph captures a dependency structure of input features for a domain of the data used by each client to create the plurality of feature dependency graphs (Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
… to create the plurality of feature dependency graphs without sharing the data (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 3: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the dependency structure identifies which features in the data are directly dependent on each other and which pairs of features in the data exhibit conditional independencies given other features. (Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 4: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
(Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
using, by each client of the plurality of clients, a graph recovery algorithm on the data of each client to … (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 5: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the merge function further includes: identifying common features of the plurality of feature dependency graphs; and providing a union of edges among the common features in the global dependency graph. (Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 6: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph. (Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 7: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea in claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the data is private data to the plurality of clients. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 8: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
(Considered directed to a mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I); Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
The broadest reasonable interpretation (BRI) includes a mathematical relationship. Also see specification paragraph 0022.)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
receiving, from a plurality of clients, a plurality of Neural Graphical Models, … (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
wherein each Neural Graphical Model is trained locally by a client using a global dependency graph and data of the client; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
training a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph, (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
and providing, to the plurality of clients, the global Neural Graphical Model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 9, the limitations are similar to those in claim 2, and are thus rejected under the same rationale.
Claim 10: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea from claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 11: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein training the global Neural Graphical Model further includes: learning an average of the distribution over input features for the domain of the data in the global dependency graph. (Considered directed to a mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I); Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 12: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein learning the average of the distribution over the features further includes: adjusting the distribution to a weighted average of the plurality of Neural Graphical Models; and adjusting a dependency structure of the global Neural Graphical Model to the dependency structure of the global dependency graph. (Considered directed to a mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I); Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 13: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein training the global Neural Graphical Model further includes: leveraging publicly available data during the training. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 14: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each client of the plurality of clients personalizes, using an algorithm, the global Neural Graphical Model to the data of each client, wherein the algorithm adds features specific to the data of a client to the global Neural Graphical Model of the client. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally recites an effect of the judicial exception or claims every mode of accomplishing that effect. Thus, claim limitations amounts to a claim that is merely adding the words "apply it" to the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 15: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each client of the plurality of clients uses the global Neural Graphical Model to perform inference tasks or sampling tasks on the data. Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally recites an effect of the judicial exception or claims every mode of accomplishing that effect. Thus, claim limitations amounts to a claim that is merely adding the words "apply it" to the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 16: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea in claim 8.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each client of the plurality of clients uses a personalized Neural Graphical Model to perform inference tasks or sampling tasks on the data. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally recites an effect of the judicial exception or claims every mode of accomplishing that effect. Thus, claim limitations amounts to a claim that is merely adding the words "apply it" to the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 17: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs (Considered directed to a mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I); Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
The broadest reasonable interpretation (BRI) includes a mathematical relationship. Also see specification paragraph 0022.)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
a memory to store data and instructions; and a processor operable to communicate with the memory, wherein the processor is operable to … a plurality of Neural Graphical Models trained locally by a client using data from the client and the global dependency graph (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
receive, from the plurality of clients, a plurality of Neural Graphical Models trained locally by a client using data from the client and the global dependency graph; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
provide, to the plurality of clients, the global dependency graph… and provide, to the plurality of clients, the global Neural Graphical Model (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
train a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph, (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).; Alternatively Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally recites an effect of the judicial exception or claims every mode of accomplishing that effect. Thus, claim limitations amounts to a claim that is merely adding the words "apply it" to the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 18, the limitations are similar to those in claims 2 and 10, and are thus rejected under the same rationale. The rejection of claim 17 is incorporated.
Regarding claim 19, the limitations are similar to those in claim 11 and are thus rejected under the same rationale. The rejection of claim 17 is incorporated.
Regarding claim 20, the limitations are similar to those in claim 14 and are thus rejected under the same rationale. The rejection of claim 17 is incorporated.
As shown above, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more” than the recited judicial exception. The claims are therefore directed to an abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 7 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hu et al. (US 20250209383, hereinafter ‘Hu’).
Regarding independent claim 1, Hu teaches a method, comprising: receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client; ([0017] Federated learning or collaborative learning may allow a central server to distribute ML models to a number of local sites (i.e., federated nodes) for training these ML models based on respective local data samples on these local sites. The distributed ML models may be separately trained on respective local sites based on corresponding local data [receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client]. The trained models and some parameters may be sent back to the central server to be aggregated into the global model… [0026] FIG. 2 illustrates an example framework 200 for using a knowledge graph system 210 to coordinate and optimize federated learning... In particular embodiments, the knowledge graph system 210 may fetch or receive information related to the central server 201 and local computing systems (e.g., 203) of the federate nodes (including local models [receiving, from a plurality of clients, a plurality of feature dependency graphs] (e.g., 204)) that directly communicate with the central server 201 to generate a global knowledge graph. In particular embodiments, the knowledge graph system 210 may receive information from the local computing systems at the local sites [wherein each client provides a feature dependency graph created using data of each client] using APIs or other interface modules through computer storages, data buses, or communication networks. In particular embodiments, the knowledge graph system 210 may receive information [receiving, from a plurality of clients, a plurality of feature dependency graphs] about the central server 201 and the local computing system 202 of a federated node from direct or indirect user inputs characterizing the server, the federated nodes, and the relationships… [0027] In particular embodiments, the knowledge graph system 210 may generate knowledge graphs [receiving, from a plurality of clients, a plurality of feature dependency graphs] based on the fetched or received information related to the central server 201 and the local computing systems (e.g., 202) at corresponding federated nodes [wherein each client provides a feature dependency graph created using data of each client;]. For example, the knowledge graph system 210 may send the fetched or received information to the graph engine 216 and use the graph engine 216 to generate, store, and manage the knowledge graphs…)
generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs, wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs; (in [0036] In particular embodiments, the federated learning process may be integrated with the graph learning process using the knowledge graph(s)... After being trained, the local models and corresponding parameters may be transmitted back to the centralized server. The central server may aggregate [generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs] all these distributed local models that have been trained on respective federated nodes and generate the final training model. The final model may be updated based on be an aggregation of a number of local models that are trained on respective federated nodes [wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graph]. The above process may correspond to an iteration of the federated learning process based on the knowledge graph system.)
and providing, to the plurality of clients, the global dependency graph. (in [0041] ... As a result, the data sharing policies may be enforced with the data privacy well protected. In particular embodiments, the federated learning system may use the knowledge graph to enforce the data sharing polices between different federated nodes or between the federated nodes and the central server. In particular embodiments, the system may allow the local data information to be accessed by the global model according to corresponding sharing polices… [0043] In particular embodiments, the federated learning system may use the global knowledge graph [providing, to the plurality of clients, the global dependency graph] to track and manage the over knowledge of all nodes at a global level (and sub-network levels) and use the knowledge graph to coordinate and optimize the federated learning process. The federated learning system may use the graph learning to generate inferred knowledge based on existing knowledge and may classify federated nodes into different categories or types… . In particular embodiments, the federated learning system may use graph learning to generate inferences related to models, parameters, and other data from a global perspective (e.g., by the central server) and to predict information for final training and updated training. In particular embodiments, the federated learning system may use graph learning to generate inference values and predictions that can be used to adjust the model training in the next iteration. In particular embodiments, the federated learning system may generate some inferences based on the global knowledge graph or based on federated knowledge graphs of respective sub-networks… )
Regarding claim 7, the rejection of claim 1 is incorporated and Hu teaches the method of method of claim 1, wherein the data is private data to the plurality of clients. (in [0019] By using a knowledge graph, particular embodiments of the system may provide an effective solution for tracking and enforcing the data sharing conditions for federated learning. By tracking and enforcing the data sharing conditions of federated learning, particular embodiments of the system may allow the data privacy of each local site [wherein the data is private data to the plurality of clients] to be better protected during the federated learning process. By generating inferences based on the knowledge graph and using these inferences to coordinate the federated learning process, particular embodiments of the system may provide an effective solution to optimize federated learning to allow the ML models to be better trained… And in [0042] In particular embodiments, because of data privacy, federated learning may be designed in a way that may not allow the local data at the federated nodes to be shared with server or other federated nodes [wherein the data is private data to the plurality of clients]. As a result, training of each local model may have to rely on the local data samples of the respective federated nodes…)
Claims 1-7 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ahmed et al. (US 12078498, hereinafter ‘Ahmed’).
Regarding independent claim 1, Amed teaches a method, comprising: (in 1:46-48: FIG. 10 is flow diagram for federated GNN training across different Transportation Network Companies (TNCs), according to some example embodiments…; And in 11:17-23: In some example embodiments, a federated learning approach is used where multiple TNCs leverage a common dataset to optimize their business model, without sharing proprietary information. Federated learning (also known as collaborative learning) is a machine-learning technique that trains an algorithm across multiple decentralized devices holding local data, without exchanging the data…)
receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client; (in 12:49-67: Agents at different TNCs 1002 [from a plurality of clients … ] can obtain 1006 different data from sensors and infrastructure and this data is kept at the local TNCs 1002 [wherein each client provides a feature dependency graph created using data of each client] and used to form signals for the graphs (e.g., users traveling through a node, demand at a node, etc.). This collected information is used to train different GNNs at each TNC to determine usage demands... To facilitate sharing of data, a federated learning (FL) training of GNNs is utilized, where each TNC collaborates to learn a common model ϕ.sup.G which can be iteratively updated from the local models ϕ.sub.i from each of the TNCs. The TNCs use the same graph structure to apply the FL approach. The aggregator 1004 initializes 1014 the common graph set at the agents of the TNCs 1002 [receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client], that is, providing the structure of the nodes and the edges, as well as the features associated with the graph. Then the aggregator 1004 shares (operation 1016) the global GNN model ϕ.sup.G with the agents… And in 16:16-20: In one example, the GNN is generated using a federated model of a plurality of transportation network companies (TNCs), wherein each of the TNCs generates a local demand model and an aggregator combines the local demand models into a general demand model [receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client]. )
generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs, wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs; (in 12:49-67: … To facilitate sharing of data, a federated learning (FL) training of GNNs is utilized, where each TNC collaborates to learn a common model ϕ.sup.G which can be iteratively updated from the local models ϕ.sub.i from each of the TNCs. The TNCs use the same graph structure to apply the FL approach. The aggregator [generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs] 1004 initializes 1014 the common graph set at the agents of the TNCs 1002 [wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs], that is, providing the structure of the nodes and the edges, as well as the features associated with the graph. Then the aggregator 1004 shares (operation 1016) the global GNN model ϕ.sup.G with the agents… And in 16:16-20: In one example, the GNN is generated using a federated model of a plurality of transportation network companies (TNCs), wherein each of the TNCs generates a local demand model and an aggregator combines the local demand models into a general demand model [generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs]. )
and providing, to the plurality of clients, the global dependency graph. (in 12:63:13:6: 13:1-6: The aggregator 1004 initializes 1014 the common graph set at the agents of the TNCs 1002, that is, providing the structure of the nodes and the edges, as well as the features associated with the graph. Then the aggregator 1004 shares (operation 1016) the global GNN model ϕ.sup.G with the agents [providing, to the plurality of clients, the global dependency graph]. The TNCs 1002 perform training updates 1008 on the model using local graph information X.sub.i and obtain the local model ϕ.sub.i. The TNCs 1002 share the calculated models with the aggregator 1004. At operation 1010, the aggregator 1004 merges the received ϕ.sub.i updates from the TNCs and recalculates the new global model ϕ.sup.G using FL.]. )… )
Regarding claim 2, the rejection of claim 1 is incorporated and Ahmed teaches the method of claim 1, wherein the global dependency graph captures a dependency structure of input features for a domain of the data used by each client to create the plurality of feature dependency graphs without sharing the data. (in 11:17-24: In some example embodiments, a federated learning approach is used where multiple TNCs leverage a common dataset to optimize their business model [wherein the global dependency graph captures a dependency structure of input features for a domain of the data used by each client to create the plurality of feature dependency graphs], without sharing proprietary information [wherein the global dependency graph captures a dependency structure of input features for a domain of the data used by each client to create the plurality of feature dependency graphs without sharing the data]. Federated learning (also known as collaborative learning) is a machine-learning technique that trains an algorithm across multiple decentralized devices holding local data [… input features for a domain of the data used by each client to create the plurality of feature dependency graphs], without exchanging the data…)
Regarding claim 3, the rejection of claim 2 is incorporated and Ahmed teaches the method of method of claim 2, wherein the dependency structure identifies which features in the data are directly dependent on each other and which pairs of features in the data exhibit conditional independencies given other features. (in 14:13-28: The training data 1112 comprises examples of values for the features 1102. In some example embodiments, the training data 1112 comprises labeled data with examples of values for the features 1102 and labels indicating the outcome, such as trip demand, trips traveled by users, sensor information, etc. The machine-learning algorithms utilize the training data 1112 to find correlations among identified features [wherein the dependency structure identifies which features in the data are directly dependent on each other] 1102 that affect the outcome [and which pairs of features in the data exhibit conditional independencies given other features as the affect the outcome feature exhibits in the training data features]. A feature 1102 is an individual measurable property of a phenomenon being observed. The concept of a feature is related to that of an explanatory variable used in statistical techniques such as linear regression. Choosing informative, discriminating, and independent features [and which pairs of features in the data exhibit conditional independencies given other features] is important for effective operation of ML in pattern recognition, classification, and regression. Features may be of different types, such as numeric features, strings, and graphs.)
Regarding claim 4, the rejection of claim 1 is incorporated and Ahmed teaches the method of method of claim 1, further comprising: using, by each client of the plurality of clients, a graph recovery algorithm on the data of each client to generate the feature dependency graph with a dependency structure of the data. (in 12:46-54: FIG. 10 is a flow diagram for federated GNN training across different Transportation Network Companies (TNCs) [ further comprising: using, by each client of the plurality of clients], according to some example embodiments. Agents at different TNCs 1002 can obtain 1006 different data from sensors and infrastructure and this data is kept at the local TNCs 1002 and used to form signals for the graphs [a graph recovery algorithm on the data of each client to generate the feature dependency graph with a dependency structure of the data] (e.g., users traveling through a node, demand at a node, etc.). This collected information is used to train different GNNs at each TNC [further comprising: using, by each client of the plurality of clients, a graph recovery algorithm on the data of each client to generate the feature dependency graph with a dependency structure of the data] to determine usage demands. )
Regarding claim 5, the rejection of claim 1 is incorporated and Ahmed teaches the method of method of claim 1, wherein the merge function further includes identifying common features of the plurality of feature dependency graphs; and providing a union of edges among the common features in the global dependency graph. (in 12:55-67: Sharing of data can be sensitive between the TNCs due to privacy concerns, but having the additional data can greatly improve the accuracy of the prediction models. To facilitate sharing of data, a federated learning (FL) training of GNNs is utilized, where each TNC collaborates to learn a common model ϕ.sup.G [wherein the merge function further includes identifying common features of the plurality of feature dependency graphs; and providing a union of edges among the common features in the global dependency graph] which can be iteratively updated from the local models ϕ.sub.i from each of the TNCs. The TNCs use the same graph structure to apply the FL approach. The aggregator 1004 initializes 1014 the common graph set at the agents of the TNCs 1002, that is, providing the structure of the nodes and the edges, as well as the features associated with the graph. Then the aggregator 1004 shares (operation 1016) the global GNN model ϕ.sup.G with the agents. )
Regarding claim 6, the rejection of claim 1 is incorporated and Ahmed teaches the method of method of claim 1, wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph. (15:18-28: Some examples of model parameters include maximum model size, maximum number of passes over the training data, data shuffle type, regression coefficients, decision tree split locations, and the like. Hyperparameters may include the number of hidden layers in a neural network, the number of hidden nodes in each layer [wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph], the learning rate (perhaps with various adaptation schemes for the learning rate), the regularization parameters, types of nonlinear activation functions, and the like. Finding the correct (or the best) set of hyperparameters can be a very time-consuming task that makes use of a large amount of computer resources... And in 2:39-54: Further, a graph neural network (GNN) machine learning (ML) architecture is presented to process travel-related historical information on a graph to predict the usage demand for rideshare applications. Road networks naturally lend to be interpreted as graph networks, with nodes indicating locations and edges representing the connecting roads. The locations can indicate the hotspots that can be the pickup or drop-off points. A signal on the graph can be defined such that each node is characterized by a feature vector [wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph] that corresponds to the attributes relating to dynamic information such as traffic density in the location, congestion incidents, user interactions (e.g., incoming and outgoing users from public transport/by foot), etc. Given a graph signal at a given time, the system predicts the signal at a future time and then labels each node based on its predicted signal as a high or low demand location spot.)
Claims 1 and 6 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chen et al. (NPL: FedE: Embedding Knowledge Graphs in Federated Setting, hereinafter ‘Chen’).
Regarding independent claim 1, Cheni teaches a method, comprising: receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client; (As depicted in Fig. 2:
PNG
media_image1.png
622
1148
media_image1.png
Greyscale
Figure 2: Overview of FedE. Illustrating the procedure of the client updating and the server operation at the round t.
2.2.2 Client Updating. At the round t, client c updates Ec and Rc according to specific knowledge graph embedding methods… After P epochs of training on the client c at the round t, updated entity embeddings Ec t+1 will be generated and uploaded to the server [receiving, from a plurality of clients, a plurality of feature dependency graphs, wherein each client provides a feature dependency graph created using data of each client]…)
generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs, wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs; (in 2.2.1 Server Operation. The server in FedE is responsible for aggregating entity embeddings from different clients and sending aggregated entity embeddings back to each client. We refer to a single server-side aggregation operation as one round of FedE. First, based on the entity table, T, the server construct a set of permutation matrices {Pc ∈ {0,1}n×nc}C c=1 and existence vec tors {vc ∈ {0,1}n×1}C c=1 for aggregating entity embeddings [generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs, wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs] from clients, whereC is the total number of clients,n is the number of all entities in the table T and nc is the number of entities from client c… After training embeddings with specific number of epochs in each c ∈ Ct, server aggregates entity embeddings [generating, using a merge function, a global dependency graph from the plurality of feature dependency graphs] Ec t+1 from client c as follows:
PNG
media_image2.png
98
570
media_image2.png
Greyscale
[wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs] …)
and providing, to the plurality of clients, the global dependency graph. (in 2.2.1 Server Operation. The server in FedE is responsible for aggregating entity embeddings from different clients and sending aggregated entity embeddings back to each client [providing, to the plurality of clients, the global dependency graph]. We refer to a single server-side aggregation operation as one round of FedE… At the end of roundt, aggregated Et+1 will be distributed back to each client c ∈ Ct [providing, to the plurality of clients, the global dependency graph] after conducting a permutation in the same way as Equation (1), so the entity embedding matrix Ec will be updated by Pc⊤Et+1.)
Regarding claim 6, the rejection of claim 1 is incorporated and Chen teaches the method of method of claim 1, wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph. (As depicted in Fig. 2:
PNG
media_image3.png
670
1386
media_image3.png
Greyscale
And in 2.2.1 Server Operation. The server in FedE is responsible for aggregating entity embeddings from different clients and sending aggregated entity embeddings back to each client [wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph]. We refer to a single server-side aggregation operation as one round of FedE. First, based on the entity table, T, the server construct a set of permutation matrices {Pc ∈ {0,1}n×nc}C c=1 and existence vectors {vc ∈ {0,1}n×1}C c=1 for aggregating entity embeddings from clients, where C is the total number of clients, n is the number of all entities in the table T and nc is the number of entities from client c [wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph]. Specifically, the permutation matrix Pc is used for mapping the entity embedding matrix from client c to the server’s entity table, where Pc i,j = 1 if the i-th entity in the entity table corresponds to the j-th entity from client c. vc i = 1 indicates that the i-th in the entity table (i.e., Ti) exists in client c… … After training embeddings with specific number of epochs in each c ∈ Ct, server aggregates entity embeddings [wherein the merge function maintains a size of a number of parameters of the global dependency graph of the same order of magnitude as clients' dependency graphs when combining the plurality of feature dependency graphs into the global dependency graph] Ec t+1 from client c as follows:
PNG
media_image4.png
78
454
media_image4.png
Greyscale
where 1 denotes an all-one vector, ⊘ denotes the element-wise division for vectors and ⊗ denotes the element-wise multiply withbroadcasting, i.e., [v ⊗ M]i,j = vi × Mi,j. At the end of round t, aggregated Et+1 will be distributed back to each client c ∈ Ct after conducting a permutation in the same way as Equation (1), so the entity embedding matrix Ec will be updated by Pc⊤Et+1 [graphs when combining the plurality of feature dependency graphs into the global dependency graph]. Chen discloses the information processing techniques for processing vector at each client c and server of the same magnitude.
Claims 8-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Xu et al. (US 20240303504, hereinafter ‘Xu’).
Regarding independent claim 8, Xu teaches a method, comprising: receiving, from a plurality of clients, a plurality of Neural Graphical Models, wherein each Neural Graphical Model is trained locally by a client using a global dependency graph and data of the client; (in [0084] At 304, in at least one embodiment, client K (e.g., client A 112) trains local model based on said current global model and neural network training data [wherein each Neural Graphical Model is trained locally by a client using a global dependency graph and data of the client] (e.g., private data described herein). In at least one embodiment, said local model can be any of local model 153 or local model 242… [0087] At 310, in at least one embodiment, said client K sends said importance value of client K to said federated server with at least one portion of neural network training information (e.g., local updates). At 312, in at least one embodiment, said federated server aggregates all local models [receiving, from a plurality of clients, a plurality of Neural Graphical Models as received local models] based on such importance values received from client 1 through client N [from a plurality of clients] to generate a new global model…; And local models as claimed Neural Graphical Models listed in [0060] In at least one embodiment, models (e.g., local models 153, 242, global models 154, 226 described herein) refers to various neural networks described herein, which include, without limitation, feedforward neural network, convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory (LSTM) network, autoencoder, generative adversarial network (GAN), restricted boltzmann machine (RBM), deep belief networks (DBN), radial basis function network (RBFN), hopfield network, self-organizing maps, perceptron's with one or more layers, modular neural networks, spiking neural networks, deep reinforcement learning networks, echo state networks, time-delay neural networks, support vector machines, attention-based neural networks, autoencoders, graph neural networks, any neural networks suitable for computer vision and/or NLP…)
training a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph, wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model; (in [0087] At 310, in at least one embodiment, said client K sends said importance value of client K to said federated server with at least one portion of neural network training information (e.g., local updates). At 312, in at least one embodiment, said federated server aggregates all local models based on such importance values received from client 1 through client N to generate a new global model [training a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph]. In at least one embodiment, federated server performs weighted averaging of all local models based on using importance as weights. For example, in at least one embodiment, said federated server uses modules (e.g., global generation module 104) to compute … where Z.sub.k=kΣ.sub.i=1.sup.NΣ.sub.k Γ.sub.k,i.sup.m is a normalization factor to ensure Σ.sub.i=1.sup.Nρ.sub.k,i.sup.m=1, and Γ.sub.k,N.sup.m, is an example contribution of client K. Then, in at least one embodiment, said modules generate new global model by performing w.sub.k+1←w.sub.k−ηE.sub.i=1.sup.Nρ.sub.k,i.sup.m.Math.∇F.sub.i(w.sub.k,i)… [0093] At 404, in at least one embodiment, said clients generate global gradients based on current global model and previous global model [training a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph]. In at least one embodiment, said current global model is derived based on global updates (e.g., global update 222 or global update 152) and previous global model is based on global updates happened in last round of federated training [wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model]. For example, in at least one embodiment, said clients perform ∇F(w.sub.k)=w.sub.k−w.sub.k−1. Alternatively, in at least one embodiment, global gradients are generated based on aggregating all local gradients received from clients [wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model]...)
and providing, to the plurality of clients, the global Neural Graphical Model. (in [0104] In at least one embodiment, model training module 512 is a module to train one or more neural networks, for example, using federated learning. In at least one embodiment, model training module 512 causes federated server (e.g., federated server 102) to send global models (e.g., global model 154, global model 226) to clients [providing, to the plurality of clients, the global Neural Graphical Model] (e.g., client A 112, client B 132, client C 142, client 202)... )
Regarding claim 9, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein the global dependency graph captures a dependency structure of input features for the domain of the data used to create the dependency structure without sharing the data. (in [0066] In at least one embodiment, each client in said federated learning framework stores its own private data, which isn't shared [… without sharing the data] by clients during training [wherein the global dependency graph captures a dependency structure of input features for the domain of the data used to create the dependency structure…]…)
Regarding claim 10, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model. (in [0139] In at least one embodiment, data center 800 may include tools, services, software or other resources to train one or more machine learning models or predict or infer information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by calculating weight parameters according to a neural network architecture [wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model] using software and computing resources described above with respect to data center 800. In at least one embodiment, trained machine learning models corresponding to one or more neural networks may be used to infer or predict information using resources described above with respect to data center 800 by using weight parameters calculated through one or more training techniques described herein [wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model]..; Examiner notes the training include neural architecture as model parameters and size of data as noted above and in [0109] In at least one embodiment, logic 615 may include, without limitation, code and/or data storage 601 to store forward and/or output weight and/or input/output data, and/or other parameters to configure neurons or layers [wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model] of a neural network trained and/or used for inferencing in aspects of one or more embodiments... [0110] In at least one embodiment, any portion of code and/or data storage 601 may be internal or external to one or more processors or other hardware logic devices or circuits… for example, or comprising DRAM, SRAM, flash or some other storage type may depend on available storage on-chip versus off-chip, latency requirements of training and/or inferencing functions being performed, batch size of data [wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model] used in inferencing and/or training of a neural network, or some combination of these factors [wherein each client sends model parameters and a size of a dataset with a Neural Graphical Model without sharing the data used to train the Neural Graphical Model].)
Regarding claim 11, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein training the global Neural Graphical Model further includes: learning an average of the distribution over input features for the domain of the data in the global dependency graph. (in [0063] In at least one embodiment, federated server 102 sends global updates 152 to clients including client A 112. In at least one embodiment, global updates 152 are generated by global model generation module 104, which receives local updates 151 from clients (e.g., client A 112, client B 132, client C 142) and aggregates all local updates 151 to generate global updates 152 using different techniques. In at least one embodiment, said techniques may include, without limitation, uniform weighting, importance weighing, majority voting. In at least one embodiment, uniform weighting refers to assigning same weight for every single local update. In at least one embodiment, one example of importance weighting is weighing such local updates based on a number of samples or type of samples that each client has. For example, in at least one embodiment, aggregation is done by performing weighted average based on number of data samples that each client has [wherein training the global Neural Graphical Model further includes: learning an average of the distribution over input features for the domain of the data in the global dependency graph]…)
Regarding claim 12, the rejection of claim 11 is incorporated and Xu teaches the method of method of claim 11, wherein learning the average of the distribution over the features further includes: adjusting the distribution to a weighted average of the plurality of Neural Graphical Models; (in [0085] At 306, in at least one embodiment, said client K computes modified global gradients based and local gradients. In at least one embodiment, client K computes modified global gradients based on global gradients and local gradients. In at least one embodiment, local gradients are generated based on difference between weights of trained local model and weights of current global model. In at least one embodiment, global gradients are based on performing weighted average of all local gradients from other clients [wherein learning the average of the distribution over the features further includes: adjusting the distribution to a weighted average of the plurality of Neural Graphical Models], for example, ∇F(w.sub.k)=Σp.sub.i∇F.sub.i(w.sub.k,i), where Σ.sub.i=1.sup.Np.sub.i=1. In at least one embodiment, client K computes modified global gradients by performing ∇F(w.sub.k.sup.−i)=(∇F(w.sub.k)−p.sub.i∇F.sub.i(w.sub.k,i))(1−p.sub.i), where p.sub.1≥0 denotes weights for weighted averaging (e.g., proportional to client sample number).)
and adjusting a dependency structure of the global Neural Graphical Model to the dependency structure of the global dependency graph. (in [0086] At 308, in at least one embodiment, said client K computes said importance value [adjusting a dependency structure of the global Neural Graphical Model to the dependency structure of the global dependency graph] (e.g., Γ.sub.k,i.sup.m or Γ.sub.k,i.sup.s described herein) of client K based on global gradients and trained model. In at least one embodiment, said importance value of client K is part of local update with said importance value 155… [0087] At 310, in at least one embodiment, said client K sends said importance value of client K to said federated server with at least one portion of neural network training information (e.g., local updates). At 312, in at least one embodiment, said federated server aggregates all local models based on such importance values received from client 1 through client N to generate a new global model [adjusting a dependency structure of the global Neural Graphical Model to the dependency structure of the global dependency graph].)
Regarding claim 13, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein training the global Neural Graphical Model further includes: leveraging publicly available data during the training. (in [0598] In at least one embodiment, pre-trained models 3606 may be stored in a data store, or registry (e.g., model registry 3524 of FIG. 35). In at least one embodiment, pre-trained models 3606 may have been trained, at least in part, at one or more facilities other than a facility executing process 3900. In at least one embodiment, to protect privacy and rights of patients, subjects, or clients of different facilities, pre-trained models 3606 may have been trained, on-premise, using customer or patient data generated on-premise… In at least one embodiment, such as where a customer or patient data has been released of privacy concerns (e.g., by waiver, for experimental use, etc.), or where a customer or patient data is included in a public data set [wherein training the global Neural Graphical Model further includes: leveraging publicly available data during the training], a customer or patient data from any number of facilities may be used to train pre-trained model 3606 on-premise and/or off premise, such as in a datacenter or other cloud computing infrastructure. )
Regarding claim 14, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein each client of the plurality of clients personalizes, using an algorithm, the global Neural Graphical Model to the data of each client, (in [0084] At 304, in at least one embodiment, client K (e.g., client A 112) trains local model [wherein each client of the plurality of clients personalizes, using an algorithm, the global Neural Graphical Model to the data of each client] based on said current global model and neural network training data (e.g., private data described herein). In at least one embodiment, said local model can be any of local model 153 or local model 242. In at least one embodiment, private data is private data 224. In at least one embodiment, said local model [local model using local data as claimed client model personalizes, using an algorithm, the global Neural Graphical Model to the data of each client,] is trained using training framework 704 [ including using an algorithm] described herein …)
wherein the algorithm adds features specific to the data of a client to the global Neural Graphical Model of the client. (in [0086] At 308, in at least one embodiment, said client K computes said importance value (e.g., Γ.sub.k,i.sup.m or Γ.sub.k,i.sup.s described herein) of client K based on global gradients and trained model. In at least one embodiment, said importance value of client K is part of local update with said importance value 155. In at least one embodiment, said contribution of client is client importance 262. In at least one embodiment, said client K computes said importance value by performing steps 406, 408, 410, 412, 414, 416 described herein. In at least one embodiment, said importance value of client K in generated based on how local model of client K contributes to next global model [wherein the algorithm adds features specific to the data of a client to the global Neural Graphical Model of the client] and how current global model is accurate for client K.)
Regarding claim 15, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein each client of the plurality of clients uses the global Neural Graphical Model to perform inference tasks or sampling tasks on the data. (in [0098] Alternatively, in at least one embodiment, said clients can generate modified global modal [the global Neural Graphical Model] based on said global gradients excluding client K. In at least one embodiment, said clients use said modified global model to generate predictions using client's private data [wherein each client of the plurality of clients uses the global Neural Graphical Model to perform inference tasks or sampling tasks on the data]. In at least one embodiment, this is to compute measurement in data space by calculating model error of modified global model on said private data. In at least one embodiment, clients compare said predictions and said ground truth by using Γ.sub.k,i(err)custom-characterε(custom-character.sub.i; w.sub.k.sup.−i), where ε(custom-character.sub.i; w.sub.k.sup.−i) denotes error on empirical distribution custom-character.sub.i.)
Regarding claim 16, the rejection of claim 8 is incorporated and Xu teaches the method of method of claim 8, wherein each client of the plurality of clients uses a personalized Neural Graphical Model to perform inference tasks or sampling tasks on the data. (in [0098] Alternatively, in at least one embodiment, said clients can generate modified global modal based on said global gradients excluding client K. In at least one embodiment, said clients use said modified global model [a personalized Neural Graphical Model] to generate predictions [wherein each client of the plurality of clients uses a personalized Neural Graphical Model to perform inference tasks or sampling tasks on the data] using client's private data. In at least one embodiment, this is to compute measurement in data space by calculating model error of modified global model on said private data. In at least one embodiment, clients compare said predictions and said ground truth by using Γ.sub.k,i(err)custom-characterε(custom-character.sub.i; w.sub.k.sup.−i), where ε(custom-character.sub.i; w.sub.k.sup.−i) denotes error on empirical distribution custom-character.sub.i. )
Regarding independent claim 17, Xu teaches a device, comprising: a memory to store data and instructions; and a processor operable to communicate with the memory, wherein the processor is operable to:
generate, using a merge function, a global dependency graph from a plurality of feature dependency graphs received from a plurality of clients, wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs; (in[0093] At 404, in at least one embodiment, said clients generate global gradients based on current global and previous global model model [… from a plurality of feature dependency graphs received from a plurality of clients]. In at least one embodiment, said current global model is derived based on global updates (e.g., global update 222 or global update 152) and previous global model is based on global updates happened in last round of federated training [generate, using a merge function, a global dependency graph from a plurality of feature dependency graphs received from a plurality of clients] . For example, in at least one embodiment, said clients perform ∇F(w.sub.k)=w.sub.k−w.sub.k−1. Alternatively, in at least one embodiment, global gradients are generated based on aggregating all local gradients received from clients. For example, in at least one embodiment, ∇F(w.sub.k)=Σp.sub.i∇F.sub.i(w.sub.k,i), where Σ.sub.i=1.sup.Np.sub.i=1. [0094] At 406, in at least one embodiment, said clients generate local gradients based on trained local model and current global model. For example, in at least one embodiment, said clients generate their own respective local gradients by performing ∇F.sub.i(w.sub.k,i)=w.sub.k,i.sup.k.sup.i−w.sub.k,i.sup.0, which is difference between weights [wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs] of trained local model and current global. And in [0109] …. In at least one embodiment, code and/or data storage 601 stores weight parameters [wherein nodes in the global dependency graph contain an intersection of features of the plurality of feature dependency graphs] and/or input/output data of each layer of a neural network trained or used in conjunction with one or more embodiments during forward propagation of input/output data and/or weight parameters during training and/or inferencing using aspects of one or more embodiments.)
provide, to the plurality of clients, the global dependency graph; (in [0104] In at least one embodiment, model training module 512 is a module to train one or more neural networks, for example, using federated learning. In at least one embodiment, model training module 512 causes federated server (e.g., federated server 102) to send global models (e.g., global model 154, global model 226) to clients [providing, to the plurality of clients, the global Neural Graphical Model] (e.g., client A 112, client B 132, client C 142, client 202)... )
receive, from the plurality of clients, a plurality of Neural Graphical Models trained locally by a client using data from the client and the global dependency graph; (in [0084] At 304, in at least one embodiment, client K (e.g., client A 112) trains local model based on said current global model [the global dependency graph] and neural network training data [a plurality of Neural Graphical Models trained locally by a client using data from the client and the global dependency graph] (e.g., private data described herein). In at least one embodiment, said local model can be any of local model 153 or local model 242… [0087] At 310, in at least one embodiment, said client K sends said importance value of client K to said federated server with at least one portion of neural network training information (e.g., local updates). At 312, in at least one embodiment, said federated server aggregates all local models [receive, from the plurality of clients, a plurality of Neural Graphical Models trained locally by a client using data from the client … as received local models] based on such importance values received from client 1 through client N [from a plurality of clients] to generate a new global model… ; And local models as claimed Neural Graphical Models listed in [0060] In at least one embodiment, models (e.g., local models 153, 242, global models 154, 226 described herein) refers to various neural networks described herein, which include, without limitation, feedforward neural network, convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory (LSTM) network, autoencoder, generative adversarial network (GAN), restricted boltzmann machine (RBM), deep belief networks (DBN), radial basis function network (RBFN), hopfield network, self-organizing maps, perceptron's with one or more layers, modular neural networks, spiking neural networks, deep reinforcement learning networks, echo state networks, time-delay neural networks, support vector machines, attention-based neural networks, autoencoders, graph neural networks, any neural networks suitable for computer vision and/or NLP…)
train a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph, wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model; (in [0087] At 310, in at least one embodiment, said client K sends said importance value of client K to said federated server with at least one portion of neural network training information (e.g., local updates). At 312, in at least one embodiment, said federated server aggregates all local models based on such importance values received from client 1 through client N to generate a new global model [train a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph, …]. In at least one embodiment, federated server performs weighted averaging of all local models based on using importance as weights. For example, in at least one embodiment, said federated server uses modules (e.g., global generation module 104) to compute … where Z.sub.k=kΣ.sub.i=1.sup.NΣ.sub.k Γ.sub.k,i.sup.m is a normalization factor to ensure Σ.sub.i=1.sup.Nρ.sub.k,i.sup.m=1, and Γ.sub.k,N.sup.m, is an example contribution of client K. Then, in at least one embodiment, said modules generate new global model by performing w.sub.k+1←w.sub.k−ηE.sub.i=1.sup.Nρ.sub.k,i.sup.m.Math.∇F.sub.i(w.sub.k,i)… [0093] At 404, in at least one embodiment, said clients generate global gradients based on current global model and previous global model [train a global Neural Graphical Model using the plurality of Neural Graphical Models and the global dependency graph]. In at least one embodiment, said current global model is derived based on global updates (e.g., global update 222 or global update 152) and previous global model is based on global updates happened in last round of federated training [wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model]. For example, in at least one embodiment, said clients perform ∇F(w.sub.k)=w.sub.k−w.sub.k−1. Alternatively, in at least one embodiment, global gradients are generated based on aggregating all local gradients received from clients [wherein the global Neural Graphical Model represents an aggregate distribution over a domain of the data used to train each Neural Graphical Model]...)
and provide, to the plurality of clients, the global Neural Graphical Model. (in [0104] In at least one embodiment, model training module 512 is a module to train one or more neural networks, for example, using federated learning. In at least one embodiment, model training module 512 causes federated server (e.g., federated server 102) to send global models (e.g., global model 154, global model 226) to clients [providing, to the plurality of clients, the global Neural Graphical Model] (e.g., client A 112, client B 132, client C 142, client 202)... )
Regarding claim 18, the limitations are similar to the limitations in claims 9 and 10, and thus rejected under the same rationale.
Regarding claim 19, the limitations are similar to the limitations in claim 11, and thus rejected under the same rationale.
Regarding claim 20, the limitations are similar to the limitations in claim 14, and thus rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wang et al. (NPL: A Graph Neural Network Learning Approach to Optimize RIS-Assisted Federated Learning): teaches an RIS-assisted over the-air FL network, where a single-antenna edge server coordinates a set K = {1,...,K} of K single-antenna de vices to train a global model with the assistance of an RIS. Each edge device k ∈ K owns a local dataset denoted by Dk = {(xkm,ykm) | 1 ≤ m ≤ Mk} for local model training, where (xkm,ykm) denotes the m-th input feature and label pair at device k, and Mk denotes the number of training samples available at device k. We assume that the training datasets at different edge devices are independent and identically distributed (i.i.d.), and have the same number of training samples, i.e., Mk = Mj,∀ k,j ∈ K, as in [24], [36]. We aim to find the optimal model parameter vector w∗ ∈ RΩ that minimizes the global loss function F(w), i.e., minw F(w) = 1 M k∈KMkFk(w) = 1 K k∈KFk(w), where M = k∈K Mk is the total number of training samples and Fk(w) is local loss function at device k. In each round t =1,...,T, the following three steps are performed. • Global model dissemination: Each edge device receives global model w(t − 1) through the downlink channel from the edge server at the beginning of round t. Since the edge server transmits with a much greater power than edge devices, it is reasonable to assume that the distortion of the global model at each device is negligible, as in [24], [37]. • Local model update: According to the received global model w(t−1), each edge device k ∈ K evaluates its lo cal stochastic gradient Υk(t) = ∇Fk w(t − 1);BFL where BFL k k , ⊂ Dk denotes the mini-batch containing |BFL k | randomly sampled data samples. • Local model aggregation: As the edge server only requires the arithmetic mean of local gradients (i.e., Υ(t) = 1 K k∈KΥk(t)) for the global model update (i.e., w(t)), we adopt AirComp to achieve low-latency uplink gradient aggregation. With AirComp, the concurrently transmitted local gradients {Υk(t)}k∈K from K edge devices can be added over-the-air, and the edge server can directly receive a summation of these gradients.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWATOSIN ALABI whose telephone number is (571)272-0516. The examiner can normally be reached Monday-Friday, 8:00am-5:00pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWATOSIN ALABI/Primary Examiner, Art Unit 2129
Prosecution Insights