Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 3 is objected to because of the following informalities: “by on a scaling parameter” should be just “on a scaling parameter”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention.
Evidence that claims 16 fails to correspond in scope with that which the inventor or a joint inventor, or for pre-AIA applications the applicant regards as the invention can be found in the reply filed on February 2, 2023. In that paper, the inventor or a joint inventor, or for pre-AIA applications the applicant has stated “A system for distributed, comprising”, and this statement indicates that the invention is different from what is defined in the claim(s) because claim 16 fails to disclose what constitutes the distribution or what is being distributed, so we will treat this claim as saying “a system for distributed machine learning” given that the claim language for claim 16 and claims 1 and 12 are similar even though claim 16 is a system claim and claims 1 and 12 are method claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
According to the first part of the analysis, claims 1-15 are method (Process) claims while claims 16-20 are system or machine claims. Etc. Therefore, claims 1-20 fall within one of the four statutory categories (a process, machine, manufacture, or composition of matter).
Regarding Claim 1:
2A Prong 1:
Computing ,…, an objective function for a dynamic machine learning model based on a relationship between the dynamic machine learning model and the static machine learning model (This step for computing an objective function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 108.);
Updating ,…, the dynamic machine learning model based on the objective function (This step for updating a dynamic machine learning model is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 114-115.).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
Obtaining ,…, a static machine learning model from a hub device (This step for obtaining a static machine learning model for a hub device is an insignificant extra-solution activity, see MPEP 2106.05(g).);
By an edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
Obtaining ,…, a static machine learning model from a hub device (This step for obtaining a static machine learning model for a hub device is receiving or transmitting data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I).);
By an edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routing, conventional and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 2:
2A Prong 1:
computing, …, a distance function between the dynamic machine learning model and the static machine learning model, wherein the objective function is based on the distance function (This step for updating a dynamic machine learning model is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 110).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 3:
2A Prong 1:
Scaling,…, the distance function by on a scaling parameter to obtain a penalty term, wherein the objective function includes the penalty term (This step for scaling a distance function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraphs 0111-0112.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 4:
2A Prong 1:
Identifying,…, a static policy function for the static machine learning model (This step for identifying a static policy function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, where the policy function can be represented as πhub, see Paragraph 110.);
Identifying,…, a dynamic policy function for the dynamic machine learning model, wherein the distance function is computed between the dynamic policy function and the static policy function (This step for identifying a dynamic policy function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 0110.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 5:
2A Prong 1:
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
Initializing,…, the dynamic machine learning model based on the static machine learning model (This step for initializing the dynamic machine learning model is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component or generic learning model as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
Initializing,…, the dynamic machine learning model based on the static machine learning model (This step for initializing the dynamic machine learning model is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component or generic learning model as a tool to perform the disclosed abstract idea above.
Regarding Claim 6:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
Collecting ,…, training data at the edge device, wherein the edge device is updated based on the training data (This step for collecting training data to update the edge device is an insignificant extra-solution activity, see MPEP 2106.05(g).);
By an edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
Collecting ,…, training data at the edge device, wherein the edge device is updated based on the training data (This step for collecting training data to update the edge device is receiving data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I) (TLI Communications LLC v. AV Auto).);
By an edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routing, conventional and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 7:
2A Prong 1:
Recommending,…, content to a user based on the dynamic machine learning model, wherein the training data comprises user interaction data with the content (This step for initializing the dynamic machine learning model is practically implementable in the human mind and is understood to be a recitation of a mental process of evaluation/judgement, a human can initialize a dynamic machine learning model.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 8:
2A Prong 1:
There are no additional elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
Training ,…, the static machine learning model based on the training data from the edge device (This step for training the static machine learning model is mere instructions to apply the exception using
a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
transmitting the training data from the edge device to the hub device (This step for transmitting data from the edge device to the hub device is an insignificant extra-solution activity, see MPEP 2106.05(g).);
By the hub device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
Training ,…, the static machine learning model based on the training data from the edge device (This step for training the static machine learning model is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
transmitting the training data from the edge device to the hub device (This step for transmitting training data to update the edge device is transmitting data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I).);
By the hub device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routing, conventional and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 9:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
collecting additional training data at an additional edge device (This step for collecting additional data from the edge device to the hub device is an insignificant extra-solution activity, see MPEP 2106.05(g).);
transmitting the additional training data from the additional edge device to the hub device, wherein the static machine learning model is trained based on the additional training data (This step for transmitting additional training data to the hub device is an insignificant extra-solution activity, see MPEP 2106.05(g).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
collecting additional training data at an additional edge device (This step for collecting additional training data is receiving data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I) (TLI Communications LLC v. AV Auto).);
transmitting the additional training data from the additional edge device to the hub device, wherein the static machine learning model is trained based on the additional training data (This step for transmitting additional training data is transmitting data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I) (TLI Communications LLC v. AV Auto).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routing, conventional to perform the disclosed abstract idea above.
Regarding Claim 10:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the static machine learning model and the dynamic machine learning model comprise reinforcement learning models (The static machine learning model and the dynamic machine learning model comprising reinforcement learning models is in the field of use for reinforcement learning, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are in the field of use to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the static machine learning model and the dynamic machine learning model comprise reinforcement learning models (The static machine learning model and the dynamic machine learning model comprising reinforcement learning models is in the field of use for reinforcement learning, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are in the field of use to perform the disclosed abstract idea above.
Regarding Claim 11:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the static machine learning model and the dynamic machine learning model comprise collaborative filtering models (The static machine learning model and the dynamic machine learning model comprising collaborative filtering models is in the field of use for collaborative filtering, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are in the field of use to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the static machine learning model and the dynamic machine learning model comprise collaborative filtering models (The static machine learning model and the dynamic machine learning model comprising collaborative filtering models is in the field of use for collaborative filtering, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are in the field of use to perform the disclosed abstract idea above.
Regarding Claim 12:
2A Prong 1:
Computing,…, a policy function of a dynamic machine learning model based on the user interaction data, wherein the dynamic machine learning model is trained based on a relationship between the dynamic machine learning and a static machine learning model from a hub device (This step for computing a policy function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 110.);
Recommending,…, content to the user based on the policy function (This step for recommending content to the user based on the policy function is practically implementable in the human mind and is understood to be a recitation of a mental process of evaluation/judgement, a human can recommend content to the user based on the policy function.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
Obtaining ,…, user interaction data for a user (This step for obtaining user data is an insignificant extra-solution activity, see MPEP 2106.05(g).);
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
obtaining ,…, user interaction data for a user (This step for obtaining user data is receiving data over a network which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I) (TLI Communications LLC v. AV Auto).);
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routine, conventional and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 13:
2A Prong 1:
Identifying,…, a state based on the user interaction data, wherein the policy function takes the state as input (This step for identifying a state based on user interaction data is practically implementable in the human mind and is understood to be a recitation of a mental process of evaluation/judgement, a human can identify a state.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
By the edge device (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 14:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the policy function comprises a neural network trained using reinforcement learning (The policy function comprising reinforcement learning model is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).).
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic learning model as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the policy function comprises a neural network trained using reinforcement learning (The policy function comprising reinforcement learning model is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).).
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are mere instructions to apply the exception using the generic learning model as a tool to perform the disclosed abstract idea above.
Regarding Claim 15:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the policy function comprises a user matrix and an item matrix trained using collaborative filtering (The policy function comprising a user matrix and an item matrix trained using collaborative filtering is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).).
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic learning model as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the policy function comprises a user matrix and an item matrix trained using collaborative filtering (The policy function comprising a user matrix and an item matrix trained using collaborative filtering is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).).
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are mere instructions to apply the exception using the generic learning model as a tool to perform the disclosed abstract idea above.
Regarding Claim 16:
2A Prong 1:
compute an objective function for a dynamic machine learning model based on a relationship between the dynamic machine learning model and the static machine learning model (This step for computing an objective function for a dynamic machine learning model based on a relationship between the dynamic machine learning model and the static machine learning model is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 108.);
update the dynamic machine learning model based on the objective function (This step for updating the dynamic machine learning model based on the objective function is defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations, see Paragraph 115.);
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
obtain a static machine learning model from a hub device (This step for obtaining a static machine learning model is an insignificant extra-solution activity, see MPEP 2106.05(g).);
an edge device including a memory and a processor, wherein the processor is configured to (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are insignificant extra-solution activities and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
obtain a static machine learning model from a hub device (This step for obtaining a static machine learning model is receiving or transmitting data over a network, which, is well-understood, routine, conventional, see MPEP 2106.05(d)(II)(I).);
an edge device including a memory and a processor, wherein the processor is configured to (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add anything significantly more as they are well-understood, routine, conventional and mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 17:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the system further comprises the hub device, and the hub device is configured to train the static machine learning model. (The system further comprising the hub device is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic learning model as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the system further comprises the hub device, and the hub device is configured to train the static machine learning model. (The system further comprising the hub device is mere instructions to apply the exception using a generic learning model as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are mere instructions to apply the exception using the generic computer component or generic learning model as a tool to perform the disclosed abstract idea above.
Regarding Claim 18:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
an additional edge device configured to train an additional dynamic machine learning model based on the static machine learning model (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
an additional edge device configured to train an additional dynamic machine learning model based on the static machine learning model (This step is mere instructions to apply the exception using a generic computer component as a tool to perform the abstract idea, see MPEP 2106.05(f).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are mere instructions to apply the exception using the generic computer component as a tool to perform the disclosed abstract idea above.
Regarding Claim 19:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the dynamic machine learning model comprises reinforcement learning models (The dynamic machine learning model comprising reinforcement learning models is in the field of use for reinforcement learning, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are in the field of use to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the dynamic machine learning model comprises reinforcement learning models (The dynamic machine learning model comprising reinforcement learning models is in the field of use for reinforcement learning, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more they are in the field of use to perform the disclosed abstract idea above.
Regarding Claim 20:
2A Prong 1:
There are no elements.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
the dynamic machine learning model comprise collaborative filtering models (The static dynamic machine learning model comprising a collaborative filtering model is in the field of use for collaborative filtering, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not integrate the judicial exception into practical application as they are in the field of use to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional Elements:
the dynamic machine learning model comprise collaborative filtering models (The static dynamic machine learning model comprising a collaborative filtering model is in the field of use for collaborative filtering, see MPEP 2106.05(h).);
The additional elements as disclosed above in combination of the abstract idea do not add
anything significantly more as they are in the field of use to perform the disclosed abstract idea above.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 2, 4, 5, 6, 16, 17, and 18 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable over Gong (US 20230087863 A1).
Regarding Claim 1,
Gong teaches a method for distributed machine learning, comprising:
obtaining, by an edge device, a static machine learning model from a hub device (See Paragraph 0046, Gong recites “Preferably, receiving, at each client of the plurality of clients from the central server, information on the updates to the global feature embedding model comprises receiving, at each client of the plurality of clients from the central server, the global feature embedding model”, where the global feature embedding model is the static machine learning model and the central server is the hub device. See Paragraph 0084, Gong recites the local client, which is the edge device. Furthermore, Gong recites that each local client is “configured to receive the global feature embedding model” from “the central server”, where the central server is the hub device and the global feature embedding model is the static machine learning model, see Paragraph 0085.);
computing, by the edge device, an objective function for a dynamic machine learning model based on a relationship between the dynamic machine learning model and the static machine learning model (See Paragraph 0047-0048, Gong recites “mapping at least a portion of the received updates to the global feature embedding model at each client of the plurality of clients on to the respective local feature embedding model, may comprise: determining, at each local client of the plurality of clients, a probability distribution for the respective local feature embedding model applied to the data set associated with a domain of the respective client”, where the local client is the edge device, the local feature embedding model is the dynamic machine learning model, and the global feature embedding model is the static machine learning model. Furthermore, Gong repeats the “probability distribution” process for the “global feature embedding model”, see Paragraph 0049. Then, Gong recites that there is “a divergence between the probability distribution for the respective local feature embedding model and the probability distribution for the global feature embedding model” where the divergence between the probability of the local and global feature embedding model is the objective function based on a relationship between the dynamic machine learning model and the static machine learning model, see Paragraph 0050.).
and updating, by the edge device, the dynamic machine learning model based on the objective function (See Paragraph 0051, Gong recites that based on the divergence, identify, “at each local client of the plurality of clients, the updates to the global feature embedding model that are relevant to the respective local feature embedding model”, where the divergence is the objective function and the local client is the edge device. Furthermore, Gong recites that “at each local client of the plurality of clients”, the “respective local feature embedding model” are updated “based on the identified relevant updates to the global feature embedding model”, see Paragraph 0052. Since the divergence was the objective function, then the local feature embedding models, or the dynamic machine learning models, are updated based on the objective function.).
Regarding Claim 2,
Gong teaches the method of claim 1, further comprising: computing, by the edge device, a distance function between the dynamic machine learning model and the static machine learning model, wherein the objective function is based on the distance function. (See Paragraph 0048-0049, Gong recites that the local client, or the edge device, computes the probability distribution of the local feature embedding model, or the dynamic machine learning model, and the global feature embedding model, or the static machine learning model. Furthermore, Gong recites that there is a “divergence between the probability distribution” of the “local feature embedding model”, or the dynamic machine learning model, and “global feature embedding model”, or the static machine learning model, which can be considered as the distance function because it measures the divergence, or difference, between the dynamic and static machine learning models, See Paragraph 050. Therefore, then the objective function is based on the distance function since the objective function is also the divergence between the probability distribution of the two models.).
Regarding Claim 4,
Gong teaches the method of claim 2, further comprising: identifying, by the edge device, a static policy function for the static machine learning model; and identifying, by the edge device, a dynamic policy function for the dynamic machine learning model, wherein the distance function is computed between the dynamic policy function and the static policy function (See Paragraph 0123, Gong recites “Prior to any optimisation, each local feature embedding model 225a, 225b and the global feature embedding model 410 may be a predetermined initialised function (or initialised network). In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialised to be the same as the global feature embedding model, as received or downloaded from the central server.”, where the initialized function for the global feature embedding model, or the static machine learning model, is the static policy function, and the initialized function for the local feature embedding model, or the dynamic machine learning model, is the dynamic policy function, that are both identified by the local client, or the edge device. Furthermore, Gong recites that a “soft probability distribution function” is calculated “by applying the present local feature embedding model and by applying the received, updated global feature embedding model to the local data set at each local client” and that “the divergence between the two probability distributions (resulting from the local feature embedding model and from the global feature embedding model) can then be obtained. From this measure of divergence, the weighted components of the domain-generalised global feature embedding model that are most relevant to the domain-specific local feature embedding model can be identified”, where the difference between the probability between the local feature embedding model and the probability of the global feature embedding model, or the static policy function, can be considered as the distance function between the two policy functions, see Paragraph 0143.)
Regarding Claim 5,
Gong teaches the method of claim 1, further comprising: initializing, by the edge device, the dynamic machine learning model based on the static machine learning model. (See Paragraph 0123, Gong recites “In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialized to be the same as the global feature embedding model, as received or downloaded from the central server”, where the local feature embedding model, or the static machine learning model, is initialized by the local client, or the edge device, based on the global feature embedding model, or the static machine learning model.).
Regarding Claim 6,
Gong teaches the method of claim 1, further comprising: collecting, by the edge device, training data at the edge device, wherein the edge device is updated based on the training data. (See Paragraph 125, Gong recites “At each local client 220a, 220b, the local feature embedding model 225a, 225b is optimised based on the associated local data set 230a, 230b. The associated local data set 230a, 230b may be a training data set relevant to the geographical domain with which the particular local client 220a, 220b is associated”, where the local feature embedding model is optimized based on the training data at the local client, or the edge device, which updates the local client. Therefore, the local client is updated based on the collected training data set.).
Regarding Claim 16,
Gong teaches A system for distributed, comprising:
an edge device including a memory and a processor, wherein the processor is configured to: obtain a static machine learning model from a hub device (See Paragraph 0121, Gong recites “Each local client comprises a processor 125a, 125b, 125c and associated data storage or memory 130a, 130b, 130c.”, where the local client is the edge device. Then, Gong recites “Preferably, receiving, at each client of the plurality of clients from the central server, information on the updates to the global feature embedding model comprises receiving, at each client of the plurality of clients from the central server, the global feature embedding model”, where the global feature embedding model is the static machine learning model and the central server is the hub device, see Paragraph 0046. Furthermore, Gong recites that each local client is “configured to receive the global feature embedding model” from “the central server”, where the central server is the hub device and the global feature embedding model is the static machine learning model, see Paragraph 0085. If the local client receives the static machine learning model, then the processor receives the static machine learning model as well.);
compute an objective function for a dynamic machine learning model based on a relationship between the dynamic machine learning model and the static machine learning model (See Paragraph 0047-0048, Gong recites “mapping at least a portion of the received updates to the global feature embedding model at each client of the plurality of clients on to the respective local feature embedding model, may comprise: determining, at each local client of the plurality of clients, a probability distribution for the respective local feature embedding model applied to the data set associated with a domain of the respective client”, where the local client is the edge device and the local feature embedding model is the dynamic machine learning model. Furthermore, Gong repeats this process for the “global feature embedding model” or the static machine learning model, see Paragraph 0049. Then, Gong recites that there is “a divergence between the probability distribution for the respective local feature embedding model and the probability distribution for the global feature embedding model” where the divergence between the probability of the local and global feature embedding model is the objective function based on a relationship between the dynamic machine learning model and the static machine learning model, see Paragraph 0050.);
and update the dynamic machine learning model based on the objective function (See Paragraph 0051, Gong recites that based on the divergence, identify, “at each local client of the plurality of clients, the updates to the global feature embedding model that are relevant to the respective local feature embedding model”, where the divergence is the objective function and the local client is the edge device. Furthermore, Gong recites that “at each local client of the plurality of clients”, the “respective local feature embedding model” are updated “based on the identified relevant updates to the global feature embedding model”, see Paragraph 0052. Since the divergence was the objective function, then the local feature embedding models, or the dynamic machine learning models, are updated based on the objective function.).
Regarding Claim 17,
Gong teaches the system of claim 16, wherein: the system further comprises the hub device, and the hub device is configured to train the static machine learning model (See Paragraph 0007, Gong recites “A central server then selects and aggregates changes to each local feature embedding model to update and improve a centralised or global feature embedding model”, where the central server is the hub device and the global feature model is the static machine learning model that is being improved, or trained, by the central server.);
Regarding Claim 18,
Gong teaches the system of claim 16, the system further comprising: an additional edge device configured to train an additional dynamic machine learning model based on the static machine learning model (“See Paragraph 0007, Gong recites “The consequent updates to the centralised or global feature embedding model may then be ‘fed-back’ to each of the local feature embedding models, update and improve the model at each local client.” Where the additional local feature embedding models, or the additional dynamic machine learning models, that is trained by the local clients, or the additional edge devices, that is updated by the data being fed by the global feature embedding model, or the static machine learning model.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
8. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
9. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.
Claim(s) 3 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Wu (Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models).
Regarding Claim 3,
Gong teaches the edge device and the distance function (See Paragraph 0048-0049, Gong recites that the local client, or the edge device, computes the probability distribution of the local feature embedding model, or the dynamic machine learning model, and the global feature embedding model, or the static machine learning model. Furthermore, Gong recites that there is a “divergence between the probability distribution” of the “local feature embedding model”, or the dynamic machine learning model, and “global feature embedding model”, or the static machine learning model, which can be considered as the distance function because it measures the divergence, or difference, between the dynamic and static machine learning models, See Paragraph 050. Therefore, then the objective function is based on the distance function since the objective function is also the divergence between the probability distribution of the two models.).
However, Gong fails to teach scaling the distance function by on a scaling parameter to obtain a penalty term, wherein the objective function includes the penalty term.
Wu teaches the method of claim 2, further comprising: scaling, by the edge device, the distance function by on a scaling parameter to obtain a penalty term, wherein the objective function includes the penalty term (Wu recites “Based on above architectures, our proposed formulation is
m
i
n
{
x
i
}
i
∈
Q
n
,
n
∈
V
x
n
n
∈
V
=
∑
n
∈
V
∑
i
∈
Q
n
(
g
i
x
i
+
γ
i
2
x
i
-
z
n
)
2
(2)
s.t.
x
i
∈
X
,
∀
i
∈
Q
n
,
n
∈
V
,
z
n
=
z
m
,
∀
(
n
,
m
)
∈
ε
where V is the set of cloud servers,
Q
n
is the set of edge devices connected to the nth cloud server,
x
i
'
s
are edge-device models,
z
n
'
s
are cloud-server models, E ⊆ (V × V) indicates communication between cloud servers and
γ
i
>
0
is the penalty parameter. In the sequel,
z
n
'
s
will be referred to as the global models and
x
i
'
s
the personalized models. Our formulation distinguishes the global models on different cloud servers, and allows (but penalizes by the quadratic penalty regularizer) the deviations among personalized models”, where the min function is the distance function, or the objective function, that allows deviation between the edge-device models and the global models on the cloud servers that includes the penalty parameter, or the scaling parameter, of
γ
i
>
0
, where the minimization function with the scaling parameter makes the penalty term that is included with the objective function, see Page 4 Paragraph 2 line 20 – Page 5 Paragraph 1 line 4.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Wu into the method of Gong to obtain the penalty term from the distance function. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Wu as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to have a scalable difference between the static machine learning model and the dynamic machine learning model.
Claim(s) 7, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Cheng (US 20230297849 A1).
Regarding Claim 7,
While Gong teaches the edge device (See Paragraph 0139, Gong recites “In the event that the convergence criteria are met for each local feature embedding model 225a, 225b at each local client 220a, 220b, then the local feature embedding models at each local client are considered optimised. As a consequence, the global feature embedding model can also be considered optimised”, where the local client is the edge device.), Gong fails to teach the method of claim 6, further comprising: recommending, by the edge device, content to a user based on the dynamic machine learning model, wherein the training data comprises user interaction data with the content.
However, Cheng teaches recommending, by the edge device, content to a user based on the dynamic machine learning model, wherein the training data comprises user interaction data with the content (See Paragraph 0128, Cheng recites for the “first computer device” which is “implemented as a plurality of terminals connected with the federated server” and the “second computer device” which is “the federated server for performing federated learning”, where “at least two first computing devices are application servers corresponding to different film and television application programs” and “each application server stores the training data corresponding to different user identifications” where “The historical interaction data is the data obtained after being authorized by the user” that can contract “decision tree models” to be transmitted to the “federated server” to be “fused by the federated server to obtain the federated learning model “ that “is configured to recommend content to users, such as recommending items that meet the interest points of the users based on the corresponding data characteristics of users”, where the first computer device is the edge device, and content is recommended to users based on the historical data authorized by the user, or the training data comprising user interaction data with the content, based on the decision tree models, where the models are the dynamic machine learning model.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Cheng into the method of Gong to have content recommended to a user. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Cheng as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to use the user interaction data to recommend personal content that can also be used to update the static machine learning model.
Regarding Claim 12,
Gong teaches obtaining, by an edge device, (See Paragraph 0007, Gong recites “Federated Person Re-Identification (FedReID) uses local data associated with, and available to, each local client of a plurality of local clients in order to optimise a local feature embedding model at each client”, where the local data is collected by the local client, or the edge device.).
Gong teaches computing, by the edge device, a policy function of a dynamic machine learning model based on the (See Paragraph 0123, Gong recites “Prior to any optimisation, each local feature embedding model 225a, 225b and the global feature embedding model 410 may be a predetermined initialised function (or initialised network). In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialised to be the same as the global feature embedding model, as received or downloaded from the central server”, where the initialized function for the local feature embedding model, or the dynamic machine learning model, is the dynamic policy function, that is computed by the local client, or the edge device. Furthermore, Gong recites that a “soft probability distribution function” is calculated “by applying the present local feature embedding model and by applying the received, updated global feature embedding model to the local data set at each local client” and that “the divergence between the two probability distributions (resulting from the local feature embedding model and from the global feature embedding model) can then be obtained. From this measure of divergence, the weighted components of the domain-generalised global feature embedding model that are most relevant to the domain-specific local feature embedding model can be identified. The identified relevant (or ‘mapped’) weightings can then be used to update the given local feature embedding model,” where the local feature embedding model is updated, or trained, from the divergence between the local feature embedding model and the global feature embedding model, or the static machine learning model, see Paragraph 0143.).
However, Gong fails to teach a method for distributed machine learning, comprising: obtaining, by an edge device, user interaction data for a user and recommending, by the edge device, content to the user based on the policy function.
However, Cheng teaches a method for distributed machine learning, comprising:
obtaining, by an edge device, user interaction data for a user (See Paragraph 0128, Cheng teaches “at least two first computing devices are application servers corresponding to different film and television application programs” where “each application server stores the training data corresponding to different user identifications, for example, the training data includes historical interaction data corresponding to the user identification” which is “obtained after being authorized by the user”, where the first computing devices are the edge devices obtaining historical interaction data authorized by the user, or user interaction data.);
and recommending, by the edge device, content to a user based on the policy function. (See Paragraph 0128, Cheng recites for the “first computer device” which is “implemented as a plurality of terminals connected with the federated server” and the “second computer device” which is “the federated server for performing federated learning”, where “at least two first computing devices are application servers corresponding to different film and television application programs” and “each application server stores the training data corresponding to different user identifications” where “The historical interaction data is the data obtained after being authorized by the user” that can contract “decision tree models” to be transmitted to the “federated server” to be “fused by the federated server to obtain the federated learning model “ that “is configured to recommend content to users, such as recommending items that meet the interest points of the users based on the corresponding data characteristics of users”, where the first computer device is the edge device, and content is recommended to users based on the historical data authorized by the user, or the user interaction data, based on the decision trees, or the policy function.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Cheng into the method of Gong to have user interaction data be obtaining and recommended for a user. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Cheng as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to recommend content to users based off of the user interaction data that can also be used to update the static machine learning model (“The federated learning model is configured to recommend content to users, such as recommending items that meet the interest points of the users based on the corresponding data characteristics of users”, see Paragraph 0128.).
Regarding Claim 13,
Gong teaches (See Paragraph 0123, Gong recites “Prior to any optimisation, each local feature embedding model 225a, 225b and the global feature embedding model 410 may be a predetermined initialised function (or initialised network). In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialized to be the same as the global feature embedding model, as received or downloaded from the central server”, where the initialized function for the local feature embedding model, or the dynamic machine learning model, is the dynamic policy function, that is computed by the local client, or the edge device. Furthermore, Gong recites that a “soft probability distribution function” is calculated “by applying the present local feature embedding model and by applying the received, updated global feature embedding model to the local data set at each local client” and that “the divergence between the two probability distributions (resulting from the local feature embedding model and from the global feature embedding model) can then be obtained. From this measure of divergence, the weighted components of the domain-generalized global feature embedding model that are most relevant to the domain-specific local feature embedding model can be identified. The identified relevant (or ‘mapped’) weightings can then be used to update the given local feature embedding model,” where the local feature embedding model is updated, or trained, from the divergence between the local feature embedding model and the global feature embedding model, or the static machine learning model, see Paragraph 0143.), but fails to teaches identifying, by the edge device, a state based on the user interaction data, wherein the policy function takes the state as input.
Cheng teaches the method of claim 12, further comprising: identifying, by the edge device, a state based on the user interaction data, wherein the policy function takes the state as input (See Paragraph 0128, Cheng recites for the “first computer device” which is “implemented as a plurality of terminals connected with the federated server” and the “second computer device” which is “the federated server for performing federated learning”, where “at least two first computing devices are application servers corresponding to different film and television application programs” and “each application server stores the training data corresponding to different user identifications” where “The historical interaction data is the data obtained after being authorized by the user” that can contract “decision tree models” to be transmitted to the “federated server” to be “fused by the federated server to obtain the federated learning model”, where the first computer device is the edge device, and content is recommended to users based on the historical data authorized by the user, or the user interaction data, based on the decision trees, or the policy function. Furthermore, “the first computing device and the second computing device may achieve a purpose of effectively mining data values” after a fusion of the “decision trees”, where data values are the states of the user interaction data with the decision trees, or policy functions, as input, see Paragraph 0131.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Cheng into the method of Gong to identify a state based on the user interaction data. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Cheng as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to recommend content to users based off of the user interaction data that can then be used to update the static machine learning model. (“The federated learning model is configured to recommend content to users, such as recommending items that meet the interest points of the users based on the corresponding data characteristics of users”, see Paragraph 0128.).
Claim(s) 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Sahin (US 20220391696 A1).
Regarding Claim 8,
Gong fails to teach the method of claim 6, further comprising: transmitting the training data from the edge device to the hub device; and training, by the hub device, the static machine learning model based on the training data from the edge device.
However, Sahin teaches the transmission of the training data from the edge device to the hub device; and training, by the hub device, the static machine learning model based on the training data from the edge device (See Claim 1, Sahin recites “a distributed machine-learning model to be trained with the update vectors received at an edge server (ES)”, which is interpreted as the machine learning model, or the static machine learning model, being trained by the edge server, or the hub device. Furthermore, Sahin recites “a machine-learning model training to process data received at an edge server (ES) as transmitted from a plurality of edge devices”, which is interpreted as the machine-learning model, or the static machine learning model, on the edge server, or the hub device, that is receiving transmitted data from the edge devices, see Claim 15.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Sahin into the method of Gong to have training data be transmitted to the hub device to train the static machine-learning model. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Sahin as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to transmit data from the edge device to the hub device to update the static machine learning model that would further train the static machine learning model on all of the latest data from the edge devices. By updating the static machine learning model, the edge devices can obtain the updated static machine learning model “from the hub device to compute an objective function between the static machine learning model and the dynamic machine learning model on the edge device”, and then update the dynamic machine learning model based on the objective function to “optimally incorporate knowledge and experience from the hub machine learning model into the edge machine learning model, which mitigates a potential drift in the edge machine learning model away from the hub machine learning model in response to learning from local data” (See Specification Paragraph 0003-0004).
Regarding Claim 9,
While Gong teaches collecting additional training data at an additional edge device (See Paragraph 124, Gong recites “At each local client 220a, 220b, the local feature embedding model 225a, 225b is optimised based on the associated local data set 230a, 230b. The associated local data set 230a, 230b may be a training data set relevant to the geographical domain with which the particular local client 220a, 220b is associated”, where the local feature embedding model is optimized based on the training data at the local client, or the additional training data at the additional edge device, which updates the local client. Therefore, the local client is updated based on the collected training data set.).
However, Gong fails to teach the transmission of the additional training data from the additional edge device to the hub device, wherein the static machine learning model is trained based on the additional training data.
Sahin teaches the transmission of the additional training data from the edge device to the hub device; and training, by the hub device, the static machine learning model based on the additional training data from the edge device (See Claim 1, Sahin recites “a distributed machine-learning model to be trained with the update vectors received at an edge server (ES)”, which is interpreted as the machine learning model, or the static machine learning model, being trained by the edge server, or the hub device. Furthermore, Sahin recites “a machine-learning model training to process data received at an edge server (ES) as transmitted from a plurality of edge devices”, which is interpreted as the machine-learning model, or the static machine learning model, on the edge server, or the hub device, that is receiving transmitted data from the edge devices, or the additional training data, see Claim 15.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Sahin into the method of Gong to have training data be transmitted to the hub device to train the static machine-learning model. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Sahin as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated perform the combination for being able to transmit additional data from the edge device to the hub device to update the static machine learning model that would further train the static machine learning model on all of the latest data from the edge devices. By updating the static machine learning model, the edge devices can obtain the updated static machine learning model “from the hub device to compute an objective function between the static machine learning model and the dynamic machine learning model on the edge device”, and then update the dynamic machine learning model based on the objective function to “optimally incorporate knowledge and experience from the hub machine learning model into the edge machine learning model, which mitigates a potential drift in the edge machine learning model away from the hub machine learning model in response to learning from local data” (See Specification Paragraph 0003-0004).
Claim(s) 10, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Balakrishnan (US 20230177349 A1).
Regarding Claim 10,
Gong fails to teach the method of claim 1, wherein: the static machine learning model and the dynamic machine learning model comprise reinforcement learning models.
However, Balakrishnan teaches that the static machine learning model and the dynamic machine learning model comprise reinforcement learning models (See Paragraph 0121, recites “In the example shown, each client computing node 1202 fetches or otherwise obtains a global model 1204 from a central server 1208 (e.g., a MEC server) coupled to an access point 1210 (e.g., a base station), updates aspects of the global model (e.g., model parameters or weights used in the global model, e.g., NN node weights) using its local data or data provided by the central server (e.g., a subset of a large training dataset D), and communicates the updates to the global model to the central server 1208”, where the central server is the hub device, the global model is the static machine learning model, the client computing node is the edge device, and the copy of the global model on each computing node is the local model. Furthermore, recites that the global model on “central server” can be a “reinforcement learning model”, and therefore since the global model is a reinforcement learning model on the central server, then the copy of the model on the client computing node will also be a reinforcement learning mode, see Paragraph 0217.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Balakrishnan into the method of Gong to have dynamic and static machine learning models comprise reinforcement learning models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Balakrishnan as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to limit the training time or updates for the global or central model to ensure model accuracy (“The following proposes efficient techniques that utilize reinforcement learning (RL) to directly learn which set of clients to select for participation in the rounds of federated learning such that key objectives can be achieved (e.g., minimizing the overall training time or achieving a minimal set of updates necessary to satisfy the central model's accuracy)”, where the central model can be the global model as suggested by Balakrishnan at Paragraph 0211.).
Regarding Claim 19,
Gong fails to teach the system of claim 16, wherein: the dynamic machine learning model comprises a reinforcement learning model.
Balakrishnan teaches that the dynamic machine learning model comprises a reinforcement learning model. (See Paragraph 0121, recites “In the example shown, each client computing node 1202 fetches or otherwise obtains a global model 1204 from a central server 1208 (e.g., a MEC server) coupled to an access point 1210 (e.g., a base station), updates aspects of the global model (e.g., model parameters or weights used in the global model, e.g., NN node weights) using its local data or data provided by the central server (e.g., a subset of a large training dataset D), and communicates the updates to the global model to the central server 1208”, where the central server is the hub device, the global model is the static machine learning model, the client computing node is the edge device, and the copy of the global model on each computing node is the local model. Furthermore, recites that the global model on “central server” can be a “reinforcement learning model”, and therefore since the global model is a reinforcement learning model on the central server, then the copy of the model on the client computing node will also be a reinforcement learning mode, see Paragraph 0217.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Balakrishnan into the method of Gong to have dynamic and static machine learning models comprise reinforcement learning models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Balakrishnan as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to limit the training time or updates for the global or central model to ensure model accuracy (“The following proposes efficient techniques that utilize reinforcement learning (RL) to directly learn which set of clients to select for participation in the rounds of federated learning such that key objectives can be achieved (e.g., minimizing the overall training time or achieving a minimal set of updates necessary to satisfy the central model's accuracy)”, where the central model can be the global model as suggested by Balakrishnan at Paragraph 0211.).
Claim(s) 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Pothula (US 20210174257 A1).
Regarding Claim 11,
Gong fails to teach the method of claim 1, wherein: the static machine learning model and the dynamic machine learning model comprise collaborative filtering models.
However, Pothula teaches the static machine learning model and the dynamic machine learning model comprising collaborative filtering models (See Paragraph 0025, recites that a “computing environment 10 may include multiple datasets (e.g. event records) 14 of multiple nodes 17 (e.g., computing systems with constraints on data sharing) of a federated machine learning model”, where “The nodes 17 may have sub-models 15 that take as inputs or training data input from datasets 14” and the sub models “use the outputs of the federated machine learning model 19 to determine to effect various actions” and “configure and dynamically adjust the federated machine learning model 19”. The computing environment is the hub device, the federated machine learning model is the static machine learning model, the nodes are the edge devices, and the sub-models are the dynamic machine learning models. Furthermore, recites “the machine learning techniques that may be used in this system” can include “collaborative filtering”, so the federated machine learning model and the sub-models can be collaborative filtering models, see Paragraph 0042.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Pothula into the method of Gong to have static and dynamic machine learning models comprise collaborative filtering models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Pothula as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to have collaborative learning between the edge devices to recommend content to each user so the static machine learning model is updated based on all of the similar data between the users that will in give consistent updates between the edge devices.
Regarding Claim 20,
Gong fails to teach the method of claim 16, wherein: the dynamic machine learning model comprises a collaborative filtering model.
However, Pothula teaches the static machine learning model and the dynamic machine learning model comprising collaborative filtering models (See Paragraph 0025, recites that a “computing environment 10 may include multiple datasets (e.g. event records) 14 of multiple nodes 17 (e.g., computing systems with constraints on data sharing) of a federated machine learning model”, where “The nodes 17 may have sub-models 15 that take as inputs or training data input from datasets 14” and the sub models “use the outputs of the federated machine learning model 19 to determine to effect various actions” and “configure and dynamically adjust the federated machine learning model 19”. The computing environment is the hub device, the federated machine learning model is the static machine learning model, the nodes are the edge devices, and the sub-models are the dynamic machine learning models. Furthermore, recites “the machine learning techniques that may be used in this system” can include “collaborative filtering”, so the federated machine learning model and the sub-models can be collaborative filtering models, see Paragraph 0042.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Pothula into the method of Gong to have the dynamic machine learning model contain a collaborative filtering models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Pothula as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to have collaborative learning between the edge devices to recommend content to each user so the static machine learning model is updated based on all of the similar data between the users that will in give consistent updates between the edge devices.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Cheng (US 20230297849 A1) further in view of Balakrishnan (US 20230177349 A1).
Regarding Claim 14,
While Gong teaches the method of claim 12, wherein: the policy function (See Paragraph 0123, Gong recites “Prior to any optimisation, each local feature embedding model 225a, 225b and the global feature embedding model 410 may be a predetermined initialised function (or initialised network). In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialised to be the same as the global feature embedding model, as received or downloaded from the central server”, where the initialized function for the local feature embedding model, or the dynamic machine learning model, is the dynamic policy function, that is computed by the local client, or the edge device. Furthermore, Gong recites that a “soft probability distribution function” is calculated “by applying the present local feature embedding model and by applying the received, updated global feature embedding model to the local data set at each local client” and that “the divergence between the two probability distributions (resulting from the local feature embedding model and from the global feature embedding model) can then be obtained. From this measure of divergence, the weighted components of the domain-generalised global feature embedding model that are most relevant to the domain-specific local feature embedding model can be identified. The identified relevant (or ‘mapped’) weightings can then be used to update the given local feature embedding model,” where the local feature embedding model is updated, or trained, from the divergence between the local feature embedding model and the global feature embedding model, or the static machine learning model, see Paragraph 0143.), Gong and Cheng fail to teach a neural network trained using reinforcement learning.
However, Balakrishnan teaches a neural network trained using reinforcement learning. (See Paragraph 0121, recites “In the example shown, each client computing node 1202 fetches or otherwise obtains a global model 1204 from a central server 1208 (e.g., a MEC server) coupled to an access point 1210 (e.g., a base station), updates aspects of the global model (e.g., model parameters or weights used in the global model, e.g., NN node weights) using its local data or data provided by the central server (e.g., a subset of a large training dataset D), and communicates the updates to the global model to the central server 1208”, where the central server is the hub device, the global model is the static machine learning model, the client computing node is the edge device, and the copy of the global model on each computing node is the local model. Furthermore, recites that the global model on “central server” can be a “reinforcement learning model”, and therefore since the global model is a reinforcement learning model on the central server, then the copy of the model on the client computing node will also be a reinforcement learning mode, see Paragraph 0217.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Balakrishnan into the method of Gong and Cheng to have dynamic and static machine learning models comprise reinforcement learning models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Balakrishnan as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to limit the training time or updates for the global or central model to ensure model accuracy (“The following proposes efficient techniques that utilize reinforcement learning (RL) to directly learn which set of clients to select for participation in the rounds of federated learning such that key objectives can be achieved (e.g., minimizing the overall training time or achieving a minimal set of updates necessary to satisfy the central model's accuracy)”, where the central model can be the global model as suggested by Balakrishnan at Paragraph 0211.).
Claim(s) 15 are rejected under 35 U.S.C. 103 as being unpatentable over Gong (US 20230087863 A1) in view of Cheng (US 20230297849 A1) further in view of Gharibi (US 20230300115 A1).
Regarding Claim 15,
While Gong teaches the method of claim 12, wherein: the policy function (See Paragraph 0123, Gong recites “Prior to any optimisation, each local feature embedding model 225a, 225b and the global feature embedding model 410 may be a predetermined initialised function (or initialised network). In other cases, for instance where a new local client is added to an existing system, the local feature embedding model at the new local client may be initialised to be the same as the global feature embedding model, as received or downloaded from the central server”, where the initialized function for the local feature embedding model, or the dynamic machine learning model, is the dynamic policy function, that is computed by the local client, or the edge device. Furthermore, Gong recites that a “soft probability distribution function” is calculated “by applying the present local feature embedding model and by applying the received, updated global feature embedding model to the local data set at each local client” and that “the divergence between the two probability distributions (resulting from the local feature embedding model and from the global feature embedding model) can then be obtained. From this measure of divergence, the weighted components of the domain-generalised global feature embedding model that are most relevant to the domain-specific local feature embedding model can be identified. The identified relevant (or ‘mapped’) weightings can then be used to update the given local feature embedding model,” where the local feature embedding model is updated, or trained, from the divergence between the local feature embedding model and the global feature embedding model, or the static machine learning model, see Paragraph 0143.), Gong and Cheng fail to teach the policy function comprising a user matrix and an item matrix trained using collaborative filtering.
Gharibi teaches the policy function comprises a user matrix and item matrix being trained using collaborative filtering (See Paragraph 0032, Gharibi recites “the concept of training a recommendation system can include initiating, at a server device, an item-vector matrix V, wherein the item-vector matrix V includes a value m related to a total number of items across one or more client devices and a value d representing a hidden dimension. The method can include transmitting the item-vector matrix V to each client device of a set of client devices, wherein each client device trains a local matrix factorization model using a respective user vector U and the item-vector matrix V to generate a respective set of gradients on each respective client device”. Next, Gharibi recites “The approach includes enabling k parties (clients 1404, 1406) to securely compute a global recommendation system that resides at the server device 1402. To achieve this, a new version of the collaborative filtering (matrix-factorization) approach is introduced”, see Paragraph 0148. Furthermore, Gharibi recites “Consider the case of one server device 1402 and one client device 1404. Assume the client device 1404 is a data holder and has collected data on their customers and their purchases” where “The matrix that captures the customer-purchase relationship can be called an interaction matrix, M, and it is of the size n x m. The rating made by user i for item j can be characterized as rij. Note that customer i rated only a subset of all available items, m. A recommendation system’s task is to find the ratings for each customer for all items and hence to predict the next item the customer is interested in”, See Paragraph 0149. Then, Gharibi recites “An algorithm is introduced to recommend top r′ items to each customer. The following steps can apply in any order. The server device 1402 initiates an item-vector matrix V C +
R
m
x
d
where m is the total number of items at all clients and d represents a hidden dimension, which can be assigned in a number of different says including an ad hoc way. The server 1402 shares the item-vector matrix V with all the clients 1404, 1406. Each client device 1404, 1406 trains their local matrix factorization model using their user vector U and item vector V using a stochastic gradient descent method. The vecture U can represent the user data local to the client. The stochastic gradient descent method is an iterative method for optimizing an objective function with suitable smoothness properties”, where the objective function is the policy function, see Paragraph 0150. Therefore, the objective function with the user vector U, which is the one-dimensional user matrix, and the item vector V, which is the one-dimensional item matrix, is trained with collaborative filtering.).
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Gharibi into the methods of Gong to have user matrixes and item matrixes being trained with collaborative filtering models. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Gharibi as all the references are in the fields of federated machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to aggregate the data from all of the edge devices to update the static machine learning model at the hub device. (“This disclosure now turns to the subject matter of the present claims. The following concepts relate to training and inference of decentralized recommendation systems. The context of this disclosure could be, for example, a chain of stores such as Walmart where each store has some similar items for sale but some different items as well, plus separate customer data that has been gathered related to purchasing histories. FIG. 14 illustrates the approach 1400 in which a server 1402 performs such processes as receiving an encrypted vector V 1410a, 1410b from respectively a first client 1404 and a second client 1406. The clients can be different stores in a chain of stores. The server aggregates the vectors and updates a global vector V and decrypts the aggregated global vector and makes it available for download 1408a, 1408b to the client devices 1404, 1406”, as suggested by Gharibi at Paragraph 0146.).
By updating the static machine learning model, the edge devices can obtain the updated static machine learning model “from the hub device to compute an objective function between the static machine learning model and the dynamic machine learning model on the edge device”, and then update the dynamic machine learning model based on the objective function to “optimally incorporate knowledge and experience from the hub machine learning model into the edge machine learning model, which mitigates a potential drift in the edge machine learning model away from the hub machine learning model in response to learning from local data” (See Specification Paragraph 0003-0004).
Conclusion
10. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Gong (US 20230087863 A1) teaches obtaining a static machine learning model from a hub device by an edge device.
11. Any inquiry concerning this communication or earlier communications from the examiner
should be directed to YASIN A HASSAN whose telephone number is (571)272-1567. The examiner can
normally be reached Mon-Fri. 8am-5pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/YASIN ABDULLAH HASSAN/
Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights