DETAILED ACTION
This office action is in response to the application filed on April 9, 2024.
Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
Step 1: The claim does not fall within one of the four statutory categories of invention (process, machine, manufacture, or composition of matter) – see MPEP 2106.03, or,
Step 2: The claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis – see MPEP 2106.04:
Step 2A, Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? - see MPEP 2106.05
MPEP 2106.04(a)(2)(I) states: “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
MPEP 2106.04(a)(2)(III) states: “Accordingly, the “mental processes” abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgements, and opinions.
Further, the MPEP states: “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g. pen and paper or a slide run) to perform the claim limitation.
Using the two-step inquiry, it is clear that Claims 1-20 are each directed to non-statutory subject matter as shown below:
With respect to Claims 1, 11, and 20:
Step 1: Claim 1 is directed to a process, Claim 11 is directed to a system, which is an article of manufacture, and Claim 20 is directed to a non-transitory computer-readable medium, which is an article of manufacture. Each claim is directed to one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“generating a first training sample from the user input;” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
“and generating an adaptive performance policy based on the application behavior prediction and the predicted adaptive performance policy.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“A method of storing data, the method comprising: receiving a user input to an application;” (Regarded as a generic computer function of receiving inputted data. Mere data gathering is considered insignificant extra-solution activity - see MPEP 2106.05(g).)
“training a first learning model on the first training sample, the first learning model further trained on a plurality of training samples from a plurality of user inputs and configured to output an application behavior prediction;” (Training a first learning model on the first training sample, where the first learning model is further trained on a plurality of training samples from a plurality of user inputs only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1). Outputting an application behavior prediction is regarded as a generic computer function of outputting data. Outputting data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
“training a second learning model on the application behavior prediction; the second learning model trained on a plurality of application behavior predictions and configured to output a predicted adaptive performance policy;” (Training a second learning model on the application behavior prediction, where the second learning model is trained on a plurality of application behavior predictions only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1). Outputting a predicted adaptive performance policy is regarded as a generic computer function of outputting data. Outputting data is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Receiving user input from an application is regarded as a generic computer function of receiving inputted data. Mere data gathering is considered insignificant extra-solution activity - see MPEP 2106.05(g).
Training a first learning model on the first training sample, where the first learning model is further trained on a plurality of training samples from a plurality of user inputs only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1). Outputting an application behavior prediction is regarded as a generic computer function of outputting data. Outputting data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
Training a second learning model on the application behavior prediction, where the second learning model is trained on a plurality of application behavior predictions only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1). Outputting a predicted adaptive performance policy is regarded as a generic computer function of outputting data. Outputting data is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claims 2 and 12:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the second learning model includes a reinforcement learning model.” (Generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. A second learning model including a reinforcement model generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claims 3 and 13:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the first learning model includes a federated learning model” (Generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. A first learning model that includes a federated learning model generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claims 4 and 14:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 3 and 13, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the plurality of training samples is obtained from decentralized data sources.” (Obtaining training samples (data) from decentralized data sources is considered insignificant extra-solution activity – see MPEP 2106.05(g).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. Obtaining training samples (data) from decentralized data sources is considered insignificant extra-solution activity – see MPEP 2106.05(g).
With respect to Claims 5 and 15:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively. An additional judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“wherein the application behavior prediction is generated according to at least one of a user behavior pattern, an application performance trend, and a potential operation bottleneck.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claims 6 and 16:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the second learning model includes a deep q-network (DQN).” (Generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. A second learning model that includes a deep q-network (DQN) generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claims 7 and 17:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the predicted adaptive performance policy includes a cache strategy.” (Generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. A predicted adaptive performance policy that includes a cache strategy generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claims 8 and 18:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 7 and 17, respectively.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the cache strategy includes at least one of a content identification policy, a content
eviction policy, and a content storage location policy.” (Generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).)
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. A cache strategy that includes at least one of a content identification policy, a content eviction policy, and a content storage location policy generally links the use of the abstract idea to a particular technological environment or field of use – see MPEP 2106.05(h).
With respect to Claims 9 and 19:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claims 1 and 11, respectively. An additional judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“generating the adaptive performance policy in real-time based on an update to at least one of the first learning model or the second learning model.” (Covers mental concepts that could be practically performed in the human mind or through the use of a pencil and paper, including observation, evaluation, and judgement - see MPEP 2106.04.)
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 10:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“training at least one of the first learning model and the second learning model in real-time based on continuous user input.” (Only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).)
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Training at least one of the first learning model and the second learning model in real-time based on continuous user input only amounts to “apply it” and mere instructions to implement an abstract idea on a computer – see MPEP 2106.05(f)(1).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e. changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 102 as being unpatentable over “Collaborative Caching in Edge Computing via Federated Learning and Deep Reinforcement Learning” by Wang et al., (non-patent literature published on December 22, 2022, hereinafter “Wang”).
With respect to Claims 1, 11, and 20:
Wang teaches:
“A method of storing data, the method comprising:
receiving a user input to an application;” (Page 2, Section 1 “Introduction” recites implementing federated learning and deep reinforcement learning techniques in order to optimize costs for content providers that provide their services through computation-intensive and time-critical applications, such as video services, interactive games, etc. Page 3, Section 4 “Problem Solution” further recites predicting a user’s content request behavior (receiving user input) from the previously mentioned content providing applications using a federated learning model.)
“generating a first training sample from the user input;” (Page 6, Section 4.1.2 recites a local model training process is designed using user request records as the input data (first training sample from the user input) for model training.)
“training a first learning model on the first training sample, the first learning model further trained on a plurality of training samples from a plurality of user inputs and configured to output an application behavior prediction;” (Page 6, Section 4.1.2 recites a federated learning local model training process that is designed using user request records as the input data (first training sample) for model training (training a first learning model). Page 7, Section 4.1.3 further recites the model aggregation and training process in more detail, where each “UE” trains its own local model (first learning model) on its own training samples created from user request records (trained on a plurality of training samples from a plurality of user inputs). Page 2, Section 1 “Introduction” recites the overall federal learning framework is used to accurately predict the content popularity in a given region (output an application behavior prediction).)
“training a second learning model on the application behavior prediction; the second learning model trained on a plurality of application behavior predictions and configured to output a predicted adaptive performance policy;” (Pages 7 and 8, Section 4.2 recites a content placement method that uses the content collection prediction results from the first federated learning model as the basis for a Dueling-DDQN deep reinforcement learning model (training a second learning model on the application behavior prediction). Page 2, Section 1 “Introduction” clarifies that the Dueling-DDQN deep reinforcement learning model is used to make caching decisions for edge nodes, where it can learn the influence of multidimensional content features (second learning model trained on a plurality of application behavior predictions) to formulate an effective caching strategy (configured to output a predicted adaptive performance policy).)
“and generating an adaptive performance policy based on the application behavior prediction and the predicted adaptive performance policy.” (Pages 7 and 8, Section 4.2 recites a content placement method that uses the content collection prediction results from the first federated learning model (application behavior prediction) as the basis for a Dueling-DDQN deep reinforcement learning model. Page 2, Section 1 “Introduction” clarifies that the Dueling-DDQN deep reinforcement learning model is used to make caching decisions for edge nodes, where it can learn the influence of multidimensional content features to formulate an effective caching strategy (predicted adaptive performance policy). This effectively generates an adaptive performance policy based off of both the application behavior prediction from the first learning model and a predicted adaptive performance policy from the second learning model.)
With respect to Claims 2 and 12:
Wang teaches:
“wherein the second learning model includes a reinforcement learning model.” (Pages 7 and 8, Section 4.2 recites a content placement method that uses the content collection prediction results from the first federated learning model as the basis for a Dueling-DDQN deep reinforcement learning model (second learning model includes a reinforcement learning model).)
With respect to Claims 3 and 13:
Wang teaches:
“wherein the first learning model includes a federated learning model” (Page 2, Section 1 “Introduction” introduces the adoption of federated learning (FL) to predict user preferences in different edge nodes in a distributed manner and apply the approach to the design of a caching policy. Page 6, Section 4.1.2 further recites a federated learning local model training process that is designed using user request records as the input data for model training (first learning model includes a federated learning model).)
With respect to Claims 4 and 14:
Wang teaches:
“wherein the plurality of training samples is obtained from decentralized data sources.” (Page 6, Section 4.1.2 recites the use of a federated learning and local model training process using user request records as the input data for model training (plurality of training samples) in order to preserve user privacy. It is inherently understood that local model training means local devices (UEs) individually train models using their own private request records, akin to using decentralized data sources.)
With respect to Claims 5 and 15:
Wang teaches:
“wherein the application behavior prediction is generated according to at least one of a user behavior pattern, an application performance trend, and a potential operation bottleneck.” (Page 2, Section 1 “Introduction” recites the overall federal learning framework is used to accurately predict the content popularity in a given region (the application behavior prediction is generated according to a user behavior pattern).)
With respect to Claims 6 and 16:
Wang teaches:
“wherein the second learning model includes a deep q-network (DQN).” (Pages 7 and 8, Section 4.2 recites a content placement method that uses the content collection prediction results from the first federated learning model as the basis for a Dueling-DDQN deep reinforcement learning model (a second learning model includes a deep q-network (DQN)).)
With respect to Claims 7 and 17:
Wang teaches:“wherein the predicted adaptive performance policy includes a cache strategy.” (Page 2,
Section 1 “Introduction” clarifies that the Dueling-DDQN deep reinforcement learning model is used to make caching decisions for edge nodes, where it can learn the influence of multidimensional content features to formulate an effective caching strategy (the predicted adaptive performance policy includes a cache strategy).)
With respect to Claims 8 and 18:
Wang teaches:
“wherein the cache strategy includes at least one of a content identification policy, a content
eviction policy, and a content storage location policy.” (Page 2, Section 1 “Introduction” clarifies that the Dueling-DDQN deep reinforcement learning model is used to make caching decisions for edge nodes, where it can learn the influence of multidimensional content features (content identification) to formulate an effective caching strategy (the cache strategy includes a content identification policy).)
With respect to Claims 9 and 19:
Wang teaches:
“generating the adaptive performance policy in real-time based on an update to at least one of the first learning model or the second learning model.” (Page 8, Section 4.2 recites the Dueling-DDQN deep reinforcement learning model method (second learning model) that generates a cache strategy (part of the adaptive performance policy), where the method involves a cache manager (CM) that makes caching decisions for all Multi-Access Edge Computing (MEC) servers, meaning real-time data updates from the Dueling-DDQN deep reinforcement learning model (second learning model) are collected and sent to all MEC servers in order to tell them what to store in real-time to maintain the best performance).)
With respect to Claim 10:
Wang teaches:
“training at least one of the first learning model and the second learning model in real-time based on continuous user input.” (Pages 6 and 7, Section 4.1.2 recites the usage of “Follow-the-regularized-leader” (FTRL) during the training of the federated learning user preference model (first learning model), where FTRL is an online optimization method based on an online gradient descent method. The FTRL algorithm allows model parameters to be instantly and iteratively updated as new user interactions are received, akin to real-time training based on continuous user input.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vibha Bhat whose telephone number is (571)-272-7091. The examiner can normally be reached on Monday – Thursday from 8:00 AM to 5:00 PM EST and every other Friday from 8:00 AM to 4:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. See MPEP § 713.01. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes, can be reached at telephone number (571)-270-1006. The fax phone number for the organization where this application or proceeding is assigned is (571)-273-8300. Information regarding the status of an application may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or (572)-272-1000.
/Vibha Bhat/Examiner
Art Unit 2142/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights