DETAILED ACTION
This action is in response to the application filed on 02/21/2024. Claims 1-9 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The examiner notes that claim 1 is a method claim with different conditions present (e.g., single-edge scenarios, multi-edge scenarios). The BRI of a method having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the conditions precedent are not met. See MPEP 2111.04. Therefore while the examiner has examined claim 1 fully in view of compact prosecution, the BRI of claim 1 is broader than written due to the presence of these contingent limitations. Given this and the numerous 112(b) issues the examiner recommends a rewrite of claim 1 that does not recite conditional limitations. The examiner cautions the applicant however to keep the amendments to the same general scope as previously claimed and not to shift inventions.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 1-9,
Independent Claims 1, 7, 8 and 9 recite the element “DRL”. The acronym regarding “DRL” is not clearly defined and thus fail to clearly define the scope of the invention.
Independent Claim 1 recites the element “QoS”. The acronym regarding “QoS” is not clearly defined and thus fail to clearly define the scope of the invention.
Independent Claims 1, 8 and 9 recite the element “FL”. The acronym regarding “FL” is not clearly defined and thus fail to clearly define the scope of the invention.
Independent Claims 2, 4, 6 recite the element “MEC”. The acronym regarding “MEC” is not clearly defined and thus fails to clearly define the scope of the invention.
Independent Claims 2-7 recite the element “ED”. The acronym regarding “ED” is not clearly defined and thus fails to clearly define the scope of the invention.
Independent Claim 5 recites the element “WPT”. The acronym regarding “WPT” is not clearly defined and thus fails to clearly define the scope of the invention.
Independent Claim 9 recites the element “FRL”. The acronym regarding “FRL” is not clearly defined and thus fails to clearly define the scope of the invention.
Claim 1 recites the limitation "the task execution delay and energy consumption" in line 3 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the optimization objectives under multiple constraints" in line 4 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the way of only optimizing local Q-value loss function in classic DRL" in line 7 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the variance of action-value estimation" in line 8 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the frequency of network updates" in line 8 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the personalized demands of smart communities on QoS and system overheads" in line 10 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the effect of local update dispersion" in line 11 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the global optimum" in line 12 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the complexity of federated aggregation" in line 13 of Claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 2 recites the limitation "the set R = {Ri …}" in line 4 of Claim 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 2 recites the limitation "the set EDi = {EDi,j …}" in line 5 of Claim 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 2 recites the limitation "the data volume … the required computational resources … the maximum tolerable delay" in line 13 of Claim 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 3 recites the limitation "the uplink data rate of Edi,j" in line 3 of Claim 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 3 recites the limitation "the available upload bandwidth … the proportion of bandwidth allocated … the transmission power of EDij … the channel gain …. The average power of Gaussian white noise … the distance between … the delay of uploading …. " in lines 4-9 of Claim 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 4 recites the limitation "the local and edge computing modes" in line 3 of Claim 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 4 recites the limitation "the delay and energy consumption" in line 5 of Claim 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 4 recites the limitation "the computing capability … the capacitance coefficient" in lines 7-8 of Claim 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 4 recites the limitation "the proportion of computational resources allocated … the available computational resources … the computing power" in lines 11-12 of Claim 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 5 recites the limitation "the amount of harvested energy by an ED during t" in lines 5-6 of Claim 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the offloading decision" in line 5 of Claim 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the maximum tolerable delay … the available battery power … the sum of the proportion of bandwidth allocated for uploading tasks … the sum of the proportion of the computational resources allocated for executed offloaded tasks" in lines 9-14 of Claim 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the DRL agent … the single-edge environment" in line 3 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the reward signals" in line 5 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the set of data volumes … the set of required computational resources for all tasks … the set of battery power … the set of distances between Mi and all Eds …. The available bandwidth and computational resources … the personalized demands for delay, energy consumption, and task success rate" in lines 7-11 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the set of offloading decisions for all tasks … the sets of the proportion of bandwidth and computational resources allocated to all tasks, respectively" in lines 12-13 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the instant reward" in line 15 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the discount factor" in line 18 of Claim 7. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the actor’s network … the target actor’s network … the training stability … the number of training epoch P, the number of time-slots H, the number of sub-slots T, the update frequencies of Fl fp, and the actor’s network fa, the replay buffer Gi, the batch size N and the learning rate" in lines 4-9 of Claim 8. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the state si(t) … the next state si(t+1) … the state-transition process … the next action … the mapping between s(t) and a(t)" in lines 12-18 of Claim 8. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the loss back-propagated ... the original loss function" in lines 25-26 of Claim 8. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation "the federated actor’s network … the number of edges participating in FRL training … the communication rounds for federated aggregation" in lines 5-7 of Claim 9. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation "the parameter set of the proximal term … the loss changes of local models … the issue of model heterogeneity … the iterative trajectories of local agents … the deviation of local models" in lines 9-11 of Claim 9. There is insufficient antecedent basis for this limitation in the claim.
Claims 1-9 are rejected as failing to define the invention in the manner required by 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
The claims are narrative in form and replete with indefinite language. The structure which goes to make up the device must be clearly and positively specified. The structure must be organized and correlated in such a manner as to present a complete operative device. The claims must be in one sentence form only. Note the format of the claims in the patent cited.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hu et al. (“An Efficient Online Computation Offloading Approach for Large-Scale Mobile Edge Computing via Deep Reinforcement Learning” [2022], hereinafter “Hu”).
Regarding Claim 1,
Hu discloses A method of joint computation offloading and resource allocation in multi-edge smart communities with personalized federated deep reinforcement learning, comprising: (Hu [Sections 3.3-3.5];
PNG
media_image1.png
932
350
media_image1.png
Greyscale
Wherein the performance of joint computation offloading and resource allocation in a community comprising multiple wireless user devices being accomplished through local computing models thus reads broadly on personalized federated deep reinforcement learning in smart-edge communities directed towards joint computation offloading and resource allocation)
design a new multi-edge smart community system consisting of communication, computing, and energy harvesting models, where the task execution delay and energy consumption are formalized as the optimization objectives under multiple constraints; (Hu [Section 6];
PNG
media_image2.png
119
350
media_image2.png
Greyscale
PNG
media_image3.png
659
345
media_image3.png
Greyscale
Wherein optimization across the plurality of wireless user devices reads on designing a new multi-edge smart community wherein task execution delay (frequency) as well as energy consumption are both constraints within the optimization problem)
for single-edge scenarios, propose an improved twin-delayed DRL-based algorithm; we design a new proximal term to improve the way of only optimizing local Q-value loss function in classic DRL, and reduce the variance of action-value estimation by decreasing the frequency of network updates (Hu [Page 675 Paragraph 2]; “To solve this problem, the twin delayed deep deterministic policy gradient [38] (TD3) employs two critic networks to estimate the Q value based on the idea of Double DQN, and takes the smallest value as the update target. Since the Q network is constantly updated during the training process, this may result in ineffective iterations of the actor, and the Q value can not be updated to the optimal value. TD3 uses delayed policy updates by setting the critic update frequency higher than the actor, so that the actor will update after the critic has been determined. Target policy smoothing is also used to randomly select a range of actions to achieve policy smoothing when calculating the Q function. That is, random noises are added to the generated action a to make the learning process more robust. In this paper, we adopt the TD3-like approach to learn scheduling policies of computation offloading and energy transmission for multiple WUDs. We improve the exploration strategy for complex high-dimensional action spaces and propose the RL-based approach for Computation Offloading and Energy Transmission (RLCOET), which alleviates the problem of slow convergence or falling into local optimal solutions caused by the difficulty of fully exploring the action space”)
for multi-edge scenarios, develop a novel personalized FL-based training framework for DRL; during the training process, consider the personalized demands of smart communities on QoS and system overheads; the proposed proximal term can attenuate the effect of local update dispersion, enabling the training to quickly converge to the global optimum; (Hu [Section 5.1];
PNG
media_image4.png
127
348
media_image4.png
Greyscale
PNG
media_image5.png
156
347
media_image5.png
Greyscale
)
design a new partial-greedy based participant selection mechanism, which reduces the complexity of federated aggregation and endows the training with sufficient exploration ability (Hu [Section 1 Subpoint 3]; “We improve the scheduling exploration and experience sampling strategy of the adopted DRL algorithm, so that the proposed RLCOET approach can achieve more efficient convergence performance and obtain near-optimal computation offloading policies for MEC networks with large-scale scheduling variables”
Hu [Page 671 Paragraph 2]; “In addition, [33] proposes a hybrid actor-critic DRL method, which decreases the long-term communication cost. However, these approaches are designed based on conventional exploration and exploitation strategies (e.g.,-greedy and normal distribution sampling), and they could suffer from low convergence efficiency. Therefore, this paper designs a novel DRL-based online offloading algorithm to address the challenge, by decomposing the original problem into two sub-problems solved by the proposed exploration strategy.” Wherein the decomposition of the original DRL approach into two sub-problems through the proposed RLCOET approach comprising the original DRL approach’s associated greedy based selection being broken down into two partial-greedy based participant selection is performed)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Federated-Reinforcement-Learning-Enabled Joint Communication, Sensing, and Computing Resources Allocation in Connected Automated Vehicles Networks” [2022] (Zhang et al.) which discloses resource allocation through the joint optimization of resources performed through a federated learning network.
“System and Method for Dynamically Allocating Processing on a Network Amongst Multiple Network Servers” (US 20030101265 A1) which discloses a method for allocating network resources amongst multiple servers.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN J KIM whose telephone number is (571)272-0523.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN J KIM/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights