DETAILED ACTION
Status of Claims
This Office action is in response to the preliminary amendment filed on 01/27/2025. Claims 1-10 have been canceled. Claims 11-30 are currently pending and are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, which was filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement submitted on 01/02/2025 is in compliance with 37 C.F.R. 1.97 and is being considered by the examiner.
Drawings
New corrected drawings in compliance with 37 CFR 1.121(d) are required in this application since several of the current drawings are illegible. Replacement drawings should be submitted, at least for FIGS. 3 and 8-9. Applicant is advised to employ the services of a competent patent draftsperson outside the Office, as the U.S. Patent and Trademark Office no longer prepares new drawings. The corrected drawings are required in reply to the Office action to avoid abandonment of the application. The requirement for corrected drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In ¶ 143 of the specification, the last sentence should be changed to “As shown in FIG. 7, the process includes step [[801]]701 and step [[802]]702” to correspond with FIG. 7.
In ¶ 161 of the specification, the phrase “repeating step 903 to step 907” should be changed to “repeating step [[903]]803 to step [[907]]807” to correspond with FIG. 8.
In ¶ 162 of the specification, “the terminal 92” should be changed to “the terminal [[92]]102” to correspond with FIG. 1.
Appropriate correction is required.
Claim Objections
Claims 27 and 29-30 are objected to because of the following informalities:
In claims 27 and 30, it appears that “the plurality of service decisions” should be changed to “the plurality of service decision[[s]] instructions.”
In claim 29, it appears that “the target decision instruction” should be changed to “the service decision instruction.”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“generating, using a decision network, a service decision instruction based on the task request and the one or more other UAV servers” (claim 11)
“wherein each of the plurality of service decision instructions is generated using a decision network based on the task request” (claim 27)
“a receiving module configured to receive a task request” (claim 29)
“a decision module configured to generate, using a decision network, a service decision instruction based on the task request” (claim 29)
“a sending module configured to transmit a task request to each of a plurality of unmanned aerial vehicle (UAV) servers within an overlapping coverage area accessible by a terminal” (claim 30)
“a receiving module configured to receive a plurality of service decision instructions from the plurality of UAV servers generated based on the task request using a decision network” (claim 30)
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification (¶ 50: “The target decision network is a pre-trained neural network.” Further, ¶ 201: “The modules in the article surveillance device described above may be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules may be embedded in or independent from a processor in a computer device in a hardware form, or may be stored in a memory in a computer device in a software form, so that the processor invokes and executes an operation corresponding to each of the modules.”) as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 29-30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 29:
The second-to-last paragraph of claim 29 recites “wherein the service decision instruction comprises an indication of whether the UAV server should service the task request and to transmit the service decision instruction to the terminal using the identifier of the terminal.” However, it is unclear what is meant to transmit the service decision instruction; for example, it is unclear whether the service decision instruction is meant to be transmitted by the receiving module, by the decision module, by another component of the service decision device, or by some outside component not included within the service decision device. For examination purposes, claim 29 is interpreted as specifying that the service decision device is configured to transmit the service decision instruction to the terminal using the identifier of the terminal. Regardless of whether this interpretation is correct, clarification is required.
Regarding claim 30:
The last paragraph of claim 30 recites “wherein each of the plurality of service decision instructions comprises an indication of whether the corresponding UAV server should service the task request and to select a UAV server from the plurality of UAV servers to service the task request based on the plurality of service decisions.” However, it is unclear what is meant to select a UAV server; for example, it is unclear whether a UAV server is meant to be selected by the sending module, by the receiving module, by another component of the service decision device, or by some outside component not included within the service decision device. For examination purposes, claim 30 is interpreted as specifying that the service decision device is configured to select a UAV server from the plurality of UAV servers to service the task request based on the plurality of service decision instructions. Regardless of whether this interpretation is correct, clarification is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 11, 14-15, and 26-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claims 11, 27, and 29-30:
Step 1: Claims 11 and 27 are each directed to a computer-implemented method. Claims 29-30 are each directed to a service decision device. Claims 11, 27, and 29-30 are each directed to at least one of the four statutory categories.
Step 2A, prong 1: Claim 11 recites the abstract idea of generating a service decision instruction. This abstract idea is described at least in claim 11 by the mental process steps of determining that the terminal lies within an overlapping service area between a UAV server and one or more other UAV servers based on the position information; and generating a service decision instruction based on the task request and the one or more other UAV servers, wherein the service decision instruction comprises an indication of whether the UAV server should service the task request. These steps fall into the mental processes grouping of abstract ideas as they include a human mentally identifying whether the terminal lies within the overlapping service area and then using pen and paper to write out an instruction indicating whether the UAV server should service the task request. The limitations as drafted are processes that, under their broadest reasonable interpretation, cover their performance in the mind if not for the recitation of generic computing components.
Claim 29 recites the abstract concept of generating a service decision instruction. This abstract idea is described at least in claim 29 by the mental process step of generating a service decision instruction based on the task request, wherein the service decision instruction comprises an indication of whether the UAV server should service the task request. This step falls into the mental processes grouping of abstract ideas because it includes a human using pen and paper to write out an instruction indicating whether the UAV server should service the task request. The limitations as drafted are processes that, under their broadest reasonable interpretation, cover their performance in the mind if not for the recitation of generic computing components.
Claims 27 and 30 recite the abstract concept of selecting a UAV server. This abstract idea is described at least in claims 27 and 30 by the mental process step of selecting a UAV server from the plurality of UAV servers to service the task request based on the plurality of service decisions. This step falls into the mental processes grouping of abstract ideas as it includes a human mentally choosing which UAV server should service the task request based on the service decision instructions. The limitations as drafted are processes that, under their broadest reasonable interpretation, cover their performance in the mind if not for the recitation of generic computing components.
With respect to claims 11, 27, and 29-30, other than reciting that the processes are “computer-implemented” and reciting “one or more processors,” “a decision network,” a “service decision device comprising: a receiving module … and a decision module,” and a “service decision device comprising: a sending module … and a receiving module,” nothing in the claims precludes the idea from practically being performed in the human mind. If not for the “computer,” “processor,” “decision network,” and “module” language, the claims encompass a human operator performing each of the mental process steps in the mind with the help of pen and paper.
Step 2A, prong 2: The claims recite elements additional to the abstract concepts. However, these additional elements fail to integrate the abstract idea into a practical application.
Claim 11 recites that the method is “computer-implemented,” which amounts to specifying the use of generic computer components (as supported by ¶ 212 of the instant specification) that are simply employed as tools for performing the abstract idea. The use of such generic computing components for executing the abstract idea does not integrate the abstract idea into a practical application (see MPEP 2106.05(f)). Claim 11 also recites a step of receiving, by one or more processors, a task request from a terminal, wherein the task request comprises an identifier of the terminal, position information of the terminal, and/or task information of the terminal. This step is insignificant extra-solution activity, as it simply gathers data necessary to perform the abstract idea (i.e., all uses of the abstract idea require such data gathering). Additionally, the step of transmitting the service decision instruction to the terminal using the identifier of the terminal is insignificant extra-solution activity, because it is a data output step that does not impose meaningful limits on the claim such that it is not nominally or tangentially related to the invention. The recitation of such insignificant extra-solution activity does not integrate the abstract idea into a practical application (see MPEP 2106.05(g)). Claim 11 additionally specifies that the service decision instruction is generated using a decision network. This amounts to general linking of the abstract idea to the particular technological field of machine learning. Using a decision network to generate the service decision instruction merely serves to employ machine learning technology to execute the abstract idea. Such general linking limitations do not integrate the abstract idea into a practical application (see MPEP 2106.05(h)).
Claim 27 recites that the method is “computer-implemented,” which amounts to specifying the use of generic computer components (as supported by ¶ 212 of the instant specification) that are simply employed as tools for performing the abstract idea. The use of such generic computing components for executing the abstract idea does not integrate the abstract idea into a practical application (see MPEP 2106.05(f)). Claim 27 also recites steps of transmitting a task request to each of a plurality of UAV servers within an overlapping coverage area accessible by a terminal, wherein the task request comprises an identifier of the terminal, position information of the terminal, and/or task information of the terminal; and receiving a plurality of service decision instructions from the plurality of UAV servers, wherein each of the plurality of service decision instructions comprises an indication of whether the corresponding UAV server should service the task request. These steps amount to insignificant extra-solution activity, because they are data gathering and data output steps that are necessary for performing the abstract idea (i.e., all uses of the abstract idea require such data gathering). Also, the step of transmitting the selection and the task request to the UAV server to service the task request is insignificant extra-solution activity, because it is a data output step that does not impose meaningful limits on the claim such that it is not nominally or tangentially related to the invention. Note that while the claim recites that the transmission is used for “the UAV server to service the task request,” this is merely the intended use of the claimed method and does not actually have to occur as part of the method. The recitation of such insignificant extra-solution activity does not integrate the abstract idea into a practical application (see MPEP 2106.05(g)). Claim 27 additionally specifies that each of the plurality of service decision instructions is generated using a decision network based on the task request. This amounts to general linking of the abstract idea to the particular technological field of machine learning. Using a decision network to generate each of the service decision instructions merely serves to employ machine learning technology to implement the abstract idea. The recitation of such general linking limitations do not integrate the abstract idea into a practical application (see MPEP 2106.05(h)).
Claim 29 recites a service decision device comprising a receiving module and a decision module, which are generic computer components (as supported by ¶ 201 of the instant specification) that are simply employed as tools to perform the abstract idea. The use of such generic computing components for executing the abstract idea does not integrate the abstract idea into a practical application (see MPEP 2106.05(f)). Claim 29 also recites the step of receiving a task request sent by a terminal in an overlapping service area between a UAV server and one or more other UAV servers, wherein the task request comprises an identifier of the terminal, position information of the terminal and/or task information of the terminal. This step is insignificant extra-solution activity, as it simply gathers data that is necessary for performing the abstract idea (i.e., all uses of the abstract idea require such data gathering). Further, the step of transmitting the service decision instruction to the terminal using the identifier of the terminal is insignificant extra-solution activity, because it is a data output step that does not impose meaningful limits on the claim such that it is not nominally or tangentially related to the invention. Note that while the claim recites that “the target decision instruction is used for the terminal to select among the UAV server and the one or more UAV servers to service the task request based on the service decision instruction and service decision instructions transmitted by the one or more other UAV servers,” this is merely the intended use of the claimed service decision device and does not actually have to occur as part of the method. The recitation of such insignificant extra-solution activity does not integrate the abstract idea into a practical application (see MPEP 2106.05(g)). Claim 29 additionally specifies that the service decision instruction is generated using a decision network based on the task request. This limitation amounts to general linking of the abstract idea to the particular technological field of machine learning. Using the decision network to generate the service decision instruction merely serves to employ machine learning technology to implement the abstract idea. The recitation of such general linking limitations do not integrate the abstract idea into a practical application (see MPEP 2106.05(h)).
Claim 30 recites a service decision device comprising a sending module and a receiving module, which are generic computer components (as supported by ¶ 201 of the instant specification) that are simply employed as tools to perform the abstract idea. The use of such generic computing components for executing the abstract idea does not integrate the abstract idea into a practical application (see MPEP 2106.05(f)). Claim 29 also recites the steps of transmitting a task request to each of a plurality of UAV servers within an overlapping coverage area accessible by a terminal, wherein the task request comprises an identifier of the terminal, position information of the terminal, and/or task information of the terminal; and receiving a plurality of service decision instructions from the plurality of UAV servers generated based on the task request, wherein each of the plurality of service decision instructions comprises an indication of whether the corresponding UAV server should service the task request. These steps are insignificant extra-solution activity, because they are data gathering and data output steps that are necessary for performing the abstract idea (i.e., all uses of the abstract idea require such data gathering). The recitation of such insignificant extra-solution activity does not integrate the abstract idea into a practical application (see MPEP 2106.05(g)). Claim 29 additionally specifies that the plurality of service decision instructions are generated using a decision network. This limitation amounts to general linking of the abstract idea to the particular technological field of machine learning. Using the decision network to generate the plurality of service decision instructions merely serves to employ machine learning technology to implement the abstract idea. The recitation of such general linking limitations do not integrate the abstract idea into a practical application (see MPEP 2106.05(h)).
Step 2B: The additional elements are re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The specification does not provide any indication that the recited computer components are anything other than generic computing components used within a conventional computer. The use of such generic and conventional computer components for executing the abstract idea does not amount to significantly more than the abstract idea itself (see MPEP 2106.05(f)).
MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere receipt or transmission of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). Accordingly, the steps of receiving the task request, transmitting the service decision instruction, transmitting the task request, receiving the plurality of service decision instructions, and transmitting the selection and task request merely amount to insignificant extra-solution activity that does not amount to significantly more than the abstract idea itself (see MPEP 2106.05(g)).
The limitations specifying that recited steps are performed by a decision network are considered general linking limitations that do not impose meaningful limits on the claims. These limitations merely serve to link the abstract idea to the technological field of machine learning and specify that certain steps of the abstract idea are executed using machine learning with no additional details of how this occurs. Therefore, these general linking limitations do not amount to significantly more than the abstract idea itself (see MPEP 2106.05(h)).
For the above reasons, the additional elements do not amount to significantly more than the abstract idea itself, whether considered individually or in combination. Therefore, when considering the combination of elements and the claimed invention as a whole, claims 11, 27, and 29-30 are not patent-eligible.
Regarding claims 14-15, 26, and 28:
Claim 14 recites the additional mental process steps of calculating decision information of the UAV server based on state information of the UAV server; and generating the service decision instruction based on the decision information. These steps fall into the mental processes grouping of abstract ideas because they encompass a human operator mentally considering the state information of the UAV server to calculate the decision information with the help of pen and paper and then writing out the service decision instruction based on the calculated decision information. The limitations as drafted encompass their performance in the human mind.
Some of the dependent claims recite limitations that further define the claimed mental process. For example, claim 15 specifies that the decision information comprises an action decision of the UAV server, available computing resources of the UAV server, available bandwidth of the UAV server, and/or an estimated execution time for a task. These limitations are considered additional mental process steps that do not preclude the mental process from being performed in the human mind with the help of pen and paper.
As explained above, dependent claims 14-15, 26, and 28 only recite additional mental process steps and limitations further defining the mental process. These additional elements fail to integrate the abstract idea into a practical application or amount to significantly more than the abstract idea itself. Therefore, when considering the combination of elements and the claimed invention as a whole, claims 14-15, 26, and 28 are not patent-eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11-20 and 23-30 are rejected under 35 U.S.C. 103 as being unpatentable over Qin et al. (CN 110177379 A), hereinafter referred to as Qin, in view of Chai et al. (CN 115827108 A), hereinafter referred to as Chai.
Regarding claim 11:
Qin discloses the following limitations:
“A computer-implemented method, comprising: receiving, by one or more processors, a task request from a terminal.” (Qin ¶ 10: “When a mobile terminal is detected to be in a multi-base station coverage area, the mobile terminal sends a connection initialization request to the multiple base stations covering the area.” Also, Qin ¶ 122: “Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware.”)
“wherein the task request comprises an identifier of the terminal, position information of the terminal, and/or task information of the terminal.” (Qin ¶ 14: “in a first possible implementation of the first aspect, the connection initialization request carries the location information of the mobile terminal, the amount of data to be processed by the mobile terminal, and the required QoS service level.” This at least teaches the task request comprising “position information of the terminal” and “task information of the terminal” as claimed. Further, Qin ¶ 12 discloses that each of the base stations send response messages to the mobile terminal, which implies that an identifier of the terminal is used for sending the response messages to the correct mobile terminal.)
“determining that the terminal lies within an overlapping service area between a… server and one or more other … servers based on the position information” and “generating… a service decision instruction based on the task request and the one or more other … servers, wherein the service decision instruction comprises an indication of whether the … server should service the task request.” (Qin ¶ 58: “the multi-base station coverage area refers to the overlapping area between the coverage areas of multiple different base stations. The connection initialization request sent by the mobile terminal is used to send information about the mobile terminal itself and the services it needs to the base station. The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Also, Qin ¶ 16 discloses that the indicator parameters can include an indication of “whether the QoS service level can be met.”)
“and transmitting the service decision instruction to the terminal using the identifier of the terminal.” (Qin ¶¶ 11-12 disclose that in response to the connection request, “The mobile terminal receives various indicator parameters sent by each of the base stations.” Additionally, Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Note that sending the parameters to the mobile terminal in response to the connection request message implies that an identifier of the mobile terminal is used for sending the response messages to the correct mobile terminal.)
The following limitations are not specifically disclosed by Qin, but are taught by Chai:
The servers are “UAV servers.” (Chai ¶ 12: “This system consists of F terminal devices and M UAVs. Each UAV carries an MEC server to offload tasks within a fixed area.”)
The service decision instruction is generated “using a decision network.” (Chai ¶ 15: “Solve the task offloading model for minimizing latency and energy consumption in the UAV-mobile edge computing system using deep reinforcement learning. The solution method is as follows: construct a task offloading model for each offloading task solved by deep reinforcement learning through a multi-objective Markov decision process.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using UAV servers as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 3 teaches that “The edge servers mounted on UAVs can expand their communication coverage and reduce geographical constraints, thereby improving deployment efficiency and user service quality. UAV-MEC has advantages such as high flexibility, wide coverage, faster response, and low cost.” Additionally, it would have been obvious to use a decision network to generate the service decision as taught by Chai, because Chai ¶ 3 teaches that “Machine learning-based methods can dynamically adjust offloading strategies in UAV-MEC environments to adapt to rapid environmental changes.”
Regarding claim 12:
The combination of Qin and Chai teaches “The computer-implemented method of claim 11,” and Qin also teaches the method “further comprising: servicing the task request in response to a terminal selection of the … server based on the service decision instruction and service decision instructions transmitted to the terminal by the one or more other … servers.” (Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Also, Qin ¶ 62: “enables the selection of the target base station with the best data processing effect among multiple base stations to establish a communication connection, which can make fuller use of network resources and improve the quality of user service.”)
As explained regarding claim 11 above, Chai teaches the servers being “UAV servers.”
Regarding claim 13:
The combination of Qin and Chai teaches “The computer-implemented method of claim 12,” and Qin further teaches “wherein the service decision instruction and the service decision instructions … only indicate one … server that should service the task request.” (Qin ¶ 45: “This allows the system to select the target base station with the best data processing performance from among multiple base stations to establish a communication connection, thereby making fuller use of network resources and improving the quality of user service.”)
Qin does not specifically disclose the servers being “UAV servers,” or that the instructions indicating one UAV server that should service the task request are “transmitted to the terminal by the one or more other UAV servers.” However, Chai does teach these limitations. (Chai ¶ 12: “This system consists of F terminal devices and M UAVs. Each UAV carries an MEC server to offload tasks within a fixed area.” Also, Chai ¶ 19: “The action space A includes the following two actions: executing tasks on the terminal device and unloading tasks to the UAV-mobile edge computing system.” Further, Chai ¶ 110 discloses to “Determine whether the training has ended, and then decide whether to output the unloading decision.” It is implied that the final decision would need to be output to the terminal device to facilitate completion of the task.)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using UAV servers as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 3 teaches that “The edge servers mounted on UAVs can expand their communication coverage and reduce geographical constraints, thereby improving deployment efficiency and user service quality. UAV-MEC has advantages such as high flexibility, wide coverage, faster response, and low cost.” Additionally, it would have been obvious to transmit the service decision instruction to the terminal as taught by Chai, because this is a simple substitution of one known element (i.e., allowing a remote system to make the decision and transmit the result to the terminal as taught by Chai) for another (i.e., allowing the terminal to make the decision and establish the connection itself as disclosed by Qin) to obtain predictable results (see MPEP 2143(I)(B)). A person having ordinary skill in the art could have substituted the transmitting a final result to the terminal with the step of the terminal making the final decision to achieve the predictable result of offloading the decision process to a remote system capable of making an optimized decision.
Regarding claim 14:
The combination of Qin and Chai teaches “The computer-implemented method of claim 11,” and Chai also teaches “wherein the generating the service decision comprises: calculating, using the decision network and based on state information of the UAV server and the task request, decision information of the UAV server; and generating the service decision instruction based on the decision information.” (Chai ¶¶ 18-19: “Step 5: The agent in deep reinforcement learning begins to interact with the MEC environment. On the one hand, the agent obtains the current state from the MEC environment… Step 6: In deep reinforcement learning, the agent obtains the current Q value through Q-network training, selects action a in the current state s from action space A, and executes the action to obtain vector value reward r and the next state s´. The action space A includes the following two actions: executing tasks on the terminal device and unloading tasks to the UAV-mobile edge computing system.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using the decision network to generate the service decision instruction based on a current state of the environment as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that this can help with “obtaining the optimal computation offloading strategy, thus optimizing system performance, meeting the ever-changing preferences of users, obtaining the optimal solution that meets user needs, improving solution efficiency and flexibility, and can be widely used for computation offloading in UAV edge computing environments.”
Regarding claim 15:
The combination of Qin and Chai teaches “The computer-implemented method of claim 14,” and Chai also teaches “wherein the decision information comprises an action decision of the UAV server, available computing resources of the UAV server, available bandwidth of the UAV server, and/or an estimated execution time for a task.” (Chai ¶ 19: “Step 6: In deep reinforcement learning, the agent obtains the current Q value through Q-network training, selects action a in the current state s from action space A, and executes the action to obtain vector value reward r and the next state s´. The action space A includes the following two actions: executing tasks on the terminal device and unloading tasks to the UAV-mobile edge computing system.” This at least teaches the decision information comprising an action decision of the UAV server as claimed.)
Note that under the broadest reasonable interpretation (BRI) of claim 15, consistent with the instant specification, the limitation “wherein the decision information comprises an action decision of the UAV server, available computing resources of the UAV server, available bandwidth of the UAV server, and/or an estimated execution time for a task” is treated as an alternative limitation. Applicant has elected to use the term “and/or” in the claim language, and therefore, the BRI covers the scenario in which only one of the limitations applies. Accordingly, while only the “action decision of the UAV server” has been addressed here, the claim is still rejected in its entirety.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using the decision network to generate an action decision of the UAV server as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that this helps with “obtaining the optimal computation offloading strategy, thus optimizing system performance, meeting the ever-changing preferences of users, obtaining the optimal solution that meets user needs, improving solution efficiency and flexibility, and can be widely used for computation offloading in UAV edge computing environments.”
Regarding claim 16:
The combination of Qin and Chai teaches “The computer-implemented method of claim 11,” and Chai further teaches the method “further comprising: training the decision network by iterating over one or more training epochs using sample environment data comprising a plurality of task requests each corresponding to each of a plurality of state information sets of the UAV server.” (Chai ¶ 18 discloses that Step 5 ends with a step to “update the preference experience pool W using the current iteration number.” Also, Chai ¶ 21: “Step 8: Train the experience samples: First, randomly select a portion of the experience samples from the experience buffer pool Φ; then, select the experience preferences from the preference experience pool W using a non-dominated ranking method. Train both the Q-network and the target Q-network simultaneously to maximize the vector value reward and obtain the optimal unloading decision. During training, let the input of the Q-network be the current state s, the experience preference, and the current preference, and output the Q-value. Let the input of the target Q-network be the next state s´, the experience preference, and the current preference, and output the target Q-value.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by training the model using iterations of experience samples as is taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art may have been motivated to do this since Chai ¶ 38 teaches that “This invention adopts a dynamic weight adjustment strategy and uses a Q-network to train and optimize the current user's preferences and the previous user preferences simultaneously. The previous user preferences are obtained from the preference experience pool through a non-dominated ranking method, which can better maintain the previously learned strategy.”
Regarding claim 17:
The combination of Qin and Chai teaches “The computer-implemented method of claim 16,” and Chai additionally teaches “wherein each state information set comprises a location of the UAV server, available computing resources of the UAV server, available bandwidth of the UAV server, and/or a number of users within a coverage area of the UAV server.” (Chai ¶ 56: “The COP (Computation Offloading Problem) is modeled as a multi-objective optimization problem with added task dependency constraints, aiming to simultaneously minimize the latency and energy consumption of the UAV-MEC system.” Also, Chai ¶¶ 69-76 teach the consideration of “channel bandwidth” when determining the “time for transmitting the task to the drone” and the “latency and energy consumption of a task.” This at least teaches that the state information comprises available bandwidth of the UAV server as claimed.)
Note that under the broadest reasonable interpretation (BRI) of claim 17, consistent with the instant specification, the limitation “wherein each state information set comprises a location of the UAV server, available computing resources of the UAV server, available bandwidth of the UAV server, and/or a number of users within a coverage area of the UAV server” is treated as an alternative limitation. Applicant has elected to use the term “and/or” in the claim language, and therefore, the BRI covers the scenario in which only one of the limitations applies. Accordingly, while only the “available bandwidth of the UAV server” has been addressed here, the claim is still rejected in its entirety.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by configuring the decision network to consider the channel bandwidth as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this based on recognizing that a UAV server should not be selected to service the task request if the UAV server is not capable of completing the task due to limitations such as not having enough bandwidth; a person having ordinary skill in the art would have recognized that selecting a UAV server that is incapable of completing the task would be inefficient and would require a reselection of another UAV server for servicing the task request, which would increase the time and resources required for completing the process.
Regarding claim 18:
The combination of Qin and Chai teaches “The computer-implemented method of claim 16,” and Chai also teaches the following limitations:
“wherein the iterating over one or more training epochs comprises: collecting experiences inside an experience pool by interacting with the sample environment data.” (Chai ¶ 21: “Step 8: Train the experience samples: First, randomly select a portion of the experience samples from the experience buffer pool Φ.”)
“updating internal weights of the evaluation network based on evaluation values obtained by the evaluation network; and updating internal weights of the decision network based on each of the collected experiences.” (Chai ¶ 38 discloses that “This invention adopts a dynamic weight adjustment strategy and uses a Q-network to train and optimize the current user's preferences and the previous user preferences simultaneously. The previous user preferences are obtained from the preference experience pool through a non-dominated ranking method, which can better maintain the previously learned strategy.” Also, Chai ¶ 116: “the present invention can quickly adjust the target weight to cope with changes in user preferences, thereby meeting user needs.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by adjusting internal weights of the decision network based on collected samples from an experience pool as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that “This invention adopts a dynamic weight adjustment strategy and uses a Q-network to train and optimize the current user's preferences and the previous user preferences simultaneously. The previous user preferences are obtained from the preference experience pool through a non-dominated ranking method, which can better maintain the previously learned strategy.”
Regarding claim 19:
The combination of Qin and Chai teaches “The computer-implemented method of claim 18,” and Chai further teaches “wherein the updating the internal weights of the decision network occurs after completing a full training epoch.” (Chai ¶¶ 21-23: “During training, let the input of the Q-network be the current state s, the experience preference, and the current preference, and output the Q-value. Let the input of the target Q-network be the next state s´, the experience preference, and the current preference, and output the target Q-value. Calculate the loss function L … Finally, the Q-network is updated using the loss function value, and the Q-network parameters are synchronized to the target Q-network every 300 generations.” Further, Chai ¶ 38: “Multiple objectives are optimized simultaneously, and the weights are dynamically adjusted to meet different user preferences.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by adjusting internal weights of the decision network after a training epoch as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that “This invention adopts a dynamic weight adjustment strategy and uses a Q-network to train and optimize the current user's preferences and the previous user preferences simultaneously. The previous user preferences are obtained from the preference experience pool through a non-dominated ranking method, which can better maintain the previously learned strategy.”
Regarding claim 20:
The combination of Qin and Chai teaches “The computer-implemented method of claim 18,” and Chai also teaches the following limitations:
“wherein the collecting experiences comprises: inputting a first sample environment data into the decision network to obtain decision information of the UAV server; evaluating the decision information using the evaluation network to obtain a final reward value.” (Chai ¶¶ 29-32: “Select action a using the Double DQN method, and determine action a using two action value functions: one for estimating the action, and the other for estimating the value of the action. … Executing action a in the current state s yields the next state s' and a vector value reward r.”)
“determining a second sample environment data by applying the decision information to the first sample environment; and storing the first sample environment data, the second sample environment data, the decision information, and the reward value inside the experience pool, wherein the experience pool comprises experiences collected from the UAV server and the one or more other UAV servers.” (Chai ¶ 18: “On the one hand, the agent obtains the current state from the MEC environment. On the other hand, the MEC environment returns the current reward vector value and the next state based on the action selected by the agent. The agent obtains the current state from the MEC environment and updates the preference experience pool. The method for updating the preference experience pool is as follows: select the current preference from the preference space Ψ and determine whether the current preference is in the encountered preference experience pool W. If it does not exist, add the current preference to the preference experience pool W. Otherwise, update the preference experience pool W using the current iteration number.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by assessing a final reward value and performing multiple iterations to provide an updated experience pool as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that “This invention adopts a dynamic weight adjustment strategy and uses a Q-network to train and optimize the current user's preferences and the previous user preferences simultaneously. The previous user preferences are obtained from the preference experience pool through a non-dominated ranking method, which can better maintain the previously learned strategy.”
Regarding claim 23:
The combination of Qin and Chai teaches “The computer-implemented method of claim 20,” and Chai also teaches the following limitations:
“wherein the evaluating comprises: calculating one or more reward and punishment values from the decision information and the first sample environment based on a plurality of constraints.” (Chai ¶ 84: “Total Energy Consumption (MUE) includes the energy consumption of TU and UAV during missions and the energy consumption of UAV during flight. In addition, during the task unloading process, we also need to follow the following task dependency constraints…” Further, Chai ¶¶ 103-105: “This invention aims to minimize latency and energy consumption, but to maximize the reward value, the opposite of latency and energy consumption is taken. … Therefore, maximizing is equivalent to minimizing total latency and total energy consumption.”)
“and aggregating the one or more calculated reward and punishment values to obtain a final reward value corresponding to the decision information.” (Chai ¶¶ 32-34 disclose the calculation of “a vector value reward r” using the equation below based on “the reward value for latency and the reward value for energy consumption.” The equation shows that a summation (i.e., aggregation) is used to calculate the final reward value.)
PNG
media_image1.png
84
425
media_image1.png
Greyscale
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by aggregating reward values to obtain a final reward value based on specified constraints as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this since Chai ¶ 38 teaches that “This invention incorporates task dependency constraints in the modeling of UAV-MEC systems, thereby improving the utilization rate of computing resources,” and that “the multi-objective Markov decision process expands the reward value into a vector value reward, where each element corresponds to an objective. Multiple objectives are optimized simultaneously, and the weights are dynamically adjusted to meet different user preferences.”
Regarding claim 24:
The combination of Qin and Chai teaches “The computer-implemented method of claim 23,” and Chai also teaches the method “further comprising: weighting the one or more calculated reward and punishment values using a reward factor.” (Chai ¶¶ 21-23 teach the calculation of a loss function L using the equations below, where “γ represents the reward discount factor.”)
PNG
media_image2.png
212
621
media_image2.png
Greyscale
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by incorporating a reward factor for weighting the rewards as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that “the multi-objective Markov decision process expands the reward value into a vector value reward, where each element corresponds to an objective. Multiple objectives are optimized simultaneously, and the weights are dynamically adjusted to meet different user preferences.”
Regarding claim 25:
The combination of Qin and Chai teaches “The computer-implemented method of claim 23,” and Chai also teaches “wherein the plurality of constraints are based on at least one of the available computing resources of the UAV server, the available bandwidth of the UAV server, the number of users within a coverage area of the UAV server, an estimated execution time for the task request by the UAV server, and an execution time of the current epoch.” (Chai ¶¶ 25-27: “the task dependency constraints include: Constraint 1: The UAV can only fly within a specified rectangular area, and the horizontal range of time slot t and the maximum distance to fly within time slot t are also specified.” This at least teaches the constraints being based on “an estimated execution time for the task request by the UAV server” as claimed.)
Note that under the broadest reasonable interpretation (BRI) of claim 25, consistent with the instant specification, the limitation “wherein the plurality of constraints are based on at least one of the available computing resources of the UAV server, the available bandwidth of the UAV server, the number of users within a coverage area of the UAV server, an estimated execution time for the task request by the UAV server, and an execution time of the current epoch” is treated as an alternative limitation. Applicant has elected to use the phrase “at least one” in the claim language, and therefore, the BRI covers the scenario in which only one of the limitations applies. Accordingly, while only the “estimated execution time for the task request by the UAV server” has been addressed here, the claim is still rejected in its entirety.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using constraints based on an estimated execution time for the task request as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 38 teaches that “This invention incorporates task dependency constraints in the modeling of UAV-MEC systems, thereby improving the utilization rate of computing resources.”
Regarding claim 26:
The combination of Qin and Chai teaches “The computer-implemented method of claim 11,” and Qin also teaches “wherein the task information comprises a data size, a computation strength, and/or a maximum allowable time delay for the task request.” (Qin ¶ 14: “the connection initialization request carries the location information of the mobile terminal, the amount of data to be processed by the mobile terminal, and the required QoS service level.” This at least teaches the task information comprising a data size as claimed.)
Note that under the broadest reasonable interpretation (BRI) of claim 26, consistent with the specification, the limitation that “the task information comprises a data size, a computation strength, and/or a maximum allowable time delay for the task request” is treated as an alternative limitation. Applicant has elected to use the term “and/or” in the claim language, and therefore, the BRI covers the scenario in which only one of the limitations applies. Accordingly, while only the “data size” has been addressed here, the claim is still rejected in its entirety.
Regarding claim 27:
Qin discloses the following limitations:
“A computer-implemented method comprising: transmitting a task request to each of a plurality of … servers within an overlapping coverage area accessible by a terminal.” (Qin ¶ 10: “When a mobile terminal is detected to be in a multi-base station coverage area, the mobile terminal sends a connection initialization request to the multiple base stations covering the area.” Also, Qin ¶ 122: “Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware.”)
“wherein the task request comprises an identifier of the terminal, position information of the terminal, and/or task information of the terminal.” (Qin ¶ 14: “in a first possible implementation of the first aspect, the connection initialization request carries the location information of the mobile terminal, the amount of data to be processed by the mobile terminal, and the required QoS service level.” This at least teaches the task request comprising “position information of the terminal” and “task information of the terminal” as claimed. Further, Qin ¶ 12 discloses that each of the base stations send response messages to the mobile terminal, which implies that an identifier of the terminal is used for sending the response messages to the correct mobile terminal.)
“receiving a plurality of service decision instructions from the plurality of … servers, wherein each of the plurality of service decision instructions comprises an indication of whether the corresponding … server should service the task request and wherein each of the plurality of service decision instructions is generated … based on the task request.” (Qin ¶ 58: “the multi-base station coverage area refers to the overlapping area between the coverage areas of multiple different base stations. The connection initialization request sent by the mobile terminal is used to send information about the mobile terminal itself and the services it needs to the base station. The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Also, Qin ¶ 16 discloses that the indicator parameters can include an indication of “whether the QoS service level can be met.”)
“selecting a … server from the plurality of … servers to service the task request based on the plurality of service decisions.” (Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.”)
“and transmitting the selection and the task request to the … server to service the task request.” (Qin ¶ 58: “the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Additionally, Qin ¶ 62: “enables the selection of the target base station with the best data processing effect among multiple base stations to establish a communication connection, which can make fuller use of network resources and improve the quality of user service.”)
The following limitations are not specifically disclosed by Qin, but are taught by Chai:
The servers are “UAV servers.” (Chai ¶ 12: “This system consists of F terminal devices and M UAVs. Each UAV carries an MEC server to offload tasks within a fixed area.”)
The service decision instructions are generated “using a decision network.” (Chai ¶ 15: “Solve the task offloading model for minimizing latency and energy consumption in the UAV-mobile edge computing system using deep reinforcement learning. The solution method is as follows: construct a task offloading model for each offloading task solved by deep reinforcement learning through a multi-objective Markov decision process.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Qin by using UAV servers as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 3 teaches that “The edge servers mounted on UAVs can expand their communication coverage and reduce geographical constraints, thereby improving deployment efficiency and user service quality. UAV-MEC has advantages such as high flexibility, wide coverage, faster response, and low cost.” Additionally, it would have been obvious to use a decision network to generate the service decision as taught by Chai, because Chai ¶ 3 teaches that “Machine learning-based methods can dynamically adjust offloading strategies in UAV-MEC environments to adapt to rapid environmental changes.”
Regarding claim 28:
The combination of Qin and Chai teaches “The computer-implemented method of claim 27,” and Qin also teaches “wherein the plurality of service decision instructions indicate a … server that should service the task request.” (Qin ¶ 45: “This allows the system to select the target base station with the best data processing performance from among multiple base stations to establish a communication connection, thereby making fuller use of network resources and improving the quality of user service.”)
As explained regarding claim 27 above, Chai teaches the servers being “UAV servers.”
Regarding claim 29:
Qin discloses the following limitations:
“A service decision device comprising: a receiving module configured to receive a task request sent by a terminal in an overlapping service area between a… server and one or more other … servers.” (Qin ¶ 122: “Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware.” Further, Qin ¶ 10: “When a mobile terminal is detected to be in a multi-base station coverage area, the mobile terminal sends a connection initialization request to the multiple base stations covering the area.” It is implied that the base stations each include a receiver for receiving the connection initialization request.)
“wherein the task request comprises an identifier of the terminal, position information of the terminal and/or task information of the terminal.” (Qin ¶ 14: “in a first possible implementation of the first aspect, the connection initialization request carries the location information of the mobile terminal, the amount of data to be processed by the mobile terminal, and the required QoS service level.” This at least teaches the task request comprising “position information of the terminal” and “task information of the terminal” as claimed. Further, Qin ¶ 12 discloses that each of the base stations send response messages to the mobile terminal, which implies that an identifier of the terminal is used for sending the response messages to the correct mobile terminal.)
“and a decision module configured to generate… a service decision instruction based on the task request, wherein the service decision instruction comprises an indication of whether the … server should service the task request.” (Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Further, Qin ¶ 16 discloses that the indicator parameters can include an indication of “whether the QoS service level can be met.”)
“and to transmit the service decision instruction to the terminal using the identifier of the terminal.” (Qin ¶¶ 11-12 disclose that in response to the connection request, “The mobile terminal receives various indicator parameters sent by each of the base stations.” Additionally, Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.” Note that sending the parameters to the mobile terminal in response to the connection request message implies that an identifier of the mobile terminal is used for sending the response messages to the correct mobile terminal.)
“and wherein the target decision instruction is used for the terminal to select among the … server and the one or more other … servers to service the task request based on the service decision instruction and service decision instructions transmitted by the one or more other … servers.” (Qin ¶ 58: “The base station calculates and feeds back the corresponding indicator parameters so that the mobile terminal can select the most suitable base station from multiple base stations to establish a connection based on these indicator parameters.”)
The following limitations are not specifically disclosed by Qin, but are taught by Chai:
The servers are “UAV servers.” (Chai ¶ 12: “This system consists of F terminal devices and M UAVs. Each UAV carries an MEC server to offload tasks within a fixed area.”)
The service decision instruction is generated “using a decision network.” (Chai ¶ 15: “Solve the task offloading model for minimizing latency and energy consumption in the UAV-mobile edge computing system using deep reinforcement learning. The solution method is as follows: construct a task offloading model for each offloading task solved by deep reinforcement learning through a multi-objective Markov decision process.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the system of Qin by using UAV servers as taught by Chai with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Chai ¶ 3 teaches that “The edge servers mounted on UAVs can expand their communication coverage and reduce geographical constraints, thereby improving deployment efficiency and user service quality. UAV-MEC has advantages such as high flexibility, wide coverage, faster response, and low cost.” Additionally, it would have been obvious to use a decision network to generate the service decision as taught by Chai, because Chai ¶ 3 teaches that “Machine learning-based methods can dynamically adjust offloading strategies in UAV-MEC environments to adapt to rapid environmental changes.”
Regarding claim 30:
Qin discloses “A service decision device comprising: a sending module … and a receiving module configured to” perform a process. (Qin ¶¶ 10-12: “the mobile terminal sends a connection initialization request to the multiple base stations covering the area. In response to the base station receiving the connection initialization request sent by the mobile terminal, the base station calculates various indicator parameters based on the connection initialization request. The mobile terminal receives various indicator parameters sent by each of the base stations.” This implies that the mobile terminal includes a sending module for sending the connection initialization request and a receiving module for receiving the indicator parameters.)
The remaining limitations of claim 30 are taught by the combination of Qin and Chai using the same rationale applied to claim 27 above, mutatis mutandis.
Claims 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Qin in view of Chai as applied to claim 20 above, and further in view of Zhan et al. (the non-patent article “Twin Delayed Multi-Agent Deep Deterministic Policy Gradient”), hereinafter referred to as Zhan.
Regarding claim 21:
The combination of Qin and Chai teaches “The computer-implemented method of claim 20,” but does not specifically teach the limitations listed below. However, these limitations are taught by Zhan:
“wherein: the evaluation network comprises a first evaluation model and a second evaluation model.” (Zhan p. 50 § C: “The model consists of multiple DDPG networks, each of which learns policy π (actor) and action value Q (critic), and has a target network for off-policy learning of Q-learning.”)
“and the updating the internal weights of the evaluation network comprises: comparing a first evaluation value by the first evaluation model and a second evaluation value by the second evaluation value to obtain a minimum evaluation value, calculating an error between the minimum evaluation value and a target evaluation value.” (Zhan p. 50 -§ E and FIG. 2 reproduced below: “With reference to the TD3 algorithm, the basic idea of this article is to use two sets of networks to represent different Q values, and by selecting the smallest one as our updated target (target Q value), the continuous overestimation is suppressed. The target network Q1(a’) and Q2(a’) in the above the Figure 2. take the minimum value min(Q1,Q2), instead of MADDPG’s Q’(a’) to calculate the update target.”)
PNG
media_image3.png
344
689
media_image3.png
Greyscale
“and updating the internal weights of the first evaluation model and the second evaluation model based on the calculated error using differential learning.” (Zhan p. 51 last paragraph: “Actor adjusts the strategy (actor neural network parameters) according to the critic’s score and tries to do better next time. The critic adjusts the score strategy (critic neural network parameters) based on the compensation provided by the system and the scores of other judges.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method disclosed by the combination of Qin and Chai by adjusting the decision network parameters based on a comparison of two evaluation models as taught by Zhan with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Zhan Abstract teaches that this “improves the actor network and critic network, optimizes the overestimation of Q value and adopts the update delayed method to make the actor training more stable.”
Regarding claim 22:
The combination of Qin and Chai teaches “The computer-implemented method of claim 20,” but does not explicitly teach “wherein the evaluation network is implemented using a multi-agent twin delayed deep deterministic policy gradient algorithm.” However, Zhan does teach this limitation. (Zhan proposes a “Twin Delayed Multi-Agent Deep Deterministic Policy Gradient,” where Zhan Abstract discloses that “Based on the traditional multi-agent reinforcement learning algorithm, this paper improves the actor network and critic network, optimizes the overestimation of Q value and adopts the update delayed method to make the actor training more stable.”)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method disclosed by the combination of Qin and Chai by using a twin delayed multi-agent deep deterministic policy gradient as taught by Zhan with a reasonable expectation of success. A person having ordinary skill in the art could have been motivated to do this because Zhan Abstract teaches that this “improves the actor network and critic network, optimizes the overestimation of Q value and adopts the update delayed method to make the actor training more stable.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kwon (US 2022/0150718 A1) ¶¶ 53 and 85 disclose a system and method in which “each of the plurality of mobile base stations 310-1 and 310-2 may be equipped with an aerial vehicle 311-1 or 311-2, flying base station 312-1 or 312-2, or mobile backhaul terminal 313-1 or 313-2,” and wherein “when identified positions of the terminals 1020-1 to 1020-3 are in an area where the first cell coverage 1014-1 and the second cell coverage 1014-2 overlap each other, the terminals 1020-1 to 1020-3 may have a movement speed of 3 kilometers or less per hour, and the second mobile base station 1010-2 may be selected as a mobile base station to be accessed.”
Wang et al. (the non-patent article “Online UAV-Mounted Edge Server Dispatching for Mobile-to-Mobile Edge Computing”) Abstract discloses that “a novel online unmanned aerial vehicle (UAV)-mounted edge server dispatching scheme is proposed to provide flexible mobile-to-MEC services. UAVs are dispatched to the appropriate hover locations by geographically merging tasks into several hot-spot areas.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Madison R Inserra whose telephone number is (571)272-7205. The examiner can normally be reached Monday - Friday: 9:30 AM - 6:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at 571-270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Madison R. Inserra/Primary Examiner, Art Unit 3662