DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/07/2022 and 10/20/2022 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-11 and 14-24 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “optimized” in claims 1-3, 9-10, 14-16 and 22-23 is a relative term which renders the claim indefinite. The term “optimized” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Dependent claims 2-11 is rejected for being dependency of claim 1.
Dependent claims 15-24 is rejected for being dependency of claim 14.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 and 14-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea and does not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Regarding claim 1
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“….provide action values as output, the state information being information indicative of a state relating to at least the communication channel.”
the action values being associated with intermediate actions, respectively, of a finite set of actions belonging to a discrete action space”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “the second neural network being trained to transform action values associated with said intermediate actions, respectively, to a corresponding optimized action belonging to said continuous action space.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. In addition, the additional elements of “a method, performed by one or more devices, for supporting provision of an optimized action, belonging to a continuous action space, for application in a wireless communication network to affect data transmission over a communication channel of the wireless communication network,… wherein the method comprises: obtaining a third neural network that is based on a combination of a trained first neural network and a trained second neural network that form a respective part of the third neural network and where output of the trained first neural network is used as input to the trained second neural network, the first neural network being trained to, based on state information as input,” as explained by the Supreme Court, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional. See MPEP 2106.05(g). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The only remaining limitation of the claim “wherein the optimized action determines a precoder to be used by a multi-antenna transmitter configured to transmit data over the communication channel said intermediate actions mapping to precoders, respectively.;” constitute storing and retrieving information in memory, which the courts have found to be well-understood, routine, and conventional. See MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. With regards to the claim limitation “wherein the method comprises: obtaining a third neural network that is based on a combination of a trained first neural network and a trained second neural network that form a respective part of the third neural network and where output of the trained first neural network is used as input to the trained second neural network, the first neural network being trained to, based on state information as input,” Snoussi teaches ensemble learning on pg. 4 left col “we propose a simpler scheme adapted to the target tracking application and efficiently implemented by the ensemble learning approach. In some sensor network target tracking applications, the node measurement is inversely proportional to the distance between the node position and the target position, in a limited sensing range. Before collecting data Yt, the best node at time t is thus the closest node to the predicted position of the target. In order to take into account all the statistical behavior of the predicted target position, the relevance of the node's data” also see section 4 “In order to illustrate the effectiveness of the proposed ensemble learning target tracking in a wireless sensor network, we have considered the tracking of simulated sinusoidal trajectory in a 2-dimensional field (figure 3). A set of 200 nodes are randomly deployed in lOOm x lOOm square area Each node has a sensing range set to 20m. At each time t, within this range the leader node obtains an observation of the target position through a range-bearing model:”)
Regarding claim 2
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the method further comprises: providing said optimized action based on output of the trained second neural network part of the third neural network while operating the third neural network in an environment of the wireless communication network with state information as input to the first neural network part of the third neural network.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 3
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the method further comprises: applying the provided optimized action in the wireless communication network to affect said data transmission over the communication channel”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 4
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the first and second neural networks have been separately trained.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 5
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the first neural network has been trained by means of reinforcement learning in the wireless communication network”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 6
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the first neural network (1503a) has been trained in said environment of the wireless communication network”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 7
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the reinforcement learning is based on a Deep Q Network, DQN, reinforcement learning algorithm and said first neural network (1503a) corresponds to a DQN.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 8
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the second neural network has been trained using a training data set with action values associated with same intermediate actions as used during training of the first neural network”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 9
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the action values in said training set map to predefined optimized actions, respectively.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 10
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements of “wherein the optimized action determines a precoder to be used by a multi-antenna transmitter configured to transmit data over the communication channel said intermediate actions mapping to precoders, respectively.;” as explained by the Supreme Court, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional. See MPEP 2106.05(g). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The only remaining limitation of the claim “wherein the optimized action determines a precoder to be used by a multi-antenna transmitter configured to transmit data over the communication channel said intermediate actions mapping to precoders, respectively.;” constitute storing and retrieving information in memory, which the courts have found to be well-understood, routine, and conventional. See MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 11
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements of “wherein obtaining the third neural network comprises obtaining the trained first neural network, obtaining the trained second neural network and providing the third neural network;” as explained by the Supreme Court, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional. See MPEP 2106.05(g). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The only remaining limitation of the claim “wherein obtaining the third neural network comprises obtaining the trained first neural network, obtaining the trained second neural network and providing the third neural network;” constitute storing and retrieving information in memory, which the courts have found to be well-understood, routine, and conventional. See MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 14-24
Claims 14-24 recites analogous limitations to claims 1-11 and therefore is rejected on the same ground as claims 1-11.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-11 and 14-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan et al. (US 2019/0014488 A1) in view of Snoussi et al. (“Ensemble Learning Online Filtering in Wireless Sensor Networks”).
Regarding claim 1 (Currently Amended)
Tan teaches a method, performed by one or more devices, for supporting provision of an optimized action, belonging to a continuous action space, for application in a wireless communication network to affect data transmission over a communication channel of the wireless communication network, (para [0066] “A state tensor of a cell may be constructed using the information described above in various structures. In some embodiments, a state tensor may be constructed as a collection of one or more state planes. Each of the state planes may indicate or include information about the cell that is obtained from one or more channels. In some embodiments, a state plane may include a two - dimensional array having a horizontal axis and a vertical axis for storing information of the cell. When constructing a state plane, in one embodiment, the cell may be first placed at the center of the horizontal axis”)
…
the action values being associated with intermediate actions, respectively, of a finite set of actions belonging to a discrete action space, (para [0047] “FIG. 5 illustrates a flowchart of an embodiment method 500 for wireless network optimization using deep reinforcement learning (DR). At step 502, the method 500 trains a deep learning network using a DRL technique for adjusting one or more parameters of a plurality of cells in a wireless network. In some embodiments, the method 500 may receive state information of the plurality of cells, select actions using the deep learning network for adjusting one or more of the plurality of cells, calculate reward values for the actions selected and taken, and update weight values of the deep learning network”)
the second neural network being trained to transform action values associated with said intermediate actions, respectively, to a corresponding optimized action belonging to said continuous action space. (Para [0038] “Various optimization techniques have been used to optimize wireless network. For example, a virtual simulation or model of a cellular network can be constructed to enable variation and optimization of network parameters in a virtual environment. In another example, network parameters are optimized iteratively by making small step adjustments and gathering real - world feedback on the effects of those adjustments on a real network until an optimal adjustment action is found”)
Tan does not teach wherein the method comprises: obtaining a third neural network that is based on a combination of a trained first neural network and a trained second neural network that form a respective part of the third neural network and where output of the trained first neural network is used as input to the trained second neural network, the first neural network being trained to, based on state information as input, provide action values as output, the state information being information indicative of a state relating to at least the communication channel.
Snoussi teaches wherein the method comprises: obtaining a third neural network that is based on a combination of a trained first neural network and a trained second neural network that form a respective part of the third neural network and where output of the trained first neural network is used as input to the trained second neural network, the first neural network being trained to, based on state information as input, provide action values as output, the state information being information indicative of a state relating to at least the communication channel. (Pg. 4 left col “we propose a simpler scheme adapted to the target tracking application and efficiently implemented by the ensemble learning approach. In some sensor network target tracking applications, the node measurement is inversely proportional to the distance between the node position and the target position, in a limited sensing range. Before collecting data Yt, the best node at time t is thus the closest node to the predicted position of the target. In order to take into account all the statistical behavior of the predicted target position, the relevance of the node's data” also see section 4 “In order to illustrate the effectiveness of the proposed ensemble learning target tracking in a wireless sensor network, we have considered the tracking of simulated sinusoidal trajectory in a 2-dimensional field (figure 3). A set of 200 nodes are randomly deployed in lOOm x lOOm square area Each node has a sensing range set to 20m. At each time t, within this range the leader node obtains an observation of the target position through a range-bearing model:”)
Tan and Snoussi are analogous art because they are both directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for deep learning and wireless network disclosed by Tan to include ensemble learning in wireless sensor network of Snoussi in order to provide a method or system that parameterization-independent ensuring the robustness of the data processing as disclosed by Snoussi (abstract “In this contribution, we propose an alternative ensemble learning (variational) approximation suitable to the communication constraints of sensor networks. The efficiency of the variational approximation relies on the fact that the online update of the filtering distribution and its compression are simultaneously performed. In addition, the variational approach has the nice property to be parameterization-independent ensuring the robustness of the data processing. The selection of the leader node is based on a trade-off between communication constraints and information content relevance of measured data.”).
Regarding claim 2 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Snoussi further teaches wherein the method further comprises: providing said optimized action based on output of the trained second neural network part of the third neural network while operating the third neural network in an environment of the wireless communication network with state information as input to the first neural network part of the third neural network. (Pg. 4 left col “we propose a simpler scheme adapted to the target tracking application and efficiently implemented by the ensemble learning approach. In some sensor network target tracking applications, the node measurement is inversely proportional to the distance between the node position and the target position, in a limited sensing range. Before collecting data Yt, the best node at time t is thus the closest node to the predicted position of the target. In order to take into account all the statistical behavior of the predicted target position, the relevance of the node's data” also see section 4 “In order to illustrate the effectiveness of the proposed ensemble learning target tracking in a wireless sensor network, we have considered the tracking of simulated sinusoidal trajectory in a 2-dimensional field (figure 3). A set of 200 nodes are randomly deployed in lOOm x lOOm square area Each node has a sensing range set to 20m. At each time t, within this range the leader node obtains an observation of the target position through a range-bearing model:”)
Regarding claim 3 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 2.
Tan further teaches wherein the method further comprises: applying the provided optimized action in the wireless communication network (Para [0062] “The method 1000 may select the plurality of cells according various criteria or rules. In this example, cells in the wireless network are divided into groups, e.g., based on geographical locations or other rules, and cells are selected for optimization based on the groups. As shown, at step 1002, the method 1000 may first select a subset of groups of cells to be optimized in the wireless network. The method 1000 may also select the entirety of the groups of cells for optimization. In some embodiments, the method 1000 may select the subset of groups sequentially so that one group is selected at a time.”)
Regarding claim 4 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Snoussi further teaches wherein the first and second neural networks have been separately trained. (Examiner notes that ensemble learning include training first and second neural network model trained separately and Snoussi teaches ensemble learning see para [0062] “The method 1000 may select the plurality of cells according various criteria or rules. In this example, cells in the wireless network are divided into groups, e.g., based on geographical locations or other rules, and cells are selected for optimization based on the groups. As shown, at step 1002, the method 1000 may first select a subset of groups of cells to be optimized in the wireless network. The method 1000 may also select the entirety of the groups of cells for optimization. In some embodiments, the method 1000 may select the subset of groups sequentially so that one group is selected at a time.”)
Regarding claim 5 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Tan further teaches wherein the first neural network has been trained by means of reinforcement learning in the wireless communication network. (Abstract “A neural network is trained using deep reinforcement learning (DRL) techniques for adjusting cell parameters of a wireless network by generating a plurality of experience tuples, and updating the neural network based on the generated experience tuples. The trained neural network may be used to select actions to adjust the cell parameters”)
Regarding claim 6 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 5.
Tan further teaches wherein the first neural network has been trained in said environment of the wireless communication network. (Para [0048] “Conventional DRL techniques typically require a large amount of experience tuples (e.g., more than 100 M experience tuples) to produce a satisfactorily trained neural network. Generating such large amount of experience tuples is generally very time-consuming for wireless communications systems, as a state of a wireless network is determined based on states of cells in the network, which in turn are based on various types of information, such as measurement reports (MRs) and key performance indicators (KPIs).”)
Regarding claim 10 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Tan further teaches wherein the optimized action determines a precoder to be used by a multi-antenna transmitter configured to transmit data over the communication channel, said intermediate actions mapping to precoders, respectively. (Para [0103] “A state of a cell comprises a setting of a base station associated with the cell, the base station providing a coverage area of the cell. A state tensor may also include information of inter-site distance (ISD), a height of a base station, an antenna azimuth, an antenna mechanical tilt (mTilt), an antenna electronic tilt (eTilt), a key performance indicator, reference signal received power (RSRP), reference signal received quality (RSRP), signal interference to noise ratio (SINR), channel quality indicator (CQI)), an objective function, a cumulative distribution function of network performance measurements, or an interference factor matrix. At step 1508, the DRL process selects an action for each of the plurality of cells. An action moves the respective cell from one state to another state. An action includes information for adjusting a setting of a base station associated with the cell.”)
Regarding claim 11 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Snoussi further teaches wherein obtaining the third neural network comprises obtaining the trained first neural network, obtaining the trained second neural network and providing the third neural network. (Examiner notes that ensemble learning include training first and second neural network model trained separately and Snoussi teaches ensemble learning see para [0062] “The method 1000 may select the plurality of cells according various criteria or rules. In this example, cells in the wireless network are divided into groups, e.g., based on geographical locations or other rules, and cells are selected for optimization based on the groups. As shown, at step 1002, the method 1000 may first select a subset of groups of cells to be optimized in the wireless network. The method 1000 may also select the entirety of the groups of cells for optimization. In some embodiments, the method 1000 may select the subset of groups sequentially so that one group is selected at a time.”)
Regarding claim 14-24
Claims 14-24 recites analogous limitations to claims 1-11 and therefore is rejected on the same ground as claims 1-11.
Claim(s) 7-9 and 20-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan et al. (US 2019/0014488 A1) in view of Snoussi et al. (“Ensemble Learning Online Filtering in Wireless Sensor Networks”) and further in view of Wang et al. (“Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks”).
Regarding claim 7 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 5.
Tan in view of Snoussi does not teach wherein the reinforcement learning is based on a Deep Q Network, DQN, reinforcement learning algorithm and said first neural network corresponds to a DQN.
Wang teaches wherein the reinforcement learning is based on a Deep Q Network, DQN, reinforcement learning algorithm and said first neural network corresponds to a DQN. (Abstract “To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics.”)
Tan, Wang and Snoussi are analogous art because they are all directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for deep learning and wireless network disclosed by Tan in view of Snoussi to include deep reinforcement leaning for dynamic multichannel using deep Q-network of Wang in order to provide “fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics” as disclosed by Wang (abstract “To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics.”).
Regarding claim 8 (Currently Amended)
Tan in view of Snoussi teaches the method as claimed in claim 1.
Tan in view of Snoussi does not teach wherein the second neural network has been trained using a training data set with action values associated with same intermediate actions as used during training of the first neural network.
Wang teaches wherein the second neural network has been trained using a training data set with action values associated with same intermediate actions as used during training of the first neural network.
Tan, Snoussi and Wang are analogous art because they are all directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for deep learning and wireless network disclosed by Tan in view of Snoussi to include deep reinforcement leaning for dynamic multichannel using deep Q-network of Wang in order to provide “fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics” as disclosed by Wang (abstract “To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics.”).
Regarding claim 9 (Original)
Tan in view of Snoussi with Wang teaches the method as claimed in claim 8.
Tan further teaches wherein the action values in said training set map to predefined optimized actions, respectively. (Para [0062] “In this example, cells in the wireless network are divided into groups, e.g., based on geographical locations or other rules, and cells are selected for optimization based on the groups. As shown, at step 1002, the method 1000 may first select a subset of groups of cells to be optimized in the wireless network. The method 1000 may also select the entirety of the groups of cells for optimization. In some embodiments, the method 1000 may select the subset of groups sequentially so that one group is selected at a time.”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN C MANG whose telephone number is (571)270-7598. The examiner can normally be reached Mon - Fri 8:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at 5712707519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN C MANG/Primary Examiner, Art Unit 2126