DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 11-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20210119881 A1 (SHIRAZIPOUR et al.)(hereinafter SHIRAZIPOUR) in view of US 20210258988 A1 (Balakrishnan et al.)(hereinafter Balakrishnan).
In re claim 11, SHIRAZIPOUR discloses a method for managing radio resources in a cellular network comprising a plurality of nodes (Fig. 2, [0018], “According to a second aspect of embodiments herein, the object is achieved by a method, performed by a third network node. The method is for handling the performance of the radio access network. The radio access network comprises the one or more radio network nodes. The first network node operates in the communications network”. [0050], “The wireless network 100 may comprise and/or interface with any type of communication, telecommunication, data, cellular, and/or radio network or other similar type of system”), wherein for each node of interest of the network, a neighborhood of this node of interest is determined, each node of said neighborhood performing a local observation of its environment and by extracting a plurality of radio characteristics ([0018], “The radio access network comprises the one or more radio network nodes. The first network node operates in the communications network. The third network node obtains data collected from a respective subset of the radio network nodes. The respective subset covers a part of a deployment of the radio access network having a first subset of at least one of: i) the one or more physical characteristics of the deployment of the radio access network, ii) the one or more radio characteristics of the radio access network, and iii) the location of users or the traffic load. The data indicates one or more parameters in the radio access network. The third network node performs a machine-implemented reinforcement learning procedure, based on the obtained data, to optimize the performance of the radio access network based on the one or more parameters”), the method comprising: each node of said neighborhood encoding each of the radio characteristics in the form of a message and transmits this message to the node of interest ([0018], “The third network node then sends an indication of an outcome of the performed machine-implemented reinforcement learning procedure to the first network node operating in the communications network”); the node of interest generating a local mapping of each radio characteristic by aggregating the messages encoding this characteristic (Fig. 11, [0104], “The one or more physical characteristics of the deployment of the radio access network 310 may be, for example a map of the city and buildings in the region of interest, multiple geographical segments of the deployment of the radio access network 310, e.g., grid, hexagon, etc., building structure information of the corresponding geographical segment, and/or the sites where an antenna is deployed”. [0127], “For example, throughput may be a valid objective for the simulation phase. In case of network planning, the objective may be to optimize overall throughput at termination of the simulation. For adaptive optimization, the throughput may be optimized for the sum of throughput over time”. [0139], “The indication may be, e.g., a message comprising one or more indicators of: the probability distribution of discrete options, e.g., increase/decrease of the parameters by a fixed amount, termination of the episode, discrete parameters, etc.”); the node of interest fusing the local mappings by means of fusion parameters to generate a hybrid local mapping of the different radio characteristics; the node of interest deciding at all times to perform an action amongst a finite set of possible actions, based on said hybrid local mapping and on a radio resource management strategy defined by a conditional probability parameterized distribution of each action, the set of fusion parameters as well as the set of the parameters of the conditional probability distribution undergoing a reinforcement learning so as to maximise a reward over time, dependent on an objective function of the network ([0139], “The indication may be, e.g., a message comprising one or more indicators of: the probability distribution of discrete options, e.g., increase/decrease of the parameters by a fixed amount...”).
SHIRAZIPOUR does not explicitly disclose the node of interest fusing the local mappings by means of fusion parameters to generate a hybrid local mapping of the different radio characteristics; the node of interest deciding at all times to perform an action amongst a finite set of possible actions, based on said hybrid local mapping and on a radio resource management strategy defined by a conditional probability parameterized distribution of each action, the set of fusion parameters as well as the set of the parameters of the conditional probability distribution undergoing a reinforcement learning so as to maximise a reward over time, dependent on an objective function of the network.
Balakrishnan discloses the node of interest fusing the local mappings by means of fusion parameters to generate a hybrid local mapping of the different radio characteristics (Fig. 12, [0071], “The encoder may include a combination of one or more of error detecting, error correcting, rate matching, and interleaving. The encoder may further include a step of scrambling. In an aspect, encoded data may be input to a modulation mapper to generate complex valued modulation symbols. The modulation mapper may map groups containing one or more binary digits, selected from the encoded data, to complex valued modulation symbols according to one or more mapping tables”. [0105], “One aspect to enable dynamic and intelligent sharing of the radio spectrum is for the networks to collaborate effectively with each other. To collaborate effectively, CIRN nodes (also referred to as agents) and networks may use a machine-learning algorithm to learn how their transmissions impact the transmissions of other CIRNs via training...”); the node of interest deciding at all times to perform an action amongst a finite set of possible actions, based on said hybrid local mapping and on a radio resource management strategy defined by a conditional probability parameterized distribution of each action ([0147], “Where R=(noRelay, amplify, decode) is the set of relaying modes supported by x, P(s,r) is the probability of the transmission being successful using relaying more r, and Utility(.Math.) is the utility function that measures the benefit of the relaying action r”. [0171], “The knowledge of the interference levels to node n may be maintained using a belief state Bn (taking values in [0,1]). The belief state Bn may indicate the conditional probability that the CIRN node interferes with node n, given the decision and observation history. The observation history may be obtained in operation 1002 over the past several sensing attempts”. [0172], “The collaborative intelligent radio network (CIRN) node can perform different control actions (as indicated below) such as reducing power levels, adding or adjusting a silence period or a backoff period, etc....Therefore, the action that maximizes the reward across all N nodes is the action that is performed by the CRIN node”), the set of fusion parameters as well as the set of the parameters of the conditional probability distribution undergoing a reinforcement learning so as to maximise a reward over time, dependent on an objective function of the network ([0172], “Therefore, the action that maximizes the reward across all N nodes is the action that is performed by the CIRN node. In the reinforcement learning approach, the reward is typically accumulated over T time slots, with each slot corresponding to a discrete time step where an action is taken. In some circumstances, however, there may be greedy index-based policies for reinforcement learning that can aim to maximize the immediate reward (i.e., over a particular time slot as opposed to an accumulation). This may increase the difficulty in exploring some of the state spaces”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of SHIRAZIPOUR with Balakrishnan to provide a system and method for managing radio resources including one or more radio nodes using machine-implemented reinforcement learning (RL) procedures to optimize the performance of the RAN based on the one or more parameters. The advantage of doing so is to efficiently use the network resources and bandwidth with the explosion of network usage.
In re claim 12, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein Balakrishnan discloses wherein the reward to be maximized corresponds to a sum of bitrates or quality-of-service levels over communications of the network to be maximized, or a handover frequency or an energy consumption to be minimized ([0135], “The schedulers may be used to one or more of: avoid interference implement listen-before-talk), take into account the different link capacities (be capacity-aware), maximize throughput in the time and/or frequency domains (dependent on the CIRN or neighboring network), or optimize rate selection in heterogeneous networks environments, among many others that take into account quality of service of the transmissions, traffic load of the different networks, etc.”. [0166], “For example, the CIRN node may observe the energy per band to observe in which bands a sudden increase in received signal energy follow a probe transmission. This may permit the CIRN node to maintain a per-band database of the interference caused by the CIRN node to other nodes. The CIRN node may monitor different frequency bands to identify the potential interference before transmission. In addition, if the CIRN node is a MIMO node, the CIRN node may monitor different frequency channels in different directions for interference”. [0182], “After determining and training the model at operation 1004, the interference caused by the CIRN node to nodes in the neighboring network can be detected. Interference mitigation/avoidance methods can then be utilized by the CIRN node at operation 1006. For example, the model may reduce the transmission power, engage frequency hopping, and/or delay transmissions through the use of a backoff procedure, which may be analogous to the WiFi backoff procedure, to reduce the possibility of collisions” (optimize for QOS, interference and energy consumption)).
In re claim 13, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein SHIRAZIPOUR discloses wherein said neighborhood of the node of interest is defined as a set of neighboring nodes of the network considering a similarity metric operating in a representation space of the local observations (Fig. 6, [0012], “Expressed more formally, a reinforcement learning agent may receive an observation from the environment in a state S and may select an action to maximize the expected future reward r. An environment may be understood as an element the agent may interact with, e.g., a network”. [0084], “In some examples of the communications network 300, which are not depicted in FIG. 3, any of the first network node 311, the second network node 312, and the one or more third network nodes 313 may be co-located, or be a same node. In some examples, all of the first network node 311, the second network node 312, and the one or more third network nodes 313 may be co-localized, or be the same node”. [0140], “By sending the indication to the first network node 311 in this Action 404, the third network node 313 may enable the first network node 311 to optimize the performance of the radio access network 310 based on the one or more parameters. The first network node 311 may gather similar indications, that is, respective indications, from, respectively, the other one or more third network nodes 313 in the communications network 300, and be therefore enabled to optimize the performance, e.g., network planning and/or optimization of network operation, of the radio access network 310, with higher confidence”. [0181], “For both supervised and unsupervised learning, the trained model can be downloaded on a local inference engine of the CIRN node or network. The local inference engine may be, for example, a Movidius neural-network processor. By downloading the model to the local inference engine, rapid inference and detection and classification of interference level caused by each CIRN node may be obtained” (neighboring nodes in space with similarity metric for local observation)).
In re claim 14, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein Balakrishnan discloses the said neighborhood of the node of interest is determined by means of a classifier trained beforehand, operating in a representation space of the local observations ([0120], “The CIRN node (potential relay node) that overhears all of the packets may apply a classifier to each of the packets. A clustering algorithm may use the classifier to categorize the packets into the different classes; for example, the clustering algorithm used on the headers/preamble may classify all the packets (labeled 1) into one category”. [0123], “After determining the source and destination nodes, at operation 906, the destination ID for a particular packet may be identified for a given data packet. The ACKs/NACKs, length, and/or different transforms of the PHY layer signal (Wavelet, FFT etc.) may be used at this point to train the classifier”. [0133], “Alternatively, the relay node may use a machine-learning classifier to identify the modulation order of the data packet” (classifier trained beforehand to categorize nodes of similar metric for local observations)).
In re claim 15, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein SHIRAZIPOUR discloses the node of interest deciding to perform an action at a time point only to the extent that one of the local mappings of a radio characteristic at this time point differs from the local mapping of the same radio characteristic at the previous time point, the difference between the two local mappings being measured using a Kullback-Leibler divergence ([0079], “the resource mapped symbols may be input to multicarrier generator which generates a time domain baseband symbol (equation)”. [0125], “A Deep Policy Network may be understood as a neural network which outputs a probability distribution of actions. As depicted in FIG. 5, the third network node 313 may interact with the simulated environment in discrete time steps t. At each time step t, the simulated environment is in a state (st_t) and sends an observation of this state, along with the current reward r (r_t) to the third network node 313. Then, the third network node 313 may choose any action (a_t) that may be available in that state. Then, the simulated environment may respond at the next time step (t+1) by moving into a new state (s_t+1), and giving the third network node 313 a corresponding reward” (perform action when moving to a new state at time t+1)). Balakrishnan also discloses ([0161], “As above, predetermined times may be used to transmit the interference levels, along with the ID of the node transmitting the information, by observing the interference levels reported at different times, the CIRN node can correlate and determine the relationship of its own transmission with the interference at other nodes in the neighbor network. Hence, each CIRN node may be able to determine an interference graph containing itself and the edges (in terms of physical distance or other nodes) to which the CIRN node is likely to cause interference”).
In re claim 16, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein SHIRAZIPOUR discloses the node of interest deciding to perform an action at a time point only to the extent that the hybrid local mapping at this time point differs from the hybrid local mapping at the previous time point, the difference between the two hybrid mappings being measured using a Kullback-Leibler divergence (See “In re claim 15”. All features discloses by claim 15. The difference is in hybrid local mapping wherein the node of interest performs action for a new state at a different time)).
In re claim 17, the combination discloses the method for managing radio resources in a cellular network according to claim 11, wherein Balakrishnan discloses the fusion of the local radio mappings using an attention mechanism with H heads, with H < K where K is the number of radio characteristics ([0162], “In some cases, it may be desirable to minimize the amount of overhead on the collaboration channel to prevent overuse of the collaboration channel. One way to minimize the overhead due to the amount of information exchanged on the collaboration channel is to restrict nodes to transmit only the interference information corresponding to a delay-sensitive transmission, such as a Voice-over-IP transmission. Alternatively, upon each re-transmission or after a predetermined number of re-transmissions, the node may set a congestion flag to warn all other active nodes in the network”. [0165], “The nodes may in addition or instead of information on the collaboration channel use a-priori knowledge. As above, the use of a-priori knowledge may assume that the nodes can differentiate between their own and another node's original transmission and retransmission. The ability to fully decode the packets means that the nodes may be able to directly infer the interference impact by identifying the affected transmissions of other nodes”).
Allowable Subject Matter
Claims 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SWATI JAIN whose telephone number is (571)270-0699. The examiner can normally be reached Mon - Fri (830 am - 530 pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pan Yuwen can be reached on 571-272-7855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SWATI JAIN/Examiner, Art Unit 2649 /YUWEN PAN/Supervisory Patent Examiner, Art Unit 2649