DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-9 are is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Melodia et al. (US 2022/0167236 A1), hereinafter referred to as D1.
Regarding claim 1, D1 discloses intelligent and learning in O-RAN for 5G and 6G cellular networks, which comprises:
a communication network having a cloud portion and an edge portion, the communication network comprising a cloud controller, a radio access network (RAN), and an edge computing device, wherein the edge computing device is co-located with at least a portion of the RAN and is configured to, within one transmission time interval (TTI) of the RAN (Note, the Examiner interprets the claim limitations as the prior art’s teaching of an O-RAN, which comprises a virtualized O-RAN with edge devices that perform near-real-time and sub-TTI control loops that collect network state information, determine network control actions, and control the actions of the O=RAN. Referring to Figures 21B, 22, and 23, open and programmable cellular architectures will heavily rely on three main ingredients: disaggregation of network elements and virtualization of their functionalities, open interfaces, and the integration of closed-control loops. Specifically, virtualization abstracts the software from the underlying hardware (cloud portion), enabling shared network operating systems running on top of a general-purpose infrastructure (L. Bonati et al.,“CellOS: Zero-touch Softwarized Open Cellular Networks,” Computer Networks, vol. 180, pp. 1-13, October 2020). In this way, the once monolithic base stations can be split into multiple virtual RAN functions, which can run in datacenters at the edge of the network (edge portion, co-located with at least a portion of the RAN), or be co-located with the radio unit deployed on the ground. See paragraphs 0240-0249. Near-real-time control loops operate on a timescale between 10 ms and 1 s (configured to within one TTI of the RAN). As shown in FIG. 22, they run between the near-real-time RIC and two components of the gNBs: the CU 2272 and the DU 2274. Because one near-real-time RIC is associated to multiple gNBs, these control loops can make decisions affecting hundreds or thousands of UEs leveraging user-session aggregated data and Medium Access Control (MAC)/physical layer Key Performance Indicators (KPIs). Machine learning-based methods are implemented as external applications, i.e., xApps, and are deployed on the near-real-time RIC to deliver specific services such as inference, classification, and prediction pipelines to optimize the per-user quality of experience, controlling load balancing and handover processes, or the resource scheduling and beamforming design. See paragraph 0254-0257. O-RAN with RIC to perform scheduling control via edge devices. See paragraphs 0258-0263.):
obtain network state information of the communication network (Referring to Figures 21B, 22, and 23, data-driven optimization outperforms fixed policies by delivering higher spectral efficiency, with gains up to 20%. This is due to the fact that eMBB traffic requires high data-rates and DRL agents are capable of dynamically adapting scheduling decisions to the current network state and traffic demand. See paragraphs 0267-0268.);
determine control actions for modifying operational settings of the RAN; and transmit the control actions to the RAN (Referring to Figures 21B, 22, and 23, determine the best scheduling policy for the corresponding slice. The reward of the agents depends on the specific slice and the corresponding KPI requirements. Specifically, eMBB and MTC DRL agents have been trained to maximize the throughput of UEs, while URLLC agents have been trained to maximize the number of resources (i.e., PRBs) allocated to each UE and to satisfy their latency requirements. To fully comply with O-RAN directives, the DRL agents were trained offline, and their effectiveness then tested on Colosseum. See paragraphs 0266-0268.)
Regarding claim 2, D1 discloses wherein the network state information includes wireless state information of the communication network and RAN state information of the RAN (Referring to Figures 21B, 22, and 23, determine the best scheduling policy for the corresponding slice. The reward of the agents depends on the specific slice and the corresponding KPI requirements. Specifically, eMBB and MTC DRL agents have been trained to maximize the throughput of UEs, while URLLC agents have been trained to maximize the number of resources (i.e., PRBs) allocated to each UE and to satisfy their latency requirements (wireless state information of the network). To fully comply with O-RAN directives, the DRL agents were trained offline, and their effectiveness then tested on Colosseum. To train the DRL agents, approximately 6 GB of training data was generated, containing various performance metrics (e.g., throughput, bit error rate), system state information (e.g., transmission queue size, signal-to-interference-plus-noise ratio, channel quality information) (RAN state information of the RAN) and resource allocation strategies (e.g., slicing policies, scheduling) by running a total of 63 hours of experiments on Colosseum. See paragraphs 0266-0268.)
Regarding claim 3, D1 discloses wherein the network state information includes applications state information of an application executing on a user equipment communicatively coupled to the RAN (Referring to Figures 21B, 22, and 23, autonomous driving applications require Ultra Reliable and Low Latency Communications (URLLC) to allow vehicles to promptly react to traffic conditions. On the other hand, high-quality multimedia content requires high data rates, but can tolerate a higher packet loss and latency. Therefore, future generations of cellular networks need to be flexible and adaptive to many different application requirements. data-driven real-time control loops between the base stations and the xApps. The RAN supports network slicing with 3 slices for different QoS: (i) eMBB, representing users requesting video traffic; (ii) MTC for sensing applications, and (iii) URLLC for latency-constrained applications. For each slice, the base stations can adopt 3 different scheduling policies independently of that of the other slices, namely, the Round Robin (RR), the Waterfilling (WF), and the Proportional Fair (PF) scheduling policies. See paragraphs 0276-027 and 0308.)
Regarding claim 4, D1 discloses wherein the wireless state information comprises a Channel Quality Indicator (CQI) and the RAN state information comprises a value of backlogged bytes remaining in a downlink queue of the RAN (Referring to Figures 6, 21B, 22, and 23, SCOPE allows users to collect large amounts of data (e.g., instantaneous through-put, transmission queue status (RAN state information comprising a value of backlogged bytes remaining in a downlink queue of the RAN), Channel Quality Information (CQI), number of PRBs allocated to each network slice, to name a few for non-limiting example). Additionally, SCOPE allows users to automatically generate datasets that can be leveraged to train machine learning and artificial intelligence models, or to design novel optimization and heuristic solutions for cellular applications. See paragraphs 0146-0148.)
Regarding claim 5, D1 discloses wherein the edge computing device determines the control actions by applying a reinforcement learning policy to the network state information (Referring to Figures 21B, 22, and 23, SCOPE stops the scenario execution and all the results collected during the experiment—which are stored in the metrics and performance dataset—are transferred to the user directory on Colosseum. This makes it possible to process results and save the dataset for future applications (e.g., to train machine learning methods). See paragraphs 0151-0152. O-RAN with RIC to perform scheduling control via edge devices according to the machine learning based on network information. See paragraphs 0258-0263.)
Regarding claim 6, D1 discloses wherein the edge computing device is implemented by first compute resources of a computing device which implements at least a portion of a Distributed Unit of the RAN via second compute resources of the computing device (Referring to Figures 21B, 22, and 23, Colosseum is equipped with an edge datacenter, with 900 TB of storage and the capability of processing RF data at a rate of 52 TB/s, putting it in a privileged position to collect data and test ML algorithms on heterogeneous networks and devices. See paragraph 0261. As shown in FIG. 22, the near-real-time RIC and two components of the gNBs: the CU 2272 and the DU 2274 (O-RAN interpreted as comprising a DU of the RAN via resources of the O-RAN). Because one near-real-time RIC is associated to multiple gNBs, these control loops can make decisions affecting hundreds or thousands of UEs leveraging user-session aggregated data and Medium Access Control (MAC)/physical layer Key Performance Indicators (KPIs). See paragraphs 0254-0257.)
Regarding claim 7, D1 discloses wherein the control actions are weights which specify a quantity of resources to allocate to a plurality of user equipments communicatively coupled to the RAN (Referring to Figures 21B, 22, and 23, determine the best scheduling policy for the corresponding slice. The reward of the agents depends on the specific slice and the corresponding KPI requirements. Specifically, eMBB and MTC DRL agents have been trained to maximize the throughput of UEs, while URLLC agents have been trained to maximize the number of resources (i.e., PRBs) allocated to each UE and to satisfy their latency requirements (wireless state information of the network). To fully comply with O-RAN directives, the DRL agents were trained offline, and their effectiveness then tested on Colosseum. To train the DRL agents, approximately 6 GB of training data was generated, containing various performance metrics (e.g., throughput, bit error rate), system state information (e.g., transmission queue size, signal-to-interference-plus-noise ratio, channel quality information) and resource allocation strategies (e.g., slicing policies, scheduling) (control actions, interpreted as weights which specify quantity of resources to allocate to UEs coupled to the RAN) by running a total of 63 hours of experiments on Colosseum. See paragraphs 0266-0268.)
Regarding claim 9, D1 discloses wherein the control actions are identifications of modulation schemes for the RAN to communicate with respective user equipments (UEs) of a plurality of UEs communicatively coupled to the RAN (Referring to Figures 6, 21B, 22, and 23, the APIs includes functionalities to change scheduling, network slicing, and transmission parameters (e.g., Modulation and Coding Scheme (MCS) and power control) in real-time without the need to recompile and redeploy the BSs (UEs in communication with BSs). See paragraph 0111. The PHY-layer configuration can be tuned by setting the downlink/uplink MCS of selected UEs (set_mcs), which directly impacts on the signal modulation and coding rate (see Section 3.2 and 3GPP. 2020. See paragraph 0140.)
Regarding claim 9, D1 discloses wherein the TTI is less than 1 millisecond (Referring to Figures 21B, 22, and 23, A fundamental part of the operations of cellular networks involves actions at a sub-10 ms—or even sub-ms—time scale. In O-RAN, these are labeled as sub-TTI control loops, and mainly concern interactions between elements in the DU. Control loops at a similar timescale, however, could also be envisioned to operate between the DU 2272 and the RU 2276, or at the UEs (although these cases are not natively covered by O-RAN). See paragraph 0255-0257.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yeh et al. (US 2022/0014963 A1) - Reinforcement learning (RL) and/or Deep RL (DRL) approaches that learn policies and/or parameters for traffic management and/or for distributing multi-access traffic through interacting with the environment are also provided. Deep contextual bandit RL techniques for intelligent traffic management for edge networks are also provided.
Ranganath et al. (US 2024/0259879 A1) - RIC-based resource management for individual RIC applications, which is based on the collection and analysis of platform telemetry data as well as measurements collected by user equipment and access network infrastructure elements.
Bai et al. (US 2024/0305533 A1) - The resilient (R)AN slicing framework includes resource planning and slice-aware scheduling, as well as signaling exchanges for provisioning resilient (R)AN slicing. The intelligent (R)AN slicing framework can realize resource isolation in a more efficient and agile manner than existing network slicing technologies.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONALD L MILLS whose telephone number is (571)272-3094. The examiner can normally be reached Monday through Friday from 9-5 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yemane Mesfin can be reached at 571-272-3927. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DONALD L. MILLS
Primary Examiner
Art Unit 2462
/Donald L Mills/Primary Examiner, Art Unit 2462
Prosecution Insights