Prosecution Insights
Last updated: August 17, 2026
Application No. 18/790,824

Methods for Reward Signal Design and Handling for UE-sided Reinforcement Learning

Non-Final OA §102§103
Filed
Jul 31, 2024
Examiner
COSME, NATASHA W
Art Unit
2465
Tech Center
2400 — Computer Networks
Assignee
InterDigital Inc.
OA Round
1 (Non-Final)
80%
Grant Probability
Favorable
1-2
OA Rounds
1y 1m
Est. Remaining
94%
With Interview

Examiner Intelligence

Grants 80% — above average
80%
Career Allowance Rate
488 granted / 609 resolved
+22.1% vs TC avg
Moderate +14% lift
Without
With
+14.2%
Interview Lift
resolved cases with interview
Typical timeline
3y 1m
Avg Prosecution
23 currently pending
Career history
629
Total Applications
across all art units

Statute-Specific Performance

§101
4.8%
-35.2% vs TC avg
§103
59.7%
+19.7% vs TC avg
§102
24.3%
-15.7% vs TC avg
§112
8.1%
-31.9% vs TC avg
Black line = Tech Center average estimate • Based on career data from 609 resolved cases

Office Action

§102 §103
DETAILED ACTION 1. Claims 1-20 are presented for examination. Notice of Pre-AIA or AIA Status 2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Information Disclosure Statement 3. The information disclosure statement (IDS) submitted on 7/31/2024 and 12/05/2025 was filed in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 102 4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 6. Claim(s) 1, 3, 7, and 9 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by R2-2302652, “AI/ML Control and other topics”, Nokia, Nokia Shanghai Bell, April 17-26, 2023, 9 pages (provided by applicant, hereafter referred to as Nokia). Regarding claim 1, Nokia teaches A Wireless Transmit/Receive Unit (WTRU) comprising: a processor (see at least Fig. 2.1.1-1, 1 of a UE “using model X” ) configured to: receive configuration information (see at least Fig. 2.1.1-1, “1. Functionality A activation”; Fig. 2.1.1-2, “1. Activation of model A), wherein the configuration information comprises an indication of an exploration rate for operating according to an exploration mode or an exploitation mode (see at least Fig. 2.1.1-1, “2. Using model X”; Fig. 2.1.1-2, “2. Using model A), wherein the configuration information further comprises an indication of a reward type for reward signals (see at least pg. 5 proposal 6, “the gNB shall be able to configure the performance monitoring at the UE side .. The UE may consider a performance monitoring KPI (Top-K/1 beam accuracy) with gNB configured threshold to determine functionality failures of the activated functionality”; the performance is a reward type; reward signals are signals indicated as either “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2); send an indication of a first action associated with the exploitation mode (see at least Fig. 2.1.1-1, “3. Functionality A performance KPI reports”; Fig. 2.1.1-2, “3. Model A performance KPI reports”); receive an out-of-range (OOR) indication (see at least “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2) and an indication of a reward signal associated with the reward type (see at least “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2); based on the reward signal being less than a threshold (see at least “4. Deactivate functionality A or “4. Deactivate mode A”) and the OOR indication indicating that an OOR condition is not detected (“activate functionality B or activate model B”), send a request associated with the exploration mode (implicitly disclosed due to the fact that the KPIs will be reported for the newly activated functionality/model); and based on the reward signal being less than the threshold (see at least “4. Deactivate functionality A .. or “4. Deactivate mode A”) and the OOR indication indicating that the OOR condition is detected (see at least “4. Deactivate functionality A or “4. Deactivate mode A”), adjust one or more actions to be performed by the WTRU (see at least Fig. 2.1.1-1, “5. Switch to functionality B” or Fig. 2.1.1-2, “5. Switch to model B”). Regarding claim 3, Nokia teaches the WTRU of claim 1. In addition, Nokia teaches wherein the reward type is associated with a beam indication (see at least pg. 4 “2.1.3 Performance monitoring for beam management”). Regarding claim 7, Nokia teaches A method performed by a Wireless Transmit/Receive Unit (WTRU), the method comprising: receiving configuration information (see at least Fig. 2.1.1-1, “1. Functionality A activation”; Fig. 2.1.1-2, “1. Activation of model A), wherein the configuration information comprises an indication of an exploration rate for operating according to an exploration mode or an exploitation mode (see at least Fig. 2.1.1-1, “2. Using model X”; Fig. 2.1.1-2, “2. Using model A), wherein the configuration information further comprises an indication of a reward type for reward signals (see at least pg. 5 proposal 6, “the gNB shall be able to configure the performance monitoring at the UE side .. The UE may consider a performance monitoring KPI (Top-K/1 beam accuracy) with gNB configured threshold to determine functionality failures of the activated functionality”; the performance is a reward type; reward signals are signals indicated as either “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2); sending an indication of a first action associated with the exploitation mode (see at least Fig. 2.1.1-1, “3. Functionality A performance KPI reports”; Fig. 2.1.1-2, “3. Model A performance KPI reports”); receiving an out-of-range (OOR) indication (see at least “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2) and an indication of a reward signal associated with the reward type (see at least “4. Deactivate functionality A and activate functionality B” in Fig. 2.1.1-1 or “4. Deactivate model A and activate model B” in Fig. 2.1.1-2); based on the reward signal being less than a threshold (see at least “4. Deactivate functionality A or “4. Deactivate mode A”) and the OOR indication indicating that an OOR condition is not detected (“activate functionality B or activate model B”), send a request associated with the exploration mode (implicitly disclosed due to the fact that the KPIs will be reported for the newly activated functionality/model); and based on the reward signal being less than the threshold (see at least “4. Deactivate functionality A .. or “4. Deactivate mode A”) and the OOR indication indicating that the OOR condition is detected (see at least “4. Deactivate functionality A or “4. Deactivate mode A”), adjust one or more actions to be performed by the WTRU (see at least Fig. 2.1.1-1, “5. Switch to functionality B” or Fig. 2.1.1-2, “5. Switch to model B”). Regarding claim 9, Nokia teaches the method of claim 7. In addition, Nokia teaches wherein the reward type is associated with a beam indication (see at least pg. 4 “2.1.3 Performance monitoring for beam management”). Claim Rejections - 35 USC § 103 7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 9. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 10. Claim(s) 2, 8, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nokia, as applied to claims 1 and 7 above, in view of R2-2211455, “General aspects of AI/ML air interface and RAN2 impact”, Intel Corporation, November 14th -18th, 2022, 14 pages (provided by applicant, hereafter referred to as Intel). Regarding claim 2, Nokia teaches the WTRU of claim 1. Nokia does not appear to specifically disclose wherein the reward type is associated with an acknowledgement or negative acknowledgement (ACK/NACK) indication. In the same field of endeavor, Intel teaches wherein the reward type is associated with an acknowledgement or negative acknowledgement (ACK/NACK) indication (see at least pg. 7 scenario 2). It would have been obvious to one having ordinary skill in the art before the effective date to modify Nokia with Intel in order to aid in system reliability. Regarding claim 8, Nokia teaches the method of claim 7. Nokia does not appear to specifically teach wherein the reward type is associated with an acknowledgement or negative acknowledgement (ACK/NACK) indication. In the same field of endeavor, Intel teaches wherein the reward type is associated with an acknowledgement or negative acknowledgement (ACK/NACK) indication (see at least pg. 7 scenario 2). It would have been obvious to one having ordinary skill in the art before the effective date to modify Nokia with Intel in order to aid in system reliability. Regarding claim 18, Nokia teaches the method of claim 7. Nokia does not appear to specifically disclose further comprising activating training of the RL model, wherein sending the indication of the first action is based on the training of the RL model being activated. In the same field of endeavor, Intel teaches further comprising activating training of the RL model, wherein sending the indication of the first action is based on the training of the RL model being activated (see at least pg. 5: “Study AI/ML model monitoring for at least the following purposes: model activation, deactivation, selection, switching, fallback, and update (including re-training)”). It would have been obvious to one having ordinary skill in the art before the effective date to modify Nokia with Intel in order to aid in system reliability. Regarding claim 19, Nokia in view of Intel teaches the method of claim 18. In the obvious combination, Intel further teaches further comprising receiving an activation command, wherein activating the training of the RL model is in response to receiving the activation command (see at least pg. 5: “Study AI/ML model monitoring for at least the following purposes: model activation, deactivation, selection, switching, fallback, and update (including re-training)”). It would have been obvious to one having ordinary skill in the art before the effective date to modify Nokia with Intel in order to aid in system reliability . Regarding claim 20, Nokia in view of Intel teaches the method of claim 18. In the obvious combination, Intel further teaches further comprising: based on the reward signal being greater than the threshold, determining whether to deactivate training of the RL model (see at least pg. 5: “Study AI/ML model monitoring for at least the following purposes: model activation, deactivation, selection, switching, fallback, and update (including re-training)”). It would have been obvious to one having ordinary skill in the art before the effective date to modify Nokia with Intel in order to aid in system reliability. 11. Claim(s) 4, 10, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nokia as applied to claims 1 and 7 above, in view of R1-2400232, “Specification support for beam management”, February 26th-March 1st, 2024, 25 pages (provided by applicant, hereafter referred to as Vivo). Regarding claim 4, Nokia teaches the WTRU of claim 1. Nokia does not appear to specifically teach wherein the processor is configured to, in response to a beam failure detection (BFD), delay or prevent a beam failure recovery (BFR) action associated with the BFD to adjust the one or more actions. In the same field of endeavor, Vivo teaches wherein the processor is configured to, in response to a beam failure detection (BFD), delay or prevent a beam failure recovery (BFR) action associated with the BFD to adjust the one or more actions (see at least section 6.1; usage of AI/ML models for beam failure detection/prediction). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Vivo in order to enhance beam management. Regarding claim 10, Nokia teaches the method of claim 7. Nokia does not appear to specifically teach wherein adjusting the one or more actions comprises: in response to a beam failure detection (BFD), delaying or preventing a beam failure recovery (BFR) action associated with the BFD. In the same field of endeavor, Vivo wherein adjusting the one or more actions comprises: in response to a beam failure detection (BFD), delaying or preventing a beam failure recovery (BFR) action associated with the BFD (see at least section 6.1; usage of AI/ML models for beam failure detection/prediction). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Vivo in order to enhance beam management. Regarding claim 17, Nokia teaches the method of claim 7. Nokia does not appear to specifically teach wherein sending the indication of the first action comprises sending an indication of one or more beam selection parameters. In the same field of endeavor, Vivo teaches wherein sending the indication of the first action comprises sending an indication of one or more beam selection parameters (see at least section 6.1; usage of AI/ML models for beam failure detection/prediction). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Vivo in order to enhance beam management. 12. Claim(s) 5, 6, and 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nokia as applied to claim 1 above, in view of US 2024/0056264 A1 by Abdelghaffar et al. (hereafter referred to as Abdelghaffar). Regarding claim 5, Nokia teaches the WTRU of claim 1. Nokia does not appear to specifically disclose wherein the processor is configured to, in response to a decoding failure, send an acknowledgement (ACK) indication to adjust the one or more actions. In the same field of endeavor, Abdelghaffar teaches wherein the processor is configured to, in response to a decoding failure, send an acknowledgement (ACK) indication to adjust the one or more actions (see at least ¶ [0181]; “The method of Aspect 11, wherein the combined transmission includes multiple bits that are jointly encoded to indicate an acknowledgment or a negative acknowledgement with an amount of the adjustment.”). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Abdelghaffar in order to aid in system reliability. Regarding claim 6, Nokia teaches the WTRU of claim 1. Nokia does not appear to specifically disclose wherein the indication is received via one or more of physical downlink shared channel (PDSCH) configuration information, an acknowledgement or negative acknowledgement (ACK/NACK) indication, a hybrid automatic repeat request (HARQ), or downlink control information (DCI). In the same field of endeavor, Abdelghaffar teaches wherein the indication is received via one or more of physical downlink shared channel (PDSCH) configuration information, an acknowledgement or negative acknowledgement (ACK/NACK) indication, a hybrid automatic repeat request (HARQ), or downlink control information (DCI) (see at least ¶ [0181]; “The method of Aspect 11, wherein the combined transmission includes multiple bits that are jointly encoded to indicate an acknowledgment or a negative acknowledgement with an amount of the adjustment.”). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Abdelghaffar in order to aid in system reliability. Regarding claim 11, Nokia teaches the method of claim 7. Nokia does not appear to specifically disclose wherein adjusting the one or more actions comprises: in response to a decoding failure, sending an acknowledgement (ACK) indication. In the same field of endeavor, Abdelghaffar teaches wherein adjusting the one or more actions comprises: in response to a decoding failure, sending an acknowledgement (ACK) indication (see at least ¶ [0181]; “The method of Aspect 11, wherein the combined transmission includes multiple bits that are jointly encoded to indicate an acknowledgment or a negative acknowledgement with an amount of the adjustment.”). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Abdelghaffar in order to aid in system reliability. Regarding claim 12, Nokia teaches the method of claim 7. Nokia does not appear to specfifcally disclose wherein the indication is received via one or more of physical downlink shared channel (PDSCH) configuration information, an acknowledgement or negative acknowledgement (ACK/NACK) indication, a hybrid automatic repeat request (HARQ), or downlink control information (DCI). In the same field of endeavor, Abdelghaffar teaches wherein the indication is received via one or more of physical downlink shared channel (PDSCH) configuration information, an acknowledgement or negative acknowledgement (ACK/NACK) indication, a hybrid automatic repeat request (HARQ), or downlink control information (DCI) (see at least ¶ [0181]; “The method of Aspect 11, wherein the combined transmission includes multiple bits that are jointly encoded to indicate an acknowledgment or a negative acknowledgement with an amount of the adjustment.”). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Abdelghaffar in order to aid in system reliability. Regarding claim 13, Nokia in view of Abdelghaffar teaches the method of claim 12. In the obvious combination, Abdelghaffar teaches further comprising determining the reward signal based on the received one or more of the PDSCH configuration information, the ACK/NACK indication, the HARQ, or the DCI (see at least abstract and ¶ [0094]). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Abdelghaffar in order to aid in system reliability. 13. Claim(s) 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nokia as applied to claim 7 above, in view of US 2022/0182175 A1 by Kvernvik et al. (hereafter referred to as Kvernvik). Regarding claim 14, Nokia teaches the method of claim 7. Nokia does appear to specifically disclose wherein the request comprises a request to enter the exploration mode. In the same field of endeavor, Kvernvik teaches wherein the request comprises a request to enter the exploration mode (see at least ¶ [0027] and [0078]; exploration mode). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Kvernvik in order to enable successful handling of changes in the environment. Regarding claim 15, Nokia teaches the method of claim 7. Nokia does appear to specifically disclose wherein the request comprises an indication of a second action associated with the exploration mode. In the same field of endeavor, Kvernvik teaches wherein the request comprises an indication of a second action associated with the exploration mode (see at least ¶ [0027] and [0078]; exploration mode). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Kvernvik in order to enable successful handling of changes in the environment. Regarding claim 16, Nokia teaches the method of claim 7. Nokia does not appear to specifically disclose wherein sending the indication of the first action comprises sending an indication of one or more channel quality parameters, wherein the one or more channel quality parameters include one or more of a channel quality indicator (CQI) or a rank indicator (RI). In the same field of endeavor, Kvernvik teaches wherein sending the indication of the first action comprises sending an indication of one or more channel quality parameters, wherein the one or more channel quality parameters include one or more of a channel quality indicator (CQI) or a rank indicator (RI) (see at least ¶ [0027], [0078], [0080]-[0084]; exploration mode). It would have been obvious to one having ordinary skill in the art before the effective filing date to modify Nokia with Kvernvik in order to enable successful handling of changes in the environment. Conclusion 14. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NATASHA W COSME whose telephone number is (571)270-7225. The examiner can normally be reached M-F 7:30-4. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ayman Abaza can be reached at 571-270-0422. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /NATASHA W COSME/Primary Examiner, Art Unit 2465
Read full office action

Prosecution Timeline

Jul 31, 2024
Application Filed
Jul 31, 2026
Non-Final Rejection mailed — §102, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12706782
ARTIFICIAL INTELLIGENCE MODEL TRAINING FOR IDLE MODE ASSISTANCE
3y 3m to grant Granted Aug 11, 2026
Patent 12706781
AI-BASED CHANNEL ESTIMATION METHOD AND APPARATUS
2y 10m to grant Granted Aug 11, 2026
Patent 12695525
SPECTRUM SURVEILLANCE
2y 10m to grant Granted Jul 28, 2026
Patent 12696336
Transmission Power Control for Beam Failure Recovery Requests
2y 5m to grant Granted Jul 28, 2026
Patent 12684390
Methods for Positioning Reference Signal (PRS) Activity Reporting
3y 1m to grant Granted Jul 14, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
80%
Grant Probability
94%
With Interview (+14.2%)
3y 1m (~1y 1m remaining)
Median Time to Grant
Low
PTA Risk
Based on 609 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month