Prosecution Insights
Last updated: November 20, 2026
Application No. 18/861,430

LEARNING DEVICE, LEARNING METHOD, AND LEARNING PROGRAM

Non-Final OA §101§103
Filed
Oct 29, 2024
Priority
May 02, 2022 — JP 2022-076195 +1 more
Examiner
JEAN GILLES, JUDE
Art Unit
2459
Tech Center
2400 — Computer Networks
Assignee
Mitsubishi Heavy Industries Ltd.
OA Round
1 (Non-Final)
93%
Grant Probability
Favorable
1-2
OA Rounds
3m
Est. Remaining
95%
With Interview

Examiner Intelligence

Grants 93% — above average
93%
Career Allowance Rate
885 granted / 953 resolved
+34.9% vs TC avg
Minimal +2% lift
Without
With
+2.5%
Interview Lift
resolved cases with interview
Typical timeline
2y 4m
Avg Prosecution
17 currently pending
Career history
976
Total Applications
across all art units

Statute-Specific Performance

§101
13.2%
-26.8% vs TC avg
§103
33.8%
-6.2% vs TC avg
§102
19.3%
-20.7% vs TC avg
§112
4.7%
-35.3% vs TC avg
Black line = Tech Center average estimate • Based on career data from 953 resolved cases

Office Action

§101 §103
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This communication is in reply to 10/29/2024. Claimed priority is granted from foreign Application NumberJP2022-076195 filed on 05/02/2022. Information Disclosure Statement The information disclosure statement (IDS) submitted on 06/30/2025 and 11/20/2024 was filed in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1, 3-5, and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over TAYLOR et al., US 20190236455 A1, in view of Zaremoodi et al., US 20210304074 A1 (hereinafter Zare). Regarding claim 1, The combination of TAYLOR and Zare teaches the invention as claimed. It discloses: A learning device comprising: a processing unit that performs learning of a learning model of an agent (TAYLOR, 0068 supplies processor/memory); 0108-0109 supplies the action model and explicit hyperparameters, including grid search), wherein the learning model includes a hyperparameter (TAYLOR, 0107-0110, teaches demonstration-action classification. Modification can be directed to applying Zare’s optimization loop, 0106, 0109, 0122-0123, to select settings based on imitation performance), the learning includes imitation learning and reinforcement learning (TAYLOR, 0058-0061 supplies reward-directed learning. Modification can be directed to applying Are’s optimization to the reinforcement-learning settings using reward performance as the objective), the imitation learning causing the processing unit to perform learning of the hyperparameter of the learning model (TAYLOR, 0108-0109 first, then Zare, 0109 and 0122 expressly teach the specified order) such that the agent executes a predetermined action in a predetermined state under a predetermined environment (TAYLOR, 0130 checks classification-loss convergence before transfer and Zare, 0116, 0123 outputs the evaluated, optimized model and settings). the reinforcement learning causing the processing unit to perform learning of the hyperparameter of the learning model such that a reward assigned to the agent under the predetermined environment is maximized (TAYLOR, 0110-0112, and 0130 transfers the pretrained model into reinforcement learning, including an all-layer transfer alternative), and the processing unit executes: a step of setting a parameter value of the hyperparameter of the learning model (TAYLOR, 0109 and Zare, 0109); a step of executing the imitation learning by using the learning model after the setting (TAYLOR, 0107-0109); a step of evaluating the learning model after the imitation learning and extracting the evaluated learning mode (TAYLOR , 0130; Zare, 0116, and 0122-0123); and a step of executing the reinforcement learning by using the extracted learning model (TAYLOR, 0110-0112 and 0130). Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify TAYLOR’s demonstration-based pretraining process using Zare’s hyperparameter tuning and evaluation procedure to select an evaluated imitation-trained model before reinforcement learning. TAYLOR permits hyperparameter grid search (0109) and uses pretrained model to initiate reinforcement learning (0130), while Zare evaluates candidate configurations and outputs an optimized model (0122-0123). The modification would predictably work because TAYLOR because TAYLOR already transfers pretrained parameters into its reinforcement-learning network (0110-0112). Likewise, regarding claims 3-5, and 7-8 Claim 3. (Currently Amended) The learning device according to Claim 1, wherein, in the step of extracting the learning model, in a case where an evaluation result of the learning model satisfies an evaluation index, one learning model that satisfies the evaluation index is extracted (Zare, 0106, 0116, and 0123 teaches satisfying constraints and outputting an optimized model. Select one passing candidate for TAYLEO’s subsequent training). Claim 4. (Currently Amended) The learning device according to Claim 1, wherein, in the step of setting the parameter value, in a case where an evaluation result in the step of evaluating the learning model does not satisfy an evaluation index, a new parameter value is set (Zare, 0122 expressly changes hyperparameter values and continues training when a constraint fails or performance is inadequate). Claim 5. (Currently Amended) The learning device according to Claim 1, wherein, in the step of extracting the learning model, evaluation based on a matching degree between a predetermined state and a predetermined action that are to be trained in the imitation learning under a predetermined environment and a predetermined state and a predetermined action in the learning model under the predetermined environment is executed (TAYLOR, 0107-0110, and 0130 evaluates imitation through action-classification loss. Use that loss on demonstration states as Zare’s selection metric: compare demonstrated actions against model outputs for those states). Claim 7. (Original) A learning method of performing learning of a learning model of an agent by using a learning device, wherein the learning model includes a hyperparameter, the learning includes imitation learning and reinforcement learning, the imitation learning causing the learning device to perform learning of the hyperparameter of the learning model such that the agent executes a predetermined action in a predetermined state under a predetermined environment, the reinforcement learning causing the learning device to perform learning of the hyperparameter of the learning model such that a reward assigned to the agent under the predetermined environment is maximized, and the learning method causes the learning device to execute: a step of setting a parameter value of the hyperparameter of the learning model; a step of executing the imitation learning by using the learning model after the setting; a step of evaluating the learning model after the imitation learning and extracting the evaluated learning model; and a step of executing the reinforcement learning by using the extracted learning model. This claim provides the same operations and sequence mapped above for claim 1, performed as a computer-implemented method. The same references, motivation and reason to combine used for the rejection of claim 1 is also valid for this claim. Claim 8. A learning program for performing learning of a learning model of an agent by using a learning device, wherein the learning model includes a hyperparameter, the learning includes imitation learning and reinforcement learning, the imitation learning causing the learning device to perform learning of the hyperparameter of the learning model such that the agent executes a predetermined action in a predetermined state under a predetermined environment, the reinforcement learning causing the learning device to perform learning of the hyperparameter of the learning model such that a reward assigned to the agent under the predetermined environment is maximized, and the learning program causes the learning device to execute: a step of setting a parameter value of the hyperparameter of the learning model; a step of executing the imitation learning by using the learning model after the setting; a step of evaluating the learning model after the imitation learning and extracting the evaluated learning model; and a step of executing the reinforcement learning by using the extracted learning model. This claim provides the same operations and sequence mapped above for claim 1, implemented by program instructions expressed in par. 0172-0173. The same references, motivation and reason to combine used for the rejection of claim 1 is also valid for this claim. Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over TAYLOR and ZARE as applied to claims 1, 3-5 and 7-8 above, and further in view of Pietquin et al, US 2020/0151562 A1. Regarding claim 2, the combination of TAYLOR-Zare teaches the learning device of claim 1, but fails to specifically disclose the detail of a learning device according to Claim 1, wherein the environment is an environment in which the reward is sparse. This teaching is well-known in the art as evidenced Pietquin. In the same field of endeavor, Pietquin expressly discloses sparse feedback, including reward only upon task completion (Pietquin, 0011, 0019, 0044 and 0054). It would have been obvious to apply TAYLOR’s pretraining process, as modified by Zare, to Pietquin’s sparse-reward environment, where reward may remain zero until task completion (Pietquin, 0044, 0054)/ Pietquin explains that dense reward sharing can produce undesirable behavior 0044, providing a reason to use task-completion rewards. TAYLOR’s demonstration-based initialization would provide a suitable starting model for subsequent reward-based learning 0130, predictably supporting training where useful reward feedback is infrequent. By this rationale, claim 2 is rejected Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over TAYLOR and ZARE as applied to claims 1, 3-5 and 7-8 above, and further in view of Meng et al., US 2020/0334538 A1. Regarding claim 6, the combination of TAYLOR-Zare teaches the learning device of claim 1, but fails to specifically disclose the detail of learning device according to Claim 5, wherein, as the matching degree, a KL divergence is used. This teaching is well-known in the art as evidenced Meng. In the same field of endeavor, Meng expressly discloses KL output-distribution comparison and convergence testing (0021, 0026, 0054-0055, 0059-0061). It would have been obvious to modify TAYLOR’s imitation-model evaluation, as combined with Zare, to use Meng’s KL-divergence comparison between reference and learned output distributions (Meng, 0026 and 0054). A skilled artisan would use that measure to quantify how closely the model’s action probabilities match the demonstration behavior before selecting it for reinforcement learning. The substitution would predictably work because TAYLOR already uses probabilistic action outputs and cross-entropy loss (0108-0110), and meng explains that minimizing KL divergence with a fixed reference distribution is equivalent to minimizing the corresponding cross-entropy objective (0055). By this rationale, claim 6 is rejected. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because its opening “A learning program” appears to claim software itself without a storage medium. If construed as a program per se, it fails the statutory-category requirement. Recasting it as a non-transitory computer-readable storage medium storing the instructions would address that issue. CONCLUSION Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jude Jean-Gilles whose telephone number is 571-272-3914. The examiner can normally be reached on Mon-Fri, from 9:00AM-5:00PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tonia Dollinger can be reached on 571-272-4170. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JUDE JEAN GILLES/Primary Examiner, Art Unit 2459 November 10, 2026
Read full office action

Prosecution Timeline

Oct 29, 2024
Application Filed
Nov 13, 2026
Non-Final Rejection mailed — §101, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12803875
SYSTEM AND METHOD OF CLOUD BASED CONGESTION CONTROL FOR VIRTUALIZED BASE STATION
2y 4m to grant Granted Nov 17, 2026
Patent 12776978
TECHNOLOGIES FOR PROTOCOL EXECUTION WITH AGGREGATION AND CACHING
1y 7m to grant Granted Oct 27, 2026
Patent 12750420
MOBILE INTERACTIVE KIOSK METHOD
1y 5m to grant Granted Sep 29, 2026
Patent 12744725
COHERENT INTERNET NETWORK BONDING SYSTEM
1y 11m to grant Granted Sep 22, 2026
Patent 12744741
SERVICE LEVEL AGREEMENT BASED DATA FORWARDING WITH LINK AGGREGATION GROUPING
1y 8m to grant Granted Sep 22, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
93%
Grant Probability
95%
With Interview (+2.5%)
2y 4m (~3m remaining)
Median Time to Grant
Low
PTA Risk
Based on 953 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month