Prosecution Insights
Browse Companies Examiners Firms Draft Your Response
Office Actions › Research & Business Foundation Sungkyunkwan University › 18488246
Last updated: July 17, 2026
Application No. 18/488,246

MULTI-TASK OFFLINE REINFORCEMENT LEARNING MODEL BASED ON SKILL REGULARIZED TASK DECOMPOSITION AND MULTI-TASK OFFLINE REINFORCEMENT LEARNING METHOD USING THE SAME

Non-Final OA §101§102§103§112
Filed
Oct 17, 2023
Priority
Nov 24, 2022 — RE 10-2022-0159388
Examiner
SIPPEL, MOLLY CLARKE
Art Unit
4100
Tech Center
4100
Assignee
Research & Business Foundation Sungkyunkwan University
OA Round
1 (Non-Final)
52%
Grant Probability
Moderate
1-2
OA Rounds
1y 0m
Est. Remaining
86%
With Interview

This examiner grants 52% of cases after interview

— +33.6% interview lift. A telephonic interview to clarify the technical implementation could significantly improve the outcome.
Based on 21 resolved cases, 2023–2026

Examiner Intelligence

SIPPEL, MOLLY CLARKE View full profile →
Grants 52% of resolved cases
52%
Career Allowance Rate
11 granted / 21 resolved
-7.6% vs TC avg
Strong +34% interview lift
Without
With
+33.6%
Interview Lift
resolved cases with interview
Typical timeline
3y 9m
Avg Prosecution
17 currently pending
Career history
41
Total Applications
across all art units

Statute-Specific Performance

§101
26.6%
-13.4% vs TC avg
§103
53.2%
+13.2% vs TC avg
§102
2.1%
-37.9% vs TC avg
§112
18.1%
-21.9% vs TC avg
Black line = Tech Center average estimate • Based on career data from 21 resolved cases

Office Action

§101 §102 §103 §112
DETAILED ACTION This action is responsive to the application filed on 10/17/2023. Claims 1-10 are pending in the case. Claims 1 and 10 are independent claims. Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Priority Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Republic of Korea on 11/24/2022. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Specification The disclosure is objected to because of the following informalities: Paragraph 0009, line 1, “a reinforce learning model” should read “a reinforcement learning model” to maintain consistency with the rest of applicant’s specification. Paragraph 0018, line 1, “a processor-implemented reinforce learning model” should read “a processor-implemented reinforcement learning model” to maintain consistency with the rest of applicant’s specification. Appropriate correction is required. Claim Objections Claims 1-10 objected to because of the following informalities: Claim 1, line 1, “a reinforce learning model” should read “a reinforcement learning model” to maintain consistency with applicant’s specification. Claim 2, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 3, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 4, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 5, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 6, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 7, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 8, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 9, line 1, “the reinforce learning model” should read “the reinforcement learning model” to maintain consistency with applicant’s specification. Claim 10, line 1, “a processor-implemented reinforce learning method” should read “a processor-implemented reinforcement learning model” to maintain consistency with applicant’s specification. Appropriate correction is required. Claim Rejections - 35 USC § 112 The following is a quotation of the first paragraph of 35 U.S.C. 112(a): (a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112: The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention. Claims 4, 6-7, and 9 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention. Regarding claim 4, the claim recites: “Equation 1: PNG media_image1.png 60 723 media_image1.png Greyscale , (where, PNG media_image2.png 62 715 media_image2.png Greyscale , PNG media_image3.png 17 14 media_image3.png Greyscale is a skill encoder, PNG media_image4.png 17 15 media_image4.png Greyscale is a skill decoder, PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image7.png 17 10 media_image7.png Greyscale is skill embedding, and PNG media_image8.png 17 13 media_image8.png Greyscale is state-action pairs from t-n to t+n-1)”. Regarding the breadth of the claims, the claims are specific to a reinforcement learning model comprising a skill regularized task decomposition model, and a data augmentation model, where the skill regularized task decomposition model performs a skill embedding operation by implementing 2n-step state-action pairs by implementing the loss function above. As written, this includes any interpretation of the variables as long as PNG media_image3.png 17 14 media_image3.png Greyscale is a skill encoder, PNG media_image4.png 17 15 media_image4.png Greyscale is a skill decoder, PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image7.png 17 10 media_image7.png Greyscale is skill embedding, and PNG media_image8.png 17 13 media_image8.png Greyscale is state-action pairs from t-n to t+n-1. Regarding the state of the art, the prior art fails to provide any evidence that a person of ordinary skill in the art would be able to implement that function. Regarding the level of one of ordinary skill in the art, it is fairly high and the level of predictability in the art is fairly low. Regarding the level of direction provided by the inventor, applicant’s specification provides little or no guidance in how a person of ordinary skill in the art could use the provided function. The current claims cover this loss function: “Equation 1: PNG media_image1.png 60 723 media_image1.png Greyscale , (where, PNG media_image2.png 62 715 media_image2.png Greyscale , PNG media_image3.png 17 14 media_image3.png Greyscale is a skill encoder, PNG media_image4.png 17 15 media_image4.png Greyscale is a skill decoder, PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image7.png 17 10 media_image7.png Greyscale is skill embedding, and PNG media_image8.png 17 13 media_image8.png Greyscale is state-action pairs from t-n to t+n-1)” which is not enabled by the applicant’s specification. No working examples are provided in the specification for the invention of claim 4. With no guidance from the specification or disclosures in the prior art, it would require an undue amount of experimentation to make and use the invention of claim 4. Regarding claim 6, the claim recites: Equation 2: PNG media_image9.png 45 796 media_image9.png Greyscale , (where, PNG media_image10.png 70 767 media_image10.png Greyscale , PNG media_image11.png 69 506 media_image11.png Greyscale , PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image15.png 14 8 media_image15.png Greyscale is a transition, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, and PNG media_image17.png 17 12 media_image17.png Greyscale is a reward sum of episodes including state-action pairs). Regarding the breadth of the claims, the claims are specific to a reinforcement learning model comprising a skill regularized task decomposition model, and a data augmentation model, where the skill regularized task decomposition model performs a skill regularization operation by implementing n-step transitions including states, actions, rewards, and next states by implementing the loss function above. As written, this includes any interpretation of the variables as long as PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image15.png 14 8 media_image15.png Greyscale is a transition, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, and PNG media_image17.png 17 12 media_image17.png Greyscale is a reward sum of episodes including state-action pairs. Regarding the state of the art, the prior art fails to provide any evidence that a person of ordinary skill in the art would be able to implement that function. Regarding the level of one of ordinary skill in the art, it is fairly high and the level of predictability in the art is fairly low. Regarding the level of direction provided by the inventor, applicant’s specification provides little or no guidance in how a person of ordinary skill in the art could use the provided function. The current claims cover this loss function: Equation 2: PNG media_image9.png 45 796 media_image9.png Greyscale , (where, PNG media_image10.png 70 767 media_image10.png Greyscale , PNG media_image11.png 69 506 media_image11.png Greyscale , PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image15.png 14 8 media_image15.png Greyscale is a transition, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, and PNG media_image17.png 17 12 media_image17.png Greyscale is a reward sum of episodes including state-action pairs)” which is not enabled by the applicant’s specification. No working examples are provided in the specification for the invention of claim 6. With no guidance from the specification or disclosures in the prior art, it would require an undue amount of experimentation to make and use the invention of claim 6. Claim 7 is rejected as being dependent upon a rejected base claim without curing any of the deficiencies. Regarding claim 9, the claim recites: “Equation 3: PNG media_image18.png 43 497 media_image18.png Greyscale , (where, PNG media_image19.png 38 133 media_image19.png Greyscale , PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, and PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder)”. Regarding the breadth of the claims, the claims are specific to a reinforcement learning model comprising a skill regularized task decomposition model and a data augmentation model, where the data augmentation model performs data augmentation by generating an imaginary demo, which is generated by the equation above. As written, this includes any interpretation of the variables as long as PNG media_image19.png 38 133 media_image19.png Greyscale , PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, and PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder. Regarding the state of the art, the prior art fails to provide any evidence that a person of ordinary skill in the art would be able to implement that function. Regarding the level of one of ordinary skill in the art, it is fairly high and the level of predictability in the art is fairly low. Regarding the level of direction provided by the inventor, applicant’s specification provides little or no guidance in how a person of ordinary skill in the art could use the provided function. The current claims cover this loss function: “Equation 3: PNG media_image18.png 43 497 media_image18.png Greyscale , (where, PNG media_image19.png 38 133 media_image19.png Greyscale , PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, and PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder) which is not enabled by the applicant’s specification. No working examples are provided in the specification for the invention of claim 9. With no guidance from the specification or disclosures in the prior art, it would require an undue amount of experimentation to make and use the invention of claim 9. The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claims 2-7 and 9 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claim 2, the claim recites: “the plurality of subtasks” in lines 5-6. There is insufficient antecedent basis for this limitation in the claim. The parent claim recites: “subtasks” in line 10. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, this limitation has been interpreted to mean “the subtasks”, referring to the previously recited claim element. Regarding claim 3, the claim recites: “the 2n-step state-action pairs of offline data” in lines 2-3. There is insufficient antecedent basis for this limitation in the claim. The parent claim recites “2n-step state-action pairs” in line 7. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, this limitation has been interpreted to mean “the 2n-step state-action pairs”, referring to the previously recited claim element. Regarding claim 4, the claim recites: “Equation 1: PNG media_image1.png 60 723 media_image1.png Greyscale , (where, PNG media_image2.png 62 715 media_image2.png Greyscale , PNG media_image3.png 17 14 media_image3.png Greyscale is a skill encoder, PNG media_image4.png 17 15 media_image4.png Greyscale is a skill decoder, PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image7.png 17 10 media_image7.png Greyscale is skill embedding, and PNG media_image8.png 17 13 media_image8.png Greyscale is state-action pairs from t-n to t+n-1)”. This limitation renders the claim indefinite for failing to particularly point out and distinctly claim the subject matter because it is unclear what “ ϕ ”, “m”, “n”, “ λ ”, “b”, and “k” are meant to represent. It is unclear if applicant is attempting to refer to previously recited claim elements or if applicant is attempting to recite new claim elements. Applicant should clearly define what each variable represents in the equation. Regarding claim 5, the claim recites: “the data” in lines 3 and 4. There is insufficient antecedent basis for this limitation in the claim. It is unclear if applicant is attempting to refer to previously recited claim elements or if applicant is attempting to recite new claim elements. For examination purposes, this limitation has been interpreted, in light of applicant’s specification, to mean “reward data” in line 3, reciting a new claim element and “the reward data” in line 4, referring to the limitation interpreted to be recited in line 3. Regarding claim 6, the claim recites: “Equation 2: PNG media_image9.png 45 796 media_image9.png Greyscale , (where, PNG media_image10.png 70 767 media_image10.png Greyscale , PNG media_image11.png 69 506 media_image11.png Greyscale , PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image15.png 14 8 media_image15.png Greyscale is a transition, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, and PNG media_image17.png 17 12 media_image17.png Greyscale is a reward sum of episodes including state-action pairs)”. This limitation renders the claim indefinite for failing to particularly point out and distinctly claim the subject matter because it is unclear what “ θ ”, “ λ ”, “ L P R ”, “ z ~ ”, “m”, “ P Z ”, “n”, “ s t i + j + 1 ”, “ s t i + j ”, “ a t i + j ”, “ s t i ”, “ a t i ”, and “ d t i ” are meant to represent. It is unclear if applicant is attempting to refer to previously recited claim elements or if applicant is attempting to recite new claim elements. Applicant should clearly define what each variable represents in the equation. Claim 7 is rejected as being dependent upon a rejected base claim without curing any of the deficiencies. Regarding claim 9, the claim recites: “Equation 3: PNG media_image18.png 43 497 media_image18.png Greyscale , (where, PNG media_image19.png 38 133 media_image19.png Greyscale , PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, and PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder)”. This limitation renders the claim indefinite for failing to particularly point out and distinctly claim the subject matter because it is unclear what “ a t ~ ”, “ s t + 1 ~ ”, “ r t ~ ”, “ p ϕ ”, and “ τ t ” are meant to represent. It is unclear if applicant is attempting to refer to previously recited claim elements or if applicant is attempting to recite new claim elements. Applicant should clearly define what each variable represents in the equation. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because they cover at least one embodiment that is software per se. In accordance with MPEP 2016.03: Products that do not have a physical or tangible form, such as information (often referred to as "data per se") or a computer program per se (often referred to as "software per se") when claimed as a product without any structural recitations. As the courts' definitions of machines, manufactures and compositions of matter indicate, a product must have a physical or tangible form in order to fall within one of these statutory categories. Digitech, 758 F.3d at 1348, 111 USPQ2d at 1719. Thus, the Federal Circuit has held that a product claim to an intangible collection of information, even if created by human effort, does not fall within any statutory category. Digitech, 758 F.3d at 1350, 111 USPQ2d at 1720 (claimed "device profile" comprising two sets of data did not meet any of the categories because it was neither a process nor a tangible product). Similarly, software expressed as code or a set of instructions detached from any medium is an idea without physical embodiment. See Microsoft Corp. v. AT&T Corp., 550 U.S. 437, 449, 82 USPQ2d 1400, 1407 (2007); see also Benson, 409 U.S. 67, 175 USPQ2d 675 (An "idea" is not patent eligible). Thus, a product claim to a software program that does not also contain at least one structural limitation (such as a "means plus function" limitation) has no physical or tangible form, and thus does not fall within any statutory category. Another example of an intangible product that does not fall within a statutory category is a paradigm or business model for a marketing company. In re Ferguson, 558 F.3d 1359, 1364, 90 USPQ2d 1035, 1039-40 (Fed. Cir. 2009). The claims do not fall within at least one of the four categories of patent eligible subject matter because Claims 1-9 recite a reinforce learning model which, under the broadest reasonable interpretation, include embodiments which are pure software per se. Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding claim 1: Step 1 Statutory Category: Claim 1 is directed to a “reinforce learning model” which is considered software per se and does not fall under one of the four statutory categories. Further: Step 2A Prong 1 Judicial exception: Claim 1 recites, in part, “perform a skill regularized task decomposition based on a determined data quality”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “perform data augmentation by generating an imaginary demo”. This limitation under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “perform a skill embedding operation by implementing 2n-step state-action pairs”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “perform a skill regularization operation by implementing n-step transitions including states, actions, rewards, and next states”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “perform an operation of decomposing a task in units of episodes into subtasks in units of n-steps”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification paragraphs 0070-0071, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case a judgment. See MPEP § 2106.04(a)(2)(III). Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “A reinforce learning model”. This limitation is an additional element that amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites: “a skill regularized task decomposition model configured to…”, “a data augmentation model configured to…”, and “wherein the skill regularized task decomposition is configured to…”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “A reinforce learning model” amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further, the additional elements: “a skill regularized task decomposition model configured to…”, “a data augmentation model configured to…”, and “wherein the skill regularized task decomposition is configured to…” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 2, the rejection of claim 1 is incorporated, and further, the claim recites: “decompose the task into the subtasks by matching the subtasks to a plurality of skills in units of action sequences”. This limitation is a continuation of the “perform an operation of decomposing a task in units of episodes into subtasks in units of n-steps” limitation identified as an abstract idea in the rejection of the parent claim. Further, the claim recites: “sharing a skill of the plurality of skills corresponding to a subtask among the plurality of subtasks”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. Further, the claim recites: “wherein the skill regularized task decomposition model is configured to…”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the claim recites: “wherein the data augmentation model is configured to perform reinforcement learning by…”. This limitation is an additional element that amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 3, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein when performing the skill embedding operation, the skill regularized task decomposition model is configured to map the 2n-step state-action pairs of offline data to a skill candidate space, and infer the 2n-step state-action pairs by implementing a mapped candidate vector”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Additionally, this limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 4, the rejection of claim 3 is incorporated, and further, the claim recites: “wherein the skill embedding operation is performed through training by implementing a skill embedding loss of Equation 1 below: Equation 1: PNG media_image1.png 60 723 media_image1.png Greyscale , (where, PNG media_image2.png 62 715 media_image2.png Greyscale , PNG media_image3.png 17 14 media_image3.png Greyscale is a skill encoder, PNG media_image4.png 17 15 media_image4.png Greyscale is a skill decoder, PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image7.png 17 10 media_image7.png Greyscale is skill embedding, and PNG media_image8.png 17 13 media_image8.png Greyscale is state-action pairs from t-n to t+n-1)”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 5, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein when performing the skill regularization operation, the skill regularized task decomposition model is configured to map the n-step transitions to a task candidate space, infer with a same task when the data is above a reference quality, solved by a same skill, and infer with another task when the data is below the reference quality”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Additionally, this limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 6, the rejection of claim 5 is incorporated, and further, the claim recites: “wherein the skill regularization operation is performed through training by implementing a skill regularized loss of Equation 2 below: Equation 2: PNG media_image9.png 45 796 media_image9.png Greyscale , (where PNG media_image10.png 70 767 media_image10.png Greyscale , PNG media_image11.png 69 506 media_image11.png Greyscale , PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image15.png 14 8 media_image15.png Greyscale is a transition, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, and PNG media_image17.png 17 12 media_image17.png Greyscale is a reward sum of episodes including state-action pairs)”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible Regarding claim 7, the rejection of claim 6 is incorporated, and further, the claim recites: “wherein when performing the operation of decomposing the task in units of episodes into the subtasks in units of n-steps, the skill regularized task decomposition model is configured to infer a subtask”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. Further, the claim recites: “…by implementing the task encoder in the skill regularized process”. This limitation is an additional element that amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 8, the rejection of claim 1 is incorporated, and further, the claim recites: “generate the imaginary demo by inferring data generated when performing a skill that is appropriate for a given task”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Additionally, this limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “and augment learning data by training by adding subtask information to an input value”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception. Further, the claim recites: “wherein the data augmentation model is configured to…” and “by implementing the skill regularized task decomposition model”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 9, the rejection of claim 8 is incorporated, and further, the claim recites: “wherein the imaginary demo is generated by Equation 3 below: Equation 3: PNG media_image18.png 43 497 media_image18.png Greyscale , (where, PNG media_image19.png 38 133 media_image19.png Greyscale , PNG media_image5.png 17 13 media_image5.png Greyscale is an action at time t, PNG media_image6.png 17 13 media_image6.png Greyscale is a state at time t, PNG media_image14.png 17 12 media_image14.png Greyscale is a reward at time t, PNG media_image16.png 14 9 media_image16.png Greyscale is a subtask embedding vector, PNG media_image12.png 17 15 media_image12.png Greyscale is a task encoder, and PNG media_image13.png 17 14 media_image13.png Greyscale is a task decoder)”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites mathematical concepts. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 10: Step 1 Statutory Category: Claim 10 is directed to a method, which falls under one of the four statutory categories. Step 2A Prong 1 Judicial exception: Claim 10 recites, in part, “performing a skill regularized task decomposition based on a determined data quality”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “performing data augmentation by generating an imaginary demo”. This limitation under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “wherein the performing of the skill regularized task decomposition comprises: performing a skill embedding operation by implementing 2n-step state-action pairs”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “performing a skill regularization operation by implementing n-step transitions including states, actions, rewards, and next states”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification, covers the recitation of a mathematical concept, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “performing an operation of decomposing a task in units of episodes into subtasks in units of n-steps”. This limitation, under the broadest reasonable interpretation, and in light of applicant’s specification paragraphs 0070-0071, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case a judgment. See MPEP § 2106.04(a)(2)(III). Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “A … reinforce learning method”. This limitation is an additional element that amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites that the method is “processor-implemented”. This limitation is an additional element that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “A … reinforce learning method” amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further, the additional element that the method is “processor-implemented” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible. Claim Rejections - 35 USC § 102 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claims 1-3, 8, and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Pertsch et al., Accelerating Reinforcement Learning with Learned Skill Priors, 10/22/2020, https://arxiv.org/pdf/2010.11944, hereinafter referred to as "Pertsch". Regarding claim 1, Pertsch teaches A reinforce learning model (Pertsch, Page 1, Abstract, Lines 9-12, “We propose a deep latent variable model that jointly learns an embedding space of skills and the skill prior from offline agent experience. We then extend common maximum entropy RL approaches to use skill priors to guide downstream learning”), comprising: a skill regularized task decomposition model (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”) configured to perform a skill regularized task decomposition based on a determined data quality (Pertsch, Page 5, Algorithm 1; Pertsch, Page 5, Paragraph 1; Because the skills are sampled based on the skill expected to maximize the return, they are considered to be “based on a determined data quality”); and a data augmentation model configured to perform data augmentation by generating an imaginary demo (Pertsch, Page 3, Section 3.1, Paragraph 2, “The downstream learning problem is formulated as a Markov decision process (MDP) defined by a tuple { S , A , T , R , p , γ } of states, actions, transition probability, reward, initial state distribution, and discount factor. We aim to learn a policy π θ a s with parameters θ that maximizes the discounted sum of rewards J θ = E π ∑ t = 0 T - 1 γ t r t   where T is the episode horizon”; Pertsch, Page 6, Figure 3; Pertsch, Page 6, Section 4.1, Paragraph 4, Line 1, “A simulated kitchen environment based on Gupta et al. [27]”), wherein the skill regularized task decomposition model is configured to: perform a skill embedding operation by implementing 2n-step state-action pairs (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”; Pertsch, Page 3, Section 3.1, Lines 1-2, “We assume access to a dataset D of pre-recorded agent experience in the form of state-action trajectories τ i = s 0 ,   a 0 ,   … , s T i ,   a T i ”; see also Pertsch, Page 4, Figure 2); perform a skill regularization operation by implementing n-step transitions including states, actions, rewards, and next states (Pertsch, Page 4, Section 3.3, Lines 1-6, “To use the learned skill embedding for downstream task learning, we employ a hierarchical policy learning scheme by using the skill embedding space as action space of a high-level policy. Concretely, instead of learning a policy over actions a ∈ A we learn a policy π θ ( z | s t ) that outputs skill embeddings, which we decode into action sequences using the learned skill decoder a t i , … , a t + H - 1 i   ~   p ( a i | z ) 1 . We execute these actions for H steps before sampling the next skill from the high-level policy”; see also Pertsch, Page 5, Algorithm 1, Steps 4-7, “for every H environment steps do / z t ~ π ( z t | s t ) / s t ' ~ p ( s t + H | s t ,   z t ) / D ← D ∪ { s t , z t , r ~ s t , z t , s t ' } ”); and perform an operation of decomposing a task in units of episodes into subtasks in units of n-steps (Pertsch, Page 5, Algorithm 1, Steps 4-7; Pertsch, Page 5, Paragraph 1, “We can cast the problem of learning the high-level policy into a standard MDP by replacing the action space A with the skill space Z, single-step rewards with H-step rewards r ~ = ∑ t = 1 H r t , and single-step transitions with H-step transitions s t + H ~ p ( s t + H | s t , z t ) . We can then use conventional model-free RL approaches to maximize the return of the high-level policy π θ ( z | s t ) ”; see also Pertsch, Page 6, Figure 3, The training data can be seen on top, each image performing a skill; the target tasks are shown below, each made up of a series of smaller tasks; for example in the “Kitchen Environment” the target task is to “manipulate a kitchen setup to reach a target configuration” and 4 smaller labeled tasks are seen in the image). Regarding claim 2, the rejection of claim 1 is incorporated, and further, Pertsch teaches wherein the skill regularized task decomposition model is configured to decompose the task into the subtasks by matching the subtasks to a plurality of skills in units of action sequences, and wherein the data augmentation model is configured to perform reinforcement learning by sharing a skill of the plurality of skills corresponding to a subtask among the plurality of subtasks (Pertsch, Page 5, Paragraph 1, “We can cast the problem of learning the high-level policy into a standard MDP by replacing the action space A with the skill space Z, single-step rewards with H-step rewards r ~ = ∑ t = 1 H r t , and single-step transitions with H-step transitions s t + H ~ p ( s t + H | s t , z t ) . We can then use conventional model-free RL approaches to maximize the return of the high-level policy π θ ( z | s t ) ”; see also Pertsch, Page 5, Algorithm 1, Steps 4-7; Each “H environment steps” are considered to be a subtask; and sampling a skill from the policy is considered to be matching the subtask to a skill; Pertsch, Page 6, Section 4.1, Paragraph 2, Lines 6-8, “The agent can transfer skills, such as traversing hallways or passing through narrow doors, but needs to learn to navigate a new maze layout for solving the downstream task”; Pertsch, Page 6, Section 4.1, Paragraph 3, Lines 5-7, “The agent can transfer skills like picking up, carrying and stacking blocks, but needs to perform a larger number of consecutive stacks than seen in the training data on a new environment with more blocks”; Pertsch, Page 6, Section 4.1, Paragraph 4, Lines 6-7, “The agent can transfer a rich set of manipulation skills, but needs to recombine them in new ways to solve the downstream task”; “transfer[ring]” the skills is considered to be “sharing” the skills). Regarding claim 3, the rejection of claim 1 is incorporated, and further, Pertsch teaches wherein when performing the skill embedding operation, the skill regularized task decomposition model is configured to map the 2n-step state-action pairs of offline data to a skill candidate space, and infer the 2n-step state-action pairs by implementing a mapped candidate vector (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”; Pertsch, Page 3, Section 3.1, Lines 1-2, “We assume access to a dataset D of pre-recorded agent experience in the form of state-action trajectories τ i = s 0 , a 0 , … , s T i , a T i ”; see also Pertsch, Page 4, Figure 2; The “2n-step state-action pairs” can be seen on the top left of the figure, the extracted skill is mapped to a “skill candidate space” represented by the “skill embedding Z” which is then passed to the “skill decoder” and the extracted skill is inferred; because the model still has the states of the “2n-step state-action pairs”, the model is considered to have inferred the “2n-step state-action pairs”). Regarding claim 8, the rejection of claim 1 is incorporated, and further, Pertsch teaches wherein the data augmentation model is configured to generate the imaginary demo by inferring data generated when performing a skill that is appropriate for a given task by implementing the skill regularized task decomposition model, and augment learning data by training by adding subtask information to an input value (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”; Pertsch, Page 4, Section 3.3, Lines 1-2, “To use the learned skill embedding for downstream task learning, we employ a hierarchical policy learning scheme by using the skill embedding space as action space of a high-level policy”; Pertsch, Page 5, Algorithm 1; Pertsch, Page 6, Section 4.1, Paragraph 2, Lines 6-8, “The agent can transfer skills, such as traversing hallways or passing through narrow doors, but needs to learn to navigate a new maze layout for solving the downstream task”; Pertsch, Page 6, Section 4.1, Paragraph 3, Lines 5-7, “The agent can transfer skills like picking up, carrying and stacking blocks, but needs to perform a larger number of consecutive stacks than seen in the training data on a new environment with more blocks”; Pertsch, Page 6, Section 4.1, Paragraph 4, Lines 6-7, “The agent can transfer a rich set of manipulation skills, but needs to recombine them in new ways to solve the downstream task”; “Transfer[ring]” the skills is considered to be “adding subtask information to an input value”). Regarding claim 10, Pertsch teaches A processor-implemented (Pertsch, Page 6, Figure 3, Pertsch, Page 6, Section 4.1, Paragraph 2, Lines 1-5, “A simulated maze navigation environment based on the D4RL maze environment [33]. The task is to navigate a point mass agent through a maze between fixed start and goal locations. We use a planner-based policy to collect 85 000 goal-reaching trajectories in randomly generated, small maze layouts and test generalization to a goal-reaching task in a randomly generated, larger maze”; A person of ordinary skill in the art would recognize that the experiments of Pertsch require a generic computer, which provides evidence that the method is “processor-implemented”) reinforce learning method (Pertsch, Page 1, Abstract, Lines 9-12, “We propose a deep latent variable model that jointly learns an embedding space of skills and the skill prior from offline agent experience. We then extend common maximum-entropy RL approaches to use skill priors to guide downstream learning”), the method comprising: performing a skill regularized task decomposition based on a determined data quality (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”; Pertsch, Page 5, Algorithm 1; Pertsch, Page 5, Paragraph 1; Because the skills are sampled based on the skill expected to maximize the return, they are considered to be “based on a determined data quality”); and performing data augmentation by generating an imaginary demo (Pertsch, Page 3, Section 3.1, Paragraph 2, “The downstream learning problem is formulated as a Markov decision process (MDP) defined by a tuple { S , A , T , R , p , γ } of states, actions, transition probability, reward, initial state distribution, and discount factor. We aim to learn a policy π θ a s with parameters θ that maximizes the discounted sum of rewards J θ = E π ∑ t = 0 T - 1 γ t r t   where T is the episode horizon”; Pertsch, Page 6, Figure 3; Pertsch, Page 6, Section 4.1, Paragraph 4, Line 1, “A simulated kitchen environment based on Gupta et al. [27]”), wherein the performing of the skill regularized task decomposition comprises: performing a skill embedding operation by implementing 2n-step state-action pairs (Pertsch, Page 3, Section 3.2, Paragraph 2, Lines 1-2, “To learn a low-dimensional skill embedding space Z, we train a stochastic latent variable model p ( a i | z ) of skills using the offline dataset (see Fig. 2)”; Pertsch, Page 3, Section 3.1, Lines 1-2, “We assume access to a dataset D of pre-recorded agent experience in the form of state-action trajectories τ i = s 0 ,   a 0 ,   … , s T i ,   a T i ”; see also Pertsch, Page 4, Figure 2); performing a skill regularization operation by implementing n-step transitions including states, actions, rewards, and next states (Pertsch, Page 4, Section 3.3, Lines 1-6, “To use the learned skill embedding for downstream task learning, we employ a hierarchical policy learning scheme by using the skill embedding space as action space of a high-level policy. Concretely, instead of learning a policy over actions a ∈ A we learn a policy π θ ( z | s t ) that outputs skill embeddings, which we decode into action sequences using the learned skill decoder a t i , … , a t + H - 1 i   ~   p ( a i | z ) 1 . We execute these actions for H steps before sampling the next skill from the high-level policy”; see also Pertsch, Page 5, Algorithm 1, Steps 4-7, “for every H environment steps do / z t ~ π ( z t | s t ) / s t ' ~ p ( s t + H | s t ,   z t ) / D ← D ∪ { s t , z t , r ~ s t , z t , s t ' } ”); and performing an operation of decomposing a task in units of episodes into subtasks in units of n-steps (Pertsch, Page 5, Algorithm 1, Steps 4-7; Pertsch, Page 5, Paragraph 1, “We can cast the problem of learning the high-level policy into a standard MDP by replacing the action space A with the skill space Z, single-step rewards with H-step rewards r ~ = ∑ t = 1 H r t , and single-step transitions with H-step transitions s t + H ~ p ( s t + H | s t , z t ) . We can then use conventional model-free RL approaches to maximize the return of the high-level policy π θ ( z | s t ) ”; see also Pertsch, Page 6, Figure 3, The training data can be seen on top, each image performing a skill; the target tasks are shown below, each made up of a series of smaller tasks; for example in the “Kitchen Environment” the target task is to “manipulate a kitchen setup to reach a target configuration” and 4 smaller labeled tasks are seen in the image). Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Pertsch in view of Nam et al., SKILL-BASED META-REINFORCEMENT LEARNING, 04/25/2022, https://arxiv.org/pdf/2204.11828, hereinafter referred to as "Nam". Regarding claim 5, the rejection of claim 1 is incorporated. Pertsch does not explicitly teach wherein when performing the skill regularization operation, the skill regularized task decomposition model is configured to map the n-step transitions to a task candidate space, infer with a same task when the data is above a reference quality, solved by a same skill, and infer with another task when the data is below the reference quality. Nam teaches wherein when performing the skill regularization operation, the skill regularized task decomposition model is configured to map the n-step transitions to a task candidate space, infer with a same task when the data is above a reference quality, solved by a same skill, and infer with another task when the data is below the reference quality (Nam, Page 5, Section 4.3, Paragraph 1, Lines 7-9, “we encode this set of transitions into a target task embedding e * ~ q ( e | c * ) . By conditioning our meta-trained high-level policy on this encoding, we can rapidly narrow its skill distribution to skills that solve the given target task: π ( z | s , e * ) ”; Nam, Page 4, Paragraph 1, “Specifically, PEARL leverages the meta-training tasks for learning a task encoder q(e|c). This encoder takes in a small set of state-action-reward transitions c and produces a task embedding e. This embedding is used to condition the actor π(a|s,z) and critic Q(s,a,e). In PEARL, actor, critic and task encoder are trained by jointly maximizing the obtained reward and the policy’s entropy H”). It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the reinforcement learning method of Pertsch to include mapping the transitions to a task candidate space and making inferences as taught by Nam. The motivation to do so would have been that the policy of Nam that results from the task embeddings is more efficient and is able to achieve high success rates (Nam, Page 5, Section 4.3, Paragraph 2). Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pertsch et al., Demonstration-Guided Reinforcement Learning with Learned Skills, 07/21/2021, https://arxiv.org/pdf/2107.10253 discloses a method that learns a set of reusable skills from large offline datasets of prior experience collected across tasks and then proposes an algorithm for demonstration-guided reinforcement learning that leverages demonstrations by following demonstrated skills instead of actions. Rana et al., Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics, 11/04/2022, https://arxiv.org/pdf/2211.02231 discloses a method for exploring the skill space using state-conditioned generative models to directly bias the high-level agent towards only sampling skills relevant to a given state based on prior experience, and enables skill adaptation enabling downstream reinforcement learning agents to adapt to unseen task variations. Adeniji et al., SKILL-BASED REINFORCEMENT LEARNING WITH INTRINSIC REWARD MATCHING, 10/17/2022, https://arxiv.org/pdf/2210.07426v2 discloses intrinsic reward matching which uses a skill discriminator to match the intrinsic and downstream task rewards and determines the optimal skill for an unseen task without environment samples. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY CLARKE SIPPEL whose telephone number is (571)272-3270. The examiner can normally be reached Monday - Friday, 7:30 a.m. - 4:30 p.m. ET.. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571)272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /M.C.S./Examiner, Art Unit 2122 /KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Read full office action

Prosecution Timeline

Oct 17, 2023
Application Filed
Jun 26, 2026
Non-Final Rejection mailed — §101, §102, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

17/824,556
Patent 12670387
SYSTEM, METHOD, AND COMPUTER-READABLE MEDIA FOR LEAKAGE CORRECTION IN GRAPH NEURAL NETWORK BASED RECOMMENDER SYSTEMS
4y 1m to grant Granted Jun 30, 2026
17/908,025
Patent 12664398
SYSTEM, METHOD AND NON-TRANSITORY COMPUTER READABLE MEDIUM
3y 9m to grant Granted Jun 23, 2026
17/727,861
Patent 12657427
Systems, Methods, and Computer Program Products for Determining Uncertainty from a Deep Learning Classification Model
4y 1m to grant Granted Jun 16, 2026
17/551,475
Patent 12632779
HYPERPARAMETER SELECTION USING BUDGET-AWARE BAYESIAN OPTIMIZATION
4y 5m to grant Granted May 19, 2026
17/945,623
Patent 12626098
METHOD AND SYSTEM FOR CREATING AN ENSEMBLE OF NEURAL NETWORK-BASED CLASSIFIERS THAT OPTIMIZES A DIVERSITY METRIC
3y 8m to grant Granted May 12, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
52%
Grant Probability
86%
With Interview (+33.6%)
3y 9m (~1y 0m remaining)
Median Time to Grant
Low
PTA Risk
Based on 21 resolved cases by this examiner. Grant probability derived from career allowance rate.

Ready to respond to this office action?

IP Author drafts your complete OA response — amendments, arguments, and claim strategy — in minutes, not days.

Start Drafting Your Response →

Data sourced from USPTO Patent File Wrapper (daily) and Office Action Archive (weekly). Analysis generated by IP Author.

Data: USPTO Patent File Wrapper (daily) • Office Action Archive (weekly) • 89M+ prosecution events

© 2026 IP Author — Browse Office Actions • Examiners • Companies • Firms

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month