DETAILED ACTION
Remarks
This office action is issued in response to communication filed on 2/2/2026 . Claims 1-20 are pending in this Office Action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 2/2/2026 have been fully considered but they are not persuasive. The examiner respectfully traverses applicant’s arguments.
Applicant argues: “To sum up, claim 1 as amended proposes a simple, flexible, extensible parameter compositional approach that provides clear separation between task-agnostic and task-specific parameters, capable of improving MTRL and mitigating its challenges both within and beyond its typical settings. The proposed approach has the benefits of clear separation between task-agnostic and task-specific components, which not only directly provides what should be shared among tasks and how to share it and minimizes inter-task interferences/conflicts, inflexibility, and catastrophic forgetting caused by parameter entanglement in standard MTRL, stabilizing and improving MTRL, but also enables natural extension to both transfer learning and continual learning without resorting to complicated design or additional data. PaCo has demonstrated clear improvement over current state- of-the-art methods on standard benchmarks.”(Applicant’s arguments at page 18)
The examiner respectfully disagrees. There is nothing in the amended claim 1 that reflects those improvements as applicant asserts. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims (i.e. proposes a simple, flexible, extensible parameter compositional approach that provides clear separation between task-agnostic and task-specific parameters, capable of improving MTRL and mitigating its challenges both within and beyond its typical settings. The proposed approach has the benefits of clear separation between task-agnostic and task-specific components, which not only directly provides what should be shared among tasks and how to share it and minimizes inter-task interferences/conflicts, inflexibility, and catastrophic forgetting caused by parameter entanglement in standard MTRL, stabilizing and improving MTRL, but also enables natural extension to both transfer learning and continual learning without resorting to complicated design or additional data (bold text emphasis added)). See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Per MPEP 2106.05(a), “After the examiner has consulted the specification and determined that the disclosed invention improves technology, the claim must be evaluated to ensure the claim itself reflects the disclosed improvement in technology”. The examiner unable to find any details that reflects those improvements. Instead, the amended claim 1 still recited at a very high level of generality. The amended claim 1 still does not provide any details how the neural network operates to generate K base policy models and T task-identified parameters group. Accordingly, the amended claim 1 does not reflect the improvement in the technology as required by per MPEP 2106.05(a).
Applicant argues: “Applicant respectfully submits that claim 1 as amended clearly recites training a neural network through multi-task reinforcement learning based on the observation signals and the T instructions, which anchors the claim to the specific technical field of machine learning. Therefore, the entity executing the method for training a multi-task model through multi-task reinforcement learning is a hardware agent with a specific computing architecture, for example, an agent in a hardware environment, a logic gate circuit, an integrated circuit chip, a processor, etc., and by conducting multi-task reinforcement learning training on the neural network, claim 1 as amended processes observation signals obtained from actual environment by various hardware sensors (which may be included in the agent), and integrates application to the specific technical field. See the present disclosure, for example, at least par [0093-0094]) (Applicant’s argument at page 18-19)
The examiner respectfully disagrees. Simply adding a general purpose computer or computer components after the fact to an abstract idea does not integrate a judicial exception into a practical application or provide significantly more per MPEP 2106.05(f). the instant specification at par [0036] defines an agent as “[0036] The term "agent" may refer to any man-made entity that chooses actions in response to observations, including for example but not limited to a robot, to a simulated robot, and to a
software agent or "bot" and the like”. Since the term agent is not limited to just “a robot” as defined by the specification, the broadest reasonable interpretation of the term “hardware agent “ is a general purpose computer and therefore, using hardware agent to perform the abstract idea does not integrate the judicial exception into practical application or provide significantly more. The examiner notes that the amended claim 1 does not recite any “sensor” as applicant asserts.
Applicant’s remaining arguments with respect to claims are substantially encompassed in the argument above, therefore examiner responds with the same rationale as stated above. For at least the foregoing reasons, the examiner maintains 101 rejection.
Applicant's arguments filed on 2/2/26 with respect to rejection of claims under 35 USC 103 have been considered but are moot in view of the new ground of rejection.
Claim Objections
Claims 1, 7 and 15 are objected to because of the following informalities: these claims recite in part “…a task-agnostic parameter set shared across the tasks”. It appears applicant intends to recite ““…a task-agnostic parameter set shared across the T tasks” Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
2. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1:
Step 1: Statutory Category ?: Yes. Claim 1 recites a method which is a statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitations:
“acquiring observation signals observed for an environment by an agent; receiving T instructions each for instructing the agent to perform one of T tasks, T being a preset positive integer greater than 1”
is a mental processes that that can be performed in the human mind using observation, evaluation, judgment and opinion including with the help with a pen and paper.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 1 recites additional element of “generating K base policy models and T-task-identified parameter groups by performing training through multi-task reinforcement learning over a neural network based on the observation signals and the T instructions, wherein the K base policy models are represented by a task-agnostic parameter set shared across the tasks, K being a preset positive integer greater than 1, wherein the K base policy models are combinable, based on each of the T task-identified parameter groups, for generating respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks”. Using the Neural network to generate policy models without any details on how the neural network operates. This additional element amounts to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea. See MPEP 2106.05(f). The additional element of “Hardware-agent” amounts to no more than mere instructions to apply the exception using a generic computer.
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 1 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the Neural network is mere instructions to apply an exception . The hardware agent is at best the equivalent of merely adding the words “apply it” to the judicial exception. Even when considered in combination, the additional element does not provide an inventive concept, claim 1 therefore is ineligible.
Claim 2 recites additional element of “wherein the generating of K base policy models includes: generating a first parameter set and a second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions, wherein the first parameter set includes first parameters shared across tasks, and the second parameter set includes second parameters for identifying the T tasks corresponding to the T instructions, wherein upon the training, the first parameter set is generated from the first parameters to include K base parameter groups and the second parameter set is generated from the second parameters to include T task-identified parameter groups, each of the K base parameter groups being for instantiating the base policy model, and each of the T task-identified parameter groups being for combining the first parameters in the first parameter set, and wherein the K base parameter groups in the first parameter set are combinable by the second parameter set for generating T task parameter groups each for instantiating a corresponding one of the respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks”. This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 2 therefore is ineligible.
Claim 3 recites additional element of “wherein the K base parameter groups in the first parameter set are matrixed to correspond to K base parameter vectors, respectively, the T task-identified parameter groups in the second parameter set are matrixed to correspond to T task-identified compositional vectors, respectively, and the T task parameter groups are matrixed to correspond to T task parameter vectors, respectively”. This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 3 therefore is ineligible.
Claim 4 recites additional element of “wherein the generating of the first parameter set and the second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions includes: presetting K trainable base parameter vectors and T trainable compositional vectors according to the T instructions; and repeatedly performing a training loop until a loss function is minimized, to obtain the K base parameter vectors and the T task-identified compositional vectors so as to generate the T task parameter vectors”. This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 4 therefore is ineligible.
Claim 5 recites additional elements of “ wherein the performing of the training loop includes: generating T trainable task parameter vectors through linearly combining the K trainable base parameter vectors by the T trainable compositional vectors; and adjusting the K trainable base parameter vectors and the T trainable compositional vectors according to the loss function established based on the T trainable task parameter vectors and the observation signals, as updated trainable base parameter vectors and updated T trainable compositional vectors for next performing of the training loop” which is mathematical calculations which fall under the mathematical concepts grouping of the abstract idea. Claim 5 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 5 is not patent eligible.
Claim 6 recites additional elements of “ wherein the environment and the agent are from virtual scenarios or real scenarios” which is mental process that can be performed in the human mind using observation, evaluation, judgment and opinion. Claim 6 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 6 is not patent eligible.
Claim 7:
Step 1: Statutory Category ?: Yes. Claim 7 recites a method which is a statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitations:
“receiving a target instruction for instructing the agent to perform a target task, and determining, by the agent, a task-specific parameter group for the target task from T task-identified parameter group according to the target instructions and determining a group of base policy models for the target task from K base policy models according to the task-specific parameter group to obtain a task policy model for the target task from the multi-task model, wherein T and K are present positive integers greater than 1 and the k base policy models are represented by a task-agnostic parameter set shared across the tasks and the task policy model is operable for instructing the agent to perform the target task”
are mental processes that that can be performed in the human mind using observation, evaluation, judgment and opinion including with the help with a pen and paper.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 7 recites additional element of “Hardware-agent” amounts to no more than mere instructions to apply the exception using a generic computer.
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 7 does not include additional elements that are sufficient to amount to significantly more than judicial exception. The hardware agent is at best the equivalent of merely adding the words “apply it” to the judicial exception. Even when considered in combination, the additional element does not provide an inventive concept, claim 7 therefore is ineligible.
Claim 8 recites additional element of “wherein the determining a group of base policy models for the target task from K base policy models according to the task- specific parameter group, to obtain a task policy model for the target task from the multi-task model includes: combining a group of base parameter groups for the target task from K base parameter groups by the task-specific parameter group to obtain a task parameter group for instantiating the task policy model for the target task from the multi-task model, wherein each of the K base parameter groups is for instantiating the base policy model” which is mental process that can be performed in the human mind using observation, evaluation, judgment and opinion. Claim 8 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 8 is not patent eligible.
Claim 9 recites additional element of “wherein the K base parameter groups are matrixed to correspond to K base parameter vectors, respectively, the T task-identified parameter groups are matrixed to correspond to T task-identified compositional vectors, respectively, and the T task parameter groups are matrixed to correspond to T task parameter vectors, respectively, wherein the determining of the task-specific parameter group includes determining a task-specific compositional vector for the target task from the T task-identified compositional vectors according to the target instruction; and wherein the combining of the group of base parameter groups includes: linearly combining a group of base parameter vectors for the target task from the K base parameter vectors by the task-specific compositional vector to obtain a task-specific parameter vector for instantiating the task policy model for the target task from the multi-task model ” which is mathematical calculations . Claim 9 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 9 is not patent eligible.
Claim 10 merely recites a non-transitory computer readable storage medium stores a computer program for implementing the method of claim 1 and therebefore being rejected for the same rationale as indicates in the above rejection of claim 1. Furthermore, claim 10 recites the additional element of “non-transitory computer readable storage medium”. The additional element of non-transitory computer readable medium which is recited at the very high level of generality such that it amounts no more than mere instructions to apply the exception using generic computer component and at best the equivalent of merely adding the words “apply it” to the judicial exception. Even when considered in combination, the additional element does not provide an inventive concept, claim 10 therefore is ineligible.
Claim 11 recites additional elements of wherein the generating of K base policy models includes: generating a first parameter set and a second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions, wherein the first parameter set includes first parameters shared across tasks, and the second parameter set includes second parameters for identifying the T tasks corresponding to the T instructions, wherein upon the training, the first parameter set is generated from the first parameters to include K base parameter groups and the second parameter set is generated from the second parameters to include T task-identified parameter groups, each of the K base parameter groups being for instantiating the base policy model, and each of the T task-identified parameter groups being for combining the first parameters in the first parameter set, and wherein the K base parameter groups in the first parameter set are combinable by the second parameter set for generating T task parameter groups each for instantiating a corresponding one of the respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks”. This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 11 therefore is ineligible .
Claim 12 recites additional elements of “ wherein the K base parameter groups in the first parameter set are matrixed to correspond to K base parameter vectors, respectively, the T task-identified parameter groups in the second parameter set are matrixed to correspond to T task-identified compositional vectors, respectively, and the T task parameter groups are matrixed to correspond to T task parameter vectors, respectively” . This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 12 therefore is ineligible.
Claim 13 recites additional elements of “wherein the generating of the first parameter set and the second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions includes: presetting K trainable base parameter vectors and T trainable compositional vectors according to the T instructions; and repeatedly performing a training loop until a loss function is minimized, to obtain the K base parameter vectors and the T task-identified compositional vectors so as to generate the T task parameter vectors” . This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 13 therefore is ineligible.
Claim 14 recites additional elements of “wherein the performing of the training loop includes: generating T trainable task parameter vectors through linearly combining the K trainable base parameter vectors by the T trainable compositional vectors; and adjusting the K trainable base parameter vectors and the T trainable compositional vectors according to the loss function established based on the T trainable task parameter vectors and the observation signals, as updated trainable base parameter vectors and updated T trainable compositional vectors for next performing of the training loop” which is mathematical calculations which fall under the mathematical concepts grouping of the abstract idea. Claim 14 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 14 is not patent eligible.
Claim 15:
Step 1: Statutory Category ?: Yes. Claim 15 recites a device which is a statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitation:
“acquiring observation signals observed for an environment by an agent; receiving T instructions each for instructing the agent to perform one of T tasks, T being a preset positive integer greater than 1”
is a mental processes that that can be performed in the human mind using observation, evaluation, judgment and opinion including with the help with a pen and paper.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 15 recites additional element of “generating K base policy models and T-task-identified parameter groups by performing training through multi-task reinforcement learning over a neural network based on the observation signals and the T instructions, wherein the K base policy models are represented by a task-agnostic parameter set shared across the tasks, K being a preset positive integer greater than 1, wherein the K base policy models are combinable, based on each of the T task-identified parameter groups, for generating respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks”. Using the Neural network to generate policy models without any details on how the neural network operates. This additional element amounts to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea. See MPEP 2106.05(f).
The additional element of memory and processor which is recited at the very high level of generality such that it amounts no more than mere instructions to apply the exception using generic computer components .
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 15 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the Neural network is mere instructions to apply an exception and the memory and processor are at best the equivalent of merely adding the words “apply it” to the exception. Even when considered in combination, the additional element does not provide an inventive concept, claim 15 therefore is ineligible.
Claim 16 recites additional elements of “wherein the generating of K base policy models includes: generating a first parameter set and a second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions, wherein the first parameter set includes first parameters shared across tasks, and the second parameter set includes second parameters for identifying the T tasks corresponding to the T instructions, wherein upon the training, the first parameter set is generated from the first parameters to include K base parameter groups and the second parameter set is generated from the second parameters to include T task-identified parameter groups, each of the K base parameter groups being for instantiating the base policy model, and each of the T task-identified parameter groups being for combining the first parameters in the first parameter set, and wherein the K base parameter groups in the first parameter set are combinable by the second parameter set for generating T task parameter groups each for instantiating a corresponding one of the respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks”. This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 16 therefore is ineligible .
Claim 17 recites additional elements of “wherein the K base parameter groups in the first parameter set are matrixed to correspond to K base parameter vectors, respectively, the T task-identified parameter groups in the second parameter set are matrixed to correspond to T task-identified compositional vectors, respectively, and the T task parameter groups are matrixed to correspond to T task parameter vectors, respectively” This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 17 therefore is ineligible.
Claim 18 recites additional elements of “wherein the generating of the first parameter set and the second parameter set by performing training through multi-task reinforcement learning over the neural network based on the observation signals and the T instructions includes: presetting K trainable base parameter vectors and T trainable compositional vectors according to the T instructions; and repeatedly performing a training loop until a loss function is minimized, to obtain the K base parameter vectors and the T task-identified compositional vectors so as to generate the T task parameter vectors” This additional elements amount to mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform the abstract idea (See MPEP 2106.05(f)). Mere instructions to apply an exception does not provide an inventive concept . Claim 18 therefore is ineligible.
Claim 19 recites additional elements of “wherein the performing of the training loop includes: generating T trainable task parameter vectors through linearly combining the K trainable base parameter vectors by the T trainable compositional vectors; and adjusting the K trainable base parameter vectors and the T trainable compositional vectors according to the loss function established based on the T trainable task parameter vectors and the observation signals, as updated trainable base parameter vectors and updated T trainable compositional vectors for next performing of the training loop” which is mathematical calculations which fall under the mathematical concepts grouping of the abstract idea. Claim 19 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 19 is not patent eligible.
Claim 20 recites additional elements of “wherein the environment and the agent are from virtual scenarios or real scenario” which is mental process that can be performed in the human mind using observation, evaluation, judgment and opinion. Claim 20 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 20 is not patent eligible.
Allowable Subject Matter
Claims 2-6,8-9,11-14 and 16-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1,10 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Pascanu et al.(US Patent Application Publication 2020/0090048 A1, hereinafter “Pascanu”) and further in view of Hwang et al.(US Patent Application Publication 2021/0256374 A1, hereinafter “Hwang”
As to claim 1, 10 and 15, Pascanu teaches a method, non-transitory computer readable medium and system for training a multi-task model through multi-task reinforcement learning, including:
acquiring observation signals observed for an environment by an agent; receiving T instructions each for instructing the agent to perform one of T tasks, T being a preset positive integer greater than 1 (Pascanu Fig.3 and par [0042]-[0043] teaches receiving state data S sub t ); and
generating K base policy models and T task-identified parameter groups (Pascanu par [0042] teaches the integer index i is used to label the tasks) by performing training through multi-task reinforcement learning over a neural network based on the observation signals and the T instructions, wherein the K base policy models are represented by a task-agnostic parameter set shared across the tasks, K being a preset positive integer greater than 1 , (Pascanu par [0006] teaches the system may employ and / or generate for each task, at least one respective task policy which indicates how the worker(s) associated with that task should perform the task. Pascanu par [0025] teaches The workers (agents) may comprise neural networks; they may share some, all, or no weights with the shared policy network. The system may have a multicolumn architecture in which worker neural network modules define at least one column of neural network layers, and in which the shared policy network defines a second column of neural network layers .)
wherein the K base policy models are combinable, based on each of the T task-identified parameter groups, for generating respective task policy models for the T tasks to obtain the multi-task model for achieving the T tasks. (Pascanu par [0024] teaches the multitask neural network system may be configured to combine the output of an adaptive system for each task with the learned multitask policy to define a task policy for the task , to enable at least one worker to perform the corresponding learned task)
Pascanu fails to expressly teach wherein the K base policy models are represented by a task-agnostic parameter set shared across the tasks.
However, Hwang teaches wherein the K base policy models are represented by a task-agnostic parameter set shared across the tasks.( Hwang par [0012] teaches grouping a plurality of adaptive parameters of the plurality of tasks into a plurality of groups . Hwang par [0013] teaches the model parameter of the current task may be determined based on the shared parameter)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Pascanu and Hwang to achieve the claimed invention. One would have been motivated to make such combination to effectively prevent catastrophic forgetting.(Hwang par [0055])
5. Claims 7 is rejected under 35 U.S.C. 103 as being unpatentable over Pascanu, in view of Silver et al.(US Patent Application Publication 2020/0244707 A1, hereinafter “Silver”) and further in view of Hwang.
As to claim 7, Pascanu teaches an operating method of a multi-task model trained through multi-task reinforcement learning, the method being implemented by a hardware agent, and , including:
receiving a target instruction for instructing the agent to perform a target task, and determining , by the agent, a task-specific parameter group for the target task from T task-identified parameter groups according to the target instructions (Pascanu par [0043] teaches generate respective action data. The action data specifies for each of the agents 21-24 a respective action a sub t) , and
determining a group of base policy models for the target task from K base policy models according to the task-specific parameter group to obtain a task policy model for the target task from the multi-task model, wherein t and k are preset positive integers greater than 1 , the K base policy models are represented by a task-agnostic parameter set shared across the tasks and the task policy model is operable for instructing the agent to perform the target task. (Pascanu par [0044] teaches the computer transmits the action data to the respective agents 21-24 and once the agents 21-24 have performed the respective actions at on the environment, the multitask computer system receives reward data for each of the tasks)
Pascanu does not teach determining a group of base policy models for the target task from K base policy models according to the task-specific parameter group to obtain a task policy model for the target task from the multi-task model
However, Silver teaches determining a group of base policy models for the target task from K base policy models according to the task-specific parameter group to obtain a task policy model for the target task from the multi-task model.(Silver par [0074] teaches the system selects one or more policies from the pool of candidate action selection policies using the match making policy for the learner policy. The system can select any number of policies that is appropriate for the type of the particular task)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Pascanu and Silver to achieve the claimed invention. One would have been motivated to make such combination to effectively control an agent to perform tasks.(Silver par [0016])
Pascanu and Silver fail to expressly teach the K base policy models are represented by a task-agnostic parameter set shared across the tasks.
However, Hwang teaches the K base policy models are represented by a task-agnostic parameter set shared across the tasks. (Hwang par [0013] teaches the model parameter of the current task may be determined based on the shared parameter)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Pascanu, Silver and Hwang to achieve the claimed invention. One would have been motivated to make such combination to effectively prevent catastrophic forgetting.(Hwang par [0055])
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HIEN DUONG whose telephone number is (571)270-7335. The examiner can normally be reached Monday-Friday 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HIEN L DUONG/Primary Examiner, Art Unit 2147
Prosecution Insights