DETAILED ACTION
This action is responsive to application filed on 05/14/2024, in which claims 1-19 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4-12, and 14-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Schwarz et al. (US 20210117786 A1)
With respect to claims 1, 10, and 11:
Schwarz teaches:
A method of performing continual learning of a plurality of tasks (para 0004), wherein the method is performed by one or more processors executing instructions from a memory that are configured to cause the one or more processors to perform the method (Para 0103), the method comprising:
A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 (para 0104).
An electronic device comprising: one or more processors; and a memory storing instructions configured to cause the one or more processors to (Para 0097):
learning a first model based on training data corresponding to a current task in a set of tasks (Para 0038 teaches training active subnetwork; also see para 0048, 0049); learning a second model based on information on the current task and information on a previous learning task in the set of tasks (Para 0039 teaches knowledge network; also see para 0050); and resetting the first model (Fig. 3 teaches re-setting values of active subnetwork).
With respect to claims 2, and 12:
Schwarz teaches the method of claim 1, and the electronic device of claim 11; and Schwarz further teaches wherein the learning of the first model is based on a reinforcement learning algorithm (See paras 0055, 0071)
With respect to claims 4, and 14:
Schwarz teaches the method of claim 1, and the electronic device of claim 11; and Schwarz further teaches further comprising: storing the information on the current task in a first buffer (Para 0050 teaches training the knowledge subnetwork while holding the trained values of active parameters of the active subnetwork, thus storing the information in buffer); and maintaining a second buffer comprising the information on the previous learning task (Para 0056 teaches holding the knowledge parameters; note, here active and knowledge, both parameters are being held, thus first and second buffers).
With respect to claims 5, and 15:
Schwarz teaches the method of claim 4, and the electronic device of claim 14; and Schwarz further teaches wherein the learning of the second model comprises: receiving the information on the current task from the first buffer (Para, “[0050] The system 100 then trains the knowledge subnetwork 140 while holding the trained values of the active parameters of the active subnetwork 170 fixed. In particular, the system trains the knowledge subnetwork 140 to generate knowledge outputs that match active outputs generated by the trained active subnetwork for training inputs for the new task. When the neural network has already been trained on one or more earlier tasks, the system 100 trains the knowledge subnetwork to attain acceptable performance at matching the active outputs while maintaining acceptable performance on the earlier machine learning tasks. Thus, once trained, the knowledge subnetwork 140 can generate high-quality outputs for both the new task and the earlier tasks. This training of the knowledge subnetwork 140 will be referred to as the “compress” phase of training and is depicted in FIG. 1B.” Also, para 0012 teaches “[0012] The described system maintains effective performance on earlier tasks even after being trained on new tasks even though the number of parameters of the neural network does not increase as it is trained on new tasks. This is because the trained active subnetwork is used to “distill” knowledge learned from a new task into the knowledge subnetwork while preventing the knowledge subnetwork from “forgetting” earlier tasks.”); and receiving the information on the previous learning task from the second buffer (Para 0012 teaches “[0012] The described system maintains effective performance on earlier tasks even after being trained on new tasks even though the number of parameters of the neural network does not increase as it is trained on new tasks. This is because the trained active subnetwork is used to “distill” knowledge learned from a new task into the knowledge subnetwork while preventing the knowledge subnetwork from “forgetting” earlier tasks.”).
With respect to claims 6, and 16:
Schwarz teaches the method of claim 5, and the electronic device of claim 15; and Schwarz further teaches further comprising, when the learning of the second model is completed: updating the second buffer based on the first buffer (Para 0012 teaches “[0012] The described system maintains effective performance on earlier tasks even after being trained on new tasks even though the number of parameters of the neural network does not increase as it is trained on new tasks. This is because the trained active subnetwork is used to “distill” knowledge learned from a new task into the knowledge subnetwork while preventing the knowledge subnetwork from “forgetting” earlier tasks.” Also, para 0057 teaches “[0057] Thus, during the progress phase of training, both the knowledge subnetwork 140 and the active subnetwork 170 process each received training input, but only the values of the active parameters are updated while the values of the knowledge parameters are fixed.” Also, see para 0066, 0073, etc.…); and resetting the first buffer (See, Fig. 3, para 0078, 0080, etc.…).
With respect to claims 7, and 17:
Schwarz teaches the method of claim 6, and the electronic device of claim 16; and Schwarz further teaches teaches wherein the updating of the second buffer comprises: storing, in the second buffer, a portion of the information on the current task stored in the first buffer (See, Fig. 4; paras 0083, 0087, 0094, etc.…)
With respect to claims 8, and 18:
Schwarz teaches the method of claim 1, and the electronic device of claim 11; and Schwarz further teaches wherein the learning of the second model comprises: determining a first loss function based on the information on the current task (See, Claim 4, paras 0089, 0091-0094) ; determining a second loss function based on the information on the previous learning task (See, Claim 4, paras 0089, 0091-0094); and performing the learning of the second model based on the first loss function and the second loss function (Para 0089 teaches “[0089] More specifically, the system can minimize a loss function that includes: (i) a first term that measures a similarity between knowledge outputs and active outputs for a given new training input and (ii) one or more second terms that penalize the knowledge subnetwork for having values of the knowledge parameters that deviate from the values of the knowledge parameters after the previous round of training.”).
With respect to claim 9:
Schwarz teaches an inference method performed by one or more processors executing instructions configured to cause the one or more processors to perform the method, the method comprising: receiving input data (See, para 0039); and outputting a task, the task corresponding to the input data among tasks in a set of tasks, by inputting the input data to a continual learning model (Para 0039 teaches “[0039] Once the neural network has been trained on a particular task, the knowledge output, i.e., the output of the knowledge subnetwork 140, can be used as the output of the neural network for inputs corresponding to that task (and to any earlier tasks in the training sequence on which the neural network has already been trained).”), wherein the continual learning model is trained based on a reinforcement learning model that is distinct from the continual learning model (Para 0038 teaches “[0038] The neural network that is being trained to perform the multiple tasks includes a knowledge subnetwork 140 and an active subnetwork 170. Both the knowledge subnetwork 140 and the active subnetwork 170 are configured to generate the same kind of output, i.e., the type of output that is required for the multiple tasks for which the neural network is being trained. For example, both subnetworks can generate a probability distribution over a set of possible classifications for each received input for supervised learning tasks or can generate a policy output, value output, or both for reinforcement learning tasks. The output generated by the knowledge subnetwork 140 will be referred to in this specification as a knowledge output and the output generated by the active subnetwork 170 will be referred to in this specification as an active output.” Note: here, active subnetwork and knowledge subnetwork are two distinct networks; also see para 0052, 0055 etc.… ), and wherein the reinforcement learning model is reset each time learning of a task in the set of tasks is completed (Para 0078 teaches “[0078] The system re-sets the values of the active parameters (step 304). For example, the system can re-set the values to predetermined initial values, e.g., zero or a fixed small positive number. As another example, the system can re-set the values by generating initial values for the active parameters in accordance with a conventional machine learning parameter initialization technique, e.g., by sampling a value for each of the active parameters from a pre-determined distribution. Importantly, the system does not re-set the values of the knowledge parameters and maintains the values of those parameters from the earlier training.”).
With respect to claim 19:
Schwarz teaches an electronic device comprising: one or more processors (See, para 0098); and a memory configured storing instructions configured to cause the one or more processors to (See, para 0097): receive input data (See, para 0039); and output a task, the task corresponding to the input data among a set of tasks, by inputting the input data to a continual learning model (Para 0039 teaches “[0039] Once the neural network has been trained on a particular task, the knowledge output, i.e., the output of the knowledge subnetwork 140, can be used as the output of the neural network for inputs corresponding to that task (and to any earlier tasks in the training sequence on which the neural network has already been trained).”), wherein the continual learning model is trained based on a reinforcement learning model that is distinct from the continual learning model (Para 0038 teaches “[0038] The neural network that is being trained to perform the multiple tasks includes a knowledge subnetwork 140 and an active subnetwork 170. Both the knowledge subnetwork 140 and the active subnetwork 170 are configured to generate the same kind of output, i.e., the type of output that is required for the multiple tasks for which the neural network is being trained. For example, both subnetworks can generate a probability distribution over a set of possible classifications for each received input for supervised learning tasks or can generate a policy output, value output, or both for reinforcement learning tasks. The output generated by the knowledge subnetwork 140 will be referred to in this specification as a knowledge output and the output generated by the active subnetwork 170 will be referred to in this specification as an active output.” Note: here, active subnetwork and knowledge subnetwork are two distinct networks; also see para 0052, 0055 etc.… ), and the reinforcement learning model is reset each time learning of a task in the set of tasks is completed (Para 0078 teaches “[0078] The system re-sets the values of the active parameters (step 304). For example, the system can re-set the values to predetermined initial values, e.g., zero or a fixed small positive number. As another example, the system can re-set the values by generating initial values for the active parameters in accordance with a conventional machine learning parameter initialization technique, e.g., by sampling a value for each of the active parameters from a pre-determined distribution. Importantly, the system does not re-set the values of the knowledge parameters and maintains the values of those parameters from the earlier training.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Schwarz et al. in view of Gupta et al. (US 20230409387 A1)
With respect to claims 3, and 13:
Schwarz teaches the method of claim 1, and the electronic device of claim 11; and Schwarz further teaches wherein the learning of the second model comprises: performing knowledge distillation from the first model to the second model (Para 0012 teaches distilling knowledge from active subnetwork in to the knowledge subnetwork).
However, Schwarz doesn’t explicitly teach and performing behavioral cloning (BC) of the second model based on the information on the previous learning task.
Gupta teaches and performing behavioral cloning (BC) of the second model based on the information on the previous learning task (Para 0052 teaches using imitation or behavior cloning for learning; also see paras 0030, 0058)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to incorporate behavior cloning as disclosed by Gupta into the teachings of Schwarz.
One would be motivated to do so to self-tune model based on observed runtime behavior. (See, para 0063)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240331367 A1: “……method for continual learning of a plurality of tasks using a machine learning model comprising a deep neural network and a preferably compact shared task-attention module. Each task is associated with a mutually different learnable task-specific token including: rehearsing learned knowledge in the deep neural network to prevent the forgetting of previous tasks; transforming latent representations of the shared task-attention module towards a task distribution, using the learnable task-specific tokens, so that memory and computational use is limited, such as substantially insignificant; and using the task-specific tokens to reduce task interference and facilitate within-task and task-id prediction by the deep neural network.” (See, Abstract)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASEEM ASHRAF whose telephone number is (571)270-3948. The examiner can normally be reached Monday-Friday 09:30 A.M-06:00 P.M.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tariq Hafiz can be reached at 571-272-5350. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASEEM ASHRAF/Supervisory Patent Examiner, Art Unit 3621
Prosecution Insights