DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending and are examined herein.
The previous grounds of rejection under 35 USC 101 and 112(b) are withdrawn in view of Applicant’s amendment.
Claims 1-20 are rejected under 35 USC 112(a).
Claims 1-9 and 11-20 are rejected under 35 USC 103.
Claim 10 is not rejected under 35 USC 102 or 103.
Response to Arguments
Applicant’s arguments filed 1/12/2026 have been fully considered, but are moot in view of the new grounds of rejection presented herein.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 14, and 19 recite “adjusting, by the one or more computing devices, one or more parameters of the respectively associated machine-learned task controller model based at least in part on an adaptive loss function that is configured to adjust the one or more parameters based on a magnitude of the feedback value.” Applicant identifies paragraphs [0029] and [0039] as providing support for the amendment. [0029] indicates that the feedback value could be an output of an objective or loss function or that it could be a loss signal backpropagated. That is, the feedback value is an output of the loss function. The loss function does not adjust the parameters based on a magnitude of the feedback value. [0039] describes the adaptive loss function, but likewise fails to provide support for adjusting parameters of the machine-learned task controller models. Dependent claims 2-13, 15-18 and 20 do not resolve the issue and are rejected with the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-9, and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over “Wierstra” (US 2019/0354868 A1) in view of “Yang” (Multi-Task Reinforcement Learning with Soft Modularization, arXiv:2003.13661v1).
Examiner Remark: Wierstra is not prior art under 35 USC 102(a)(2) by virtue of the exception under 35 USC 102(b)(2); however, Wierstra is prior art under 35 USC 102(a)(1) by virtue of its publication date.
Regarding claim 1, Wierstra teaches
A computer-implemented method for generating a machine-learned multitask model configured to perform a plurality of tasks, the method comprising: (Wierstra, Abstract, Figures 1A, 1B, and 3-4.)
obtaining, by one or more computing devices, a machine-learned multitask search model comprising a plurality of candidate nodes; (Wierstra, Figure 1A provides an example of a super neural network comprising a plurality of different layers. [0049] indicates that the layers may be convolutional layers, fully connected layers, or a combination of the above, each of which includes nodes/neurons. [0067-0069] indicates that the network may be iteratively trained. Any of these iterations could be interpreted as “obtaining” the network.)
obtaining, by the one or more computing devices, the plurality of tasks and one or more machine-learned task controller models associated with the plurality of tasks; (Wierstra, [0073] describes training task-specific path/sub-network selection. The algorithm which performs path selection is being interpreted as a the machine-learned task controller model as it is trained using reinforcement learning. See also Figure 4.);
for each task of the plurality of tasks: (Wierstra, [0067]-[0071]: "To configure the super neural network 110 for use in performing multiple machine learning tasks, the neural network system 100 trains the super neural network 110 on each of the tasks")
using, by the one or more computing devices, the machine-learned task controller model respectively associated with the task to generate a routing that specifies a subset of the plurality of candidate nodes of the machine-learned multitask search model for inclusion in a machine-learned task submodel for the corresponding task; (Wierstra, the task controller model is the selection mechanism. Figure 3, elements 302 and 304, [0089-0092]. The path specifies the routing. The nodes of the layers which are selected as part of the path are being interpreted as having been included in the submodel for that task.)
inputting, by the one or more computing devices, task input data associated with the task to the corresponding machine-learned task submodel to obtain a task output; (Wierstra, Figure 3, element 306, described at [0093-0096].)
generating, by the one or more computing devices using the task output, a feedback value based on an objective function; and (Wierstra, Figure 3, step 308, described at [0097] describes determining a reward (i.e., feedback value) based on a fitness measure (i.e., objective function).)
adjusting, by the one or more computing devices, one or more parameters of the respectively associated machine-learned task controller model (Wierstra, Figure 3, steps 310-314, described at [0098]-[0100])
Wierstra does not appear to explicitly teach (portion not taught by Wierstra emphasized by italics)
adjusting, by the one or more computing devices, one or more parameters of the respectively associated machine-learned task controller model based at least in part on an adaptive loss function that is configured to adjust the one or more parameters based on a magnitude of the feedback value.
However, Yang—directed to analogous art--teaches
adjusting, by the one or more computing devices, one or more parameters of the respectively associated machine-learned task controller model based at least in part on an adaptive loss function that is configured to adjust the one or more parameters based on a magnitude of the feedback value. (Yang, Abstract describers performing multi-task learning by using a routing network (analogous to the claims machine-learned task controller model) a base network (analogous to the machine learned multi-task search model). Section 4 provides the implementation details. Section 4, first paragraph indicates that the routing network and the base network are trained together. Section 4.2 provides an examples of the loss function that is used to perform the training. In particular the weights w depend on the temperature parameter (see section 3.1), meaning that the loss functions provided after equation (10) are adaptive. These are based at least on the computed J values (i.e., feedback values as made clear from equations (1) and (2)).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wierstra by Yang because the multi-task learning approach “improves the sample efficiency as well as the success rate over the baselines by a large margin. The advantage becomes more obvious when given more diverse tasks. This shows that soft modularization allows effective sharing and reusing network components across tasks, which opens up future opportunities to generalize the policy to unseen tasks in a zero-shot manner” (Yang, Conclusion).
Regarding claim 2, the rejection of claim 1 is incorporated herein. Furthermore, Wierstra teaches
wherein the method further comprises generating, by the one or more computing devices, the machine-learned multitask model, wherein the machine-learned multitask model comprises a combination of at least a subset of machine-learned task submodels of the plurality of machine-learned task submodels. (Wierstra, [0067] describes configuring/generating the super neural network which may perform a plurality of tasks and which comprises a plurality of layers, some of which are selected by each path/task.)
Regarding claim 3, the rejection of claim 2 is incorporated herein. Furthermore, Wierstra teaches
inputting, by the one or more computing devices, multitask training data associated with a machine-learned task submodel of the at least the subset of machine-learned task submodels to the machine-learned multitask model to obtain a multitask training output; and adjusting, by the one or more computing devices, one or more parameters of the machine- learned multitask model based at least in part on the multitask training output. (Wierstra, Figure 3, step 306, described at [0093-0096]. Note that at least stochastic gradient descent adjusts the parameters of the model based on a gradient obtained using the output of the model and an objective function.)
Regarding claim 4, the rejection of claim 1 is incorporated herein. Furthermore, Wierstra teaches
the feedback value comprises a reward value; and the objective function comprises a reinforcement learning reward function. (Wierstra, [0097] describes generating a reward value based on a fitness function. Any function that is used to generate a reward to be used in reinforcement learning (as the fitness function is) could be reasonably interpreted as a “reinforcement learning reward function”.)
Regarding claim 6, the rejection of claim 1 is incorporated herein. Furthermore, Wierstra teaches
for each task of the plurality of tasks:
inputting, by the one or more computing devices, the task input data associated with the task to the corresponding machine-learned task submodel to obtain the task output further comprises inputting, by the one or more computing devices, training data associated with the task to the corresponding machine-learned task submodel to obtain a training output; (Wierstra, Figure 3, element 306, described at [0093-0096].)
generating, by the one or more computing devices using the task output, the feedback value based on the objective function further comprises generating, by the one or more computing devices using the training output, a loss value based on a task loss function: and (Wierstra, [0097] indicates that the objective function may be a classification error function. The error would be the “loss value”.)
the method further comprises adjusting, by the one or more computing devices, the one or more parameters of at least one candidate node of the machine-learned multitask search model based on a plurality of loss values respectively associated with the plurality of tasks. (Wierstra, [0097-0098] indicates that the feedback values are used to determine the paths for the tasks. [0104] indicates that steps 304-314 may be iterated. This includes the training step 306, described at [0093-0096], where the parameters of the nodes are determined. Since this training is based on the paths for the tasks and the paths are based on the plurality of loss values, the training of the nodes is consequently also based on the plurality of loss values.)
Regarding claim 7, the rejection of claim 6 is incorporated herein. Furthermore, Wierstra teaches
the task input data comprises the training data; and the task output comprises the training output. (Wierstra, Figure 1B shows the input being a training point and the outputs being task outputs. This aspect is further described at [0094-0096].)
Regarding claim 8, the rejection of claim 6 is incorporated herein. Furthermore, Wierstra teaches
the task input data comprises image data, and (Wierstra, Figure 1B, element 160, [0037, 0044, 0073])
the task output comprises at least one of:
image classification data; (Wierstra, [0037, 0044, 0096])
image recognition data; (Wierstra, [0037, 0044, 0096])
object recognition data corresponding to one or more objects depicted in the image data; and (Wierstra, [0037])
object segmentation data.
Regarding claim 9, the rejection of claim 6 is incorporated herein. Wierstra does not appear to explicitly teach
a respective task weight is associated with each task of the plurality of tasks; and at least the objective function is configured to evaluate the task weight associated with the respective task.
However, Yang—directed to analogous art--teaches
a respective task weight is associated with each task of the plurality of tasks; and at least the objective function is configured to evaluate the task weight associated with the respective task. (Yang, section 4.2. describes performing multi-task learning by assigning a weight to each of M different tasks where the weights are used in the optimization objectives in equations (13) and (14).)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wierstra by Yang because the weights allow for the training to balance the learning across different tasks as described by Yang in Section 4.2., first paragraph.
Regarding claim 13, the rejection of claim 1 is incorporated herein. Furthermore, Wierstra teaches
wherein at least one of the plurality of tasks comprises:
an image generation task;
a sound signal description task, wherein the task output of the sound signal description task comprises data describing a sound signal; (Wierstra, [0038])
a text translation task, wherein the task output of the text translation task comprises a translation of text in a first natural language to a second natural language; or (Wierstra, [0039])
a control data generation task, wherein the task output of the control data generation task comprises control data for controlling an agent which operates in a real-world environment. (Wierstra, [0040])
Regarding claim 14, Wierstra teaches
A computing system, comprising: (Wierstra, [0006])
a machine-learned multitask model configured to generate a plurality of outputs for a respectively associated plurality of tasks, wherein the machine-learned multitask model comprises a plurality of candidate nodes, wherein each node of the plurality of candidate nodes is included in the machine-learned multitask model based at least in part on their inclusion in one or more of a plurality of machine-learned task submodels respectively associated with the plurality of tasks; (Wierstra, Figure 1A provides an example of a super neural network comprising a plurality of different layers. [0049] indicates that the layers may be convolutional layers, fully connected layers, or a combination of the above, each of which includes nodes/neurons. [0067-0069] indicates that the network may be iteratively trained. Any of these iterations could be interpreted as “obtaining” the network. Figure 3, elements 302 and 304, [0089-0092]. The path specifies the routing. The nodes of the layers which are selected as part of the path are being interpreted as having been included in the submodel for that task.)
one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising: (Wierstra, [0123])
The remainder of claim 14 is substantially similar to claim 1; claim 14 is rejected with the same rationale.
Regarding claim 15, the rejection of claim 14 is incorporated herein. Furthermore, Wierstra teaches
the first task input data and the second task input data comprises image data; the first task output comprises image classification data; and the second task output comprises object recognition data corresponding to one or more objects depicted in the image data. (Wierstra, Figure 1B, element 160, [0037, 0044, 0073]. Note in particular that [0037] indicates that the tasks may include distinct image processing tasks in which case the inputs for both tasks would be images.)
Regarding claim 16, the rejection of claim 14 is incorporated herein. Furthermore, Wierstra teaches
wherein each node of the plurality of nodes of the machine-learned multitask model is selected for inclusion in the one or more of the plurality of machine-learned task submodels by one or more associated machine-learned task controller models. (Wierstra, [0016-0019] indicates that many different candidate paths are attempted for each of the tasks with the candidate paths being chosen randomly in some embodiments. Consequently, in the normal course of operation, each of the nodes would eventually be selected at least once for inclusion in one of the paths. See MPEP 2112.02(I).)
Regarding claim 17, the rejection of claim 14 is incorporated herein. Furthermore, Wierstra teaches
the machine-learned multitask model comprises one or more neural networks: and each of the plurality of nodes comprises at least one of: one or more neurons: or one or more functions. (Wierstra, Abstract, Figures 1A-1B. A neural network necessarily comprises neurons which in turn necessarily include at least an activation function as would be understood by a person of ordinary skill in the art.)
Regarding claim 18, the rejection of claim 14 is incorporated herein. Furthermore, Wierstra teaches
the first task input data is processed by at least a first node of the machine-learned multitask model; and the second task input data is processed by the first node of the machine-learned multitask model. (Wierstra, [0060] indicates that different paths may share one or more modular neural networks, in which case each of the nodes of that network is used when executing the inputs for the two tasks that share the modular neural network.)
Regarding claim 19, Wierstra teaches
One or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising: (Wierstra, [123])
The remainder of claim 19 is substantially similar to claim 14 and is rejected with the same rationale.
Regarding claim 20, the rejection of claim 19 is incorporated herein. Claim 20 recites substantially similar subject matter to claims 15 and 16 (together) and is rejected with the same rationale.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over “Wierstra” (US 2019/0354868 A1) in view of “Yang” (Multi-Task Reinforcement Learning with Soft Modularization, arXiv:2003.13661v1), in view of “Guo” (Learning to Branch for Multi-Task Learning, arXiv:2006.01895v2).
Regarding claim 5, the rejection of claim 1 is incorporated herein. Wierstra does not appear to explicitly teach
wherein adjusting, by the one or more computing devices, the one or more parameters of the respectively associated machine-learned task controller model based at least in part on the feedback value comprises backpropagating the objective function through the corresponding machine-learned task submodel to reach the respectively associated machine-learned task controller model.
However, Guo—directed to analogous art--teaches
wherein adjusting, by the one or more computing devices, the one or more parameters of the respectively associated machine-learned task controller model based at least in part on the feedback value comprises backpropagating the objective function through the corresponding machine-learned task submodel to reach the respectively associated machine-learned task controller model. (Guo, Abstract, and sections 3.1.-3.4 describe determining the multi-task topology (i.e., routes through the neural network corresponding to different tasks) by performing backpropagation.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wierstra by Guo because “In this work, we propose a tree-structured network design space that can automatically learn how to branch a network such that the overall multi-task loss is minimized” (see Guo, section 1. Introduction).
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over “Wierstra” (US 2019/0354868 A1) in view of “Yang” (Multi-Task Reinforcement Learning with Soft Modularization, arXiv:2003.13661v1), in view of “Klinger” (US 2020/0125955 A1).
Regarding claim 11, the rejection of claim 1 is incorporated herein. Furthermore, Wierstra teaches
... generate a first routing that specifies a first subset of the plurality of the candidate nodes;...generate a second routing that specifies a second subset of the plurality of the candidate nodes; and (Wierstra, the task controller model is the selection mechanism. Figure 3, elements 302 and 304, [0089-0092]. The path specifies the routing. The nodes of the layers which are selected as part of the path are being interpreted as having been included in the submodel for that task. [0082] indicates that different tasks have different paths.)
the first subset of the plurality of candidate nodes and the second subset of the plurality of candidate nodes contain at least one shared candidate node. (Wierstra, [0060] indicates that the paths may share one or more modular neural networks.)
Wierstra does not appear to explicitly teach the plurality of controller models:
the one or more machine-learned task controller models comprise a plurality of task controller models respectively associated with the plurality of tasks;
a first machine-learned task controller model associated with a first task is used to generate a first routing that specifies a first subset of the plurality of the candidate nodes;
a second machine-learned task controller model associated with a second task is used to generate a second routing that specifies a second subset of the plurality of the candidate nodes; and
However, Klinger—directed to analogous art--teaches
the one or more machine-learned task controller models comprise a plurality of task controller models respectively associated with the plurality of tasks; a first machine-learned task controller model associated with a first task is used to generate a first routing that specifies a first subset of the plurality of the candidate nodes; a second machine-learned task controller model associated with a second task is used to generate a second routing that specifies a second subset of the plurality of the candidate nodes; and (Klinger, [0038] describes using N agents (i.e., “task controller models”) to determine routes through a neural network for corresponding N tasks and in which different routes for different tasks may re-use/share portions of the neural network.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wierstra by Klinger because this allows for models to be trained to maximize accuracy and efficiency as described by Klinger at [0038].
Regarding claim 12, the rejection of claim 11 is incorporated herein. Wierstra does not appear to explicitly teach
wherein, for each task of the plurality of tasks, the one or more parameters of the respectively associated machine-learned task controller model are adjusted based at least in part on an evaluation of a loss function.
However, Klinger—directed to analogous art--teaches
wherein, for each task of the plurality of tasks, the one or more parameters of the respectively associated machine-learned task controller model are adjusted based at least in part on an evaluation of the adaptive loss function. (Klinger, [0038] as applied to claim 11 also indicates that the routing agents are trained based on a computed reward or penalty (i.e., a loss function).)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have combined these references in this way for the same reasons given above with respect to claim 11.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Markus A Vasquez whose telephone number is (303)297-4432. The examiner can normally be reached Monday to Friday 10AM to 2PM PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARKUS A. VASQUEZ/ Primary Examiner, Art Unit 2121