DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on January 23rd, 2026 has been entered.
Response to Arguments
Applicant's arguments filed January 23rd, 2026 have been fully considered but they are not persuasive.
Applicant’s arguments with respect to the “shared weights” limitations on pages 8-10 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant argues that there is a point of distinction between a multi-headed forecast network which is specialized and different from the general predictive structure disclosed by Ring (Remarks, pg. 7-8). However, this argument treats the phrase “multi-headed forecast network” as if it imposes a specialized architecture not actually required by the claim or supported by Applicant’s own disclosure. Applicant’s Figure 1A shows a state input S provided to a single common network body that branches into multiple forecast outputs f1(S), f2(S), f3(S), and f4(S). Nothing in that depicted “multi-headed” structure requires a special forecasting technique, a special head construction, or anything more than a common network portion producing plural output branches. Thus, under the broadest reasonable interpretation consistent with the specification, a “multi-headed forecast network” is simply a network having shared upstream processing and multiple forecast-specific outputs. Applicant’s attempt to distinguish Ring on the ground that Ring is not “directed to” a multi-headed forecast network therefore elevates an implementation label over the actual claimed structure. Implementing Ring’s plural forecasts using that ordinary multi-output structure would have predictably yielded the claimed arrangement.
Applicant’s arguments regarding the forecast ID are not persuasive (Remarks, 11-12). Applicant points to an embodiment in which forecast IDs g1 through g4 are supplied as an input vector, with only one value turned ‘on,’ to request a particular forecast. That example may illustrate one possible implementation of a forecast ID, but it does not limit the claims to that particular format, encoding, or example. The claims do not require one-hot encoding, nor require forecast IDs labeled g1 through g4, and do not require the specific red/green/yellow block example described in the specification. Under the broadest reasonable interpretation, a forecast ID is simply information that identifies, selects, or distinguishes a particular forecast. Applicant’s cited passage therefore does not establish a patentable distinction; it merely describes one embodiment falling within the broader claim language.
Applicant’s arguments regarding the skill ID are unpersuasive for the same reason (Remarks, 11-12). Applicant relies on an embodiment in which skill IDs k1 through k4 are supplied to a network to compute forecast values for different skills, such as run-to-door, walk-to-door, skip-to-door, or crawl-to-door. But the claims are not limited to those particular skills, to that notation, or to that specific input format. The cited passage shows that a skill ID may identify which skill, action, policy, or behavior-conditioned forecast is being evaluated. Ring’s action and policy information likewise identifies the behavior or action-conditioned process associated with the forecast. Applicant has not identified claim language requiring the narrower embodiment described in the specification, and the argument appears to improperly import exemplary disclosure into the claims rather than addressing the full scope of the claimed forecast ID and skill ID limitations.
Applicant submits that Ring does not fairly teach a “learned reduced vector representation (Remarks, pg. 12-13).” Ring teaches embedding state information into a learned reduced vector representation. In Ring’s specification ¶ 0123, Ring explains that the learning agent must learn the values of forecast predictions. In Ring’s specification ¶ 0126, Ring creates, optimizes, and evaluates forecasts incrementally and uses forecast values to split states into sets. In Ring’s specification ¶ 0127, Ring states that the N forecast values form a set of features and that, to evaluate those features as a state representation, they are combined with the agent’s observations into a feature vector. In Ring’s specification ¶ 0131, Ring further explains that states are aggregated into classes according to what the forecasts can disambiguate and that the feature vector consists of one binary feature per class. Thus, Ring does not merely disclose a raw vector of observation functions; Ring teaches a forecast-derived feature vector that represents state information using learned forecast values and aggregated state classes. This reasonably teaches a learned reduced vector representation.
Examiner’s Interpretation
Sener teaches that the representation function corresponds to shared network weights. In Section 1, Sener explains that multi-task learning commonly uses a “parametrized hypothesis class” that “shares some parameters across tasks,” and that those parameters are learned by solving an optimization problem (Sener, pg. 1, ¶2). In Section 2, Sener identifies the relevant architecture as hard parameter sharing, where “a subset of the parameters is shared between tasks while other parameters are task-specific,” and expressly states that the paper focuses on hard parameter sharing with gradient-based optimization for deep MTL (Sener, pg. 2, ¶1). In Section 3, Sener formalizes this by defining each task model as having shared parameters θsh and task-specific parameters θt, i.e., some parameters are shared between tasks and some are task-specific. Sener then links those shared parameters to the representation function in Section 3.3, Efficient Optimization for Encoder-Decoder Architectures, where the task model is decomposed as ft(g(x; θsh); θt), and Sener states that g is “the representation function shared by all tasks,” while ft are task-specific functions taking that representation as input (Sener, pg. 5, ¶2).
Because the shared representation function is written as g(x; θsh), it is the portion of the neural network parameterized by θsh. In a neural-network implementation, those learned shared parameters are the weights/biases of the shared layers. Sener’s concrete architectures confirm this interpretation: in Section 4.1, MultiMNIST, Sener states that it treats “all layers except the last” as the representation function g and uses two fully connected layers as task-specific functions (Sener, pg. 6, ¶5).
Accordingly, Sener teaches sharing weights in all but a last layer. The shared representation function corresponds to the common lower network layers whose parameters are shared among tasks, while the independent fully connected layers correspond to task-specific output heads.
In claim 7 and claim 13, the weights are stated to be shared among the forecasts (i.e. and the method further comprises sharing, among each of the plurality of forecasts, weights of the multi-headed forecast network, in all but a last layer of the multi-headed forecast network)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6, 13, 15, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over US Pre-Grant Patent 2016/0012338 (Ring et al; Ring) in view of “End-to-End Multi-Task Learning with Attention,” arXiv, 2018, Liu et al; Liu, further in view of “Multi-Task Learning as Multi-Objective Optimization,” arXiv, 2018, Sener et al; Sener.
Regarding claim 1:
Ring teaches:
1. A multi-headed forecast network performing a multi-headed forecast method of creating artificial intelligence in machines and computer-based software applications, the method comprising: receiving input from the environment as state information into a multi-forecast network;
(Ring, ¶0128, Figs. 1a and 1b)
“FIGS. 1a and 1b illustrate two grid worlds, for training and testing an agent in accordance with an embodiment of the present invention [i.e. A multi-headed forecast network performing a multi-headed forecast method of creating artificial intelligence in machines and computer-based software applications,]. FIG. 1a is a cross shaped corridor 105 with 23 positions (92 states) and a reward 110 (not visible to the agent) in one of three identical-looking arms. The fourth arm can potentially be distinguished by the agent and used for orientation. FIG. 1b is an 82-position (328 state) world with 7 identical-looking rooms 115; each with two exits, one marked by a protruding wall 120 (dot invisible to the agent). In FIGS. 1a and 1b, the agent has two actions: go forward or rotate 90 degrees left (|A|=z). In each case, the state space consists of position and orientation, so |S|=4p, where P is the number of positions. The agent observes just one bit, namely whether it has a wall immediately in front of it (|O|=1), and is rewarded for visiting an (invisible) goal position 110 [i.e. receiving input from the environment as state information into a multi-forecast network;].”
2. and outputting, from the multi-headed forecast network a plurality of forecasts based on the input received from the environment as state information, each of the plurality of forecasts corresponding to a different state information feature, wherein:
(Ring, ¶0126, Algorithm 1)
“Forecasts were created, optimized, and evaluated in an incremental process detailed in Algorithm 1, below, starting with forecast f.sup.1 [i.e. and outputting, from the multi-headed forecast network a plurality of forecasts based on the input] For simplicity, all agent observations in all the tests are binary, and the algorithm begins with a vector of observation functions O where each function produces a binary value in each state, oεO:S.fwdarw.{0,1} [i.e. received from the environment as state information, each of the plurality of forecasts corresponding to a different state information feature,].”
3. the state information feature includes a physical element of an observable object;
(Ring, ¶0128, Fig. 1a)
“FIGS. 1a and 1b illustrate two grid worlds, for training and testing an agent in accordance with an embodiment of the present invention. FIG. 1a is a cross shaped corridor 105 with 23 positions (92 states) and a reward 110 (not visible to the agent) in one of three identical-looking arms. The fourth arm can potentially be distinguished by the agent and used for orientation [i.e. the state information feature includes a physical element of an observable object;].”
Ring does not explicitly teach:
1. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts corresponding to each of the different state information features
2. and weights of the multi-forecast network in all but a last layer of the multi-forecast network are shared among each of the plurality of forecasts.
Liu teaches:
1. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts corresponding to each of the different state information features
(Liu, pg. 4, Sect. 3.1, ¶7)
“MTAN consists of two components: a single shared network, and K task-specific attention networks. The shared network can be designed based on the particular task (in our case, SegNet, for image-to-image predictions) [i.e. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts], whilst each task specific network consists of a set of attention modules, which link with the shared network. The attention modules apply a soft attention mask to the shared network, to determine the importance of each feature for the particular task.”
Neither Ring nor Liu teaches:
1. and the method further comprises sharing, among each of the plurality of forecasts, weights of the multi-headed forecast network, in all but a last layer of the multi-headed forecast network.
Sener teaches:
1. and the method further comprises sharing, among each of the plurality of forecasts, weights of the multi-headed forecast network, in all but a last layer of the multi-headed forecast network.
(Sener, pg. 6, Sect. 4.1, ¶6)
“We treat all layers except the last as the representation function and put two fully-connected layers as task-specific functions [i.e. and the method further comprises sharing, among each of the plurality of forecasts, weights of the multi-headed forecast network, in all but a last layer of the multi-headed forecast network].”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to modify Ring, Liu, and Sener. Using shared lower layers would allow the network to learn common state representations once and reuse those learned features across the plurality of forecasts, while separate last-layer heads would preserve the ability to produce distinct forecast outputs. This is a predictable application of Sener and Liu’s known multi-task neural-network design to Ring’s known plurality of forecast outputs.
The combination would have provided recognized technical benefits. Sharing the common layers reduces redundant computation and reduces the number of independently trained parameters relative to maintaining a fully separate network for each forecast. It also allows training signals from the plurality of forecasts to improve the shared representation of the underlying state information. At the same time, keeping the last layer forecast-specific maintains specialization for each individual forecast. Thus, the proposed combination does not change Ring’s principle of operation; it merely implements Ring’s multiple forecasts using a known and suitable neural-network architecture for related predictive tasks.
Accordingly, it would have been obvious to combine Ring with Liu, as further supported by Sener, to obtain a multi-headed forecast network in which the plurality of forecasts share weights in the common lower layers while using forecast-specific weights in the last layer. Ring supplies the plurality of forecasts based on state/action information, Liu supplies the shared multi-task neural-network architecture, and Sener expressly teaches the all-but-last-layer sharing arrangement. The resulting system is no more than the predictable use of a known shared-representation or multi-head neural-network architecture to implement multiple related forecast outputs.
Regarding claim 6:
Ring, Liu, and Sener teach the method of claim 1.
Ring teaches:
1. further comprising inputting at least one of a plurality of skill IDs and a plurality of forecast IDs to provide a hybrid network, wherein the plurality of forecasts is output for a set of similar skills or forecasts based, respectively on the plurality of skill IDs and the plurality of forecast IDs.
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, [i.e. further comprising inputting] but not limited to, observation signals (O), actions (A), calculated features (Φ) as above [i.e. one of a plurality of skill IDs], existing forecasts (F) [i.e. and a plurality of forecast IDs] and existing policies (P). In a step 510, the process may build a forecast [i.e. wherein the plurality of forecasts is output].”
“In a step 515, the agent may perform an action a in A, possibly according to a policy p in P, associated with a forecast fin F [i.e. to provide a hybrid network].” In a step 520, the agent may evaluate the environment upon completion of the action a, possibly involving receiving and/or updating its observation signals (O), calculated features (Φ) and/or estimated (predicted) values for forecasts (F) [i.e. for a set of similar skills or forecasts based, respectively on the plurality of skill IDs and the plurality of forecast IDs].”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Liu and Sener. The motivation is the same as claim 1.
Regarding claim 13:
Ring teaches:
1. A multi-forecast network performing a method of creating artificial intelligence in machines and computer-based software applications, the method comprising: receiving input from the environment as state information into a multi-forecast network;
(Ring, ¶0128, Figs. 1a and 1b)
“FIGS. 1a and 1b illustrate two grid worlds, for training and testing an agent in accordance with an embodiment of the present invention [i.e. A forecast network method of creating artificial intelligence in machines and computer-based software applications, the method comprising,]. FIG. 1a is a cross shaped corridor 105 with 23 positions (92 states) and a reward 110 (not visible to the agent) in one of three identical-looking arms. The fourth arm can potentially be distinguished by the agent and used for orientation. FIG. 1b is an 82-position (328 state) world with 7 identical-looking rooms 115; each with two exits, one marked by a protruding wall 120 (dot invisible to the agent). In FIGS. 1a and 1b, the agent has two actions: go forward or rotate 90 degrees left (|A|=z). In each case, the state space consists of position and orientation, so |S|=4p, where P is the number of positions. The agent observes just one bit, namely whether it has a wall immediately in front of it (|O|=1), and is rewarded for visiting an (invisible) goal position 110 [i.e. : receiving input from the environment as state information into a multi-forecast network;].”
2. receiving additional input from at least one of forecast IDs, skill IDs and parameter values;
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O) [i.e. and parameter values;], actions (A), calculated features (Φ) as above [i.e. skill IDs], existing forecasts (F) [i.e. receiving additional input from at least one of forecast IDs] and existing policies (P).”
3. embedding the additional input into a learned reduced vector representation before being inputted to the multi-forecast network;
(Ring, ¶0123)
“And besides these, there is also the agent's estimate of the ideal value {circumflex over (f)}(s), because, just as with PSRs and TD Nets, the learning agent must learn the values of those predictions.”
(Ring, ¶0126)
“Forecasts were created, optimized, and evaluated in an incremental process detailed in Algorithm 1, below, starting with forecast f.sup.1. For simplicity, all agent observations in all the tests are binary, and the algorithm begins with a vector of observation functions O [i.e. embedding the additional input into a learned reduced vector representation before being inputted to the multi-forecast network;] where each function produces a binary value in each state, oεO:S.fwdarw.{0,1}.”
4. and outputting a forecast for each learned reduced vector representation based on the input received from the environment as state information,
(Ring, ¶0127)
“The N ideal values for forecasts f.sup.1 to f.sup.N form a set of features whose quality is evaluated. To evaluate a set of features as a state representation for a reinforcement-learning agent, they are combined with the agent's observations into a feature vector which is used to compute the following three measurements: (1) The mean-squared error (MSE) between the true value function {circumflex over (V)} (computed with a perfect model and full state information) and {circumflex over (V)}, the best linear approximation of V based on the feature vector [i.e. based on the input received from the environment as state information,]. This value is designated “MSE” in the graphs. (2) The average value of each state according to the true value function for policy {circumflex over (π)}.sub.f, where {circumflex over (π)}.sub.f is the best policy that can be computed as a linear function of the feature vector.”
(Ring, ¶0131)
“That is, each state belongs to exactly one class, and two states belong to the same class if and only if they cannot be distinguished by any PSR test (left) or forecast (right). In these graphs, the feature vector consists of one binary feature per class, where each feature value is 1 in exactly those states that belong to the class [i.e. and outputting a forecast for each learned reduced vector representation].”
5. outputting a plurality of state information feature forecasts, each of the plurality of state information forecasts corresponding to a different state information feature;
(Ring, ¶0126, Algorithm 1)
“Forecasts were created, optimized, and evaluated in an incremental process detailed in Algorithm 1, below, starting with forecast f.sup.1 [i.e. outputting a plurality of state information feature forecasts,] For simplicity, all agent observations in all the tests are binary, and the algorithm begins with a vector of observation functions O where each function produces a binary value in each state, oεO:S.fwdarw.{0,1} [i.e. each of the plurality of state information forecasts corresponding to a different state information feature;].”
6. and inputting at least one of a plurality of skill IDs and a plurality of forecast IDs
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O), actions (A), calculated features (Φ) as above [i.e. and inputting at least one of a plurality of skill IDs], existing forecasts (F) [i.e. and a plurality of forecast IDs] and existing policies (P).”
7. to provide a hybrid network, wherein the plurality of forecasts is output for a set of similar skills or forecasts based, respectively on the plurality of skill IDs and the plurality of forecast IDs,
(Ring, ¶0146)
“In a step 515, the agent may perform an action a in A, possibly according to a policy p in P, associated with a forecast fin F [i.e. to provide a hybrid network].” In a step 520, the agent may evaluate the environment upon completion of the action a, possibly involving receiving and/or updating its observation signals (O), calculated features (Φ) and/or estimated (predicted) values for forecasts (F) [i.e. wherein the plurality of forecasts is output for a set of similar skills or forecasts based, respectively on the plurality of skill IDs and the plurality of forecast IDs].”
8. wherein the state information feature includes a physical element of an observable object;
(Ring, ¶0128, Fig. 1a)
“FIGS. 1a and 1b illustrate two grid worlds, for training and testing an agent in accordance with an embodiment of the present invention. FIG. 1a is a cross shaped corridor 105 with 23 positions (92 states) and a reward 110 (not visible to the agent) in one of three identical-looking arms. The fourth arm can potentially be distinguished by the agent and used for orientation [i.e. the state information feature includes a physical element of an observable object;].”
Ring does not explicitly teach:
1. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts corresponding to each of the different state information features;
2. and weights of the multi-forecast network in all but a last layer of the multi-forecast network are shared among each forecast provided as output.
Liu teaches:
1. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts corresponding to each of the different state information features;
(Liu, pg. 4, Sect. 3.1, ¶7)
“MTAN consists of two components: a single shared network, and K task-specific attention networks. The shared network can be designed based on the particular task (in our case, SegNet, for image-to-image predictions) [i.e. a single one of the multi-headed forecast network predicts the physical element for each of the forecasts], whilst each task specific network consists of a set of attention modules, which link with the shared network. The attention modules apply a soft attention mask to the shared network, to determine the importance of each feature for the particular task.”
Neither Ring nor Liu explicitly teach:
1. and the method further comprises sharing, among each forecast provided as output, and weights of the multi-forecast network in all but a last layer of the multi-forecast network are shared among each forecast provided as output.
Sener teaches:
(Sener, pg. 6, Sect. 4.1, ¶6)
“We treat all layers except the last as the representation function and put two fully-connected layers as task-specific functions [i.e. and the method further comprises sharing, among each forecast provided as output, and weights of the multi-forecast network in all but a last layer of the multi-forecast network are shared among each forecast provided as output].”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Sener and Liu. The motivation is the same as claim 1.
Regarding claim 15:
The combination of Ring, Liu, and Sener teaches the method of claim 13.
Sener teaches:
1. wherein weights or parameters of the multi-forecast network in all but a last layer of the network are shared among each of the plurality of state information feature forecasts.
(Sener, pg. 6, Sect. 4.1, ¶6)
“We treat all layers except the last as the representation function and put two fully-connected layers as task-specific functions [i.e. wherein weights or parameters of the multi-forecast network in all but a last layer of the network are shared among each of the plurality of state information feature forecasts].”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Liu and Sener. The motivation is the same as claim 1.
Regarding claim 19:
The combination of Ring, Liu, and Sener teaches the method of claim 13.
Ring teaches:
1. wherein the additional input includes a plurality of skill IDs.
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O), actions (A), calculated features (Φ) as above [i.e. wherein the additional input includes a plurality of skill IDs], existing forecasts (F) and existing policies (P).”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring, Liu, and Sener. The motivation is the same as claim 7.
Claims 7-8, 10, 12 are rejected under 35 U.S.C. 103 as being unpatentable over US Pre-Grant Patent 2016/0012338 (Ring et al; Ring) in view of “Multi-Task Learning as Multi-Objective Optimization,” arXiv, 2018, Sener et al; Sener.
Regarding claim 7:
Ring teaches:
1. A multi-input forecast network performing a multi- input forecast method of creating artificial intelligence in machines and computer-based software applications, the method comprising: receiving input from the environment as state information into the multi-input forecast network;
(Ring, ¶0128, Figs. 1a and 1b)
“FIGS. 1a and 1b illustrate two grid worlds, for training and testing an agent in accordance with an embodiment of the present invention [i.e. A multi-input forecast method of creating artificial intelligence in machines and computer-based software applications,]. FIG. 1a is a cross shaped corridor 105 with 23 positions (92 states) and a reward 110 (not visible to the agent) in one of three identical-looking arms. The fourth arm can potentially be distinguished by the agent and used for orientation. FIG. 1b is an 82-position (328 state) world with 7 identical-looking rooms 115; each with two exits, one marked by a protruding wall 120 (dot invisible to the agent). In FIGS. 1a and 1b, the agent has two actions: go forward or rotate 90 degrees left (|A|=z). In each case, the state space consists of position and orientation, so |S|=4p, where P is the number of positions. The agent observes just one bit, namely whether it has a wall immediately in front of it (|O|=1), and is rewarded for visiting an (invisible) goal position 110 [i.e. receiving input from the environment as state information into a multi-forecast network;].”
2. receiving additional input from at least one of forecast IDs, skill IDs and parameter values;
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, [i.e. further comprising inputting] but not limited to, observation signals (O), actions (A), calculated features (Φ) as above, existing forecasts (F) and existing policies (P) [i.e. and parameter values]. In a step 510, the process may build a forecast.”
3. and outputting from the multi-input forecast network, one or more forecasts based on the input received from the environment as state information and the additional input,
(Ring, ¶0126, Algorithm 1)
“Forecasts were created, optimized, and evaluated in an incremental process detailed in Algorithm 1, below, starting with forecast f.sup.1 [i.e. and outputting a plurality of forecasts]. For simplicity, all agent observations in all the tests are binary, and the algorithm begins with a vector of observation functions O where each function produces a binary value in each state, oεO:S.fwdarw.{0,1} [i.e. based on the input received from the environment as state information and the additional input]”
4. wherein each of the one or more forecasts corresponds to a flagged one of the at least one forecast IDs, skill IDs and parameter values,
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O), actions (A), calculated features (Φ) as above, existing forecasts (F) and existing policies (P). In a step 510, the process may build a forecast [i.e. wherein each of the one or more forecasts corresponds to a flagged one of the at least one forecast IDs, skill IDs and parameter values,].”
Ring does not explicitly teach:
1. the method further comprises sharing, among each of the one or more forecasts, weights of the multi-input forecast network, in all but a last layer of the multi-input forecast network
Sener teaches:
1. the method further comprises sharing, among each of the one or more forecasts, weights of the multi-input forecast network, in all but a last layer of the multi-input forecast network
(Sener, pg. 6, Sect. 4.1, ¶6)
“We treat all layers except the last as the representation function and put two fully-connected layers as task-specific functions [i.e. the method further comprises sharing, among each of the one or more forecasts, weights of the multi-input forecast network, in all but a last layer of the multi-input forecast network].”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to modify Ring and Sener. Using shared lower layers would allow the network to learn common state representations once and reuse those learned features across the plurality of forecasts, while separate last-layer heads would preserve the ability to produce distinct forecast outputs. This is a predictable application of Sener’s known multi-task neural-network design to Ring’s known plurality of forecast outputs.
The combination would have provided recognized technical benefits. Sharing the common layers reduces redundant computation and reduces the number of independently trained parameters relative to maintaining a fully separate network for each forecast. It also allows training signals from the plurality of forecasts to improve the shared representation of the underlying state information. At the same time, keeping the last layer forecast-specific maintains specialization for each individual forecast. Thus, the proposed combination does not change Ring’s principle of operation; it merely implements Ring’s multiple forecasts using a known and suitable neural-network architecture for related predictive tasks.
Accordingly, it would have been obvious to combine Ring and Sener, to obtain a multi-headed forecast network in which the plurality of forecasts share weights in the common lower layers while using forecast-specific weights in the last layer. Ring supplies the plurality of forecasts based on state/action information, and Sener expressly teaches the all-but-last-layer sharing arrangement.
Regarding claim 8:
The combination of Ring and Sener teaches the method of claim 7.
Ring teaches:
1. wherein the additional input includes a plurality of forecast IDs, wherein the forecast outputted is a forecast value for the forecast ID supplied as an input.
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O), actions (A), calculated features (Φ) as above, existing forecasts (F) [i.e. a plurality of forecast IDs, wherein the forecast outputted is a forecast value for the forecast ID supplied as an input] and existing policies (P).”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Sener. The motivation is the same as claim 7.
Regarding claim 10:
The combination of Ring and Sener teaches the method of claim 7.
Ring teaches:
1. wherein the additional input includes a plurality of skill IDs.
(Ring, ¶0146)
“In the present invention, an autonomous agent may include, but not limited to, observation signals (O), actions (A), calculated features (Φ) as above [i.e. wherein the additional input includes a plurality of skill IDs], existing forecasts (F) and existing policies (P).”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Liu. The motivation is the same as claim 7.
Regarding claim 12:
The combination of Ring and Sener teaches the method of claim 7.
Ring teaches:
1. wherein the additional input includes a variable input parameter that affects behavior.
(Ring, ¶0145)
“In the present invention, an autonomous agent may include, but not limited to, binary observation signals (O), binary actions (A) [i.e. wherein the additional input includes a variable input parameter that affects behavior], calculated features (Φ), existing forecasts (F) and existing policies (P).”
One of ordinary skill in the art, at the time the invention was filed, would have been motivated to combine Ring with Sener. The motivation is the same as claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL JUSTIN BREENE whose telephone number is (571)272-6320. Examiner
interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-
based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO
Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached on 303-297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786 9199 (IN USA OR CANADA) or 571-272-1000.
/P.J.B./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights