DETAILED ACTION
This action is written in response to the RCE filed 4/9/26 . The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
In view of the Applicant’s latest claim amendments, all outstanding rejections under §§ 101 and 112 are withdrawn.
The Applicants argue that the previous art of record does not anticipate or render obvious the claims as currently amended. The Examiner provides updated prior art rejections below necessitated by the current amendments.
Subject Matter Eligibility
In determining whether the claims are subject matter eligible, the examiner has considered and applied guidance from MPEP § 2106. The examiner finds that the independent claims are directed to the practical application of training and implementing a hierarchical ensemble machine learning model.
Furthermore, the combination of steps performed in the recited method cannot be practically performed as a mental process.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, eg, In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-2, 4-5, 8-8, 11-13, 15-16 and 18-20 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over the corresponding claims of US patent application 17/827400. Although the claims at issue are not identical, they are not patentably distinct from each other for the reasons outlined in the table below. This is a provisional rejection because the claims of the copending application have not been allowed and issued.
This application – 17/827364
Copending application – 17/827400
1. A method implemented on at least one processor, a memory, and a communication platform for integrated targeting, comprising:
1. A method implemented on at least one processor, a memory, and a communication platform for predicting user segment via machine learning, comprising:
constructing an expert hierarchy comprising an initial expert layer and one or more augmented expert layers,
wherein the initial expert layer has a plurality of initial experts and an augmented expert layer has at least one augmented expert for prediction, wherein each expert in a layer of the expert hierarchy is trained based on training data directed thereto and an expert output from an expert at any lower layer of the expert hierarchy so that an expert at any of the one or more augmented expert layers augments the expertise of experts at any lower layer of the expert hierarchy;
creating an initial expert layer of an expert hierarchy with a plurality of initial experts trained for prediction;
deriving at least one augmented expert layer for the expert hierarchy with one or more augmented experts at each of the at least one augmented expert layer, wherein each augmented expert at any of the at least one augmented expert layer augments the plurality of initial experts and is trained, via machine learning for the prediction,using training data and outputs from all of experts from a lower expert layer, wherein each of the outputs is generated by each of the experts from the lower expert layer based on the training data as an input to the expert;
wherein each of the initial experts and the augmented experts is a machine learning model;
Id.
training each initial expert based on first training data;
Id.
training each augmented expert at a first augmented expert layer based on second training data and predictions generated by all trained initial experts based on the second training data;
3. The method of claim 1, wherein when the expert hierarch has multiple augmented expert layers, each augmented expert at an augmented expert layer higher than a first augmented expert layer additionally augments any augmented expert at a lower augmented expert layer.
training each augmented expert at a next augmented expert layer based on third training data and predictions generated by all trained experts at all expert levels lower than the next augmented expert layer based on the third training data; and
generating an augmented expert at a first augmented expert layer based on first training data and a plurality of predictions generated by the plurality of initial experts based on the first training data, and
generating an augmented expert at an augmented expert layer above the first augmented expert layer based on second training data, a plurality of predictions generated by the plurality of initial experts based on the second training data, and one or more predictions generated by respective one or more augmented experts at any lower augmented expert layer based on the second training data
obtaining a nonlinear integration model, via machine learning, for combining expert predictions from the trained initial and augmented experts in the expert hierarchy hierarch based on an input to generate an integrated expert prediction in response to the input.
7. The method of claim 1, further comprising:
accessing a nonlinear integration model provided for integrating different expert predictions; combining, in accordance with the nonlinear integration model, expert predictions from the initial and augmented experts in the expert hierarchy generated based on the input; and
generating an integrated expert prediction based on a result of the combining.
As illustrated in the table above, every limitation in this application has a corresponding equivalent or more specific limitation in the ‘400 application. Thus, the ‘400 application anticipates this claim.
Independent claims 8 and 15 recite an analogous medium and system, respectively.
The correspondence in dependent claims is illustrated in the table below.
This application – 17/827364
Copending application – 17/827400
2. The method of claim 1, wherein the initial experts of the initial expert layer are heterogeneous experts.
2. The method of claim 1, wherein the plurality of initial experts are heterogeneous experts.
4. The method of claim 1, wherein the step of obtaining the nonlinear integration model comprises:
configuring the nonlinear integration model via a plurality of parameters; and
learning values of the plurality of parameters via machine learning to capture nonlinear relationships among the experts in the expert hierarchy.
These further limitations are not in the claims of the ‘400 application. However, they are taught by the Jordan reference, see claim mapping in §102 rejections infra. At the time of filing, it would have been obvious to a person of ordinary skill to combine the techniques disclosed by Jordan with the method of ‘400 because this would improve classification performance through iterative training.
5. The method of claim 4, wherein the nonlinear integration model corresponds to an artificial neural network (ANN) with the plurality of parameters related to the ANN, including embeddings of the ANN.
These further limitations are not in the claims of the ‘400 application. However, they are taught by the Jordan reference, see claim mapping in §102 rejections infra. At the time of filing, it would have been obvious to a person of ordinary skill to combine the techniques disclosed by Jordan with the method of ‘400 because this would improve classification performance through iterative training.
As illustrated in the table above, every claim limitation above in this application has a corresponding equivalent or more specific limitation in the ‘400 application. Thus, the ‘400 application anticipates these claim. (Obviousness instead of anticipation applies where noted above.)
Dependent claims 9-13 and 16-20 are analogous to dependent claims 2 and 4-5.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Jordan (Jordan, Michael I., and Robert A. Jacobs. "Hierarchical mixtures of experts and the EM algorithm." Neural computation 6, no. 2 (1994): 181-214.)
Shazeer (Shazeer, Noam, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. "Outrageously large neural networks: The sparsely-gated mixture-of-experts layer." arXiv preprint arXiv:1701.06538. 2017. Cited by Applicant in IDS dated 6/24/25.)
Zhou (Zhou, Zhi-Hua. "When semi-supervised learning meets ensemble learning." Frontiers of Electrical and Electronic Engineering in China 6.1 (2011): 6-16.)
Claims 1, 4-8, 11-15 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Jordan and Zhou.
Regarding claims 1, 8 and 15, Jordan discloses a method implemented on at least one processor, a memory, and a communication platform for integrated targeting, comprising:
constructing an expert hierarchy comprising an initial expert layer and one or more augmented expert layers, wherein the initial expert layer has a plurality of initial experts and each of the one or more augmented expert layers has at least one augmented expert for prediction, wherein each expert in a layer of the expert hierarchy is trained based on training data directed thereto and an expert output from an expert at any lower layer of the expert hierarchy so that an expert at any of the one or more augmented expert layers augments the expertise of experts at any lower layer of the expert hierarchy; …
P. 3, fig. 1 (reproduced below). Caption: “A two-level hierarchical mixture of experts. To form a deeper tree, each expert is expanded recursively into a gating network and a set of sub-experts.”
PNG
media_image1.png
598
742
media_image1.png
Greyscale
The Examiner notes that the Applicant provides no explicit definition for the terms ‘expert’, ‘augment’ or ‘augmented expert’.
The examiner interprets ‘augmented expert’ according to its broadest reasonable interpretation as encompassing every expert network above the first (ie leaf) layer. According to Jordan, this hierarchical mixture-of-experts (HME) architecture “yielded a modest improvement” over previous efforts. Thus, the additional hierarchical experts have ‘augmented’ the performance of the base experts. Each expert in the described model is a model which is trained in a supervised manner, ie using labeled training data.
P. 2 “The algorithms that we discuss in this paper are supervised learning algorithms.” (Emphasis added.) The Examiner notes that supervised learning means that the models are trained with labeled training data, ie “training data directed thereto”.
obtaining a nonlinear integration model, via machine learning, for combining expert predictions from the trained initial and augmented experts in the expert hierarchy based on an input to generate an integrated expert prediction in response to the input.
PP. 3-4, eqns. 1-3. “Then the ith output of the top-level gating network is the “softmax” function of the ξi”. (Emphasis added.) The Examiner notes that the softmax function is non-linear.
Cf. Applicant’s specification at [0063], including eqn. (8), which includes the softmax function.
Zhou discloses the following further limitation which Jordan does not disclose:
wherein each of the initial experts and the augmented experts is a machine learning model;
See generally p. 8, sec. 3, discussing classifier combination for semi-supervised classification.
training each initial expert based on first training data;
P. 8, second col., “Process At first, we train two initial learners h01 and h02 using L which contains l labeled examples. Then, h01 selects u number of unlabeled instances from U to label, and puts these newly labeled examples into the data set σ2 which contains all the examples in L; at the same time, h02 selects u number of unlabeled instances from U to label, and puts these newly labeled examples into the data set σ1 which contains all the examples in L. Then, h11 and h12 are trained from σ1 and σ2, respectively. After that, h11 selects u number of unlabeled instances to label, and uses these newly labeled examples to update σ2; while h12 also selects u number of unlabeled instances to label, and uses these newly labeled examples to update σ1. Such a process is repeated for a pre-set number of learning rounds.”
‘initial experts’ :: initial learners h01 and h02
training each augmented expert at a first augmented expert layer based on second training data and predictions generated by all trained initial experts based on the second training data;
‘augment experts’ :: h11 and h12
‘predictions generated by all trained initial experts’ :: σ1 and σ2
training each augmented expert at a next augmented expert layer based on third training data and predictions generated by all trained experts at all expert levels lower than the next augmented expert layer based on the third training data; and
Id. “Such a process is repeated for a pre-set number of learning rounds.”
At the time of filing, it would have been obvious to a person of ordinary skill to apply the hierarchical semi-supervised learning technique described by Zhou with the hierarchical system of Jordan because this can help improve prediction results, particularly when there are very few labeled data (see generally Zhou, p. 10, sec. 4.)
Regarding independent claim 8, the recited computing hardware (ie a machine readable and non-transitory medium) are inherent throughout the Jordan disclosure.
Regarding claims 4, 11 and 18, Jordan discloses the further limitation wherein the step of obtaining the nonlinear integration model comprises:
configuring the nonlinear integration model via a plurality of parameters; and
P. 2, “Expert network (i; j) produces its output μij as a generalized linear function of the input x: μij = f(Ujix) … where Uij is a weight matrix and f is a fixed continuous nonlinearity.” (Emphasis added.)
learning values of the plurality of parameters via machine learning to capture nonlinear relationships among the experts in the expert hierarchy.
Id.
Regarding claims 5 and 12, Jordan discloses the further limitations wherein the nonlinear integration model corresponds to an artificial neural network (ANN) with the plurality of parameters related to the ANN, including embeddings of the ANN.
P. 8, “EM is an iterative approach to maximum likelihood estimation. Each iteration of an EM algorithm is composed of two steps: an Estimation (E) step and a Maximization (M) step. The M step involves the maximization of a likelihood function that is redefined in each iteration by the E step. If the algorithm simply increases the function during the M step, rather than maximizing the function, then the algorithm is referred to as a Generalized EM (GEM) algorithm. The Boltzmann learning algorithm (Hinton & Sejnowski, 1986) is a neural network example of a GEM algorithm. GEM algorithms are often significantly slower to converge than EM algorithms.” (Emphasis added.)
Regarding claims 6, 13 and 19, Jordan discloses the further limitation wherein the step of learning comprises:
initializing the values of the plurality of parameters;
Initialization of weight parameters is inherent in any neural network system. Cf. pp. 17-18, describing on-line supervised training: “In this section we present the equations for the on-line algorithm. These equations involve an update not only of the parameters in each of the networks,6 but also the storage and updating of an inverse covariance matrix for each network”
See also footnote 6: “Note that in this section we use the term “parameters" for the variables that are traditionally called “weights" in the neural network literature. We reserve the term “weights" for the observation weights”.
See also p. 18, “λ was initialized to 0.99”.
receiving fourth training data having pairs of data, wherein each of the pair includes an input feature vector and a corresponding ground truth label; and’
Id.
See also p. 21, ‘supervised learning’.
for each of the pairs in the fourth training data, receiving the outputs from the experts in the expert hierarchy based on the input feature vector in the pair, generating an integrated output of the received outputs based on current values of the plurality of parameters of the nonlinear function, determining a loss based on a discrepancy between the integrated output and the ground truth label in the pair, updating the current values of the plurality of parameter based on the loss, and repeating the steps of receiving, generating, determining, and updating until a convergence condition is satisfied.
P. 17, “It can be shown, however, that R(t)ij is an estimate of the inverse Hessian of the least-squares cost function (Ljung & Sooderstrom, 1986), thus Equation 32 is in fact a stochastic approximation to a Newton-Raphson method rather than a gradient method.” (Emphasis added.)
See also p. 17, first paragraph, discussing least squares learning, which inherently involves a lost / cost function.
Iterative repetition of the described on-line training algorithm is assumed. See p. 18, fig. 5, (reproduced below) illustrating relative error vs. training epochs (ie number of training cycles).
PNG
media_image2.png
554
818
media_image2.png
Greyscale
Regarding claims 7, 14 and 20, Jordan discloses the further limitation comprising:
receiving the input;
P. 2, “These networks receive the vector x as input”.
sending the input to the experts at different layers of the expert hierarchy to facilitate each of the experts in the expert hierarchy to generate a prediction based on the input; and
P. 3, fig. 1 (reproduced below). Caption: “A two-level hierarchical mixture of experts. To form a deeper tree, each expert is expanded recursively into a gating network and a set of sub-experts.”
PNG
media_image1.png
598
742
media_image1.png
Greyscale
combining, via the nonlinear integration model, predictions generated by the experts in the expert hierarchy to output an integrated expert prediction in response to the input.
Id.
Claims 2, 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Jordan, Zhou and Shazeer.
Regarding claims 2, 9 and 16, Shazeer discloses the following further limitation which Jordan/Zhou does not disclose wherein the initial experts of the initial expert layer are heterogeneous experts.
P. 13, “Experiments: We trained a set of models with identical architecture (the MoE-256 model described in Appendix C), using different values of wimportance and wload. We trained each model for 10 epochs, then measured perplexity on the test set.” (Emphasis added.)
The Examiner notes that the experts described are heterogenous with respect to their models (ie their model weights).
At the time of filing, it would have been obvious to a person of ordinary skill to combine the features disclosed by Shazeer with the Jordan/Zhou system. There are only two possibilities: either all expert models in a MOE model are identical, or they are heterogeneous. If all experts comprise identical models, there would be no advantage gained from creating an ensemble (ie a mixture) thereof.
Additional Relevant Prior Art
The following references were identified by the Examiner as being relevant to the disclosed invention, but are not relied upon in any particular prior art rejection:
Eto discloses a hierarchical mixture of experts system. See eg fig. 2 and abstract. (US 2021/0150388 A1)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Gonzales whose telephone number is (571) 270-3837. The examiner can normally be reached on Monday-Friday 7 a.m. to 4 p.m. MT. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang, can be reached at (571) 270-7092.
Information regarding the status of an application may be obtained from the USPTO Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
/Vincent Gonzales/Primary Examiner, Art Unit 2124
Prosecution Insights