Notice of Pre-AIA or AIA Status
This Non-Final communication is in response to application no. 18/752,058 filed on 6/24/2024. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-20 are within the four statutory (a process, machine, manufacture or composition of
matter.) Claims 8-20 describe a machine and 1-7 describes a process.
With respect to claim 1:
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG
generating a path embedding representing a decision path comprising a set of touchpoints to obtain a defined outcome, each touchpoint comprising an electronic interaction between electronic devices; (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
generating an attention path embedding based on the path embedding …, the attention path embedding comprising a set of aggregated attention weights for the set of touchpoints in the decision path; (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
generating a set of touchpoint contribution values corresponding to the set of touchpoints based on the attention path embedding, a touchpoint contribution value from the set of touchpoint contribution values representing a level of contribution made by a touchpoint from the set of touchpoints to obtain the defined outcome; and (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application
Additional elements:
using an attention network of a machine learning model (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
providing a recommendation for a connections networking system based on the set of touchpoint contribution values. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception
The additional element “using an attention network of a machine learning model” is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
The additional element “providing…” adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
When considered in combination, these additional elements represent insignificant extra-solution activity and mere instructions to apply an expectation, which do not provide an inventive concept.
Therefore, claim 1 is ineligible.
With respect to claim 2:
Step 2A Prong 1: claim 2, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating a modified path embedding based on the path embedding using a path interpolation layer of the machine learning model, the modified path embedding comprising the set of touchpoints and an unobserved touchpoint interpolated from the set of touchpoints by the path interpolation layer; and (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
generating the attention path embedding based on the modified path embedding using the attention network of the machine learning model. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 2 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 2 does not recite an additional element.
Therefore, claim 2 is ineligible.
With respect to claim 3:
Step 2A Prong 1: claim 3, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating a positional encoded path using a positional encoding layer of the machine learning model, the positional encoded path comprising position information associated with the set of touchpoints to allow for order differentiation between touchpoints in the set of touchpoints; and (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
generating the attention path embedding based on the positional encoded path using the attention network of the machine learning model. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 3 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 3 does not recite an additional element.
Therefore, claim 3 is ineligible.
With respect to claim 4:
Step 2A Prong 1: claim 4, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating the set of aggregated attention weights for the set of touchpoints in the decision path using a set of self-attention structures for a multi-head model of the machine learning model, each self-attention structure to generate a set of attention weights, and an aggregation layer to aggregate each set of attention weights to form the set of aggregated attention weights. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 4 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 4 does not recite an additional element.
Therefore, claim 4 is ineligible.
With respect to claim 5:
Step 2A Prong 1: claim 5, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating the set of aggregated attention weights for the set of touchpoints in the decision path using a transformer model, the transformer model comprising a plurality of transformer blocks, each transformer block comprising a multi-head model comprising multiple self-attention structures, each transformer block to generate a set of attention weights, and an aggregation layer to aggregate each set of attention weights to form the set of aggregated attention weights. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 5 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 5 does not recite an additional element.
Therefore, claim 5 is ineligible.
With respect to claim 6:
Step 2A Prong 1: claim 6, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating a context path embedding based on the attention path embedding and a contextual embedding using a concatenation layer of the machine learning model, the contextual embedding comprising a member embedding, a company embedding, or a campaign embedding; and (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
generating the set of touchpoint contribution values corresponding to the set of touchpoints based on the context path embedding. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 6 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 6 does not recite an additional element.
Therefore, claim 6 is ineligible.
With respect to claim 7:
Step 2A Prong 1: claim 7, which incorporates the rejection of claim 1, recites an additional abstract idea:
generating a set of calibrated values for the set of touchpoint contribution values using a calibration layer of the machine learning model, the calibration layer using a secondary mix model. (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 7 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 7 does not recite an additional element.
Therefore, claim 7 is ineligible.
With respect to claim 8:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 8. Therefore, claim 8 is ineligible.
With respect to claim 9:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 9. Therefore, claim 9 is ineligible.
With respect to claim 10:
The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 10. Therefore, claim 10 is ineligible.
With respect to claim 11:
The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 11. Therefore, claim 11 is ineligible.
With respect to claim 12:
The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis that was utilized for claim 5, as described above, is equally applicable to claim 12. Therefore, claim 12 is ineligible.
With respect to claim 13:
The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis that was utilized for claim 6, as described above, is equally applicable to claim 13. Therefore, claim 13 is ineligible.
With respect to claim 14:
The claim recites similar limitations as corresponding to claim 7. Therefore, the same subject matter analysis that was utilized for claim 7, as described above, is equally applicable to claim 14. Therefore, claim 14 is ineligible.
With respect to claim 15:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 15. Therefore, claim 15 is ineligible.
With respect to claim 16:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 16. Therefore, claim 16 is ineligible.
With respect to claim 17:
The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 17. Therefore, claim 17 is ineligible.
With respect to claim 18:
The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 18. Therefore, claim 18 is ineligible.
With respect to claim 19:
The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis that was utilized for claim 5, as described above, is equally applicable to claim 19. Therefore, claim 19 is ineligible.
With respect to claim 20:
The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis that was utilized for claim 6, as described above, is equally applicable to claim 20. Therefore, claim 20 is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6, 8-9, 13, 15-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable by Yan (US 2019/0278378 A1) in view of Weng (US 2019/0385069 A1).
Regarding claim 1, Yan teaches:
A method, comprising: ([0006] “systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify significant touchpoints and/or measure performance of touchpoints in digital content campaigns (i.e., campaigns that utilize multiple media channels and multiple touchpoints to provide digital content to user client devices).”)
generating a path embedding representing a decision path comprising a set of touchpoints to obtain a defined outcome, each touchpoint comprising an electronic interaction between electronic devices; ([0022] “To illustrate, in one or more embodiments, deep learning attribution system generates training touchpoint paths, where each training touchpoint path is specific to a particular user and reflects a digital training touchpoint sequence and a training conversion indication. The deep learning attribution system can then train the touchpoint attributional neural network based on the training touchpoint paths.” And [0099] “To illustrate, the embedding layer 406 transforms the encoded touchpoint vectors 404 (e.g., changes the dimensionality of the input) to dense vectors 408, … Indeed, the deep learning attribution system 104 applies the embedding matrix W.sub.e to the encoded touchpoint vectors 404 (i.e., W.sub.ex.sub.t=e.sub.T) to obtain dense vector latent representations of each touchpoint in a training touchpoint sequence. Notably, the t.sup.th column of the embedding matrix W.sub.e is a vector having the dimension v.sub.e and is a continuous representation of the t.sup.th touchpoint in the training touchpoint sequence.”)
generating an attention path embedding based on the path embedding using an attention network of a machine learning model, the attention path embedding comprising a set of aggregated attention weights for the set of touchpoints in the decision path; ([0108] “As shown in FIG. 4A, the touchpoint attribution attention neural network 400a includes the attention layer 414 (also referred to as the attention mechanism). In various embodiments, the attention layer 414 determines attention weights 418 for each touchpoint in a training touchpoint sequence. As mentioned above, the attention weight reflects the relative conversion importance or significance of each digital target touchpoint in a training touchpoint sequence given the previous touchpoints in the training touchpoint sequence. Indeed, not all touchpoints contribute equally to the representation of a user's conversion journey. Hence, the deep learning attribution system 104 trains the attention layer 414 to extract touchpoints that are more significant to a conversion such that the attention weights 418 capture the incremental importance of each touchpoint.”)
generating a set of touchpoint contribution values corresponding to the set of touchpoints based on the attention path embedding, a touchpoint contribution value from the set of touchpoint contribution values representing a level of contribution made by a touchpoint from the set of touchpoints to obtain the defined outcome; and ([0153] “Turning now to FIGS. 5A and 5B, additional detail is provided regarding employing a trained touchpoint attribution attention neural network to determine touchpoint attributions and predict conversions for target touchpoint sequences. In particular, FIG. 5A illustrates the deep learning attribution system 104 employing the trained touchpoint attribution attention neural network to determine touchpoint attributions for a target touchpoint sequence.”)
Yan does not teach:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values.
However, Weng does:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values. ([0036] “For example, analysis apparatus 204 may apply weights, coefficients, and/or operations associated with machine learning models 208 to features associated with each offline and/or nearline candidate to produce a score representing the likelihood that the member will connect with the candidate after the candidate is outputted as a connection recommendation to the member.”
Yan and Weng are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning prediction and recommendation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng. One would be motivated to do this for the benefit of using relative importance of a user’s interactions to improve the recommendation in a connection network.
Regarding claim 2, Yan in view of Weng teaches claim 1 as outlined above. Yan further teaches:
generating a modified path embedding based on the path embedding using a path interpolation layer of the machine learning model, the modified path embedding comprising the set of touchpoints and an unobserved touchpoint interpolated from the set of touchpoints by the path interpolation layer; and ([0161] “In one or more embodiments, the deep learning attribution system 104 adds a potential touchpoint to the target touchpoint sequence 512 to create a first modified target touchpoint sequence. The deep learning attribution system 104 provides the first modified target touchpoint sequence to the trained touchpoint attribution attention neural network 504.”)
generating the attention path embedding based on the modified path embedding using the attention network of the machine learning model. ([0162] “Using the various layers, as described above, the trained touchpoint attribution attention neural network 504 determines a conversion probability for the first modified target touchpoint sequence. In particular, the trained touchpoint attribution attention neural network 504 determines touchpoint attention weights for the first modified target touchpoint sequence. Then, based on the touchpoint attention weights, the trained touchpoint attribution attention neural network 504 determines a first conversion probability for the first modified target touchpoint sequence.”)
Regarding claim 6, Yan in view of Weng teaches claim 1 as outlined above. Yan further teaches:
generating a context path embedding based on the attention path embedding and a contextual embedding using a concatenation layer of the machine learning model, the contextual embedding comprising a member embedding, a company embedding, or a campaign embedding; and ([0054] “In one or more embodiments, the content management system 102 employs the deep learning attribution system 104 to facilitate the various digital content campaigns. In alternative embodiments, the content management system 102 hosts (or communicates with) a separate digital content system that manages and facilitates various digital content campaigns. In these embodiments, the marketing system can communicate with the deep learning attribution system 104.” Campaigns can be included in the embeddings).
generating the set of touchpoint contribution values corresponding to the set of touchpoints based on the context path embedding. ([0153] “Turning now to FIGS. 5A and 5B, additional detail is provided regarding employing a trained touchpoint attribution attention neural network to determine touchpoint attributions and predict conversions for target touchpoint sequences. In particular, FIG. 5A illustrates the deep learning attribution system 104 employing the trained touchpoint attribution attention neural network to determine touchpoint attributions for a target touchpoint sequence.”)
Regarding claim 8, Yan teaches:
A computing apparatus, comprising: a memory component; and circuitry coupled to the memory component, the circuitry to: ([0006] “systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify significant touchpoints and/or measure performance of touchpoints in digital content campaigns (i.e., campaigns that utilize multiple media channels and multiple touchpoints to provide digital content to user client devices).” And [0228] “Embodiments of the present disclosure may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory”)
generating a path embedding representing a decision path comprising a set of touchpoints to obtain a defined outcome, each touchpoint comprising an electronic interaction between electronic devices; ([0022] “To illustrate, in one or more embodiments, deep learning attribution system generates training touchpoint paths, where each training touchpoint path is specific to a particular user and reflects a digital training touchpoint sequence and a training conversion indication. The deep learning attribution system can then train the touchpoint attributional neural network based on the training touchpoint paths.” And [0099] “To illustrate, the embedding layer 406 transforms the encoded touchpoint vectors 404 (e.g., changes the dimensionality of the input) to dense vectors 408, … Indeed, the deep learning attribution system 104 applies the embedding matrix W.sub.e to the encoded touchpoint vectors 404 (i.e., W.sub.ex.sub.t=e.sub.T) to obtain dense vector latent representations of each touchpoint in a training touchpoint sequence. Notably, the t.sup.th column of the embedding matrix W.sub.e is a vector having the dimension v.sub.e and is a continuous representation of the t.sup.th touchpoint in the training touchpoint sequence.”)
generating an attention path embedding based on the path embedding using an attention network of a machine learning model, the attention path embedding comprising a set of aggregated attention weights for the set of touchpoints in the decision path; ([0108] “As shown in FIG. 4A, the touchpoint attribution attention neural network 400a includes the attention layer 414 (also referred to as the attention mechanism). In various embodiments, the attention layer 414 determines attention weights 418 for each touchpoint in a training touchpoint sequence. As mentioned above, the attention weight reflects the relative conversion importance or significance of each digital target touchpoint in a training touchpoint sequence given the previous touchpoints in the training touchpoint sequence. Indeed, not all touchpoints contribute equally to the representation of a user's conversion journey. Hence, the deep learning attribution system 104 trains the attention layer 414 to extract touchpoints that are more significant to a conversion such that the attention weights 418 capture the incremental importance of each touchpoint.”)
generating a set of touchpoint contribution values corresponding to the set of touchpoints based on the attention path embedding, a touchpoint contribution value from the set of touchpoint contribution values representing a level of contribution made by a touchpoint from the set of touchpoints to obtain the defined outcome; and ([0153] “Turning now to FIGS. 5A and 5B, additional detail is provided regarding employing a trained touchpoint attribution attention neural network to determine touchpoint attributions and predict conversions for target touchpoint sequences. In particular, FIG. 5A illustrates the deep learning attribution system 104 employing the trained touchpoint attribution attention neural network to determine touchpoint attributions for a target touchpoint sequence.”)
Yan does not teach:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values.
However, Weng does:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values. ([0036] “For example, analysis apparatus 204 may apply weights, coefficients, and/or operations associated with machine learning models 208 to features associated with each offline and/or nearline candidate to produce a score representing the likelihood that the member will connect with the candidate after the candidate is outputted as a connection recommendation to the member.”
Yan and Weng are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning prediction and recommendation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng. One would be motivated to do this for the benefit of using relative importance of a user’s interactions to improve the recommendation in a connection network.
Regarding claim 9, Yan in view of Weng, teaches claim 8 as outlined above. Claim 9 recites similar limitations corresponding to claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 13, Yan in view of Weng, teaches claim 8 as outlined above. Claim 13 recites similar limitations corresponding to claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding claim 15, Yan teaches:
A non-transitory computer-readable medium storing executable instructions, which when executed by circuitry, causes the circuitry to: ([0006] “systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify significant touchpoints and/or measure performance of touchpoints in digital content campaigns (i.e., campaigns that utilize multiple media channels and multiple touchpoints to provide digital content to user client devices).”)
generating a path embedding representing a decision path comprising a set of touchpoints to obtain a defined outcome, each touchpoint comprising an electronic interaction between electronic devices; ([0022] “To illustrate, in one or more embodiments, deep learning attribution system generates training touchpoint paths, where each training touchpoint path is specific to a particular user and reflects a digital training touchpoint sequence and a training conversion indication. The deep learning attribution system can then train the touchpoint attributional neural network based on the training touchpoint paths.” And [0099] “To illustrate, the embedding layer 406 transforms the encoded touchpoint vectors 404 (e.g., changes the dimensionality of the input) to dense vectors 408, … Indeed, the deep learning attribution system 104 applies the embedding matrix W.sub.e to the encoded touchpoint vectors 404 (i.e., W.sub.ex.sub.t=e.sub.T) to obtain dense vector latent representations of each touchpoint in a training touchpoint sequence. Notably, the t.sup.th column of the embedding matrix W.sub.e is a vector having the dimension v.sub.e and is a continuous representation of the t.sup.th touchpoint in the training touchpoint sequence.”)
generating an attention path embedding based on the path embedding using an attention network of a machine learning model, the attention path embedding comprising a set of aggregated attention weights for the set of touchpoints in the decision path; ([0108] “As shown in FIG. 4A, the touchpoint attribution attention neural network 400a includes the attention layer 414 (also referred to as the attention mechanism). In various embodiments, the attention layer 414 determines attention weights 418 for each touchpoint in a training touchpoint sequence. As mentioned above, the attention weight reflects the relative conversion importance or significance of each digital target touchpoint in a training touchpoint sequence given the previous touchpoints in the training touchpoint sequence. Indeed, not all touchpoints contribute equally to the representation of a user's conversion journey. Hence, the deep learning attribution system 104 trains the attention layer 414 to extract touchpoints that are more significant to a conversion such that the attention weights 418 capture the incremental importance of each touchpoint.”)
generating a set of touchpoint contribution values corresponding to the set of touchpoints based on the attention path embedding, a touchpoint contribution value from the set of touchpoint contribution values representing a level of contribution made by a touchpoint from the set of touchpoints to obtain the defined outcome; and ([0153] “Turning now to FIGS. 5A and 5B, additional detail is provided regarding employing a trained touchpoint attribution attention neural network to determine touchpoint attributions and predict conversions for target touchpoint sequences. In particular, FIG. 5A illustrates the deep learning attribution system 104 employing the trained touchpoint attribution attention neural network to determine touchpoint attributions for a target touchpoint sequence.”)
Yan does not teach:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values.
However, Weng does:
providing a recommendation for a connections networking system based on the set of touchpoint contribution values. ([0036] “For example, analysis apparatus 204 may apply weights, coefficients, and/or operations associated with machine learning models 208 to features associated with each offline and/or nearline candidate to produce a score representing the likelihood that the member will connect with the candidate after the candidate is outputted as a connection recommendation to the member.”
Yan and Weng are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning prediction and recommendation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng. One would be motivated to do this for the benefit of using relative importance of a user’s interactions to improve the recommendation in a connection network.
Regarding claim 16, Yan in view of Weng, teaches claim 15 as outlined above. Claim 16 recites similar limitations corresponding to claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 20, Yan in view of Weng, teaches claim 15 as outlined above. Claim 20 recites similar limitations corresponding to claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Claims 3-5, 10-12, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable by Yan in view of Weng and Vaswani (NPL: ‘Attention Is All You Need’ (2017)).
Regarding claim 3, Yan in view of Weng teaches claim 1 as outlined above. Yan further teaches:
generating the attention path embedding based on the positional encoded path using the attention network of the machine learning model. ([0108] “As shown in FIG. 4A, the touchpoint attribution attention neural network 400a includes the attention layer 414 (also referred to as the attention mechanism). In various embodiments, the attention layer 414 determines attention weights 418 for each touchpoint in a training touchpoint sequence. As mentioned above, the attention weight reflects the relative conversion importance or significance of each digital target touchpoint in a training touchpoint sequence given the previous touchpoints in the training touchpoint sequence. Indeed, not all touchpoints contribute equally to the representation of a user's conversion journey. Hence, the deep learning attribution system 104 trains the attention layer 414 to extract touchpoints that are more significant to a conversion such that the attention weights 418 capture the incremental importance of each touchpoint.”)
Neither Yan nor Weng teaches:
generating a positional encoded path using a positional encoding layer of the machine learning model, the positional encoded path comprising position information associated with the set of touchpoints to allow for order differentiation between touchpoints in the set of touchpoints; and
However, Vaswani does:
generating a positional encoded path using a positional encoding layer of the machine learning model, the positional encoded path comprising position information associated with the set of touchpoints to allow for order differentiation between touchpoints in the set of touchpoints; and (Section 3.5 “To this end, we add "positional encodings" to the input embeddings at the bottoms of the encoder and decoder stacks. The positional encodings have the same dimension d_model as the embeddings, so that the two can be summed. There are many choices of positional encodings, learned and fixed”)
Yan, Weng and Vaswani are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng with the positional encoding of Vaswani. One would be motivated to do this since the touchpoint sequence order of Yan is important in affecting the prediction outcome.
Regarding claim 4, Yan in view of Weng teaches claim 1 as outlined above. Neither Yan nor Weng teaches the limitations of claim 4. However, Vaswani does:
generating the set of aggregated attention weights for the set of touchpoints in the decision path using a set of self-attention structures for a multi-head model of the machine learning model, each self-attention structure to generate a set of attention weights, and an aggregation layer to aggregate each set of attention weights to form the set of aggregated attention weights. (Section 3.2.2 “Multi-head attention allows the model to jointly attend to information from different representation subspaces at different positions. With a single attention head, averaging inhibits this.”)
Yan, Weng and Vaswani are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng with the positional encoding of Vaswani. One would be motivated to do this since the touchpoint sequence order of Yan is important in affecting the prediction outcome.
Regarding claim 5, Yan in view of Weng teaches claim 1 as outlined above. Neither Yan nor Weng teaches the limitations of claim 5. However, Vaswani does:
generating the set of aggregated attention weights for the set of touchpoints in the decision path using a transformer model, the transformer model comprising a plurality of transformer blocks, each transformer block comprising a multi-head model comprising multiple self-attention structures, each transformer block to generate a set of attention weights, and an aggregation layer to aggregate each set of attention weights to form the set of aggregated attention weights. (Section 3.2.3 “The Transformer uses multi-head attention in three different ways: In "encoder-decoder attention" layers, the queries come from the previous decoder layer, and the memory keys and values come from the output of the encoder. This allows every position in the decoder to attend over all positions in the input sequence. This mimics the typical encoder-decoder attention mechanisms in sequence-to-sequence models such as [31, 2, 8]. The encoder contains self-attention layers. In a self-attention layer all of the keys, values and queries come from the same place, in this case, the output of the previous layer in the encoder. Each position in the encoder can attend to all positions in the previous layer of the encoder. Similarly, self-attention layers in the decoder allow each position in the decoder to attend to all positions in the decoder up to and including that position. We need to prevent leftward information flow in the decoder to preserve the auto-regressive property. We implement this inside of scaled dot-product attention by masking out (setting to −∞) all values in the input of the softmax which correspond to illegal connections. See Figure 2.”)
Yan, Weng and Vaswani are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng with the positional encoding of Vaswani. One would be motivated to do this since the touchpoint sequence order of Yan is important in affecting the prediction outcome.
Regarding claim 10, Yan in view of Weng, teaches claim 8 as outlined above. Claim 10 recites similar limitations corresponding to claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 11, Yan in view of Weng, teaches claim 8 as outlined above. Claim 11 recites similar limitations corresponding to claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 12, Yan in view of Weng, teaches claim 8 as outlined above. Claim 12 recites similar limitations corresponding to claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding claim 17, Yan in view of Weng, teaches claim 15 as outlined above. Claim 17 recites similar limitations corresponding to claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 18, Yan in view of Weng, teaches claim 15 as outlined above. Claim 18 recites similar limitations corresponding to claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 19, Yan in view of Weng, teaches claim 15 as outlined above. Claim 19 recites similar limitations corresponding to claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable by Yan in view of Weng and Chittilappilly (US 2016/0210661 A1)
Regarding claim 7, Yan in view of Weng teaches claim 1 as outlined above. Neither Yan nor Weng teaches the limitations of claim 7. However, Chittilappilly does:
generating a set of calibrated values for the set of touchpoint contribution values using a calibration layer of the machine learning model, the calibration layer using a secondary mix model. ([0036] “The herein disclosed techniques address such problems using technological techniques for managing digital media spend allocation using calibrated user-level response data. More specifically, the techniques described herein discuss (1) determining channel spend allocation values for a plurality of media channels based on a channel response predictive model; (2) receiving a stream of touchpoint attribute records characterizing user responses to the media channels; (3) calibrating a portion of the touchpoint attribute records using selected channel spend allocation values to provide calibrated touchpoint attribute records; (4) generating a touchpoint response predictive model derived from the calibrated touchpoint attribute records; and (5) providing access to the calibrated touchpoint response predictive model for use by a media spend planning application to enable a marketing manager to specify media spend allocations.”)
Yan, Weng and Chittilappilly are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention neural network and touchpoint embedding of Yan with the connection recommendation of Weng with the calibration of Chittilappilly. One would be motivated to do this to improve the accuracy of the system (Chittilappilly background).
Regarding claim 14, Yan in view of Weng, teaches claim 8 as outlined above. Claim 14 recites similar limitations corresponding to claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL P GRUSZKA whose telephone number is (571)272-5259. The examiner can normally be reached M-F 9:00 AM - 6:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL GRUSZKA/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights