DETAILED ACTION
This action is in response the communications filed on 07/08/2026 in which claims 1-4, 7, 11-13, and 15-20 are amended, and claims 1-20 are pending.
--
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
-
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Step 1: Claims 1-14 recite a method. Claims 15-17 recite a system. Claims 18-20 recite a computer program product. Therefore, claims 1-14 are directed to a process, and claims 15-17 are directed to a machine, and claims 18-20 are directed to a manufacture.
With respect to claims 1, 15 and 18:
2A Prong 1: The claim recites a judicial exception.
generating… a self-attention matrix representing dependencies between the past events; (mathematical concepts - mathematical equations, in light of spec [0021])
the machine learning model implementing an intensity function comprising a kernel function, wherein the kernel function determines, based on the self-attention matrix, an instance-level causal strength value between a pair of events of the past events, and the kernel function contributes to an additive function that sums intensity of repeated individual events of the past events; and (mathematical concepts - mathematical equations, in light of spec [0028])
2A Prong 2: The judicial exception is not integrated into a practical application.
(claim 15) a processor; a data bus coupled to the processor; a memory coupled to the data bus; and a computer-usable medium embodying computer program code, the computer program code comprising instructions executable by the processor and configured for (claim 18) the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components)
accessing time-series data describing past events (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
… using a transformer model… (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a model)
inputting the time-series data into a machine learning model (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
generating, using the machine learning model, an output based on the inputting of the time-series data, wherein the output indicates an instance-level causality that links two or more events of the past events, and the output is useable for an event diagnosis task or an event consolidation task. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using the model to generate an output)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 15) a processor; a data bus coupled to the processor; a memory coupled to the data bus; and a computer-usable medium embodying computer program code, the computer program code comprising instructions executable by the processor and configured for (claim 18) the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components)
accessing time-series data describing past events (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
… using a transformer model… (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a model)
inputting the time-series data into a machine learning model (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
generating, using the machine learning model, an output based on the inputting of the time-series data, wherein the output indicates an instance-level causality that links two or more events of the past events, and the output is useable for an event diagnosis task or an event consolidation task. (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using the model to generate an output)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 2, 16 and 19:
2A Prong 1: The claim recites a judicial exception.
wherein the intensity refers to a probability density of an occurrence of an event, and wherein the event is different from the past events. (mathematical concepts - mathematical equations, in light of spec [0028])
With respect to claims 3, 17 and 20:
2A Prong 1: The claim recites a judicial exception.
wherein the transformer model implements self-attention for the generating of the self-attention matrix. (mathematical concepts - mathematical equations, self-attention mechanism)
With respect to claim 4:
2A Prong 1: The claim recites a judicial exception.
wherein the self-attention is implemented via A(x, xj) = exp(xTKxj) I(t>tj) / Σ exp (xTKxl) where t is a timestamp associated with x, and l(t> tj) is an indicator function that assumes a value 1 if an argument t > tj, is true and 0 otherwise, and a transformer, K is a parameter matrix learned from data (mathematical concepts - mathematical equations)
With respect to claim 5:
2A Prong 1: The claim recites a judicial exception.
wherein the transformer, K, corresponds to WQWI, where WQ and WK are transformation matrices for queries and keys, respectively (mathematical concepts - mathematical equations, self-attention mechanism)
With respect to claim 6:
2A Prong 1: The claim recites a judicial exception.
wherein the transformer model implements a key-value-query formalism (mathematical concepts - mathematical equations, self-attention mechanism)
With respect to claim 7:
2A Prong 1: The claim recites a judicial exception.
wherein the intensity function further comprises a decay function that further contributes to the additive function. (mathematical concepts - mathematical equations or relationships)
With respect to claim 8:
2A Prong 1: The claim recites a judicial exception.
wherein the decay function comprises a decay distribution φ(t – tj|x, xj) that models a time decay of causal influence (mathematical concepts - mathematical equations, a decay distribution)
With respect to claim 9:
2A Prong 1: The claim recites a judicial exception.
wherein the decay distribution φ(t – tj|x, xj) is a truncated Gaussian mixture (mathematical concepts - mathematical equations, Gaussian mixture)
With respect to claim 10:
2A Prong 1: The claim recites a judicial exception.
wherein φ(t – tj|x, xj) = γ(x, xj) e ^ -γ(x, xj)(t-tj), where γ(x, xj) is a decay rate function (mathematical concepts - mathematical equations, a decay rate function)
With respect to claim 11:
2A Prong 1: The claim recites a judicial exception.
further comprising: adopting an instance-aware parameterization of the kernel function, wherein: each event is associated with a latent embedding vector x = g(t, k) and an embedding function g(t, k) is defined as: g(t, k) = MLP[t - ti ⨁ MLP(k)] with an event type provided as a K-dimensional one-hot vector and a time difference provided as ti - ti-i , for xi, and one multilayer perceptron (MLP) layer is used to embed the K-dimensional one-hot vector for the event type; and concatenating the one MLP layer with the time difference to form a M-dimensional embedding vector for each event (t, k) (mathematical concepts - mathematical equations g(t, k) = MLP[t - ti ⨁ MLP(k)] )
With respect to claim 12:
2A Prong 1: The claim recites a judicial exception.
wherein an intensity function for a latent event vector embedding x is in a form of:
λ (x I Ht) = μ (x I Ht) + Σ α(x, xj) φ(t – tj|x, xj)
where μ is a background intensity, a function α (x, xj) ∈ R+, where R+ denotes a set of nonnegative real numbers, and the function is the kernel function, and (mathematical concepts - mathematical equations)
a decay distribution φ(t – tj|x, xj) models a time decay of causal influence (mathematical concepts - mathematical equations, a decay distribution)
With respect to claim 13:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the output of the machine learning model is used, without any post-processing, to provide the output that indicates the instance-level causality. (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the output of the machine learning model is used, without any post-processing, to provide the output that indicates the instance-level causality. (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 14:
2A Prong 1: The claim recites a judicial exception.
wherein the additive function is disposed inside a nonlinear function. (mathematical concepts - mathematical equations, a nonlinear function)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6-10 and 13-20 rejected under 35 U.S.C. 103 as being unpatentable over Shou ("Influence-Aware Attention for Multivariate Temporal Point Processes" 20230317) in view of Zhang ("Self-Attentive Hawkes Process" 20200214)
In regard to claims 1, 15 and 18, Shou teaches: A computer-implemented method for discovering causality, the method comprising: (Shou, p. 8, Likelihood "The conditional intensity for type-k event takes the following form: λ(t|Ht) [λ(t|Ht): discovering causality] = fk(...) (13) where Ht = {(tj , yj) : tj < t} is the history up to t… the posterior quantifies the extent of influence which is similar to methods of using an infectivity matrix for Granger causality.")
accessing time-series data describing past events; (Shou, p. 8, Likelihood "The conditional intensity for type-k event takes the following form: λ(t|Ht) = fk(...) (13) where Ht = {(tj , yj) : tj < t} is the history up to t [time-series data describing past events]")
Shou does not teach, but Zhang teaches: generating, using a transformer model, a self-attention matrix representing dependencies between the past events; (Zhang, p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention. [a transformer model] This generates a hidden vector [a self-attention matrix] that summarizes the influence of all previous events: h_u,i+1 = … (10)" [dependencies between the past events]; self-attention inherently generates a matrix, also see Vaswani, or a vector can be viewed as a matrix)
inputting the time-series data into a machine learning model, (Zhang, p. 1, Abstract "This study attempts to fill the gap by designing a self-attentive Hawkes process (SAHP). The SAHP [a machine learning model] employed self-attention to summarize influence from history events and compute the probability of the next event."; p. 4, 4. Self-Attentive Hawkes Process "In this section, we describe how to adapt the self-attention mechanism to Hawkes processes as Figure 2 shows. Event type embedding. The input sequence [the time-series data] is made up of events... "; also see Fig. 2 (v1, t1) (v2, t2)... (vi, ti)...) the machine learning model implementing an intensity function comprising a kernel function, (Zhang, p. 5, Figure 2 "An event stream and the SAHP for one event type (u). The intensity function (λu(t)) [an intensity function] is determined by a sequence of past events via the SAHP."; p. 3, 3.1. Temporal Point Processes and Hawkes Process "the definition of the intensity function is given as: λ*(t) = μ + Σ (v', t')∈Ht φ(t – t'), (4)... φ(t) is a triggering kernel [a kernel function] containing the peer influence of different event types.")
wherein the kernel function determines, based on the self-attention matrix, an instance-level causal strength value between a pair of events of the past events, and (Zhang, p. 4, Historical hidden vector "As an event consists of its type and timestamp, we add the positional encoding to the event type embedding in order to obtain the representation of the event (vi; ti) [a pair of events of the past events]: xi = ... (9)"; p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention. This generates a hidden vector [the self-attention matrix] that summarizes the influence of all previous events: h_u,i+1 = (10) where xi+1 is like query(q) in the attention terminology, xj is the key(k)…"; p. 4, Intensity function "we compute three parameters of the the intensity function based on the history hidden vector hu,i+1 via the following three non-linear transformations: μ = gelu (huWμ) (12) η = gelu (huWη) (13) [μ and η are based on hu, based on the self-attention matrix]... Finally, we express the intensity function as follows: λ*(t) [an intensity function] = softplus (μ + (η-μ) exp(-γ(t-ti)) [η-μ: an instance-level causal strength value, (η-μ) exp(-γ(t-ti)): the kernel function]), (15) The decaying speed is decided by (η-μ) that can be both positive and negative. This enables us to capture both excitation and inhibition effects. With inhibition we mean the effect when past events reduce the likelihood of future events [an instance-level causal strength value]..."; self-attention to compute attention vector/matrix between each event and the earlier events in the sequence, which are pairwise, instance-level dependencies used in the intensity function)
the kernel function contributes to an additive function that sums intensity of repeated individual events of the past events; and (Zhang, p. 4, Self-Attention "This generates a hidden vector that summarizes the influence of all previous events: h_u,i+1 = (Σ f(xi+1, xj) [intensity f(·,·), the past events xj] g(xj)… (10)"; p. 4, Intensity function "we compute three parameters of the the intensity function based on the history hidden vector hu,i+1 via the following three non-linear transformations: μ = gelu (huWμ) (12) [μ is based on hu, therefore is also history dependent; μ is an additive function that sums intensity of repeated individual events of the past events] η = gelu (huWη) (13)... Finally, we express the intensity function as follows: λ*(t) = softplus (μ [an additive function] + (η-μ) exp(-γ...(t-ti)) [the kernel function]), (15)")
generating, using the machine learning model, an output based on the inputting of the time-series data, wherein the output indicates an instance-level causality that links two or more events of the past events, and (Zhang, p. 5, Figure 2 "An event stream and the SAHP for one event type (u). The intensity function (λu(t)) [generating an output, an instance-level causality] is determined by a sequence of past events [based on the inputting of the time-series data, two or more events of the past events] via the SAHP. [using the machine learning model]")
PNG
media_image1.png
690
1112
media_image1.png
Greyscale
the output is useable for an event diagnosis task or an event consolidation task. (Zhang, p. 11, MIMIC-II (MMC) "The Multiparameter Intelligent Monitoring in Intensive Care (MIMIC-II) dataset is developed based on an electric medical record system… According to the clinical history, a temporal point process is supposed to capture the dynamics of when a patient will visit doctors and what the diagnose result will be. [an event diagnosis task or an event consolidation task]")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Shou to incorporate the teachings of Zhang by including Hawkes process with self-attention. Doing so would improve the capability and interpretability of the model. (Zhang, p. 1, Abstract "Experiments on goodness-of-fit and prediction tasks showed the improved capability of SAHP. Furthermore, the SAHP is more interpretable than RNN-based counterparts because the learnt attention weights revealed contributions of one event type to the happening of another type.")
Claims 15 and 18 recite substantially the same limitation as claim 1, therefore the rejection applied to claim 1 also apply to claims 15 and 18. In addition, Shou teaches: (claim 15) a processor; a data bus coupled to the processor; a memory coupled to the data bus; and a computer-usable medium embodying computer program code, the computer program code comprising instructions executable by the processor and configured for (claim 18) the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to (Shou, p. 11, 4.3. Ablation Studies "Training is on Google Colab with GPU and high-RAM, more specifically with Tesla P100-PCIE-16GB and 52GB RAM.")
In regard to claims 2, 16 and 19, Shou does not teach, but Zhang teaches: wherein the intensity refers to a probability density of an occurrence of an event, and wherein the event is different from the past events. (Zhang, p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention. his generates a hidden vector that summarizes the influence of all previous events: h_u,i+1 = (Σ f(xi+1, xj) [the intensity f(·,·), an event, the event is different from the past events x_i+1] g(xj)… (10) where xi+1 is like query(q) in the attention terminology, xj is the key(k)… the similarity function f(,) is specified as an embedded Gaussian [a probability density (Gaussian)]: f(xi+1, xj) = … (11)")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claims 3, 17 and 20, Shou does not teach, but Zhang teaches: wherein the transformer model implements self-attention for the generating of the self-attention matrix. (Zhang, p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention. [the transformer model, self-attention]")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 6, Shou does not teach, but Zhang teaches: wherein the transformer model implements a key-value-query formalism. (Zhang, p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention. [the transformer model]This generates a hidden vector that summarizes the influence of all previous events: h_u,i+1 = (Σ f(xi+1, xj)g(xj)/Σ f(xi+1, xj) (10)... where xi+1 is like query (q) in the attention terminology, xj is the key (k) and g(xj) is the value (v). [a key-value-query formalism]")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 7, Shou does not teach, but Zhang teaches: wherein the intensity function further comprises a decay function that further contributes to the additive function. (Zhang, p. 4, Intensity function "Finally, we express the intensity function as follows: λ*(t) = softplus (μ [the additive function] + (η-μ) exp(-γ(t-ti)) [exp(-γ(t-ti)): a decay function]), (15)")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 8, Shou does not teach, but Zhang teaches: wherein the decay function comprises a decay distribution φ(t - tj|x, xj) that models a time decay of causal influence. (Zhang, p. 4, Intensity function "Finally, we express the intensity function as follows: λ*(t) = softplus (μ + (η-μ) exp(-γ(t-ti)) [exp(-γ(t-ti)): a decay distribution]), (15)… When t increases from ti, the intensity decays exponentially [a time decay]")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 9, Shou does not teach, but Zhang teaches: wherein the decay distribution φ(t – tj|x, xj) is a truncated Gaussian mixture. (Zhang, p. 4, Self-Attention "This generates a hidden vector that summarizes the influence of all previous events: h_u,i+1 = (Σ f(xi+1, xj)g(xj)… (10)… the similarity function f(,) is specified as an embedded Gaussian: [Gaussian] f(xi+1, xj) = … (11)"; p. 4, Intensity function "γu = softplus(huWγ) (14)… Finally, we express the intensity function as follows: λ*(t) = softplus (μ + (η-μ) exp(-γ(t-ti)) [exp(-γ(t-ti)): the decay distribution]), (15) for t ∈ (ti, ti+1]"; the decay function exp(-γ(t-ti)) is based on hu, which is based on f, a Gaussian, therefore hu or γ is a weighted Gaussian with specific time intervals [a truncated Gaussian mixture])
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 10, Shou does not teach, but Zhang teaches: wherein φ(t – tj|x, xj) = γ(x, xj) e ^ -γ(x, xj)(t-tj), where γ(x, xj) is a decay rate function. (Zhang, p. 4, Intensity function "γu = softplus(huWγ) (14)… [a decay rate function] softplus is used for the decaying parameter since γ needs to be constrained to strictly positive values... Finally, we express the intensity function as follows: λ*(t) = softplus (μ + (η-μ) exp(-γ(t-ti))), (15) for t ∈ (ti, ti+1]")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 13, Shou does not teach, but Zhang teaches: wherein the output of the machine learning model is used, without any post-processing, to provide the output that indicates the instance-level causality. (Zhang, p. 5, Figure 2 "An event stream and the SAHP for one event type (u). The intensity function (λu(t)) [an output of the machine learning model] is determined by a sequence of past events via the SAHP."; Zhang doesn't recite anything post-processing, therefore the output is used without post-processing)
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
In regard to claim 14, Shou does not teach, but Zhang teaches: wherein the additive function is disposed inside a nonlinear function. (Zhang, p. 4, Intensity function "Finally, we express the intensity function as follows: λ*(t) = softplus [a nonlinear function] (μ [an additive function] + (η-μ) exp(-γ...(t-ti))), (15)"; μ is inside softplus function)
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
Claims 4-5 rejected under 35 U.S.C. 103 as being unpatentable over Shou and Zhang, as applied to claim 3, and in further view of Gupta ("Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences" 20220217)
In regard to claim 4, Shou does not teach, but Zhang teaches: where t is a timestamp associated with x, and l(t> tj) is an indicator function that assumes a value 1 if an argument t > tj, is true and 0 otherwise (Zhang, p. 4, Self-Attention "Given a series of historical events until ti... [timestamp] we compute the pairwise influence of one previous event to the next event by employing self-attention... The temporal information is provided to the model during training by preventing the model to learn about future events via masking. We implement this in the attention mechanism by masking out all values in the input sequence that correspond to future events. [l(t> tj) is an indicator function] Hence, the intensity of one event is obtained only based on its history.")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
PNG
media_image2.png
110
604
media_image2.png
Greyscale
Shou and Zhang do not teach, but Gupta teaches: wherein the self-attention is implemented via A(x, xj) = exp(xTKxj) I(t>tj) / Σ exp (xTKxl)… K is a parameter matrix learned from data. (Gupta, p. 4, Attention layer "The state-of-the-art works on attention models… often refer s, k, v as query, key and value vectors respectively, Similarly, we call the trainable weights Ws, Wk, and Wv as the Query, Key and Value matrices [a parameter matrix], respectively")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Shou and Zhang to incorporate the teachings of Gupta by including attention mechanism with trainable weights. Doing so would allow to capture the interaction between events. (Gupta, p. 4, Attention layer "The second layer models the interaction between all the query events and the past corpus events")
In regard to claim 5, Shou and Zhang do not teach, but Gupta teaches: wherein the transformer, K, corresponds to WQWKT, where WQ and WK are transformation matrices for queries and keys, respectively. (Gupta, p. 4, Attention layer "The state-of-the-art works on attention models… often refer s, k, v as query, key and value vectors respectively, Similarly, we call the trainable weights Ws, Wk, and Wv as the Query, Key and Value matrices, respectively")
The rationale for combining the teachings of Shou, Zhang and Gupta is the same as set forth in the rejection of claim 4.
Claim 11 rejected under 35 U.S.C. 103 as being unpatentable over Shou and Zhang, as applied to claim 1, and in further view of Bian ("Iterative convolutional enhancing self-attention Hawkes process with time relative position encoding" 20230204)
In regard to claim 11, Shou does not teach, but Zhang teaches: further comprising: adopting an instance-aware parameterization of the kernel function, (Zhang, p. 4, Intensity function "Finally, we express the intensity function as follows: λ*(t) = softplus (μ + (η-μ) exp(-γ(t-ti)) [(η-μ) exp(-γ(t-ti)): the kernel function]), (15)"; η_u, μ_u, γ_u are all for the type-u event [an instance-aware parameterization]) wherein: each event is associated with a latent embedding vector x = g(t, k) and an embedding function g(t, k) is defined as: g(t, k) = MLP[t - ti ⊕ MLP(k)] with an event type provided as a K-dimensional one-hot vector and a time difference provided as ti - ti-i , for xi, and one multilayer perceptron (MLP) layer is used to embed the K-dimensional one- hot vector for the event type; and (Zhang, p. 4, Event type embedding "To obtain a unique dense embedding for event type, we use a linear embedding layer [one multilayer perceptron (MLP) layer]…evWE (7)… ev is a one-hot vector of the type-v and WE is the embedding matrix. [a K-dimensional one-hot vector]"; p. 5, see Figure 2 all events are embedded to generate x1, x2... [a latent embedding vector])
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
Shou and Zhang do not teach, but Bian teaches: concatenating the one MLP layer with the time difference to form a M-dimensional embedding vector for each event (t, k). (Bian, 3 Model "Given a sequence of events e... with the length L, where each arrival time ti, corresponds to a category mi {1, 2, ... M} with a total number of M categories... Input Layer Event sequences need to be embedded as dense vectors to be used as signal inputs. Event embedding is used to implement this process by exerting a linear embedding layer in category one-hot vectors and then concatenating it with time interval vectors: Cem = cWc, Eem = Concat (Cem, Δt)")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Shou and Zhang to incorporate the teachings of Bian by including the concatenation of embedding of events. Doing so would enable the model to learn relationships between the specific timing of an event.
Claim 12 rejected under 35 U.S.C. 103 as being unpatentable over Shou, Zhang and Bian, as applied to claim 11, and in further view of Lee ("Hawkes Processes with Stochastic Excitations" 20160922)
In regard to claim 12, Shou does not teach, but Zhang teaches: wherein an intensity function for a latent event vector embedding x is in a form of: λ (x I Ht) = μ (x I Ht) + Σ α(x, xj) φ(t – tj|x, xj) where μ is a background intensity... a decay distribution φ(t – tj|x, xj) models a time decay of causal influence. (Zhang, p. 4, Intensity function "Finally, we express the intensity function as follows: λ*(t) [an intensity function] = softplus (μ [a background intensity] + (η-μ) exp(-γ(t-ti)) [exp(-γ(t-ti)): a decay distribution]), (15)")
The rationale for combining the teachings of Shou and Zhang is the same as set forth in the rejection of claim 1.
Shou, Zhang and Bian do not teach, but Lee teaches: a function α (x, xj) ∈ R+, where R+ denotes a set of nonnegative real numbers, and the function is the kernel function, and (Lee, p. 2, 2.2. Proposed Model Specification and Interpretation "λ(t) = … Y(Ti)… (1)… Y is a stochastic process"; p. 3, 2.2.2. THE CONTAGION PROCESS (Yi)i=1,2,... "The levels of excitation Y measure the impact of clustering or contagion of the event times. To see this, observe in Equation (1) that whenever Y is high and of positive value, it imposes a greater value to the intensity λ")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Shou, Zhang and Bian to incorporate the teachings of Lee by including positive strength of excitation. Doing so would increase the probability of generating an event in a shorter period of time, thereby causing the clustering phenomena. (Lee, p. 2, 2.2. Proposed Model Specification and Interpretation "To see this, observe in Equation (1) that whenever Y is high and of positive value, it imposes a greater value to the intensity λ, thus increasing the probability of generating an event in a shorter period of time, thereby causing the clustering phenomena.")
Response to Arguments
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 101 have been fully considered but they are not persuasive:
Argument: (p. 10-11) Regarding Prong One of Step 2A, the Applicant respectfully submits that amended independent claim 1 of the instant application recites the features of: "… generating, using a transformer model, a self- attention matrix representing dependencies between the past events ... the kernel function determines, based on the self-attention matrix, an instance-level causal strength value between a pair of events of the past events ... " MPEP 2106.04(a)(2)(1) states that claims are not directed to mathematical concepts if they apply mathematical operations and merely involve mathematical operations or include additional features that are more than mere mathematical concepts. Further, the Applicant submits that generating self-attention metric of past events… is not directed to a mathematical concept but rather applies mathematical operations.
Response: The claim recites “generating… a self-attention matrix… am intensity function… a kernel function,” therefore the claim recites mathematical concepts. Further, reciting them with a transformer or any other machine learning model is mere instructions to apply an exception – MPEP(f).
Argument: (p. 11-15) Regarding Prong Two of Step 2A of the 2019 Revised Patent Subject Matter Eligibility Guidance… For example, amended independent claim 1 recites a combination of additional elements of "generating, using the machine learning model, an output based on the inputting of the time-series data ... the output indicates an instance-level causality that links two or more events of the past events ... the output is useable for an event diagnosis task or an event consolidation task". Determining instance-level causality that links two or more events of past events is neither routine nor conventional. Instead, they represent a technological improvement by reducing computational overhead and enabling direct utilization of discovered instance-level causal relationships in event diagnosis or event consolidation tasks, rather than a mere presentation of information using conventional computer functionality.
Response: The claim recites using the machine learning model to generate an output, which is mere instructions to apply an exception – MPEP 2106.05(f). Therefore, the feature does not integrate the exception into a practical application and is not sufficient to be significantly more than the judicial exception. Further, the claim recites the output is useable for a task, which is an interned result, and not given weight.
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 103 have been fully considered but they are not persuasive:
Argument: (p. 18-19) Zhang describes summarizing an influence of all previous events using self- attention by generating a hidden vector, where the hidden vector is used to compute parameters p and ri using non-linear transformations… Consequently, the individual contributions of historical event instances are aggregated into a latent representation before the intensity function is computed. Accordingly, Zhang is directed to the conventional neural point-process approach, by typically embedding event history in the form of a latent state vector, losing instance-wise information. Therefore, Zhang does not teach or suggest…
Response: Zhang uses self-attention to compute attention vector/matrix between each event and the earlier events in the sequence, which are pairwise, instance-level dependencies used in the intensity function. (Zhang, p. 4, Self-Attention "we compute the pairwise influence of one previous event to the next event by employing self-attention… where xi+1 is like query(q) in the attention terminology, xj is the key(k)… ")
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Vaswani ("Attention Is All You Need" 2017) teaches (Vaswani, p. 4, 3.2 Attention " We compute the matrix of outputs as: Attention (Q, K, V) … (1)")
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SU-TING CHUANG whose telephone number is (408)918-7519. The examiner can normally be reached Monday - Thursday 8-5 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.C./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
