CTNF 18/404,881 CTNF 101938 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Preliminary Amendment The preliminary amendment filed on 2024/01/04 has been entered. Claims 1-20 are pending in the application. Information Disclosure Statement The information disclosure statement (IDS) submitted on 2024/01/04, 2025/06/27, 2025/09/03, 2025/11/04, and 2026/04/28. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding independent claims 1 and 11 Step 1 -- whether the claim falls within any statutory category. See MPEP 2106.03. Claim 1 is drawn to a computer-implemented method and claim 11 is drawn to a system. Therefore, each of these claims falls under one of the four categories of statutory subject matter (process; machine, manufacture, or composition of matter). (Step 1: YES). Step 2A Prong One -- whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding independent claim 1, the claim is directed to a computer-implemented method comprising: receiving a request to predict a data entry value based on tabular data comprising a set of features; predicting, using a deep learning network comprising a sequential multistep architecture, the data entry value by, for each respective decision step of the multistep architecture: selecting a subset of features from the set of features, the selected subset of features relevant for predicting the data entry value at the respective decision step; and processing the selected subset of features to generate a decision step output; and generating a final decision output by aggregating the decision step output generated from each respective decision step. The limitations of “selecting a subset of features from the set of features, the selected subset of features relevant for predicting the data entry value at the respective decision step” and “predicting … the data entry value by, for each respective decision step …” , under their broadest reasonable interpretation, recite a mental process. Selecting from a set of features those features relevant to a prediction, and forming a prediction at each step, are concepts practically performed in the human mind, including observation, evaluation, judgment, and opinion. A person evaluating tabular data can, with the aid of pen and paper, identify which columns (features) bear on a predicted entry and form a prediction therefrom. See MPEP 2106.04(a)(2), subsection III. The specification confirms this character of the selection at [0032]-[0033] (feature selection “refers to a process of selecting a subset of features from a larger pool of features based on how useful each feature is towards a given prediction”). The limitations of “processing the selected subset of features to generate a decision step output” and “generating a final decision output by aggregating the decision step output generated from each respective decision step” additionally recite mathematical concepts. Processing feature values to compute a decision step output, and aggregating the per-step outputs into a final output, are mathematical calculations and the organization and manipulation of information through mathematical operations. See MPEP 2106.04(a)(2), subsection I. The specification describes the aggregation as a summation/concatenation of decision step outputs ([0030], [0038]-[0040], e.g., the decision embedding constructed as a sum of activations over the decision steps). Because the claim recites both a mental process and a mathematical concept, and it is not clear that the claim recites distinct exceptions, the limitations are considered together as a single abstract idea rather than parsed into multiple exceptions. See MPEP 2106.04, subsection II.B (discussing Bilski v. Kappos , 561 U.S. 593 (2010)). (Step 2A Prong One: YES). Independent claim 11 is a system claim reciting limitations corresponding to those of claim 1 (receive a request; predict using a deep learning network comprising a sequential multistep architecture by selecting a subset of features and processing the selected subset to generate a decision step output; and generate a final decision output by aggregating). Claim 11 recites the same abstract idea for the same reasons. Step 2A Prong Two -- whether the claim as a whole integrates the recited judicial exception into a practical application. See MPEP 2106.04(d). Regarding independent claim 1, this claim recites the additional elements of “data processing hardware,” “a deep learning network comprising a sequential multistep architecture,” and “receiving … a request to predict a data entry value.” The “data processing hardware” is recited at a high level of generality and is no more than a generic computer used as a tool to perform the recited abstract idea. This amounts to mere instructions to apply the exception on a generic computer. See MPEP 2106.05(f). The specification confirms the generic nature of the hardware, describing it as residing on a user device or a remote system ([0023]-[0025]) and as a generic computing device having a processor, memory, and storage ([0044]-[0045]). The “deep learning network comprising a sequential multistep architecture” merely recites the technological environment--namely, neural networks--in which the abstract idea is performed. The claim recites only the outcome of predicting a data entry value and does not recite any specific technical means by which a technical improvement is achieved; the selecting, processing, and aggregating are stated as results without reciting how those results are accomplished. This additional element therefore amounts to mere instructions to implement the abstract idea on a generic computer, see MPEP 2106.05(f), and generally links the use of the exception to a particular technological environment or field of use, see MPEP 2106.05(h). This treatment is consistent with the 2024 Guidance Update on Patent Subject Matter Eligibility (Example 47, claim 2), in which “using the trained ANN” to detect and analyze data was found to be mere instructions to apply the abstract idea and to generally link the exception to the neural-network environment. The limitation “receiving … a request to predict a data entry value based on tabular data comprising a set of features” is mere data gathering recited at a high level of generality and constitutes insignificant extra-solution activity. See MPEP 2106.05(g). All uses of the recited exception require receiving such input data; this limitation imposes no meaningful limit on the claim. Although the specification asserts benefits of interpretability and parameter efficiency ([0021]-[0022], [0031]-[0032]), these asserted benefits are not reflected in the claim as a specific technical improvement to the functioning of a computer or to another technology; rather, they are characteristics of the abstract idea itself (feature selection and interpretation). See MPEP 2106.04(d)(1) and 2106.05(a). Considered individually and in combination, the additional elements do not impose a meaningful limit on the judicial exception and do not integrate the exception into a practical application. (Step 2A Prong Two: NO; Step 2A: YES). The claim is directed to the abstract idea. Independent claim 11 recites the additional elements of “data processing hardware” and “memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to” perform the recited operations. These are generic computer components that merely act as a tool on which the abstract idea operates, see MPEP 2106.05(f), and claim 11 is not integrated into a practical application for the same reasons set forth for claim 1. (Step 2A Prong Two: NO; Step 2A: YES). Step 2B -- whether the claim provides an inventive concept (significantly more). See MPEP 2106.05. The additional elements identified above are re-evaluated, individually and as an ordered combination, to determine whether they add an inventive concept. The generic data processing hardware (and, in claim 11, the memory hardware storing instructions) and the generically recited deep learning network amount to no more than mere instructions to apply the exception using a generic computer and generic neural-network components, which cannot supply an inventive concept. See MPEP 2106.05(f). The receiving of the request, found above to be insignificant extra-solution activity, is further determined to be well-understood, routine, and conventional. Receiving or obtaining data input is a function the courts have recognized as well-understood, routine, and conventional activity. See MPEP 2106.05(d), subsection II, and 2106.05(g). The specification describes the receipt and handling of tabular data using generic computing resources ([0023]-[0027], [0044]-[0045]) and does not indicate that any additional element, or the combination of additional elements, operates in an unconventional manner. Considered individually and in combination, the additional elements do not amount to significantly more than the judicial exception. (Step 2B: NO). Claims 1 and 11 are ineligible. Regarding dependent claims 2-10 and 12-20 Claims 2-10 and 12-20 merely narrow the previously cited abstract idea or add generically recited neural-network components and generic computer hardware. For the reasons described above with respect to independent claims 1 and 11, the judicial exception of these claims is not meaningfully integrated into a practical application and the claims do not amount to significantly more than the abstract idea. Step 1 -- See MPEP 2106.03. Claims 2-10 depend from claim 1 (process) and claims 12-20 depend from claim 11 (system). Each falls within a statutory category. (Step 1: YES). Step 2A Prong One -- whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding claims 2 and 12, these claims recite “for each respective decision step … determining, using an attentive transformer of the deep learning network, an aggregate of how many times each feature in the selected subset of features has been processed in each preceding decision step.” Determining an aggregate (i.e., a count) of how many times each feature has been processed in preceding steps recites a mathematical concept and a mental process--a tabulation that can be performed by mathematical calculation or in the human mind with pen and paper. See MPEP 2106.04(a)(2), subsections I and III. Regarding claims 4 and 14, these claims recite “processing the selected subset of features to generate information for a next decision step …; and providing the information to the next decision step.” Generating information for, and providing information to, a subsequent step recites the iterative mental and mathematical evaluation by which the abstract idea is carried out across steps, and narrows the abstract idea identified for the parent claims. See MPEP 2106.04(a)(2), subsections I and III. Regarding claims 5 and 15, these claims recite “providing the information to an attentive transformer of the deep learning network that determines, based on the provided information, an aggregate of how many times each feature in the selected subset of features has been processed in each preceding decision step,” which recites the same mathematical concept and mental process set forth for claims 2 and 12. Regarding claims 9 and 19, these claims recite “training the deep learning network using supervised learning for a particular task.” Training a network using supervised learning encompasses mathematical concepts; the training of a neural network is performed through mathematical calculations (e.g., optimization that iteratively adjusts parameters). See MPEP 2106.04(a)(2), subsection I; see also the 2024 Guidance Update on Patent Subject Matter Eligibility (Example 47), in which training using a backpropagation algorithm and a gradient descent algorithm was found to encompass mathematical concepts. Claims 3, 6, 7, 8, 10 and their respective system counterparts 13, 16, 17, 18, 20 do not recite an additional judicial exception. Rather, they recite additional elements--a “fully connected layer and batch normalization” (claims 3, 13); a “feature transformer” (claims 6, 16); “a plurality of neural network layers each including a fully-connected layer, batch normalization, and a generalized linear unit (GLU) nonlinearity” (claims 7, 17); a “rectified linear unit (ReLU)” (claims 8, 18); and “data processing hardware reside[d] on a user device or a remote system” (claims 10, 20)--that narrow the parent abstract idea and are addressed under Step 2A Prong Two below. (Step 2A Prong One: YES for the parent abstract idea narrowed by these claims.) Step 2A Prong Two -- whether the claim as a whole integrates the recited judicial exception into a practical application. See MPEP 2106.04(d). The dependent claims recite the additional elements of an attentive transformer (claims 2, 5, 12, 15), a fully connected layer and batch normalization (claims 3, 13), a feature transformer (claims 6, 16), a plurality of neural network layers each including a fully-connected layer, batch normalization, and a GLU nonlinearity (claims 7, 17), a rectified linear unit (claims 8, 18), supervised training “for a particular task” (claims 9, 19), and generic data processing hardware residing on a user device or a remote system (claims 10, 20). These additional elements are generically recited neural-network building blocks and generic computer hardware. Each amounts to mere instructions to implement the abstract idea using a generic computer and generic neural-network components, see MPEP 2106.05(f), and/or generally links the abstract idea to a particular technological environment (neural networks) or field of use, see MPEP 2106.05(h). The recitation in claims 9 and 19 of training “for a particular task” generally links the exception to a field of use, see MPEP 2106.05(h), and the hardware of claims 10 and 20 is a generic computer recited as a tool, see MPEP 2106.05(f). The specification describes these components as known and generic--the attentive transformer as a single fully connected layer performing batch normalization ([0035]), the feature transformer layers as fully-connected layers followed by batch normalization and a gated linear unit nonlinearity ([0037]-[0038]), the ReLU as a conventional activation function ([0038]), and the hardware generically ([0023]-[0025], [0044]-[0045]). No dependent claim recites a specific technical improvement reflected in the claim that integrates the exception into a practical application. (Step 2A Prong Two: NO; Step 2A: YES). Step 2B -- whether the claim provides an inventive concept. See MPEP 2106.05. Re-evaluated individually and as an ordered combination, the additional elements of the dependent claims are well-understood, routine, and conventional neural-network components and generic computer hardware (fully connected layers, batch normalization, GLU and ReLU nonlinearities, attentive and feature transformers, and a generic processor/memory), as confirmed by the specification cited above, and amount to mere instructions to apply the abstract idea. See MPEP 2106.05(d) and 2106.05(f). They do not, alone or in combination, add an inventive concept. (Step 2B: NO). Accordingly, claims 1-20 recite an abstract idea that is not integrated into a practical application and do not amount to significantly more than the judicial exception, and are rejected under 35 U.S.C. 101. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-103 AIA The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim s 1, 2, 4-6, 8, 9, 11, 12, 14-16, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ravishankar (US 2019/0130247 A1) in view of Petschulat (US 2014/0280066 A1), and further in view of Laukien (WO 2017/205850 A1) . As to independent Claim 1 , Ravishankar teaches a computer-implemented method that when executed on data processing hardware causes the data processing hardware to perform operations comprising ( Ravishankar , ¶[0046]–[0047], disclosing one or more processors 152 and a memory 156 storing the executable routines that carry out the described functions): predicting, using a deep learning network, the data entry value ( Ravishankar , ¶[0023]: “the multi-task feature ranking network 116 is in the form of a multi-task deep learning algorithm in which input features 102 … are processed by K neural networks or branches 120 of a neural network for predicting the K tasks 122 (data entry value)”; selecting a subset of features from the set of features, the selected subset of features relevant for predicting the data entry value at the respective sequential processing step ( Ravishankar , ¶[0023]: “a broadcast layer 126 is provided which selects and/or filters the input features 102 before they propagate through the separate, parallel branches 120 … the weights w of the broadcast layer 126 reveal the relevance of feature 102 for different tasks 122 … which may facilitate subsequent selection of one or more features as being relevant”; ¶[0030]: “Importance(n,k)=abs(W(n))”; ¶[0034]: “automatically selecting a subset of relevant features based upon the largest weights”). “Selecting a subset of features” reads on the broadcast layer’s selecting and filtering of the input features, and “relevant for predicting the data entry value” reads on Ravishankar ’s use of the learned weights to reveal each feature’s relevance and to keep the highest-importance features as the selected subset; processing the selected subset of features to generate a decision step output ( Ravishankar , ¶[0023], where the filtered features “are processed by K neural networks or branches 120”; claim 1: “two or more separate and parallel branches … configured to receive a set of filtered inputs from the broadcast layer”). The filtered subset is processed by a branch of the network, and that branch’s output corresponds to the recited “decision step output”; and generating a final decision output by aggregating the decision step output generated from each respective sequential processing step ( Ravishankar , ¶[0003] and claim 1: “an output layer downstream from the branches, wherein the output layer is configured to provide an output of the feature ranking neural network”). The downstream output layer combines the branch outputs into the network’s output, which corresponds to the recited “final decision output.” Ravishankar receives a set of input features at an input layer and predicts an output from them ( Ravishankar , ¶[0004]: “a plurality of input samples is provided as a training data set to an input layer … [e]ach input sample is characterized by one or more features”), but it does not frame that input as tabular data or describe receiving a request to predict a particular data-entry value of such data. Ravishankar therefore does not teach “receiving a request to predict a data entry value based on tabular data comprising a set of features.” In the same field of endeavor, Petschulat teaches receiving a request to predict a data entry value based on tabular data comprising a set of features. Petschulat takes in a tabular dataset arranged as columns and rows, locates the null (unknown) entries within it, and receives a request to run a predictive query that supplies those entries ( Petschulat , Abstract: “receiving a tabular dataset from a user as input, the tabular dataset having data values organized as columns and rows; identifying a plurality of null values within the tabular dataset … the null values depicted as unknown values”; ¶[0017]: “FIG. 8 depicts an exemplary tabular dataset”; ¶[0036]–[0041]: FIGS. 16A–F “illustrate usage of the PREDICT command”; and the predictive-query flow, step “Receiving a request for a predictive query or a latent structure query,” and FIG. 16, step “PREDICT a single target Column” → “Output”). The columns of the dataset are the recited “set of features,” the dataset itself is the recited “tabular data,” and a predicted null entry is the “data entry value” that the received request asks the system to predict. Ravishankar and Petschulat are analogous to the claimed invention as both are from the same field of endeavor of machine-learning prediction over tabular and otherwise structured feature data. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply the feature-selecting deep neural network of Ravishankar to the tabular-value prediction of Petschulat , so that the network receives the request and predicts the unknown tabular data-entry value. The motivation to combine Ravishankar and Petschulat is supplied by Ravishankar itself, which teaches that selecting the relevant subset of features yields “efficient models leading to better stability & regularization along with reduced compute and memory” ( Ravishankar , Abstract) and improves the interpretability of the prediction ( Ravishankar , ¶[0033]); a skilled artisan would have drawn on those benefits to predict Petschulat ’s unknown tabular entries with an interpretable, parameter-efficient model, with a reasonable expectation of success because both references operate on the same kind of column/feature-vector input. The combination of Ravishankar and Petschulat , however, does not teach “predicting, using a deep learning network, the data entry value by, for each respective sequential processing step of multiple sequential processing steps” performing the selecting and processing , and “ generating a final decision output by aggregating the decision step output generated from each respective sequential processing step. ” Ravishankar spreads its processing across parallel branches rather than a sequence of steps whose per-step outputs are accumulated. In the same field of endeavor, Laukien teaches predicting by passing the input through a series of sequential stages and combining their per-stage outputs into a single output. Laukien discloses “first through Nth processing stages, each processing stage having a respective encoder and a respective decoder,” in which “encoders are coupled output-to-input, with the input of the first encoder receiving an overall processing input,” and “each decoder provid[es] feedback to the previous decoder and the first decoder provid[es] an overall processing output” ( Laukien , Abstract; FIG. 3). The first-through-Nth stages, coupled output-to-input, read on the recited “multiple sequential processing steps,” and the chaining of the stage decoders to produce one “overall processing output” reads on “generating a final decision output by aggregating the decision step output generated from each respective sequential processing step.” Ravishankar , Petschulat , and Laukien are analogous to the claimed invention as all are from the same field of endeavor of machine-learning prediction from input data using neural and predictive network architectures, Laukien being classified in IPC G06N 3/04. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to perform the feature selection and processing of Ravishankar , as applied to the tabular prediction of Petschulat , at each of the multiple sequential stages taught by Laukien , and to aggregate the per-stage outputs into the final decision output. The motivation to combine Ravishankar , Petschulat , and Laukien is provided by Laukien , which teaches that arranging prediction as successive stages whose encodings are “based at least in part on history” and whose decoder outputs are combined into a single overall output improves predictive performance ( Laukien , Abstract); a skilled artisan would accordingly have arranged the feature selection of Ravishankar as successive steps per Laukien and aggregated the resulting outputs to improve the prediction of Petschulat ’s tabular data-entry value, with a reasonable expectation of success. As to independent Claim 11, Claim 11 recites the same operative limitations as Claim 1 in system form and is rejected on the same grounds as Claim 1 over Ravishankar in view of Petschulat and further in view of Laukien . Ravishankar additionally teaches the recited “data processing hardware” and “memory hardware in communication with the data processing hardware, the memory hardware storing instructions” ( Ravishankar , ¶[0046]: “one or more processors 152”; ¶[0047]: “the memory 156 may encompass any tangible, non-transitory medium for storing data or executable routines”). As to independent Claim 11 , Claim 11 recites the same operative limitations as Claim 1 in system form and is rejected on the same grounds as Claim 1 over Ravishankar in view of Petschulat and further in view of Laukien . Ravishankar additionally teaches the recited “data processing hardware” and “memory hardware in communication with the data processing hardware, the memory hardware storing instructions” ( Ravishankar , ¶[0046]: “one or more processors 152”; ¶[0047]: “the memory 156 may encompass any tangible, non-transitory medium for storing data or executable routines”). Regarding claims 2 and 12 The limitations of claims 1 and 11, from which claims 2 and 12 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 2 and 12 further recite: for each respective sequential processing step of the multiple sequential processing steps, determining, using an attentive transformer of the deep learning network, an aggregate of how many times each feature in the selected subset of features has been processed in each preceding sequential processing step of the multiple sequential processing steps ( Ravishankar , ¶[0023]–[0024], [0030]; Laukien , Abstract). Ravishankar teaches determining, using an attentive transformer of the deep learning network, a per-feature aggregate that governs feature selection. Specifically, Ravishankar discloses a learned, element-wise feature-weighting layer--"the broadcast layer 126 is an element-wise multiplicative layer whose weights, w, are learned during neural network training," and those weights "reveal the relevance of feature 102 for different tasks 122" ( Ravishankar , ¶[0023])--expressly identifies that layer as "the feature weighting layer (i.e., broadcast layer 126)" ( Ravishankar , ¶[0024]), and computes a per-feature relevance quantity, "Importance(n,k)=abs(W(n))" ( Ravishankar , ¶[0030]). The recited "attentive transformer of the deep learning network" reads on this learned feature-weighting layer, and the recited "determining … an aggregate" of per-feature usage reads on Ravishankar 's accumulation of per-feature weight/importance that governs whether each feature is selected. However, Ravishankar does not teach: (1) performing the determining "for each respective sequential processing step of the multiple sequential processing steps"; and (2) that the aggregate is "an aggregate of how many times each feature in the selected subset of features has been processed in each preceding sequential processing step of the multiple sequential processing steps." Ravishankar performs its feature weighting once, at a single broadcast layer feeding parallel branches, and does not accumulate the per-feature quantity across multiple preceding sequential steps. In the same field of endeavor, Laukien teaches the foregoing sub-limitations that Ravishankar lacks. Laukien discloses "first through Nth processing stages," which read on the recited "multiple sequential processing steps," and teaches that the processing at each stage depends on the preceding stages, in that "[e]ach encoded output is based upon both (1) a current input value and (2) one or more previous input values, such that encodings are based at least in part on history" ( Laukien , Abstract). Applying Ravishankar 's feature-weighting (attentive) layer at each of Laukien 's successive, history-based stages provides, at each step, the recited aggregate of how each feature in the selected subset has been processed in the preceding steps. Ravishankar and Laukien are analogous to the claimed invention as both are from the same field of endeavor of machine-learning prediction using neural-network architectures. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Laukien such that Ravishankar 's attentive feature-weighting layer is applied at each of Laukien 's successive sequential processing steps and accumulates, at each step, an aggregate of each feature's usage in the preceding steps. The motivation to combine Ravishankar and Laukien is that Ravishankar teaches that weighting and selecting the relevant features yields "efficient models leading to better stability & regularization along with reduced compute and memory" ( Ravishankar , Abstract) and improves interpretability ( Ravishankar , ¶[0033]), while Laukien teaches that basing each successive stage on the history of the preceding stages improves predictive performance ( Laukien , Abstract); accumulating Ravishankar 's per-feature weighting across Laukien 's preceding sequential steps predictably refines the per-step feature selection, with a reasonable expectation of success. Regarding claims 4 and 14 The limitations of claims 1 and 11, from which claims 4 and 14 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 4 and 14 further recite: for each respective sequential processing step of the multiple sequential processing steps, processing the selected subset of features to generate information for a next sequential processing step of the multiple sequential processing steps; and providing the information to the next sequential processing step ( Ravishankar , ¶[0023]; Laukien , Abstract and FIG. 3). Ravishankar teaches processing the selected subset of features, in that the filtered subset of features output by the broadcast layer "are processed by K neural networks or branches 120 of a neural network" ( Ravishankar , ¶[0023]). The recited "processing the selected subset of features" reads on Ravishankar 's processing of the filtered feature subset through the network. However, Ravishankar does not teach: (1) processing the selected subset of features "to generate information for a next sequential processing step of the multiple sequential processing steps"; and (2) "providing the information to the next sequential processing step." Ravishankar processes the filtered features through parallel branches and does not generate, from one step, information that is forwarded to and used by a next sequential processing step. In the same field of endeavor, Laukien teaches the foregoing sub-limitations that Ravishankar lacks. Laukien discloses "first through Nth processing stages," which read on the recited "multiple sequential processing steps," and teaches that each stage generates output information that is forwarded to the next stage, in that "[e]ncoders are coupled output-to-input, with the input of the first encoder receiving an overall processing input" ( Laukien , Abstract; FIG. 3). The recited generating of "information for a next sequential processing step" reads on Laukien 's generation of a stage's encoded output, and the recited "providing the information to the next sequential processing step" reads on Laukien 's output-to-input coupling, by which the output of one stage is supplied as the input to the next stage. Ravishankar and Laukien are analogous to the claimed invention as both are from the same field of endeavor of machine-learning prediction using neural-network architectures. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Laukien such that the selected subset of features processed by Ravishankar generates, at each step, information that is forwarded to a next step in accordance with Laukien 's output-to-input coupling of successive stages. The motivation to combine Ravishankar and Laukien is that Ravishankar teaches efficient, feature-selective processing within a neural network ( Ravishankar , Abstract, ¶[0023]), while Laukien teaches that coupling successive stages output-to-input, such that each stage's processing is "based at least in part on history" of the preceding stages, improves predictive performance ( Laukien , Abstract); forwarding the per-step processed feature information to the next step as taught by Laukien predictably enables the multistep refinement of the prediction, with a reasonable expectation of success. Regarding claims 5 and 15 The limitations of claims 4 and 14, from which claims 5 and 15 respectively depend, are rejected under the same rationale set forth above for claims 4 and 14. Claims 5 and 15 further recite: providing the information to the next sequential processing step comprises providing the information to an attentive transformer of the deep learning network that determines, based on the provided information, an aggregate of how many times each feature in the selected subset of features has been processed in each preceding sequential processing step of the multiple sequential processing steps ( Ravishankar , ¶[0023]–[0024], [0030]; Laukien , Abstract). Ravishankar teaches the attentive transformer that determines a per-feature aggregate, in that the learned, element-wise feature-weighting (broadcast) layer determines a per-feature relevance quantity, "Importance(n,k)=abs(W(n))," that governs feature selection ( Ravishankar , ¶[0023]–[0024], [0030]). However, Ravishankar does not teach: (1) that the information is "provided … to the next sequential processing step"; and (2) that the aggregate determined by the attentive transformer is "an aggregate of how many times each feature … has been processed in each preceding sequential processing step of the multiple sequential processing steps." In the same field of endeavor, Laukien teaches these sub-limitations. Laukien discloses "first through Nth processing stages" coupled output-to-input, such that the output of one stage is provided to the next ( Laukien , Abstract; FIG. 3), and teaches that each stage's processing is "based at least in part on history" of the preceding stages ( Laukien , Abstract). The recited provision of the information "to the next sequential processing step" reads on Laukien 's output-to-input coupling, and the recited aggregate "in each preceding sequential processing step" reads on Laukien 's history-based dependence on the preceding stages. Ravishankar and Laukien are analogous to the claimed invention as both are from the same field of endeavor of machine-learning prediction using neural-network architectures. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Laukien such that Ravishankar 's attentive feature-weighting layer receives the information provided to the next step in accordance with Laukien 's output-to-input coupling and determines therefrom an aggregate of each feature's usage in the preceding steps. The motivation to combine Ravishankar and Laukien is that Ravishankar teaches efficient, interpretable feature weighting and selection ( Ravishankar , Abstract, ¶[0033]), while Laukien teaches that coupling successive stages output-to-input and basing each stage on the history of the preceding stages improves predictive performance ( Laukien , Abstract); providing the per-step information to the attentive transformer as taught by Laukien predictably refines the per-step feature selection, with a reasonable expectation of success. Regarding claims 6 and 16 The limitations of claims 1 and 11, from which claims 6 and 16 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 6 and 16 further recite: processing the selected subset of features to generate the decision step output comprises processing the selected subset using a feature transformer of the deep learning network ( Ravishankar , ¶[0023], [0038]). Ravishankar teaches this limitation. Ravishankar discloses that the selected, filtered subset of features "are processed by K neural networks or branches 120 of a neural network" ( Ravishankar , ¶[0023]), the branches being implemented as fully-connected/hidden neural-network layers ( Ravishankar , ¶[0038]). The recited "feature transformer of the deep learning network" reads on Ravishankar 's branch layers that process the selected subset of features to produce the decision step output. Regarding Claims 8 and 18 The limitations of claims 1 and 11, from which claims 8 and 18 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 8 and 18 further recite: the decision step output generated by processing the selected subset of features passes through a rectified linear unit (ReLU) of the deep learning network ( Ravishankar , ¶[0017]). Ravishankar teaches this limitation. Ravishankar expressly discloses "an activation function, such as rectified linear unit (ReLU)" within the neural network ( Ravishankar , ¶[0017]). The recited passing of the decision step output "through a rectified linear unit (ReLU) of the deep learning network" reads on Ravishankar 's use of the ReLU activation function in the network that produces the output. Regarding Claims 9 and 19 The limitations of claims 1 and 11, from which claims 9 and 19 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 9 and 19 further recite: training the deep learning network using supervised learning for a particular task ( Ravishankar , ¶[0004], [0038]). Ravishankar teaches this limitation. Ravishankar discloses that "[t]he input samples are processed to train the respective weight for each task" ( Ravishankar , ¶[0004]) and that "[a] multilayer perceptron (MLP) neural network was trained to predict the presence of IMI" ( Ravishankar , ¶[0038]), IMI detection being a particular task. The recited "training the deep learning network using supervised learning for a particular task" reads on Ravishankar 's supervised training of the network, using labeled input samples, to perform a particular task . 07-21-aia AIA Claim s 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Ravishankar , Petschulat , in view of Laukien , and further in view of Ioffe (Ioffe & Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv:1502.03167), published March 2, 2015, and relied upon at pages 1-11 . The limitations of claims 2 and 12, from which claims 3 and 13 respectively depend, are rejected under the same rationale set forth above for claims 2 and 12. Claims 3 and 13 further recite: the attentive transformer comprises a fully connected layer and batch normalization ( Ravishankar , ¶[0023], [0038]; Ioffe , Abstract). Ravishankar teaches that the attentive transformer comprises a fully connected layer. As set forth for claims 2 and 12, the recited "attentive transformer" reads on Ravishankar 's learned feature-weighting (broadcast) layer, which is an element-wise/fully-connected layer whose weights are learned during training ( Ravishankar , ¶[0023]), and Ravishankar further discloses that the network is implemented with fully-connected layers--"A multilayer perceptron (MLP) neural network … Three hidden layers" ( Ravishankar , ¶[0038]). The recited "fully connected layer" therefore reads on Ravishankar 's fully-connected weighting/hidden layer. However, Ravishankar does not teach that the attentive transformer comprises batch normalization. In the same field of endeavor, Ioffe teaches batch normalization. Ioffe discloses a technique that addresses internal covariate shift "by normalizing layer inputs … performing the normalization for each training mini-batch" ( Ioffe , Abstract). The recited "batch normalization" reads on Ioffe 's normalization of layer inputs performed for each training mini-batch, which is applied to the inputs of a neural-network layer such as the fully-connected layer of Ravishankar . Ravishankar and Ioffe are analogous to the claimed invention as both are from the same field of endeavor of constructing and training deep neural networks. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Ioffe such that the fully-connected weighting layer of Ravishankar 's attentive transformer is provided with the batch normalization of Ioffe . The motivation to combine Ravishankar and Ioffe is that Ioffe teaches that batch normalization "allows us to use much higher learning rates and be less careful about initialization" and accelerates and stabilizes training by reducing internal covariate shift ( Ioffe , Abstract); applying Ioffe 's batch normalization to the fully-connected layer of Ravishankar is the use of a known technique to improve a similar neural-network layer in the same predictable way, with a reasonable expectation of success . 07-21-aia AIA Claim s 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ravishankar , Petschulat , and Laukien , and further in view of Ioffe and Dauphin (Dauphin et al., "Language Modeling with Gated Convolutional Networks," arXiv:1612.08083), published September 8, 2017, and relied upon at pages 1-9 . The limitations of claims 6 and 16, from which claims 7 and 17 respectively depend, are rejected under the same rationale set forth above for claims 6 and 16. Claims 7 and 17 further recite: the feature transformer of the deep learning network comprises a plurality of neural network layers each including a fully-connected layer, batch normalization, and a generalized linear unit (GLU) nonlinearity ( Ravishankar , ¶[0038] and claim 6; Ioffe , Abstract; Dauphin , § 2 (Approach), Eq. (1)). Ravishankar teaches that the feature transformer comprises a plurality of fully-connected neural network layers, disclosing "A multilayer perceptron (MLP) neural network … Three hidden layers … each … 200 neurons" ( Ravishankar , ¶[0038]) and "a plurality of hidden layers" ( Ravishankar , claim 6). The recited "plurality of neural network layers each including a fully-connected layer" reads on Ravishankar 's plurality of fully-connected hidden layers. However, Ravishankar does not teach that each such layer includes batch normalization. In the same field of endeavor, Ioffe teaches batch normalization. Ioffe discloses a technique that addresses internal covariate shift by "normalizes layer inputs … performing the normalization for each training mini-batch" ( Ioffe , Abstract). The recited “batch normalization” reads on Ioffe’s normalization of layer inputs performed for each training mini-batch. Ravishankar and Ioffe are analogous to the claimed invention as both are from the same field of endeavor of constructing and training deep neural networks. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Ioffe such that each fully-connected layer of Ravishankar’s feature transformer includes the batch normalization of Ioffe . The motivation to combine Ravishankar and Ioffe is that Ioffe teaches that batch normalization “allows us to use much higher learning rates and be less careful about initialization” and accelerates and stabilizes training by reducing internal covariate shift ( Ioffe , Abstract), with a reasonable expectation of success. The combination of Ravishankar and Ioffe , however, does not teach a generalized linear unit (GLU) nonlinearity. In the same field of endeavor, Dauphin teaches the gated linear unit nonlinearity, h(X) = (X ∗ W+b) ⊗ σ(X ∗ V+c), "σ being the sigmoid function and ⊗ … the element-wise product," which Dauphin expressly "dubs … Gated Linear Units (GLU)" ( Dauphin , § 2 (Approach), Eq. (1)). The recited "generalized linear unit (GLU) nonlinearity" reads on Dauphin 's gated linear unit. Ravishankar , Ioffe , and Dauphin are analogous to the claimed invention as all are from the same field of endeavor of constructing and training deep neural-network layers. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar , Ioffe , and Dauphin such that each fully-connected layer of Ravishankar 's feature transformer includes the batch normalization of Ioffe and the gated linear unit of Dauphin . The motivation to combine Ravishankar and Ioffe is that Ioffe teaches that batch normalization "allows us to use much higher learning rates and be less careful about initialization" and accelerates and stabilizes training ( Ioffe , Abstract); and the motivation to combine Ravishankar and Dauphin is that Dauphin teaches that gated linear units provide "deterministic gates that reduce the vanishing gradient problem" while retaining nonlinear capabilities ( Dauphin , § 3 (Gating Mechanisms)). Incorporating these known layer components into Ravishankar 's fully-connected layers is the use of known techniques to improve a similar neural-network layer in the same predictable way, with a reasonable expectation of success . 07-21-aia AIA Claim s 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ravishankar , Petschulat , and Laukien , and further in view of Dzhulgakov (US 2019/0073580 A1) . The limitations of claims 1 and 11, from which claims 10 and 20 respectively depend, are rejected under the same rationale set forth above for claims 1 and 11. Claims 10 and 20 further recite: the data processing hardware resides on a user device or a remote system ( Ravishankar , ¶[0046]–[0047]; Dzhulgakov , Abstract). Ravishankar teaches the data processing hardware, disclosing "one or more processors 152" and a memory storing executable routines ( Ravishankar , ¶[0046]–[0047]). However, Ravishankar does not teach that the data processing hardware "resides on a user device or a remote system." In the same field of endeavor, Dzhulgakov teaches this sub-limitation. Dzhulgakov discloses distributing the neural-network computation between a local user machine and a remote machine --"a local machine that receives user" input and a "remote machine that stores embedding matrices and parameters" ( Dzhulgakov , Abstract). The recited residence of the data processing hardware "on a user device or a remote system" reads on Dzhulgakov 's local user machine and remote machine. Ravishankar and Dzhulgakov are analogous to the claimed invention as both are from the same field of endeavor of deploying neural-network computation on computing hardware. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine Ravishankar and Dzhulgakov such that the data processing hardware of Ravishankar resides on a user device or a remote system as taught by Dzhulgakov . The motivation to combine Ravishankar and Dzhulgakov is the predictable distribution of computation and storage between a resource-limited user device and a remote system, as Dzhulgakov teaches, so as to leverage remote resources while serving the user locally, with a reasonable expectation of success. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG VAN LE whose telephone number is (571)270-0164. The examiner can normally be reached 8 a.m. - 5 p.m.. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HUNG VAN LE/Examiner, Art Unit 2145 /CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145 Application/Control Number: 18/404,881 Page 2 Art Unit: 2145 Application/Control Number: 18/404,881 Page 3 Art Unit: 2145 Application/Control Number: 18/404,881 Page 4 Art Unit: 2145 Application/Control Number: 18/404,881 Page 5 Art Unit: 2145 Application/Control Number: 18/404,881 Page 6 Art Unit: 2145 Application/Control Number: 18/404,881 Page 7 Art Unit: 2145 Application/Control Number: 18/404,881 Page 8 Art Unit: 2145 Application/Control Number: 18/404,881 Page 9 Art Unit: 2145 Application/Control Number: 18/404,881 Page 10 Art Unit: 2145 Application/Control Number: 18/404,881 Page 11 Art Unit: 2145 Application/Control Number: 18/404,881 Page 12 Art Unit: 2145 Application/Control Number: 18/404,881 Page 13 Art Unit: 2145 Application/Control Number: 18/404,881 Page 14 Art Unit: 2145 Application/Control Number: 18/404,881 Page 15 Art Unit: 2145 Application/Control Number: 18/404,881 Page 16 Art Unit: 2145 Application/Control Number: 18/404,881 Page 17 Art Unit: 2145 Application/Control Number: 18/404,881 Page 18 Art Unit: 2145 Application/Control Number: 18/404,881 Page 19 Art Unit: 2145 Application/Control Number: 18/404,881 Page 20 Art Unit: 2145 Application/Control Number: 18/404,881 Page 21 Art Unit: 2145 Application/Control Number: 18/404,881 Page 22 Art Unit: 2145 Application/Control Number: 18/404,881 Page 23 Art Unit: 2145 Application/Control Number: 18/404,881 Page 24 Art Unit: 2145 Application/Control Number: 18/404,881 Page 25 Art Unit: 2145
Prosecution Insights