DETAILED ACTION
This action is responsive to the Claims filed on 03/07/2024. Claims 1-15 are canceled. Claims 16-35 are pending in the case. Claims 16, 23 and 30 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 16-35 are rejected under 35 U.S.C. 101 because the claim are directed to an abstract idea without significantly more.
Regarding Claim 16/23/30:
Under step 1, the claim is directed to a method which is directed to a process, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations:
to perform operations for implementing a neural network system… process the system input to generate a system output… for generating a corresponding data value based on which the system output is generated… combining the plurality of inputs according to a set of weights defined by the side information to generate an initial output;… generating, from the initial output, a node probability output that defines a probability distribution over possible values for the corresponding data value; and providing as output the node probability output for the corresponding data value
Each of these amount to mental evaluation because they describe manipulation of abstract data. Performing operations of a neural network system and combining and generating output data based on input data are all evaluations which can be performed in the mind. Examiner notes it is not the neural network itself which can be performed in the mind, but rather the claimed operations which the neural network performs are mental evaluations.
Under step 2A Prong 2, The claim recites the following additional element(s):
A system comprising: one or more computers, and one or more storage devices on which are stored instructions, that are operable, when executed by the one or more computers, to cause the one or more computers… wherein each gated linear network is used … wherein each node in each gated linear layer is configured to perform operations comprising: [claim 23] A method performed by a neural network system configured…[claim 30] One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, causes the one or more computers to perform operations for implementing a neural network system (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”)
Further, the additional elements: wherein the neural network system comprises one or more neural networks, wherein the one or more neural networks comprise one or more gated linear network… is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Further, the additional elements: … to receive a system input…wherein each gated linear network comprises a plurality of layers arranged in a hierarchy of layers, wherein the plurality of layers comprises a plurality of gated linear layers, wherein each gated linear layer has one or more nodes… receiving a plurality of inputs from nodes in a layer below the gated linear layer in the hierarchy of layers; receiving side information for the node … that amounts to adding insignificant extra-solution activity to the judicial exception, because the limitation describe mere data gathering. See MPEP 2106.05(g)
Therefore the claim is directed to a judicial exception.
Further, additional element … to receive a system input… receiving side information for the node … is well understood, routine, and conventional activity because it amounts to “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) )
The additional elements of wherein each gated linear network comprises a plurality of layers arranged in a hierarchy of layers, wherein the plurality of layers comprises a plurality of gated linear layers, wherein each gated linear layer has one or more nodes… receiving a plurality of inputs from nodes in a layer below the gated linear layer in the hierarchy of layers are insignificant extra-solution activities that are considered well-understood, routine, conventional activities. In accordance with the MPEP, the following factual determination is based on the technical publication: Eddington et al., "INTEGRATED SENSOR-ARRAY PROCESSOR", US Document ID US 20170309292A1 (PTO-892). Eddington para 054 “Note that the aforementioned neural network may include any one or more of the well-known classes of neural algorithms, including but not limited to perceptrons, feedforward neural networks, deep neural networks (DNNs),” discloses that deep neural networks are well-known (corresponding to routine and conventional). By definition, a deep neural network is a network of nodes arranged in a hierarchy of layers which receive input from nodes a layer beneath them.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 17/24/31:
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the system output comprises an image comprising a plurality of pixels…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 18/25/32
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the system input comprises an input image, and wherein the system output comprises a classification output that classifies the input image into one of a pre-determined plurality of classes…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 19/26/33
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the system input comprises a sequence of data items, and wherein the system output specifies a probability density function for the sequence of data items…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 20/27/34
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the sequence of data items represents one of: a still or moving image; sound data; text data; object position data, environment state data, action data, or a combination thereof, or atomic position data…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 21/28/35
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the system output comprises: control data for controlling an agent moving in a simulated or real-world environment; or data predicting a future image or video sequence seen by a real or virtual camera associated with a physical object or the agent in the simulated or real-world environment…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 22/29
The claim depends on the rejected base claim
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the one or more gated linear networks are implemented in parallel across different special-purpose hardware…is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 16, 21, 23, 28, 30 and 35 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jordan et al. “Hierarchical mixtures of experts and the EM algorithm”
Claim 16/23/30
Jordan teaches, [claim 16] A system comprising: one or more computers, and one or more storage devices on which are stored instructions, that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations [claim 23] A method performed by a neural network system configured [claim 30] One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, causes the one or more computers to perform operations for implementing a neural network system (abstract “We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's)… Comparative simulation results are presented” pg 5 Section 2.5.1 “CART and
MARS required similar CPU time as compared to the HME algorithm”)
to receive a system input and process the system input to generate a system output, (pg 1 Section 2 “The algorithms that we discuss in this paper are supervised learning algorithms. We explicitly ad- dress the case of regression, in which the input vectors are elements of Rm and the output vectors are elements of Rn” further figure 1 depicts the system receiving input which is processed to generate system output.)
wherein the neural network system comprises one or more neural networks, (pg 1 Section 1 “In this paper we present a neural network architecture that is a close cousin to architectures such as CART and MARS.”)
wherein the one or more neural networks comprise one or more gated linear networks, (pg 1 Section 2 “The hierarchical mixture-of-experts (HME) architecture is shown in Figure l.’ The architecture is a tree in which the gating networks sit at the non-terminals of the tree.” Pg 2 Section 2 “all of the expert networks in the tree are linear” therefore the network is both gated and linear)
wherein each gated linear network is used for generating a corresponding data value based on which the system output is generated, (pg 1 Section 1 “The architecture is a tree in which the gating networks sit at the non- terminals of the tree. These networks receive the vector x as input and produce scalar outputs” also shown in figure 1)
wherein the plurality of layers comprises a plurality of gated linear layers, wherein each gated linear layer has one or more nodes, and wherein each node in each gated linear layer is configured to perform operations comprising ( pg 2 ¶ 05 “The hierarchical mixture-of-experts (HME) architecture is shown in Figure 1.1 The architecture is a tree in which the gating networks sit at the non terminals of the tree. These networks receive the vector x as input and produce scalar outputs that are a partition of unity at each point in the input space. The expert networks sit at the leaves of the tree. Each expert produces an output vector uij for each input vector” and figure 1
PNG
media_image1.png
326
380
media_image1.png
Greyscale
the figure depicts a network comprise a hierarchy of layers, there are at least two gated linear layers which are regulated by the gating network modules to perform operations)
receiving a plurality of inputs from nodes in a layer below the gated linear layer in the hierarchy of layers; receiving side information for the node (figure 1 depicted above, each node shown receives a plurality of at least two inputs from a prior layer, side information is mediated by the gating network block)
combining the plurality of inputs according to a set of weights defined by the side information to generate an initial output ( pg 5 ¶2 “The output vector at each nonterminal of the tree is the weighted output of the experts below that nonterminal. That is, the output at the I th nonterminal in the second layer of the two-level tree is:
PNG
media_image2.png
39
148
media_image2.png
Greyscale
and the output at the top level of the tree is:
PNG
media_image3.png
61
149
media_image3.png
Greyscale
the output at each node is the weighted sum of its inputs, the input are from a previous layer below the hierarchy.)
generating, from the initial output, a node probability output that defines a probability distribution over possible values for the corresponding data value (pg 5 ¶02 “The output vector at each nonterminal of the tree is the weighted output of the experts below that nonterminal” The output at each node is the output of the gated linear network, it is a weighted sum of the respective data value input to the network pg6 ¶02 “The hierarchy can be given a probabilistic interpretation… our choice of parameterization … corresponds to a multinomial logit probability model at each nonterminal of the tree” the model can also be interpreted as a conditional probability according to the dispersion parameters
PNG
media_image4.png
29
126
media_image4.png
Greyscale
”)
and providing as output the node probability output for the corresponding data value. (pg 7 Given these assumptions, the total probability of generating y from x is the mixture of the probabilities of generating y from each of the component densities, where the mixing proportions are multinomial probabilities:
PNG
media_image5.png
56
422
media_image5.png
Greyscale
…We also utilize Equation 9 without the superscripts to refer to the probability model defined by a particular HME architecture,” the output probability is a product of the parameters and the corresponding data value)
Claim 21/28/35
Jordon teaches claim 16/23/30
Jordon teaches, wherein the system output comprises: control data for controlling an agent moving in a simulated or real-world environment; or data predicting a future image or video sequence seen by a real or virtual camera associated with a physical object or the agent in the simulated or real-world environment. (pg 4 Section 2.5.1 “We tested the algorithm on a nonlinear system identification problem. The data were obtained from a simulation of a four-joint robot arm moving in three-dimensional space. The network must learn the forward dynamics of the arm; a mapping from twelve coupled input variables to four output variables” the forward dynamics output is for controlling an agent moving in a simulated environment)
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 17-20, 24-27, 31-34 are rejected under 35 U.S.C. § 103 as being unpatentable over Jordon further in view of Gregor et al. “DRAW: A Recurrent Neural Network For Image Generation”
Claim 17/24/31
Jordon teaches claim 16/23/30
Jordon does not explicitly teach, wherein the system output comprises an image comprising a plurality of pixels.
Gregor when addressing image generation via neural networks teaches, wherein the system output comprises an image comprising a plurality of pixels (Abstract “This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural network architecture for image generation…. The system substantially improves on the state of the art for generative models on MNIST, and, when trained on the Street View House Numbers dataset, it generates images that cannot be distinguished from real data with the naked eye” an image as described in the art consists of a plurality of pixels)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify neural network of Jordon which is a system for mapping input data to output data to comprise the image generation of Gregor. One would have been motivated to make such a combination because both references describe the mapping of input data to output data via trained networks. Further, Gregor notes “The system substantially improves on the state of the art for generative models on MNIST” (Gregor abstract)
Claim 18/25/32
Jordon teaches claim 16/23/30
Jordon does not explicitly teach, wherein the system input comprises an input image, and wherein the system output comprises a classification output that classifies the input image into one of a pre-determined plurality of classes.
Gregor when addressing image generation via neural networks as well as classification teaches, wherein the system input comprises an input image, and wherein the system output comprises a classification output that classifies the input image into one of a pre-determined plurality of classes. ( Section 4.1 pg 5 “Our model consists of an LSTM recurrent network that receives a 12 × 12 ‘glimpse’ from the input image at each time-step, using the selective read operation defined in Section 3.2. After a fixed number of glimpses the network uses a softmax layer to classify the MNIST digit”)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify neural network of Jordon which is a system for mapping input data to output data to comprise the image generation and classification of Gregor. One would have been motivated to make such a combination because both references describe the mapping of input data to output data via trained networks. Further, Gregor notes “This paper introduced the Deep Recurrent Attentive Writer (DRAW)neural network architecture… improving on the best known results for binarized MNIST generation… is beneficial not only to image generation, but also to image classification” (Gregor Conclusion)
Claim 19/26/33
Jordon teaches claim 16/23/30
Jordon does not explicitly teach, wherein the system input comprises a sequence of data items, and wherein the system output specifies a probability density function for the sequence of data items.
Gregor when addressing image generation via neural networks as well as classification teaches, wherein the system input comprises a sequence of data items (pg 2 Section 2.1 “At each time-step t, the encoder receives input from both the image x and from the previous decoder hidden vector”) and wherein the system output specifies a probability density function for the sequence of data items. ( pg 2-3 Section 2.1 “The precise form of the encoder input depends on a read operation, which will be defined in the next section. The output henc t of the encoder is used to parameterise a distribution Q(Zt|henc t ) over the latent vector zt. In our experiments the latent distribution is a diagonal Gaussian” pg 3 Section 2.3 “An image ˜x can be generated by a DRAW network by iteratively picking latent samples ˜zt from the prior P” the latent distribution corresponding to the probability density function specifies the system output as it is sampled from to generate images.)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify neural network of Jordon which is a system for mapping input data to output data to comprise the image generation of Gregor. One would have been motivated to make such a combination because both references describe the mapping of input data to output data via trained networks. Further, Gregor notes “The system substantially improves on the state of the art for generative models on MNIST” (Gregor abstract)
Claim 20/27/34
Jordon/Gregor teaches claim 19/26/33
Gregor teaches, wherein the sequence of data items represents one of: a still or moving image; sound data; text data; object position data, environment state data, action data, or a combination thereof, or atomic position data. (pg 5 Section 4.1 “Our model consists of an LSTM recurrent network that receives a 12 × 12 ‘glimpse’ from the input image at each time-step” images at each time step are a sequence. All images can be considered either moving or still as claimed)
Jordon/Gregor are combined for the reasons set forth in the rejection of claim 19/26/33
Claim(s) 22 and 29 are rejected under 35 U.S.C. § 103 as being unpatentable over Jordon further in view of Udo Seiffert “Artificial neural networks on massively parallel computer hardware”
Claim 22/29
Jordon teaches claim 16/23/30
Jordon does not explicitly teach, wherein the one or more gated linear networks are implemented in parallel across different special-purpose hardware.
Seiffert however when addressing parallel implementation of neural networks teaches, wherein the one or more gated linear networks are implemented in parallel across different special-purpose hardware. (pg 12 Section 4.4 “From the parallel computing point of view there is an elegant way to handle this by running several instances with differently initialized weight sets on parallel processors (Fig. 5). All instances themselves are run sequentially or maybe in smaller but optimal parallel sub-clusters as a combination with the parallel programming model described above. In fact, this is not a parallel programme in the true sense but takes enormous advantage from all considered hardware architectures because there is no data exchange between the separately trained nets… In general, this can be done with every type of neural network requiring this or a similar non-interactive procedure”)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify neural network of Jordon which is a system for mapping input data to output data to comprise parallel implementation described by Seiffert. One would have been motivated to make such a combination because both references describe the neural network systems. Further Seifert notes “Spending a lot of additional time and extra money to implement a particular algorithm on parallel hardware is often considered as the ultimate solution to all existing time problems for some… In that light artificial neural networks are, depending on their special characteristics, rather easily viable on parallel hardware of all sorts… Every effort has been made to get numerically optimised computer programmes for serial as well as parallel computer systems” (Introduction)
Conclusion
Prior art not relied upon:
Gehring et al. “Convolutional Sequence to Sequence Learning” describes another sequence to sequence neural network composed of gated linear units
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNATHAN R GERMICK whose telephone number is (571)272-8363. The examiner can normally be reached M-F 9:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.R.G./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights