DETAILED ACTION
This action is in response to the original filing of 12-21-2021. Claims 1-20 are pending and have been considered below:
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 11-12 and 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Towards deep learning with segregated dendrites”, Guergiuev et al. (“Guergiuev”), pages 1-41, 4-7-2017 in view of Volkovs et al. (“Volkovs” 20210255862 A1) and Birdwell et al. (“Birdwell” 20150106316 A1).
Claim 1: Guergiuev discloses a method performed by one or more data processing apparatus (abstract), the method comprising:
obtaining a network input comprising a respective data element at each input position in a sequence of input positions (Page 6; Paragraph 1 “In the data presented in this paper, all 60,000 images in the MNIST training set were presented to the network one at a time, and each exposure to the full set of images was considered an epoch of training.” ; sequence of inputs);
and processing the network input using a neural network to generate a network output that defines a prediction related to the network input (Page 6; Paragraph 1; processing input to classify (predict) related to inputs),
Guergiuev provides encoders/decodes (Page 4, Paragraphs 2-4 and Pag3 5, Paragraphs 2-4; image translated into a pattern of spikes).
However may not explicitly disclose wherein the neural network comprises a sequence of encoder blocks and a decoder block, wherein each encoder block has a respective set of encoder block parameters and performs operations comprising:
receiving a respective current embedding for each input position; processing the current embeddings for the input positions, in accordance with the set of encoder block parameters (Guergiuev does provide embedding of inputs Page 3 Paragraph 4 and Page Paragraph 4),
to update the respective current embedding for each input position, comprising applying an attention operation to the current embeddings for the input positions;
wherein the decoder block has a set of decoder block parameters and performs operations comprising: receiving the respective current embedding for each input position from a final encoder block in the sequence of encoder blocks; and processing the current embeddings for the input positions, in accordance with the set of decoder block parameters, to generate the network output.
Volkovs discloses architecture for training a transformer neural networks (abstract). Provided within this architecture are a sequence of encoder/decoder blocks (Figure 2 and Paragraph 20). The system receives an input (query/question) which transforms it into a sequence of input embeddings which can provide embedding output according to a set of parameters (Paragraph 26). The system also provides an attention block in order to generate attention representations within the encoder/decoder (Paragraphs 4 and 25, 28 and 30; updates associations). The decoder sequenced embeddings (positioning) are processed in order to generate an output (make a prediction) (Paragraphs 22-25, 35 and 50). Therefore it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply a known technique to a known device ready for improvement and provide an architecture with a set of encoders/decoders and attention blocks in the system of Guergiuev. One would have been motivated to provide the architecture as an improved method of training for computational efficiency (Volkovs: Paragraph 5).
Last, Guergiuev may not explicitly disclose wherein the set of encoder block parameters comprises a plurality of brain emulation parameters that, when initialized, represent biological connectivity between a plurality of biological neuronal elements in a brain of a biological organism;
Birdwell is provided because it discloses brain emulation parameters that represent biological connections between a plurality of biological neuronal elements in a brain of a biological organism (Paragraphs 119 and 255; establishes how the weight matrix is used to represent synaptic connectivity and how it maps biological connectivity).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply a known technique to a known device ready for improvement and provide the brain emulation neural network as taught by Birdwell. One would have been motivated to provide the modification because it improves accuracy and leads to better generalization (Birdwell: Paragraph 6).
Claim 2: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein at least one encoder block in the sequence of encoder blocks includes a feed forward module, and wherein the feed forward module includes one or more brain emulation neural network layers having the plurality of brain emulation parameters that, when initialized, represent biological connectivity between the plurality of biological neuronal elements in the brain of the biological organism (Birdwell: Paragraphs 116 and 119, biological organism and connection via synapses and Guergiuev: Training Paradigm: Pages 18-19; encoded images provided in a feed forward module).
Claim 3: Guergiuev, Volkovs and Birdwell disclose a method of claim 2, wherein the feed forward module (Guergiuev: Training Paradigm: Pages 18-19; encoded images provided in a feed forward module) is configured to: for each input position in the sequence of input positions: receive an input at the input position; and apply a sequence of transformations to the input at the input position using the one or more brain emulation neural network layers to generate an output for the input position (Volkovs: Paragraphs 22-25; apply sequence transformations for inputs and Birdwell: Paragraphs 116, 119 and 125; brain emulation NN provides connected neurons via synapses).
Claim 4: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein at least one encoder block in the sequence of encoder blocks includes an attention module that includes: (i) a query sub-network configured to process the respective current embedding for each input position to generate a query vector, (ii) a key sub-network configured to process the respective current embedding for each input position to generate a key vector, and (iii) a value sub-network configured to process the respective current embedding for each input position to generate a value vector (Birdwell: Paragraph 33; attention network Volkovs: Paragraphs 18 embeddings vectors and 22-26; matrix and sequence provide positioning, and further provides key and value matrices).
Claim 5: Guergiuev, Volkovs and Birdwell disclose a method of claim 4, wherein the query sub-network, the key sub-network, and the value sub-network, each include one or more brain emulation neural network layers having the plurality of brain emulation parameters that, when initialized, represent biological connectivity between the plurality of biological neuronal elements in the brain of the biological organism (Volkovs: Paragraphs 22-26; networks for query, key and value and Birdwell: Paragraphs 116, 119 and 125; brain emulation neural network layers utilize neurons connected via synapses).
Claim 6: Guergiuev, Volkovs and Birdwell disclose a method of claim 5, wherein the attention module is configured to perform the attention operation, and wherein the attention operation comprises (Birdwell: Paragraph 33; attention network), for each input position in the sequence of input positions: processing the respective current embedding for the input position using the one or more brain emulation neural network layers (Birdwell: Paragraphs 116, 119 and 125; brain emulation neural network layers) included in the query sub-network to generate a query vector; processing the respective current embedding for the input position using the one or more brain emulation neural network layers (Birdwell: Paragraphs 116, 119 and 125; brain emulation neural network layers) included in the key sub-network to generate a key vector; processing the respective current embedding for the input position using the one or more brain emulation neural network layers (Birdwell: Paragraphs 116, 119 and 125; brain emulation neural network layers) included in the value sub-network to generate a value vector (Volkovs: Paragraphs 18 embeddings vectors and 25 attention block processing of query, key and value matrices, Paragraph 23; neural network);
determining a respective input-position specific weight for each of the input positions by applying a compatibility function between the query vector for the input position and the key vectors (Volkovs: Paragraphs 30-34; self-attention block provides embedding weighting);
and determining the updated current embedding for the input position by determining a weighted sum of the value vectors weighted by the corresponding input-position specific weights for the input positions (Volkovs: Figure 2:270 Paragraphs 30-34; after the self-attention block with weighting and multi-layer perceptron performs weighted sum for update (each input is multiplied by a weight, then added together with other weighted inputs and a bias term to produce a single value) and Birdwell: Paragraph 37, weighting parameters added up).
Claim 7: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein the set of decoder block parameters includes the plurality of brain emulation parameters that, when initialized, represent biological connectivity between the plurality of biological neuronal elements in the brain of the biological organism (Volkovs: Paragraphs 30-34; decoder Birdwell: Paragraphs 119 and 125; incorporate brain emulation neural network layers).
Claim 8: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein a data type of the network input includes an image data type, a text data type, or an audio data type (Guergiuev: Page 6, Paragraph 1; 60,000 images and Volkovs: Paragraphs 18-19; query/text data).
Claim 11: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein the plurality of brain emulation parameters are held static during training of the neural network (Birdwell: Paragraphs 116 (fixed threshold and period) and 145; neurons hold a constant value).
Claim 12: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein the plurality of brain emulation parameters are determined prior to training of the neural network based on weight values associated with biological connections between the plurality of biological neuronal elements in the brain of the biological organism (Birdwell: Paragraph 116; neurons threshold and refractory period (parameter) fixed for network instead of trained/learned).
Claim 14: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein each biological neuronal element of the plurality of biological neuronal elements is a biological neuron, a part of a biological neuron, or a group of biological neurons (Birdwell: Paragraphs 6-7 and 188; biological neural networks).
Claim 15: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein the plurality of brain emulation parameters are arranged in a two-dimensional weight matrix having a plurality of rows and a plurality of columns, wherein each row and each column of the weight matrix corresponds to a respective biological neuronal element from the plurality of biological neuronal elements (Birdwell: Paragraph 85; row and column of weight matrix correspond to biological neural element),
and wherein each brain emulation parameter in the weight matrix corresponds to a respective pair of biological neuronal elements in the brain of the biological organism, the pair comprising: (i) the biological neuronal element corresponding to a row of the brain emulation parameter in the weight matrix, and (ii) the biological neuronal element corresponding to a column of the brain emulation parameter in the weight matrix (Birdwell: abstract and Paragraph 85; form linked rows and columns of synapse circuit).
Claim 16: Guergiuev, Volkovs and Birdwell disclose a method of claim 15, wherein initializing the plurality of brain emulation parameters comprises performing a matrix multiplication of: (i) the two-dimensional weight matrix of brain emulation parameters representing synaptic connectivity between the plurality of biological neuronal elements in the brain of the biological organism, and (ii) the current embeddings for the input positions (Volkovs: Paragraph 60; matrix multiplication and Birdwell: Paragraphs 85 119 and 196; provides biological neuronal elements for incorporating).
Claim 17: Guergiuev, Volkovs and Birdwell disclose a method of claim 15, wherein each brain emulation parameter of the weight matrix has a respective value that characterizes synaptic connectivity in the brain of the biological organism between the respective pair of biological neuronal elements corresponding to the brain emulation parameter (Birdwell: Paragraphs 85 (matrix) 119 and 196; distance between two neurons and a weight (strength) of synaptic connection).
Claim 18: Guergiuev, Volkovs and Birdwell disclose a method of claim 17, wherein each brain emulation parameter of the weight matrix that corresponds to a respective pair of biological neuronal elements that are not connected by a biological connection in the brain of the biological organism has value zero, and each brain emulation parameter of the weight matrix that corresponds to a respective pair of biological neuronal elements that are connected by a biological connection in the brain of the biological organism has a respective non-zero value characterizing an estimated strength of the biological connection (Birdwell: Paragraphs 85 (matrix) 119 and 196; weights can be positive, zero or negative).
Claims 19-20 are similar in scope to claim 1 and therefore rejected under the same rationale.
Claims 9-10 is/are rejected under 35 U.S.C. 103 as being
unpatentable over “Towards deep learning with segregated dendrites”, Guergiuev et al. (“Guergiuev”, pages 1-41, 4-7-2017, Volkovs et al. (“Volkovs” 20210255862 A1) and Birdwell et al. (“Birdwell” 20150106316 A1) in further view of “Complex brain networks: graph theoretical analysis of structural and functional systems”, Bullmore et al. (“Bullmore”) pages 186-198, 2-2009.
Claim 9: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, however may not explicitly disclose wherein the plurality of brain emulation parameters are determined from a synaptic connectivity graph that represents biological connectivity between the plurality of biological neuronal elements in the brain of the biological organism.
Bullmore is provided because it discloses complex brain networks with graph theoretical analysis(abstract). Further, Bullmore discloses graph theory which is utilized to represent connectivity between neuronal elements of a biological organism (Pages 189-190, Structural brain networks-mapping networks in animal models and human brain, Box 4).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply a known technique to a known device ready for improvement and provide graph theory to the modified Guergiuev. One would have been motivated to provide the analysis because it provides new ways of quantifying functional systems (Bullmore: Page 186, Paragraph 1).
Claim 10: Guergiuev, Volkovs, Birdwell and Bullmore disclose a method of claim 9, wherein the synaptic connectivity graph comprises a plurality of nodes and edges, each edge connects a pair of nodes, each node corresponds to a respective neuronal element in the brain of the biological organism, and each edge connecting a pair of nodes in the synaptic connectivity graph corresponds to a biological connection between a pair of biological neuronal elements in the brain of the biological organism (Bullmore: Pages 189-190, Structural brain networks-mapping networks in animal models and human brain, Box 4).
Claim 13 is/are rejected under 35 U.S.C. 103 as being
unpatentable over “Towards deep learning with segregated dendrites”, Guergiuev et al. (“Guergiuev”, pages 1-41, 4-7-2017, Volkovs et al. (“Volkovs” 20210255862 A1) and Birdwell et al. (“Birdwell” 20150106316 A1) in further view of “Development of Scheme and Tools to Construct a standard moth brain for neural network simulation”, Ikeno et al. (“Ikeno”), Pages 1-10, 7-10-2012.
Claim 13: Guergiuev, Volkovs and Birdwell disclose a method of claim 1, wherein the plurality of brain emulation parameters are determined;
determining a respective value of each brain emulation parameter, comprising: setting a value of each brain emulation parameter that corresponds to a pair of biological neuronal elements in the brain that are not connected by a biological connection to zero (Birdwell: Paragraph 196);
and setting a value of each brain emulation parameter that corresponds to a pair of biological neuronal elements in the brain that are connected by a biological connection based on a proximity of the pair of biological neuronal elements in the brain (Paragraph 85).
Guergiuev may not explicitly disclose from a synaptic resolution image of at least a portion of the brain of the biological organism, the determining comprising: processing the synaptic resolution image to identify: (i) the plurality of biological neuronal elements, and (ii) a plurality of biological connections between pairs of biological neuronal elements;
Ikeno is provided because it discloses constructing a moth brain for neural network simulations (abstract). Further Ikeno discloses parameters determined by a synaptic resolution image of biological organism, while identifying neuronal elements and biological connections (Page 2, Paragraph 2 “In brain science, standard brain maps are used to integrate and compare morphological data taken from different subjects at different times. Standard brains have already been developed for various insect species, such as Drosophila [12– 14], the honeybee [15, 16], two moth species [17, 18], and the locust [19, 20]. These standard brains have been employed for various morphological analyses of neurons and brain regions. For example, possible synaptic connections between identified neurons have been analyzed using the honeybee standard brain [21], while morphological development of the optic lobes has been studied in a moth brain [22]”, Page 8, Paragraph 2; brain image data can be registered and Figure 8; registration of neurons).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply a known technique to a known device ready for improvement and provide mapping of a synaptic image in the modified Guergiuev. One would have been motivated to provide the mapping because it provides very effective neuronal registration for computational neuroscience (Ikeno: abstract).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
20230019839 A1 [0061]
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERROD L KEATON whose telephone number is (571)270-1697. The examiner can normally be reached on MONDAY -FRIDAY 9:30-5. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MICHELLE BECHTOLD can be reached on 571-431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-3800. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHERROD L KEATON/Primary Examiner, Art Unit 2148 9-22-2025