Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to submission of application on 3/10/2023.
Claims 1-4, and 9-15 are pending.
Subject Matter Eligibility
In determining whether the claims are subject matter eligible, the examiner has considered and applied the 2019 USPTO Patent Eligibility Guidelines, as well as guidance in the MPEP chapter 2106. The examiner finds that the independent claims do not recite a judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, and 9-11 are rejected under 35 U.S.C. § 013 as being unpatentable over Ohta, et al (US 5,095,459, Optical Neural Network, herein Ohta), Zhang, et al (US 2020/0026992 A1, Hardware Neural Network Conversion Method, Computing Device, Compiling Method and Neural Network Software and Hardware Collaboration System, herein Zhang), Shen, Y. (Novel Angular and Frequency Manipulation of Light in Nano-scaled Dielectric Photonic Systems, herein Shen), and Leedumrongwatthanakun, S. (Quantum information processing with a multimode fibre, herein Leedumrongwatthanakun).
Regarding claim 1,
Ohta teaches a frequency multiplexed neural network (Ohta, Fig. 4, and, abstract line 1 “An optical neural network which imitates a biological neural network, to provide an associative and/or pattern recognition function, is made of light emitting elements to represent an input neuron state vector,” and, column 9, line 17 “Each input unit 24 is divided into P partial units 25, each of which is modulated with a frequency of ωi, ..., ωp. The input signal is binary; a logic 1 if there is a frequency modulated signal and a logic 0 if there is no such a signal. These signals are multiplexed in the input unit 24, and the output propagates to respective hidden units 26.”
PNG
media_image1.png
609
389
media_image1.png
Greyscale
In other words, frequency modulated signal that is multiplexed is frequency multiplexed, and neural network is neural network.) , comprising:
an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies (Ohta, Fig. 4, and column 9, line 15 “In FIG. 4, this multilayer neural network includes an input layer 21, a hidden layer 22, and an output layer 23. Each input unit 24 is divided into P partial units 25, each of which is modulated with a frequency of ωi, ..., ωp. The input signal is binary; a logic 1 if there is a frequency modulated signal and a logic 0 if there is no such a signal. These signals are multiplexed in the input unit 24, and the output propagates to respective hidden units 26. The input section 26 of the hidden layer 22 adds signals from respective input units 24 to provide an output, the frequencies of which are separated by band pass filters 27 having a central frequency of ω1,..., ωp, followed by a threshold process in the comparator 28. When the output of each comparator 28 is a 1, it is modulated with a frequency of ω1, ..., ωp and transmitted to the output layer 23. The output frequencies of the output unit 30 are separated by band-pass filters 31. The peak value of each frequency is held by a peak hold circuit 32. The respective outputs are input into a comparator 33 to be subjected to a threshold process for outputting. If the number of modulation frequencies is P, the capacity of a network having N units is enhanced to an equivalent of N X P units. Although the ordinary multilayer neural network model is discussed in the above embodiment, it is easy to expand it to a back propagation model which has a hierarchical structure. Although the light emitting element (neuron state vector) is modulated with a frequency multiplexed signal, the sensitivity of a light receiving element may be modulated with a frequency multiplexed signal to produce substantially the same results. This may be achieved by using a phototransistor, for example, as the light receiving element.” In other words, input layer is input layer, unit is node, logic 1 and logic 0 is input values, input unit 24 is divided into P partial units is input nodes receive a plurality of input values, and, each of which is modulated with a frequency of ωi ,..., ωp is each of the plurality of input values provided at a respective one of a plurality of different frequencies.);
a plurality of hidden layers (Ohta, Fig, and column 9, line 15 “In FIG. 4, this multilayer neural network includes an input layer 21, a hidden layer 22, and an output layer 23.” In other words, hidden layer is hidden layer, and from Fig. 4, 22 (26-29) shows a plurality of hidden layers. Examiner notes that it is known in the art that any layer that is not the input or output layers is called a hidden layer. Further, any sublayer that is not the input or output layer is also a hidden layer.); to provide
[a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith] and
an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers (Ohta, Fig. 4, and column 9, line 15 “In FIG. 4, this multilayer neural network includes an input layer 21, a hidden layer 22, and an output layer 23.” In other words, output layer is output layer, and from Fig. 4, output layer 23 shows a plurality of output nodes coupled to at least one of the plurality of hidden layers.) ,
each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies (Ohta, Fig. 4, and column 9, line 23 “The input section 26 of the hidden layer 22 adds signals from respective input units 24 to provide an output, the frequencies of which are separated by band pass filters 27 having a central frequency of ω1 ,..., ωp, followed by a threshold process in the comparator 28.When the output of each comparator 28 is a 1, it is modulated with a frequency of ωl , ..., ωp and transmitted to the output layer 23. The output frequencies of the output unit 30 are separated by band-pass filters 31. The peak value of each frequency is held by a peak hold circuit 32. The respective outputs are input into a comparator 33 to be subjected to a threshold process for outputting.” In other words, output layer is output layer that includes a plurality of output nodes, from Fig 4, the output nodes are operably coupled to at least one of the hidden layers, and output frequencies is each of the plurality of output values at a respective one of the plurality of frequencies.), wherein
[the plurality hidden layers comprise a plurality of weight factor matrices]; wherein
the plurality of weight factor matrices comprises
[a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix;] wherein
[decomposition of an m x n weight factor matrix comprises decomposing the m x n weight factor matrix into a product of three matrices UZV, where; U includes an m x m unitary matrix, Σ includes an m x n rectangular diagonal matrix, and V includes an n x n unitary matrix]; and wherein
[the decomposition of the m x m unitary matrix U and the n x n unitary matrix V] comprises
[decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters]
[using at least one of the Reck-Zeilinger method or the Clements method].
Thus far, Ohta does not explicitly teach a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith.
Zhang teaches a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith (Zhang, Figures 10 and 11, and paragraph [0028], line 1 “As shown in FIG. 11, by arranging lines into the crossbar and connecting them at intersection points with memristors, setting a conductance value (a reciprocal of a resistance) of the memristor to a matrix element value of a weight matrix, and inputting a voltage value at an input end, the matrix vector multiplication may be completed at an output end.” And, paragraph [0040], line 1 “…recoding inter-layer data with an autoencoder, wherein, the autoencoder is a neural network, consisting of three layers of neurons, including an input layer, a hidden layer and an output layer, the number of nodes of the output layer is equal to the number of nodes of the input layer, and the number of nodes of the hidden layer is greater than dimensionality of inter-layer vector data;”
PNG
media_image2.png
388
569
media_image2.png
Greyscale
PNG
media_image3.png
422
297
media_image3.png
Greyscale
In other words, weight matrix is weight matrix, FIG. 10 shows a plurality of hidden layers is a plurality of hidden layers, at each layer is for each of the plurality of hidden layers, and from prior mapping, at each layer…Wk is a two dimensional weight matrix is hidden layers having at least one weight factor associated therewith.)
Zhang teaches the plurality hidden layers comprise a plurality of weight factor matrices (Zhang, paragraph [0164], line 18 “…the autoencoder is inserted between layers of the neural network, to replace the original inter-layer vector, as denoted by reference sign 2; and 4) with respect to each connection, a decoder of an input node, a weight matrix of the connection, and an encoder of an output node thereof will be combined into a larger-size connection matrix…” In other words, layers is at each layer, and replace the original inter-layer vector… with a weight matrix is the plurality of hidden layers comprise a plurality of weight factor matrices.);
Both Ohta and Zhang are directed to neural network implementations, among other things. Ohta teaches a frequency multiplexed neural network comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers, and an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies; but does not explicitly teach a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith. Zhang teaches a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith.
In view of the teaching of Ohta, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Zhang into Ohta. This would result in a frequency multiplexed neural network, comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers, to provide a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith, and an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies.
One of ordinary skill in the art would be motivated to do this in order to speed up execution of ever increasing computational loads of neural networks. (Zhang, paragraph [0002], line 1 “In recent years, a deep learning technology has made a breakthrough progress, and has achieved a very high precision rate in image recognition, speech recognition, natural language processing and many other fields; however, deep learning requires massive computing resources, and it is very difficult for a traditional general-purpose processor to fulfill computing requirements of deep learning, so hardware conversion of deep learning and its application specific integrated circuit (ASIC) design have become an important direction of development.”)
Thus far, the combination of Ohta and Zhang does not explicitly teach a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix.
Shen teaches a plurality of weight factor matrices generated by decomposition of an
m x n weight factor matrix (Shen, Figure S1, and page 78, paragraph 3, line 1 “ Artificial neural network architecture contains an input layer, at least one hidden layers, and an output layer. In each layer, information propagate through neural network via linear combination (e.g. matrix multiplication) followed by nonlinear activation applied to the result from linear combination.” And, page 78, paragraph 4, line 2 “Firstly, matrix multiplication in our ONNW is implemented using optical interference unit, and we will show that our optical system can implement any weighting matrix ωi multiplication with real number entries.” And, page 79, paragraph 3, line 1 “Singular Value Decomposition, suppose M is an m x n matrix whose entries are real numbers K. Then there exists a factorization, called a singular value decomposition of M [1641, of the form
M = UΣV* (5.1)
where
U is an m x m, unitary matrix,
Σ is an m x n diagonal matrix with non-negative real numbers on the diagonal, and
V* is an n x n, unitary matrix over K. (If K = R, unitary matrices are orthogonal matrices) V* is the conjugate transpose of the n x m unitary matrix, V.”
PNG
media_image4.png
356
521
media_image4.png
Greyscale
In other words, weighting matrix, is weight factor matrix, and decomposition of M is decomposition of an m x n weight factor matrix.)
Shen teaches decomposition of an m x n weight factor matrix comprises decomposing the m x n weight factor matrix into a product of three matrices UΣV, where; U includes an m x m unitary matrix, Σ includes an m x n rectangular diagonal matrix, and V includes an n x n unitary matrix (Shen, See above mapping. In other words, decomposition of M is decomposition of m x n weight factor matrix, and, UΣV is UΣV where U includes an m x m unitary matrix, Σ includes an m x n rectangular diagonal matrix, and V includes an n x n unitary matrix.)
Shen teaches the decomposition of the m x m unitary matrix U and the n x n unitary matrix V comprises (Shen, see above mapping. In other words, U is an m x m unitary matrix, and V is an n x n unitary matrix. Examiner notes that Shen implicitly shows the decomposition is into a plurality of beam splitters and a plurality of phase shifters in Figure S1. However, for clarity, this portion of the limitation is mapped to Leedumrongwatthanakun. See below for mapping. )
Both Shen and the combination of Ohta and Zhang are directed to optical neural networks, among other things. The combination of Ohta and Zhang teaches a frequency multiplexed neural network, comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers to provide a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith, and an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies; but does not explicitly teach a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix where the decomposition of the m x m unitary matrix U and the n x n unitary matrix V comprises decomposition of the U and V matrices. Shen teaches a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix where the decomposition of the m x m unitary matrix U and the n x n unitary matrix V comprises decomposition of the U and V matrices.
In view of the teaching of the combination of Ohta and Zhang, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Shen into the combination of Ohta and Zhang. This would result in a frequency multiplexed neural network, comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers to provide a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith, and an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies using a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix where the decomposition of the m x m unitary matrix U and the n x n unitary matrix V comprises decomposition of the U and V matrices.
One of ordinary skill in the art would be motivated to do this because optical computing has great potential for improvements in both size and speed of computation requiring new methods of manipulating light. (Shen, page 15, paragraph 1, line 4 “Traditional optical devices based on geometrical optics such as lenses, mirrors and fibers have done an excellent job of manipulating light in the past decades. However, just as electronics has seen a dramatic miniaturization over the last 40 years, which enabled data density to be doubled every 18 months (the so-called Moore's law), the feature sizes of photonic devices have rapidly decreased too. Unfortunately, such a trend has unavoidably slowed down because we are already at the boundary where geometrical optics breaks down (feature size ~ 0ipm). In order to catch up with the density of electronic devices (feature size ~10nm), the need to manipulate light at the micro-scale or even nano-scale is increasingly desirable for the optical community.”)
Thus far, the combination of Ohta, Zhang ,and Shen does not explicitly teach decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using at least one of the Reck-Zeilinger method or the Clements method.
Leedumrongwatthanakun teaches decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using at least one of the Reck-Zeilinger method or the Clements method (Leedumrongwatthanakun, Figure 1.8, and, page 50, paragraph 2, line 1 “Moreover, machine-learning-based approaches have also been applied to tackle the imaging problem through fibres with high reliability [Aisawa et al., 1991, Marusarz and Sayeh, 2001, Takagi et al., 2017, Caramazza et al., 2019]. Most techniques use artificial neural networks e.g., convolutional neural network, to predict input images from the intensity only output speckle pattern, for example, reconstructing handwritten digits from the MNIST database [Borhani et al., 2018, Rahmani et al., 2018, Turpin et al., 2018, Li et al., 2018, Fan et al., 2019].” And, page 20, paragraph 2, line 7 “To implement a k-dimensional unitary transformation U(k), one typically needs to decompose a k-dimensional unitary transformation into a large number of two-dimensional unitary transformations [Reck et al., 1994]. The reconfigurability of two-dimensional unitary transformation is provided by Mach-Zehnder (MZ) interferometer. Each MZ interferometer consists of two beamsplitters and two tunable phase shifters, as shown in Fig. 1.8d.” And, page 21, paragraph 2, line 5 “The most well-known decomposition is proposed by M. Reck et al. in 1994 [Reck et al., 1994], and is known as the triangular architecture (Reck-Zeilinger design) as depicted in Fig 1.8b. Another decomposition is proposed by [Clements et al., 2016], as depicted in Fig. 1.8c, and we called it the rectangular decomposition (Clements design).”
PNG
media_image5.png
377
878
media_image5.png
Greyscale
In other words, at least two beamsplitters is plurality of photonic beam splitters, two tunable phase shifters is a plurality of phase shifters, and decomposition using the Reck-Zeilinger design is decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using at least one of the Reck-Zeilinger method or the Clements method .)
Both Leedumrongwatthanakun and the combination of Ohta, Zhang, and Shen are directed to information processing using photonics, among other things. The combination of Ohta, Zhang, and Shen teaches a frequency multiplexed neural network, comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers to provide a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith, an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, where each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies, and the plurality hidden layers comprise a plurality of weight factor matrices, wherein the plurality of weight factor matrices comprises a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix, wherein decomposition of an m x n weight factor matrix comprises decomposing the m x n weight factor matrix into a product of three matrices UZV, where; U includes an m x m unitary matrix, Σ includes an m x n rectangular diagonal matrix, and V includes an n x n unitary matrix; but does not explicitly teach decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using one of the Reck-Zeilinger or Clements methods for the decomposition. Leedumrongwatthanakun teaches decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using one of the Reck-Zeilinger or Clements methods for the decomposition.
In view of the teaching of the combination of Ohta, Zhang, and Shen, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Leedumrongwatthanakun into the combination of Ohta, Zhang, and Shen. This would result in a frequency multiplexed neural network, comprising an input layer that includes a plurality of input nodes, each of the plurality of input nodes to receive a plurality of input values, each of the plurality of input values provided at a respective one of a plurality of different frequencies, a plurality of hidden layers to provide a weight matrix operably coupled to the input layer, each of the plurality of hidden layers having at least one weight factor associated therewith, an output layer that includes a plurality of output nodes operably coupled to at least one of the plurality of hidden layers, where each of the plurality of output nodes to provide a respective one of a plurality of output values, each of the plurality of output values at a respective one of the plurality of frequencies, and the plurality hidden layers comprise a plurality of weight factor matrices, wherein the plurality of weight factor matrices comprises a plurality of weight factor matrices generated by decomposition of an m x n weight factor matrix, wherein decomposition of an m x n weight factor matrix comprises decomposing the m x n weight factor matrix into a product of three matrices UZV, where; U includes an m x m unitary matrix, Σ includes an m x n rectangular diagonal matrix, and V includes an n x n unitary matrix, and decomposition of the U and V matrices into a plurality of photonic beam splitters and a plurality of phase shifters using one of the Reck-Zeilinger or Clements methods for the decomposition.
One of ordinary skill in the art would be motivated to do this in order to speed up computation by using efficient methods for optical computation. (Leedumrongwatthanakun, page 2, paragraph 2, line 5 “Likewise to the requirement for high-performance information processing, the requirement in the high capacity of optical communication also rise exponentially at a faster rate than Moore’s law (doubling over nine months) due to the use of internet [Agrawal, 2016].”)
Regarding claim 2,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim I wherein each of the hidden layers includes a plurality of nodes,
each of the nodes having the same weight factor for each of the plurality of frequencies (Leedumrongwatthanakun, page 96, paragraph 1, line 3 In case of classical fully-mixing of coherent states, i.e., equally-weighted incoherent summation of coherent light source, the probability density function (PDF) of intensity speckle is
PNG
media_image6.png
60
605
media_image6.png
Greyscale
where d represents a number of coherent sources, is the gamma function, and I¯ is
an average intensity.” In other words, equally-weighted incoherent summation of coherent light source is each of the nodes having the same weight factor for each of the plurality of frequencies.)
It would be obvious to combine Leedumrongwatthanakun into the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun, at least for the reasons used to combine Leedumrongwatthanakun into the combination of Ohta, Zhang, and Shen used in claim 1.
Regarding claim 4,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim I wherein
each of the hidden layers performs at least one matrix multiplication and accumulation operation (Zhang, paragraph [0093], line 2 “ These chips usually consist of a plurality of processing cores, each processing core may accept M inputs, which are subjected to a matrix vector multiplication with an M N matrix, resulting in N results, which undergo a hardware built-in activation function or a built-in hardware neuron model, resulting in final N outputs.” And, paragraph [0184], line 31 “…the virtual core used for reduction accumulates output data of respective small matrices with respect to a same neuron, to obtain a final output, which is an output of N*B bits as shown in FIG. 9.” In other words, matrix vector multiplication is multiplication, and accumulates data…of small matrices is accumulates. Examiner notes that hidden layers is previously mapped.)
It would be obvious to combine Zhang into the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun at least for the reasons used to combine Zhang into Ohta described in the mapping of claim 1.
Regarding claim 9,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim I wherein
one or more of the plurality of photonic beam splitters and one or more of the plurality of phase shifters are grouped into Mach Zehnder Interferometers (MZIs) (Leedumrongwatthanakun, Figure 1.8, and, page 50, paragraph 2, line 1. See the mapping of claim 1, office action, page 13. And, page 35, paragraph 2, line 1 “Programmable linear optical network is the heart of these implementations. It allows implementing different experiments, simulations and information processing tasks on the same optical platform. The famous architecture of the programmable linear optical network is a cascade of the MZ interferometers that consists of many beamsplitters and tunable phase shifters.” In other words, many beam splitters is plurality of photonic beam splitters, phase shifters is one or more of a plurality of phase shifters, and MZ interferometers is Mach Zehnder interferometers.)
Regarding claim 10,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim I wherein
each of the plurality of frequencies includes matched optical path lengths through the plurality of hidden layers (Ohta, FIG. 4, and, column 1, line 10 “This optical neurocomputer includes an array of light emitting elements 1, a correlation matrix mask 2 for performing optical modulation, an array of light receiving elements 3 for receiving optical signals, and a threshold element 5 for comparing the output of the light receiving array 3 with the threshold value.” In other words, as shown in Figure 4, light emitting elements is plurality of frequencies, and optical signals is optical path lengths, and from Figure 4, 22, (26-29) through a plurality of hidden layers.)
Regarding claim 11,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim I wherein
the plurality of hidden layers comprise an m x n weight matrix (Zhang, FIG. 4, FIG. 9, and paragraph [0183], line 1 “In FIG. 9, M and N define a matrix size that the virtual core is capable of processing, and A and B define a size of an actual large-size matrix relative to the matrix size of the virtual core; and for convenience of representation in the diagram, it is assumed that M=N, A=B=2.” In other words, M and N define a matrix size is m x n weight matrix, and from FIG. 4, shows a plurality of hidden layers.).
It would be obvious to combine Zhang into the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun at least for the reasons used to combine Zhang into Ohta described in the mapping of claim 1.
Claim 3 is rejected under 35 U.S.C. §103 as being unpatentable over Ohta, Zhang, Shen, Leedumrongwatthanakun, and Chen, et al (An optical diffractive deep neural network with multiple frequency-channels, herein Chen).
Regarding claim 3,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim 1 wherein
[each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies.]
Thus far, the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun does not explicitly teach each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies.
Chen teaches each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies (Chen, Algorithm 1, and page 3, paragraph 6, line 1 “At the output layer, we get frequency-channel at different frequency. Then we merge these channels to get final result with weighting coefficient. The loss function is defined in formula (4).
Loss = 𝑙𝑜𝑠𝑠(Σ 𝑤𝑓 × 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 ) (4)
where
𝑙𝑜𝑠𝑠 is the function to get classification error. For example, cross entropy function or mean squared error.
𝑤𝑓 is the weighting coefficient for each 𝑐ℎ𝑎𝑛𝑛𝑒𝑙, which is also learned by MF_DNET.”
In other words, wf is the weight coefficient for each channel, which is also learned is each of the nodes having a different weight factor for each of at last two of the plurality of frequencies.)
Both Chen and the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun are directed to photonic neural networks, among other things. The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the frequency multiplexed neural network of claim 1, but does not explicitly teach each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies. Chen teaches each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies.
In view of the teaching of the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Chen into the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun. This would result in the frequency multiplexed neural network of claim 1, wherein each of the nodes having a different weight factor for each of the plurality of the at least two of the plurality of frequencies .
One of ordinary skill in the art would be motivated to do because machine learning with optical transmission could provide fast predictions while consuming less power. (Chen, abstract, line 1 “Diffractive deep neural network (DNNet) is a novel machine learning framework on the modulation of optical transmission. Diffractive network would get predictions at the speed of light. It's pure passive architecture, no additional power consumption. We improved the accuracy of diffractive network with optical waves at different frequency.”
Claims 12-15 are rejected under 35 U.S.C. §103 as being unpatentable over Ohta, Zhang, Shen, Leedumrongwatthanakun, and Carolan, et al (US 2017/0351293 A1, Apparatus and Methods for Optical Neural Network, herein Carolan)
Regarding claim 12,
The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the neural network of claim 11 further comprising
[one or more splitter elements to split each of a plurality of input signals equally into in paths upstream of the m x n weight matrix.]
Thus far, the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun does not explicitly teach one or more splitter elements to split each of a plurality of input signals equally into in paths upstream of the m x n weight matrix.
Carolan teaches one or more splitter elements to split each of a plurality of input signals equally into m paths upstream of the m x n weight matrix (Carolan, paragraph [0045], line 1 “In another example, the optical interference unit 200 includes an array of interconnected Mach-Zehnder Interferometers (MZIs) 220b. Each MZI splits input optical signals into a first arm and a second arm and then combines the optical signals from the two arms for interference.” In other words, an array of interconnected Mach-Zehnder Interferometers is one or more splitters, and splits input optical signals into a first arm and a second arm is split each of a plurality of input signals equally into m paths. Examiner notes that m x n weight matrix is previously mapped in claim 1.)
Both Carolan and the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun are directed to optical (photonic) neural networks, among other things. The combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun teaches the frequency multiplexed neural network of claim 1, but does not explicitly teach one or more splitter elements to split each of a plurality of input signals equally into m paths upstream of the m x n weight matrix. Carolan teaches one or more splitter elements to split each of a plurality of input signals equally into m paths upstream of the m x n weight matrix.
In view of the teaching of the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Carolan into the combination of Ohta, Zhang, Shen, and Leedumrongwatthanakun. This would result in the frequency multiplexed neural network of claim 1, with splitter elements to split each of a plurality of input signals equally into m paths upstream of the m x n weight matrix.
One of ordinary skill in the art would be motivated to do this because electronic artificial neural networks are limited by clock rates whereas optical neural networks are not and may provide more speed with less power consumption, thus requiring new methods to implement the photonic signals. (Carolan, paragraph [0004], line 8 “Conventional artificial neural networks usually use electronic architectures, such as application- specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs ). However, the computational speed and power efficiency achieved with these hardware architectures are still limited by electronic clock rates and ohmic losses.”)
Regarding claim 13,
The combination of Ohta, Zhang, Shen, Leedumrongwatthanakun, and Carolan teaches the neural network of claim 12 wherein the one or more splitter elements comprise
at least one of: one or more 1-to-m multimode interferometers; one or more Y-junction arrays; or one or more directional couplers (Carolan, see mapping of claim 12. And, paragraph [0046], line 1 “In yet another example, the optical interference unit 200 can include a multimode interferometer (MMI) 220c. An MIMI can include an array of single mode waveguides to receive input optical signals and a multimode waveguide for the received optical signals to interference with each other.” In other words, can include a multimode interferometer (MMI) is one of: one or more 1-to-m multimode interferometers; one or more Y-junction arrays; or one or more directional couplers.)
Regarding claim 14,
The combination of Ohta, Zhang, Shen, Leedumrongwatthanakun, and Carolan teaches the neural network of claim 12 further comprising
one or more accumulator elements to combine each of a plurality of output signals downstream of the m x n weight matrix (Zhang, paragraph [0155], line 8 “…output vectors Yi, . . . , Y6 , then remaining Y7 is projected within a linear space spanned by Y1 , ... , Y6 , with projection values a1 , ... , a6 respectively on Y1 , ... , Y6 , then the original 8x6x6 neurons are in full connection with 32 neurons at edge 2-4, and a connection weight between 6x6 neurons corresponding to Y7 and 32 neurons is multiplied by at and accumulated onto a connection weight between 6x6 neurons corresponding to Yi and 32 neurons.” In other words, Yi is output, accumulated into a connection weight is one or more accumulator elements to combine the output signals downstream of the m x n matrix.)
Regarding claim 15,
The combination of Ohta, Zhang, Shen, Leedumrongwatthanakun, and Carolan teaches the neural network of claim 14 wherein the one or more accumulator elements comprise
at least one of: one or more m-to-1 multimode interferometers; one or more Y-junction arrays; or one or more directional couplers (Carolan, see mapping of claim 13, paragraph [0046], line 1. In other words, can include a multimode interferometer (MMI) is one of: one or more m-to-1 multimode interferometers; one or more Y-junction arrays; or one or more directional couplers.)
The prior art made of record and not used is considered pertinent to applicant’s disclosure:
Kenney, et al (WO 2020/176538 A1) “Hybrid Analog-Digital Matrix Processors” discloses techniques for computing matrix operations for arbitrarily large matrices on a finite-sized hybrid analog-digital matrix processor.
Schuld, et al “Supervised Learning with Quantum Computers” Chapter 8 “Learning with Quantum Models” discloses quantum models for machine learning which either have no direct equivalent in classical machine learning, or which are quantum extensions of classical models with a new quality of dynamics.
Zhang, et al “Efficient training and design of photonic neural network through neuroevolution” discloses a novel learning strategy based on neuroevolution to design and train ONNs (optical neural networks) using two typical neuroevolution algorithms to determine the hyper-parameters of ONNs and to optimize the weights (phase shifters) in the connections.
Zipp, et al (US 12,340,301 B2) “Photonic Neural Network” discloses a photonic neural network device with a planar waveguide; a layer having a changeable refractive index adjacent to the planar waveguide; and a plurality of electrodes. Each electrode may be electrically coupled to the layer having the changeable refractive index at a corresponding location of the layer having the changeable refractive index.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BART RYLANDER whose telephone number is (571)272-8359. The examiner can normally be reached Monday - Thursday 8:00 to 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Bart I Rylander/Examiner, Art Unit 2124