Detailed Office Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Election/Restriction
Applicant’s election without traverse of claims 1-11 in the reply filed on 14 January 2026 is acknowledged.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6 and 9-11
Claims 1-6 and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Ashtiani et al. (An on-chip photonic deep neural network for image classification. Nature 606, 501–506 (2022); “Ashtiani”) in view of Zhang et al. (An optical neural chip for implementing complex-valued neural network. Nat Commun 12, 457 (2021); “Zhang”).
Regarding claim 1, Ashtiani discloses in figure 2, and related figures and text, for example, Ashtiani – Selected Text, a waveguide Supply light to receive an input optical signal Ps encoded with a first vector, and output an output optical signal Pout; one or more optical-to-electrical (O/E) converters PDn receive one or more optical signals In encoded with one or more vectors and generate one or more electrical signals Isum based on the received one or more optical signals; one or more optical modulators (shown as a micro-ring MRM) optically coupled to the waveguide and electrically coupled TIA to the one or more O/E converters, the one or more optical modulators to modulate an intensity of the input optical signal on the waveguide based on the one or more electrical signals. Ashtiani, abstract (“In each neuron, linear computation is performed optically and the non-linear activation function is realized opto-electronically…”).
Ashtiani – Figure 2
PNG
media_image1.png
450
914
media_image1.png
Greyscale
Ashtiani – Selected Text
Deep neural networks with applications from computer vision to medical diagnosis1–5 are commonly implemented using clock-based processors6–14, in which computation speed is mainly limited by the clock frequency and the memory access time. In the optical domain, despite advances in photonic computation15–17, the lack of scalable on-chip optical non-linearity and the loss of photonic devices limit the scalability of optical deep networks. Here we report an integrated end-to-end photonic deep neural network (PDNN) that performs sub-nanosecond image classification through direct processing of the optical waves impinging on the on-chip pixel array as they propagate through layers of neurons. In each neuron, linear computation is performed optically and the non-linear activation function is realized opto-electronically, allowing a classification time of under 570 ps, which is comparable with a single clock cycle of state-of-the-art digital platforms. A uniformly distributed supply light provides the same per-neuron optical output range, allowing scalability to large-scale PDNNs. Two-class and four-class classification of handwritten letters with accuracies higher than 93.8% and 89.8%, respectively, is demonstrated. Direct, clock-less processing of optical data eliminates analogue-to-digital conversion and the requirement for a large memory module, allowing faster and more energy efficient neural networks for the next generations of deep learning systems.
The structure of a photonic neuron with N optical inputs (Ini) and one optical output is shown in Fig. 2a, in which linear computation (that is, the weighted sum of the input signals) is performed optically and the non-linear activation function is realized opto-electronically. First, an array of 500-μm-long P-doped–intrinsic–N-doped (PIN) current-controlled attenuators is used to individually adjust the optical power in each input nanophotonic waveguide of the neuron. The cross section of the PIN attenuator as well as its microphotograph are shown in Fig. 2b. By forward biasing the PIN junction and injecting carriers, the power of the optical wave (that is, the signal weight) of each neuron input can be adjusted (Fig. 2c). To add the weight-adjusted signals, the outputs of attenuators are photodetected using silicon– germanium (SiGe) photodiodes (PDs) and the resulting photocurrents are combined to generate the weighted sum of the neuron inputs, isum. The microphotograph of the SiGe PD is shown in Fig. 2d. To generate the neuron output, the weighted sum of the neuron inputs is passed through a non-linear activation function. Here the rectified linear unit (ReLU) function, offering fast convergence11,12, is used as the non-linear activation function and is realized by using the electro-optic non-linear response of a PN junction micro-ring modulator (MRM)37 (Fig. 2e). In Fig. 2a, the electrical current, isum (that is, the weighted sum of the inputs), is amplified and converted to a voltage using a linear transimpedance amplifier (TIA). The input voltage of the MRM (driving the forward-biased PN junction), VM, is generated by adding a DC voltage, Vb, to the TIA output voltage, VTIA. The power of a laser, coupled into the chip, is equally distributed among all neurons (within all layers), providing the supply light to the input of the MRM in each neuron.
Further regarding claim 1, Zhang discloses in figure 1, and related figures and text, embodiments of optical neural network devices comprising optically encoded input vectors traversing cascaded Mach-Zehnder Interferometers configured as ‘an input layer, multiple hidden layers and an output layer. In the complex valued architecture, light signals are encoded and manipulated by both optical magnitude and phase during the initial input signal preparation and network evolution.’ Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
Zhang – Figure 1
PNG
media_image2.png
629
708
media_image2.png
Greyscale
Zhang
Fig. 1 The composition of complex-valued coherent optical neural network. a An optical neural network is composed of an input layer, multiple hidden layers and an output layer. In our complex-valued design, the light signals are encoded and manipulated by both magnitude and phase during the initial input preparation and network evolution. b The schematic of the ONC in implementing complex-valued networks. The input preparation, weight multiplication and coherent detection are all integrated onto a single chip. The division and modulation of the light signals (i1–i6) are realized by the MZIs marked in red. The green marked MZI separates the reference light that will later be used for coherent detection. The MZIs used to implement the 6 × 6 complex-valued weight matrix are marked in blue. The remaining grey marked MZIs are used for on-chip coherent detection. c The workflow of the ONC system. A coherent laser at 1550 nm is used to generate signal light and reference light. The signal light on each path is modulated by its magnitude and phase according to the machine learning (ML) task. The weighted sum operation is accomplished passively through light inference. The measurement results are sent to the electrical interface for processing, including the application of activation function and the calculation of cost function. The ONC chip are then reconfigured accordingly by the updated weight matrices.
Results Design and fabrication.
Figure 1a shows the architecture of the optical neural network, which is composed of an input layer, multiple hidden layers and an output layer. In the complex valued architecture, light signals are encoded and manipulated by both optical magnitude and phase during the initial input signal preparation and network evolution. Figure 1b shows the schematic of the ONC to implement complex-valued neural networks. The input preparation, weight multiplication and coherent detection are all integrated onto a single chip. A coherent laser (wavelength 1550 nm) is used to generate the input signals. The ONC is essentially a multiport interferometer, in which Mach–Zehnder interferometers (MZIs) are arranged in a specific manner50–52. Each MZI consists of two beam splitter (BS)–phase shifter (PS) pairs. The transmissivity of the BS is fixed at 50:50, and the PS is thermally modulated to tune the phase. In the diagram, MZIs marked with different colours have different functionalities. The coherent laser is coupled into the chip from the bottom port. Input light division and modulation are realized by the chain of MZIs marked in red. The green marked MZI separates the reference light that will be used for coherent detection. The on-chip light division makes sure that the light signals propagating along different optical paths have the same polarization and share a stable relative phase. The input modulation is dictated by the machine learning task. For tasks with real-valued inputs, the light signals are modulated by the magnitude, and the relative phases between different paths are set to zero. For complex-valued inputs, the modulation includes both magnitude modulation and path-dependent phase rotations. All light signals, as well as the reference light, are generated on chip from a single coherent laser and are modulated by the same chain of PSs. Stringent control is required over the phases of the light signals, when implementing either complex-valued or real-valued networks on the coherent chip. The integration of the light division and modulation effectively avoids the possible phase fluctuations which take place when coupling external light signals to the chip. After the input preparation, six light signals and a reference light are available. Then, the light signals travel through the 6 × 6 optical neural network marked blue in Fig. 1b. An N-mode network realizes the weight matrix multiplication by transforming the input states into output states according to Sout = U(N)Sin. U(N) is a N × N unitary matrix that represents the product of multiple rotation matrices {Tpq} and a diagonal matrix D, such that U N ð Þ¼QNp ¼2 Qp_1 q¼1 TpqD, where the modulus of complex elements on the diagonal of D equal to one, and Tpq is defined as the N-dimensional identity matrix … where θ is defined as the internal PS between two BSs and ϕ is the external PS. An optical network with N inputs realizes an arbitrary N × N unitary weight matrices U(N) by adjusting the tuneable PSs on the MZIs. Detection-based implementation of activation functions are adopted in our demonstrations.
The MZIs marked in grey are used for on-chip coherent detection. The output light signals of the optical chip contain information in both magnitude and phase, while conventional intensity detection techniques only access magnitude information. Our integrated chip is capable of both intensity and coherent detection. The goal of coherent detection is to determine the phase angle ϕs between the reference light and signal light. By connecting photodiodes at both outputs in a balanced way, the obtained output current is II / 2AsAlcosϕs, where As and Al are the respective magnitudes of the signal and the reference light. Similarly, by adding a phase shift of π/2 to the reference light, the output current is IQ / 2AsAlsinϕs. The ϕs is then determined from the ratio of II and IQ, which also helps eliminate the physical noise from the optical components. The choice of detection method is determined by the activation function. Intensity detection is naturally adopted for the activation function M(z) = ||z||, meanwhile the coherent detection is adopted for the activation function … . The detected photocurrents are converted into voltage signals by a transimpedance amplifier (TIA), and then collected and processed by a classical processor with an analogue-to-digital converter (DAC). Feedback signals can be generated and sent back to the ONC to adjust the chip configurations as shown in Fig. 1c.
Classification of nonlinear datasets
Circle and Spiral.
Here, we highlight the capability of complex networks in forming nonlinear decision boundaries, in comparison to their real counterparts. Two nonlinear datasets are studied, namely the Circle and the Spiral. The dataset visualization, model construction (real/ complex-valued for comparison) and chip measurement results are shown from left to right in Fig. 5a. Both datasets are entangled and linearly inseparable. They have two real-valued inputs and two classification classes. For each task, a complex model and an equivalent (same number of layers and neurons) real model are compared. For classification of the Circle, a single complex/real layer with two neurons is adopted. Intensity detection is performed at the output ports of the chip. For the Spiral, a two-layer network is adopted. The first layer has four neurons and are designed as complex-/real-valued for comparison. Intensity detection is performed at this layer, which is equivalent to the application of the activation function M(z) = ||z||. The second layer is a real-valued linear mapping from the four hidden nodes to the two output nodes. The classification results can be interpreted from the chip outputs y1, y2 by a simple manner of Argmax: if y1 ≥ y2, the corresponding instance belongs to the class blue, otherwise if y1 < y2, it belongs to the class pink.
In chip implementation, the real-valued input vectors are encoded by the magnitude of the light signals, while keeping their phases identical. The outputs are complex values and are acquired by coherent detection.
Discussion.
Complex values in neural networks host a number of performance advantages but were burdened by the heavy computational costs of complex multiplication in conventional computers. Here, we have demonstrated the implementation of genuine complex valued neural networks on a single ONC, where complex multiplication can be realized passively by optical interference. The resulting ONCs have significant performance advantages over real-valued counterparts in a range of tasks at both the single neuron and the network level. Notably, a single complex-valued neuron is able to solve certain nonlinear tasks that cannot be done by its real-valued counterpart. Moreover, complex-valued networks on our ONCs demonstrate marked improvements in classification of nonlinear datasets and handwriting recognition tasks. The advantages are briefly concluded as: (a) It offers doubled number of trainable free parameters, using the same physical chip as real-valued networks. (b) It is capable of classifying nonlinear patterns with simple architectures, e.g., fewer layers and neurons, as well as achievable activation function (M(z) = ||z|| by intensity detection). Thus, we have illustrated the potential of complex-valued optical neural networks to feature versatile representations, easy optimization and rapid learning.
We can extend our ONC into a fully fledged multilayer neural network by cascading the optical circuits. Thus our proposal satisfy all the criteria of cascadable photonic neural networks, including isomorphism, physical cascadability, gain cascadability, and noise cascadability22. The isomorphism and physical cascadability are inherently satisfied since each neuron has a hardware counterpart on the ONC, and the output is in the same physical format as the input. Our multilayer proposal is assisted by an electrical interface that matches the gain of input and output with a good gain cascadability, which is distinct from all optical configurations that demand high-gain optical-to-optical nonlinearity.
Consequently, it would have been obvious to one of ordinary skill in the art to modify Ashtiani to disclose: an optical device, comprising: a waveguide to receive an input optical signal encoded with a first vector, and output an output optical signal; one or more optical-to-electrical (O/E) converters to receive one or more optical signals encoded with one or more vectors and generate one or more electrical signals based on the received one or more optical signals; one or more optical modulators optically coupled to the waveguide and electrically coupled to the one or more O/E converters, the one or more optical modulators to modulate an intensity of the input optical signal on the waveguide based on the one or more electrical signals; and wherein the output optical signal is encoded with a product of the first vector and the one or more vectors; Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text; because the resultant configuration would facilitate designing, fabricating, and deploying optical neural networks capable of optical matrix multiplication. Zhang – Selected Text.
Regarding claims 2-6 and 9-11, it would have been obvious to one of ordinary skill in the art to modify Ashtiani in view of Zhang, as applied in the rejection of claim 1, to disclose:
2. The optical device of claim 1, wherein the one or more O/E converters are photodetectors. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
3. The optical device of claim 1, wherein each of the one or more optical modulators comprises: a phase shifter electrically coupled to a respective O/E converter; and a Mach-Zehnder interferometer (MZI) optically coupled to the waveguide and to the phase shifter. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
4. The optical device of claim 3, further comprising: one or more linear transformation devices electrically connected to and between the one or more O/E converters and the one or more optical modulators, wherein the one or more linear transformation devices are configured to convert photocurrents generated by the one or more O/E converters to the one or more electrical signals, wherein the phase shifters are configure to apply a phase change that is proportional to the one or more electrical signals. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
5. The optical device of claim 4, wherein the one or more linear transformation devices are one or more resistors. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
6. The optical device of claim 3, wherein the MZI comprises a micro-ring assisted MZI (RAMZI), wherein the phase shifter is disposed on a micro-ring of the RAMZI.
9. The optical device of claim 3, wherein the MZI comprises a first branch, a second branch, and a branch length difference between lengths of the first branch and the second branch. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
10. The optical device of claim 1, wherein the one or more O/E converters comprises at least: a first O/E converter to receive a first optical signal encoded with a second vector, and a second O/E converter to receive a second optical signal encoded with a third vector, wherein the output optical signal is encoded with an elementwise product of the first vector and the second and third vectors. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
11. The optical device of claim 9, wherein the one or more optical modulators comprises at least: a first optical modulator optically coupled to the waveguide and electrically coupled to the first O/E converter, and a second optical modulator optically coupled to the waveguide and electrically coupled to the second O/E converter. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text.
because the resultant configurations would facilitate designing, fabricating, and deploying optical neural networks capable of optical matrix multiplication. Zhang – Selected Text.
Claims 7 and 8
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Ashtiani et al. (An on-chip photonic deep neural network for image classification. Nature 606, 501–506 (2022); “Ashtiani”) in view of Zhang et al. (An optical neural chip for implementing complex-valued neural network. Nat Commun 12, 457 (2021); “Zhang”), as applied in the rejection of claims 1-6 and 9-11, and further in view of Sacher et al. (Dynamics of microring resonator modulators, Opt. Express 16, 15741-15753 (2008); “Sacher”).
Regarding claims 7 and 8, Sacher discloses embodiments of microring modulators for which, “at higher frequencies, the modulation depth is slightly larger for the over-coupled (σ < a) ring.” Sacher, Numerical Results and Sacher, figure 1, and related text, for example, 2. Time-dependent microring transmission (“The situation when σ = a is referred to as critical coupling. At critical coupling, the wave in the bus waveguide destructively interferes with the wave coupled out of the ring to result in zero transmission [11, 12]. To have complete extinction of the input wave, the modulator must thus operate near the critical coupling condition. Moreover, to use small changes in the index, loss, or coupling to cause large changes in the output intensity, the Q of the resonator must be high (a, σ ≈ 1), so that a circulating wave can, in essence, experience any small changes in device parameters many times before being dissipated.”).
Consequently, it would have been obvious to one of ordinary skill in the art to modify Ashtiani in view of Zhang, as applied in the rejection of claims 1-6 and 9-11, to disclose:
7. The optical device of claim 6, wherein the micro-ring is overcoupled to the MZI. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text; Sacher, figure 1, and related text.
8. The optical device of claim 7, wherein a coupling coefficient between the micro-ring and the MZI is between 0.89 and 0.94. Ashtiani, figure 2, and related figures and text, for example, Ashtiani – Selected Text; Zhang, figure 1, and related figures and text, for example, Zhang – Selected Text; Sacher, figure 1, and related text.
because the resultant configurations would facilitate designing, fabricating, and deploying optical neural networks capable of optical matrix multiplication; Zhang – Selected Text; while predictably tailoring intensity modulation characteristics. Sacher, abstract.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER RADKOWSKI whose telephone number is (571)270-1613. The examiner can normally be reached on M-Th 9-5. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Hollweg, can be reached on (571) 270-1739. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, See http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at (866) 217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/PETER RADKOWSKI/Primary Examiner, Art Unit 2874