Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 7-14, 16, 18-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Application Publication NO. 20210357737 “Hamerly”.
Claim 1:
Hamerly teaches an optical device for implementing an optical neural network (i.e. para. [0008], One of these optical architectures is an optical neural network that includes at least one coherent light source, first and second optical fan-out elements, a two-dimensional array of homodyne receivers in optical communication with the first and second optical fan-out elements, electronic circuitry operably coupled to the two-dimensional array of homodyne receivers, and a light source operably coupled to the electronic circuitry), the optical device comprising: a plurality of input waveguides configured to receive a plurality of input optical signals (i.e. para. [0015], for each image in a set of images that represents an input to a layer in the digital optical neural network, breaking the image into a set of patches); a first transform device coupled to the plurality input waveguides and configured to perform a first transform of the input optical signals to output a plurality of intermediate optical signals (i.e. para. [0126], “Since the convolutions in the method 800 of FIG. 8 are taken on spatially-encoded optical data, they can be performed using Fourier optics: convert the image to its Fourier transform with a lens”, wherein it is noted that an input image may be converted to a frequency domain); a plurality of intermediate waveguides that receive the plurality of intermediate optical signals from the first transform device, each intermediate waveguide comprising a phase modulator and an amplitude modulator (i.e. para. [0038], FIG. 9A shows conversion of a point source to a plane wave in the Fourier plane, whose amplitude and/or phase are modified with a spatial light modulator (SLM)); a second transform device coupled to the intermediate waveguides and that receives the plurality of intermediate optical signals, the second transform device configured to perform a second transform of the plurality of intermediate optical signals and output a plurality of output optical signals (i.e. para. [0155], “the convolution may be computed by an elementwise multiplication in Fourier space (1204) followed for an inverse Fourier transform (1206) back to real space”, wherein the transform from the frequency domain back into the real spatial domain would encompasses modifying the image waveguides while in the frequency domain with amplitude and phase modulators); and an array of non-linear activation devices and photodetectors that receive the plurality of output optical signals (i.e. para. [0065], Circuitry 228 coupled to the differential homodyne detectors 226 applies a nonlinear function (e.g., a Sigmoid function) in the electrical domain to each detector output, which are read out and used to serially modulate a light source 230, such as a laser, that emits a coherent optical carrier modulated with the layer output), wherein the optical device is monolithically formed on a substrate (i.e. para. [0172], “FIG. 15B illustrates one way to realize a many-channel WDM on-chip. This uses a traditional WDM, such as an arrayed waveguide grating (AWG) or cascaded Mach-Zehnder filters 1552, to do the (de)multiplexing at coarse channels, while high-Q ring resonators 1554 provide the fine resolution”, wherein the BRI for monolithic encompasses how the Mach-Zehnder modulator implemented as a filter is on a single “on-chip”).
Claim 2:
Hamerly teaches the optical device of claim 1, wherein the second transform is an inverse of the first transform (i.e. para. [0126], convert the image to its Fourier transform with a lens, pass the Fourier-transformed image through an SLM, and perform an inverse Fourier transform on the SLM output with another lens).
Claim 3:
Hamerly teaches the optical device of claim 1, wherein the input signals are provided in a spatial domain and the intermediate optical signals are provided in a spatial frequency domain 9i.e. para. [0160], “The CONV layer 1220 includes a coherent transceiver array 1230 in the image plane and a coherent transceiver array 1240 in the Fourier plane”, wherein it is noted in Fig. 14A-B that an input image plane is in the spatial domain before being converted into the Fourier or Frequency domain).
Claim 7:
Hamerly teaches the optical device of claim 1, wherein the amplitude modulators comprise one or more of a Mach-Zehnder Interferometer and a micro-ring resonator (i.e. para. [0172], FIG. 15B illustrates one way to realize a many-channel WDM on-chip. This uses a traditional WDM, such as an arrayed waveguide grating (AWG) or cascaded Mach-Zehnder filters 1552, to do the (de)multiplexing at coarse channels, while high-Q ring resonators 1554 provide the fine resolution)..
Claim 8:
Hamerly teaches the optical device of claim 1, wherein the first transform device, the phase modulators, the amplitude modulators, and the second transform device are configured to perform a convolution of input data encoded onto the input optical signals (i.e. para. [0059], Since the optical processor performs the convolution as a matrix-matrix (rather than matrix-vector) operation, it is possible to obtain energy savings even without running the neural network on large batches of data).
Claim 9:
Hamerly teaches the optical device of claim 1, wherein the first transform device and the second transform device are formed at a common region of the substrate (i.e. para. [0172], “FIG. 15B illustrates one way to realize a many-channel WDM on-chip. This uses a traditional WDM, such as an arrayed waveguide grating (AWG) or cascaded Mach-Zehnder filters 1552, to do the (de)multiplexing at coarse channels, while high-Q ring resonators 1554 provide the fine resolution”, wherein the BRI for a common region of the substrate encompasses how the Mach-Zehnder modulator implemented as a filter is on a single “on-chip”).
Claim 10:
Hamerly teaches a method of implementing an optical neural network (i.e. para. [0008], One of these optical architectures is an optical neural network that includes at least one coherent light source, first and second optical fan-out elements, a two-dimensional array of homodyne receivers in optical communication with the first and second optical fan-out elements, electronic circuitry operably coupled to the two-dimensional array of homodyne receivers, and a light source operably coupled to the electronic circuitry), comprising: performing, by a first transform device, a first transformation on a plurality of input optical signals, in a first domain, encoded with data to provide a plurality of first intermediate optical signals in a second domain (i.e. para. [0126], “Since the convolutions in the method 800 of FIG. 8 are taken on spatially-encoded optical data, they can be performed using Fourier optics: convert the image to its Fourier transform with a lens”, wherein it is noted that an input image in the spatial domain may be converted to a frequency domain); applying a filter to the plurality of first intermediate optical signals in the second domain to provide a plurality of second intermediate optical signals , wherein the filter comprises a plurality of phase modulators and a plurality of amplitude modulators (i.e. para. [0038], FIG. 9A shows conversion of a point source to a plane wave in the Fourier plane, whose amplitude and/or phase are modified with a spatial light modulator (SLM)) encoded according to weights of the optical neural network (i.e. para. [0129], Each kernel weight is called W′H′ times, however, the fan-out in FIG. 8 is (W′/K.sub.x)(H′/K.sub.y). While still ≳20 for typical CNN layers, this is much less than the input fan-out ratio); performing, by a second transform device, a second transformation on the plurality of second intermediate optical signals to provide a plurality of output optical signals in the first domain (i.e. para. [0156], the convolution may be computed by an elementwise multiplication in Fourier space (1204) followed for an inverse Fourier transform (1206) back to real space); and classifying the data based on optical power of the plurality of output optical signals detected by one or more photodetectors (i.e. para. [0023], FIG. 3B illustrates classification of an MNIST image classified by the three-layer neural network in FIG. 3A).
Claim 11:
Claim 11 is the method claim reciting similar limitations to Claim 1 and is rejected for similar reasons.
Claim 12:
Claim 12 is the method claim reciting similar limitations to Claim 9 and is rejected for similar reasons.
Claim 13:
Hamerly teaches The method of claim 10, further comprising: tuning the plurality phase modulators and the plurality of amplitude modulators based on a weight matrix comprising the weights (i.e. para. [0129], Each kernel weight is called W′H′ times, however, the fan-out in FIG. 8 is (W′/K.sub.x)(H′/K.sub.y). While still ≳20 for typical CNN layers, this is much less than the input fan-out ratio).
Claim 14:
Claim 14 is the method claim reciting similar limitations to Claim 3 and is rejected for similar reasons.
Claim 16:
Hamerly teaches the method of claim 10, wherein classifying the data based optical power of the plurality of output optical signals detected by one or more photodetectors comprises: activating one or more non-linear activation devices based on the optical power of the output optical signals, wherein the one or more non-linear activation devices output one or more activation signals to the one or more photodetectors (i.e. para. [0065], Circuitry 228 coupled to the differential homodyne detectors 226 applies a nonlinear function (e.g., a Sigmoid function) in the electrical domain to each detector output, which are read out and used to serially modulate a light source 230, such as a laser, that emits a coherent optical carrier modulated with the layer output); detecting an optical power of the one or more activation signals by the one or more photodetectors (i.e. para. [0066], This allows the weights and optical input signals for a given layer 110 to interfere at the homodyne detectors 226 in that layer 110. This can be accomplished by using a single pulsed laser to generate light that is distributed and modulated to provide weights and optical input/output signals for all of the layers); and responsive to the detected optical power exceeding a threshold, classifying the data according to a class associated with the one or more photodetectors (i.e. para. [0079], “ the size of the inputs, two inner layers, and outputs scales as (784.fwdarw.1000.fwdarw.1000.fwdarw.10). These networks are trained on the MNIST dataset as shown in FIG. 3B.”, wherein it is noted that the BRI for a threshold encompasses how a specific class for identifying a digit in NIST is the result of the cumulative processing of signals across all network layer, wherein the highest resulting signal would indicate the predicted class).
Claim 18:
Claim 18 is the method claim reciting similar limitations to Claim 7 and is rejected for similar reasons.
Claim 19:
Claim 19 is the method claim reciting similar limitations to Claim 8 and is rejected for similar reasons.
Claim 20:
Hamerly teaches an optical convolution neural network (i.e. para. [0008], One of these optical architectures is an optical neural network that includes at least one coherent light source, first and second optical fan-out elements, a two-dimensional array of homodyne receivers in optical communication with the first and second optical fan-out elements, electronic circuitry operably coupled to the two-dimensional array of homodyne receivers, and a light source operably coupled to the electronic circuitry), comprising: a substrate (i.e. para. [0172], “FIG. 15B illustrates one way to realize a many-channel WDM on-chip. This uses a traditional WDM, such as an arrayed waveguide grating (AWG) or cascaded Mach-Zehnder filters 1552”, wherein the BRI for a substrate encompasses a chip); an optical Fourier transformation device formed on the substrate and configured to transform input optical signals encoded with an input image into Fourier transformed optical signals in a spatial frequency domain (i.e. para. [0126], “Since the convolutions in the method 800 of FIG. 8 are taken on spatially-encoded optical data, they can be performed using Fourier optics: convert the image to its Fourier transform with a lens”, wherein it is noted that an input image in the spatial domain may be converted to a frequency domain); a plurality of modulation devices formed on the substrate and optically coupled to the optical Fourier transformation device, the plurality of modulation devices (i.e. para. [0038], FIG. 9A shows conversion of a point source to a plane wave in the Fourier plane, whose amplitude and/or phase are modified with a spatial light modulator (SLM)) configured to apply elements of a weight matrix to each of the Fourier transformed optical signals in the spatial frequency domain (i.e. para. [0129], Each kernel weight is called W′H′ times, however, the fan-out in FIG. 8 is (W′/K.sub.x)(H′/K.sub.y). While still ≳20 for typical CNN layers, this is much less than the input fan-out ratio); and an inverse optical Fourier transformation device formed on the substrate and optically coupled to the plurality of modulation devices, the inverse optical Fourier transformation device configured to transform the Fourier transformed optical signals to a spatial domain and output convolved optical signals (i.e. para. [0155], “the convolution may be computed by an elementwise multiplication in Fourier space (1204) followed for an inverse Fourier transform (1206) back to real space”, wherein the transform from the frequency domain back into the real spatial domain would encompasses modifying the image waveguides while in the frequency domain with amplitude and phase modulators), wherein the input image is labeled with a class based on the convolved optical signals (i.e. para. [0023], “FIG. 3B illustrates classification of an MNIST image classified by the three-layer neural network in FIG. 3A”, wherein the BRI for a label with a class encompasses labeling the image as a number from 0-9).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4-5 & 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20210357737 “Hamerly” and further in light of U.S. Patent Application Publication NO. 20200250532 “Shen”.
Claim 4:
Hamerly teaches the optical device of claim 1.
While Hamerly teaches using the principals of light interference within the architecture of an optical neural network, Hamerly may not explicitly teach, wherein the first transform device comprises one of: one or more star-couplers and one or more multimode interferometers
However, Shen teaches wherein the first transform device comprises one of: one or more star-couplers and one or more multimode interferometers (i.e. para. [0359], “The optical power splitter may be, for example, a 1:N multimode interference (MMI) splitter, a multi-stage splitter including multiple 1:2 MMI splitter or directional-couplers, or a star coupler”, wherein it is noted that within a single common substrate a plurality of optical devices such as splitters and optical amplitude modulators, and electrical devices such as photodetectors and operational amplifiers (op-amps) can be fabricated on the common substrate).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add one or more star-couplers and one or more multimode interferometers, to the optical-electronic convolutional neural networks of Hamerly, with how one or more star-couplers and one or more multimode interferometers may be used in an optical CNN bound to a single substrate, as taught by Shen. One would have been motivated to combine Shen with Hamerly, and would have had a reasonable expectation of success in doing so, because such a combination provides greater integration density may be beneficial in fabrication of the OMNI unit, as the OMNI unit typically includes 10's to 100's of optical components such as power splitters and phase shifters.
Claim 5:
Hamerly and Shen teach the optical device of claim 4.
Shen further teaches wherein the second transform device comprises one of: one or more star-couplers and one or more multimode interferometers (i.e. para. [0359], “The optical power splitter may be, for example, a 1:N multimode interference (MMI) splitter, a multi-stage splitter including multiple 1:2 MMI splitter or directional-couplers, or a star coupler”, wherein it is noted that within a single common substrate a plurality of optical devices such as splitters and optical amplitude modulators, and electrical devices such as photodetectors and operational amplifiers (op-amps) can be fabricated on the common substrate).
Claim 15:
Claim 15 is the method claim reciting similar limitations to Claims 4-5 and is rejected for similar reasons.
Claim(s) 6 & 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20210357737 “Hamerly” and further in light of Reed, G. T., Thomson, D. J., Gardes, F. Y., Hu, Y., Fedeli, J.-M., & Mashanovich, G. Z. (2014). High-speed carrier-depletion silicon Mach-Zehnder optical modulators with lateral PN junctions. Frontiers in Physics, 2. https://doi.org/10.3389/fphy.2014.00077, hereinafter “Reed”.
Claim 6:
Hamerly teaches the optical device of claim 1.
While Hamerly teaches a phase modulator in the form of a spatial light modulator (SLM) that can convert a point source to a plane wave in the Fourier plane and back to real space, Hamerly may not explicitly teach
wherein the phase modulators comprise one or more of a resistor, a PN diode, a metal-oxide-semiconductor capacitor.
However, Reed teaches
wherein the phase modulators comprise one or more of a resistor, a PN diode, a metal-oxide-semiconductor capacitor (i.e. pg. 2, The layout of the phase modulator cross-section is shown in Figure 1. Half of the rib section of the waveguide and the slab to one side is formed of p-type silicon. The other half of the waveguide rib and the slab region to the other side is formed of n-type silicon).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add wherein the phase modulators comprise one or more of a resistor, a PN diode, a metal-oxide-semiconductor capacitor, to the optical-electronic convolutional neural networks of Hamerly, with how wherein the phase modulators comprises a PN diode, as taught by Reed. One would have been motivated to combine Reed with Hamerly, and would have had a reasonable expectation of success in doing so, because such a combination provides a scheme for achieving high modulation efficiency whilst retaining self-aligned formation of the PN junction.
Claim 17:
Claim 17 is the method claim reciting similar limitations to Claim 6 and is rejected for similar reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent Application Publication NO. 20230360385 “Lam” teaches in para. [0082], the neural-network-based ensemble includes a convolutional-neural-network-based ensemble model, which has a first convolutional neural network arranged to process the magnitude information to extract magnitude features, a second convolutional neural network arranged to process the phase information to extract phase features, and a concatenate unit arranged to combine magnitude features extracted by the first convolutional neural network and phase features extracted by the second convolutional neural network for identification and/or classification of the object.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TAN whose telephone number is (571)272-7433. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T./Examiner, Art Unit 2145
/CHAU T NGUYEN/Primary Examiner, Art Unit 2145
Prosecution Insights