DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In communications filed on 07/15/2026. Claims 10-20 cancelled. Claims 1-9 are pending in this examination.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. This examination is in response to US Patent Application No. 18/472,649.
Response to Restriction Requirement
In response, Applicant hereby elects :
Elections:
I. Claims 1-9
II. Claims 10-16
III. Claims 17-20
Without acquiescing to the propriety of this restriction requirement, Applicants hereby elect Group I, which corresponds to claims 1-9 without traverse, in order to expedite prosecution. Applicants herewith submit an amendment canceling claims 10-20.
Examiner, therefore, has examined claims 1-9 in this instant office action.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent No. 2022/0366253 issue to Ozcan in view of US Patent Application No. (12,524,932) to Shao et al (“Shao”)
Regarding claim 1 Ozcan discloses an optical device for implementing a neural network, the optical device comprising: [0016] FIG. 1 schematically illustrates one embodiment of a D.sup.2NN that is used in transmission mode according to one embodiment. A source of light (which may be natural or artificial) directs light onto an object which reflect (or passes through the object in other embodiments) and is directed through the D.sup.2NN. In this mode, light passes through the individual substrate layers that form the D.sup.2NN. The light that passes through the D.sup.2NN is then detected by an optical detection device (e.g., optical sensor).
a plurality of photon directing devices disposed along a common optical axis, the plurality of photon directing devices configured to receive light from an input and direct the light according to a trained machine learning model, each photon directing device of the plurality of photon directing devices corresponding to a layer of the trained machine learning model [ see FIG 1 and corresponding text for more details, ¶¶ 96-100, FIG. 1 schematically illustrates one embodiment of a Diffractive Deep Neural Network D.sup.2NN 10 that is used in transmission mode according to one embodiment. A source of light 12 directs light onto an object 14 (or multiple objects 14) which reflects and is directed through the D.sup.2NN 10…the source of light 12 may act as an excitation light source and the D.sup.2NN 10 receives fluorescent light that is emitted from the object 14. The source of light 12 may include a natural light source such as the sun( equated to photon emission) … The plurality of optically transmissive layers 16 arranged along the optical path 11 collectively define a trained mapping function between an input optical image or input optical signal 20 to the plurality layers 16 and an output optical image or output optical signal 22 created by optical diffraction through the plurality of substrate layers 16….].
a plurality of lenses provided for each photon directing device of the plurality of photon directing device, the plurality of lenses of a respective photon directing device being formed based on weights of a layer of the trained machine learning model corresponding to the respective photon directing device [0098] Each substrate layer 16 of the D.sup.2NN 10 has a plurality of physical features 18 formed on the surface of the substrate layer 16 or within the substrate layer 16 itself that collectively define a pattern of physical locations along the length and width of each substrate layer 16 that have varied complex-valued transmission coefficients (or varied complex-valued transmission reflection coefficients for the embodiment of FIG. 2). The physical features 18 formed on or in the layers 16 thus create a pattern of physical locations within the layers 16 that have different complex-valued transmission coefficients as a function of lateral coordinates (e.g., length and width and in some embodiments depth) across each substrate layer 16. In some embodiments, each separate physical feature 18 may define a discrete physical location on the substrate layer 16 while in other embodiments, multiple physical features 18 may combine or collectively define a physical region with a particular complex-valued transmission coefficient. The plurality of optically transmissive layers 16 arranged along the optical path 11 collectively define a trained mapping function between an input optical image or input optical signal 20 to the plurality layers 16 and an output optical image or output optical signal 22 created by optical diffraction through the plurality of substrate layers 16
and a plurality of optical sensors to receive the light directed by the plurality of photon directing devices, each of the plurality of optical sensors corresponding to an inference of the trained machine learning model 0100] As seen in FIG. 1, the output optical image or output optical signal 22 is captured by one or more optical sensors 26. The optical sensor 26 may include, for example, an image sensor (e.g., CMOS image sensor or image chip such as CCD), photodetectors (e.g., photodiode such as avalanche photodiode detector (APD)), photomultiplier (PMT) device, and the like… In other embodiments, the optical sensor 26 may be integrated within a device such as a camera that is configured to acquire, store, process, manipulate, and/or transfer the output optical image or output optical signal 22. For example, the D.sup.2NN 10 may integrated inside a camera according to one embodiment.].
While Ozcan discloses photon directing devices as: [0135] Among other techniques, laser lithography based on two-photon polymerization can provide a desired solution for creating such monolithic D.sup.2NNs 10.
Ozcan does not explicitly disclose; however, Shao discloses photon directing devices
[claim 1, A method for reconstructing an image of one or more internal structures using a network of machine learning models, the method comprising: providing projection data from a single-photon emission computerized tomography (SPECT) scan representing the image of the one or more internal structures, and generated using SPECT scan apparatus, as an input to a first machine learning model having one or more fully-connected layers that have been trained to generate first output data that represents an initial reconstruction of a first image based on processing of the projection data from the SPECT scan by the first machine learning model], and [claim 7. The method of claim 1, wherein the one or more internal structures include a brain, a heart, or other biological component].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Ozcan by incorporating “single-photon emission computerized tomography (SPECT) projection ”, as taught by Shao. One could have been motivated to do so in order for reconstructing an image of one or more internal structures using a network of machine learning models [ Shao, claim 1].
Regarding claim 2 , Ozcan discloses the optical device of claim 1, wherein each lens of the plurality of lenses comprises a focal power defined by the weights.
[0002] The technical field generally relates to an optical deep learning physical architecture or platform that can perform, at the speed of light, various complex functions and tasks that current computer-based neural networks can implement. The optical deep learning physical architecture or platform has applications in image analysis, feature detection, object classification, camera designs, and other optical components that can learn to perform unique functions or tasks , and [ 0100-0101] In other embodiments, the optical sensor 26 may be integrated within a device such as a camera that is configured to acquire, store, process, manipulate, and/or transfer the output optical image or output optical signal 22. For example, the D.sup.2NN 10 may integrated inside a camera according to one embodiment… in FIG 2, Like the FIG. 1 embodiment, the output optical image or output optical signal 22 is captured by one or more optical sensors 26. The one or more optical sensors 26 may be coupled to a computing device 27 as noted or integrated into a device such as a camera as noted above.].
Regarding claim 3 , Ozcan discloses, the optical device of claim 1, wherein each photon directing device is provided as a sheet of material, and wherein the plurality of lenses comprises deformations within the material of the respective photon directing device.
[0102-103] FIG. 4 illustrates one embodiment of how different physical features 18 are formed in the substrate layer 16. In this embodiment, a substrate 16 has different thicknesses (t) of material at different lateral locations along the substrate layer 16. In one embodiment, the different thicknesses (t) modulates the phase of the light passing through the substrate layer 16. This type of physical feature 18 may be used, for instance, in the transmission mode embodiment of FIG. 1. The different thicknesses of material in the substrate layer 16 forms a plurality of discrete “peaks” and “valleys” that control the complex-valued transmission coefficient of the neurons 24 formed in the substrate layer 16… FIG. 5 illustrates another embodiment in which the physical features 18 are created or formed within the substrate 16. In this embodiment, the substrate 16 may have a substantially uniform thickness but have different regions of the substrate 16 have different optical properties. For example, the complex-valued refractive index of the substrate layers 16 may altered by doping the substrate layers 16 with a dopant (e.g., ions or the like) to form the regions of neurons 24 in the substrate layers 16 with controlled transmission properties. In still other embodiments, optical nonlinearity can be incorporated into the deep optical network design using various optical non-linear materials (crystals, polymers, semiconductor materials, doped glasses, polymers, organic materials, semiconductors, graphene, quantum dots, carbon nanotubes, and the like) that are incorporated into the substrate 16. A masking layer or coating that partially transmits or partially blocks light in different lateral locations on the substrate 16 may also be used to form the neurons 16 on the substrate layers 16].
Regarding claim 4 , Ozcan discloses, wherein the deformations are defined by the weights of the layer of the trained machine learning model corresponding to the respective photon directing device.
[0102] FIG. 4 illustrates one embodiment of how different physical features 18 are formed in the substrate layer 16. In this embodiment, a substrate 16 has different thicknesses (t) of material at different lateral locations along the substrate layer 16. In one embodiment, the different thicknesses (t) modulates the phase of the light passing through the substrate layer 16. This type of physical feature 18 may be used, for instance, in the transmission mode embodiment of FIG. 1. The different thicknesses of material in the substrate layer 16 forms a plurality of discrete “peaks” and “valleys” that control the complex-valued transmission coefficient of the neurons 24 formed in the substrate layer 16.
Regarding claim 5 , Ozcan discloses, wherein each lens of the plurality of lenses comprises an optical axis, wherein one or more of the optical axes are tilted with respect to the common optical axis based on the weights.
[ se FIG. 2 and corresponding text for more details, [0101] FIG. 2 schematically illustrates one embodiment of a D.sup.2NN 10 that is used in reflection mode according to one embodiment…].
Regarding claim 6 , Ozcan discloses, wherein each lens of the plurality of lenses corresponds to a neuron of a corresponding layer of the trained machine learning model and formed to mimic connections between the neuron of the corresponding layer and one or more neurons of an adjacent layer of the trained machine learning model.
[0101]…As seen in the embodiment of FIG. 2, the optical path 11 is a folded optical path as a result of the reflections off the plurality of substrate layers 16. The number of substrate layers 16 may vary depending on the particular function or task that is to be performed as noted above. Each substrate layer 16 of the D.sup.2NN 10 has a plurality of physical features 18 formed on the surface of the substrate layer 16 or within the substrate layer 16 itself that collectively define a pattern of physical locations along the length and width of each substrate layer 16 that have varied complex-valued reflection coefficients. Like the FIG. 1 embodiment, the output optical image or output optical signal 22 is captured by one or more optical sensors 26. The one or more optical sensors 26 may be coupled to a computing device 27 as noted or integrated into a device such as a camera as noted above.
Regarding claim 7 , Ozcan discloses, wherein the plurality of optical sensors are configured to detect an intensity of light from the plurality of photon directing devices and activate the neural network to label the input based on detected intensity.
[0095] FIGS. 38A and 38B illustrate the hybrid system training process. FIG. 38A illustrates the first stage of the hybrid system training. FIG. 38B illustrates the second stage of the hybrid system training starts with the already trained diffractive layers (first 5 layers) from FIG. 38A and an electronic neural network, replacing the operations after intensity detection at the sensor], and [00124 … (FIG. 15A), and to quantify the match between these numerical testing results and the experiments, 50 handwritten digits i.e., 5 different inputs per digit, selected among the same 91.75% of the test images were 3D printed and demonstrated that numerical testing was successful. For each input object that is uniformly illuminated with the THz source, the output plane was imaged of the D.sup.2NN to map the intensity distribution for each detector region that is assigned to a digit. The results illustrated in FIG. 15B demonstrate the success of the 3D-printed diffractive neural network 10 and its inference capability: the average intensity distribution at the output plane of the network for each input digit clearly reveals that the 3D-printed D.sup.2NN 10 was able to focus the input energy of the beam and achieve a maximum signal at the corresponding detector region that was assigned for that digit], and [0138, 0145, 0158].
Regarding claim 8 , Ozcan discloses, wherein the plurality of photon directing devices corresponds to a plurality of machine learning models based on a plurality of polarizations of light.
[0101]…non-passive components may be incorporated in into the substrates 16 such as spatial light modulators (SLMs). SLMs are devices that imposes spatial varying modulation of the phase, amplitude, or polarization of a light.
Regarding claim 9 , Ozcan discloses, wherein the plurality of photon directing devices corresponds to a plurality of machine learning model based on a plurality of electromagnetic wavelengths.
[0097] The source of light 12 that illuminates the object 14 may have any number of wavelengths including visible light (e.g., light with a wavelength in the range of about 380 nm to about 740 nm) as well as light outside the perception range of humans. For example, the wavelength operating range may extend beyond the visible perception range of humans (e.g., from about 300 nm to about 1,000 nm). The long wavelength light used in the experiments described herein was used due to the coarse resolution of the physical features 18 contained in the layers 16 during the 3D printing process used to fabricate the D.sup.2NN 10. Shorter wavelengths of light may be used for D.sup.2NN 10 with smaller physical feature 18 sizes.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
See submitted 892 for more relevant references.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHRIAR ZARRINEH whose telephone number is (571)272-1207. The examiner can normally be reached Monday-Friday, 8:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge Ortiz-Criado can be reached at 571-272-7624. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHRIAR ZARRINEH/Primary Examiner, Art Unit 2496
Prosecution Insights