Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5-11 and 15-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Image-guided Computational Holographic Wavefront Shaping to Haim et al., hereinafter, “Haim”.
Claim 1. Haim teaches A system for imaging through an obscurant, the system comprising: [Abstract] we introduce a guide-star free noninvasive approach …using just 100 holographically measured scattered random light fields.
[Introduction] Examples include scattering in biological tissues, which limits the penetration depth of optical microscopes to less than a millimeter, and scattering in dense fog…Examiner understands biological tissues and fog to be obscurants – spec [0063].
a spatial light modulator (SLM) or a deformable mirror array (DMA) configured to modulate light; [Abstract] emulating an image-guided wavefront-shaping experiment, where several ’virtual SLMs’ are simultaneously optimized to maximize the reconstructed image quality
one or more sensors configured to capture an image; [Fig 1, 3, 4 5] camera
a processor; [page 5] This is done in a straightforward fashion using a GPU-optimized Python library developed for the training of deep neural networks
[page 15, para 4] The measured running time of the results presented in Fig.2, running on a consumer NVIDIA GeForce RTX 3090 GPU, was 6 minutes for 500 iterations.
and a memory including instructions stored thereon which, when executed by the processor, cause the system to: [page 5, para 3] The object image is reconstructed by digitally reversing the physical propagation through the complex medium, … at each iteration of the algorithm…efficient automatic-differentiation based gradient-descent search after the phase pattern yielding the optimal image quality can be performed… using a GPU-optimized Python library
[page 15] For the implementation of the gradient-descent algorithm, the PyTorch library was used [32], in order to leverage its automatic-differentiation and GPU-enabled capabilities.
incoherently illuminate a target by a light, [page 12, 1st para] the diffused illumination illuminates an area …the object, resulting in very low contrast incoherently-compounded initial images
wherein the obscurant scatters the light creating an optical aberration; [Introduction] when the scattering is tilt-invariant to a sufficient degree, also known as the ’optical memory-effect’ [2, 16], decomposition of the reflection matrix [13, 14], or correction of the guide-star wavefront distortion
[Methods] The forward physical model connecting the desired object pattern with the field measured by the camera at each speckle illumination is obtained by propagating the object field, On(r) = O(r)Sn, through the scattering medium
modulate the scattered light by the SLM or DMA; [Experimental imaging through highly scattering layers] …effectively correct the wavefront distortion using a virtual SLM with 600×600 DoFs
[Methods] the computational image guided correction can be easily extended to multiple virtual SLMs conjugated to different planes in the thick scattering sample
capture, by the one or more sensors, an image of the target as illuminated by the modulated light; [Fig.1] the object image
generate a simulated image by a differential model; [page 12] by leveraging state-of-the-art automatic-differentiation optimization tools used for the training of neural-networks, our physical-model based method allows an efficient parallel multi-conjugate correction
[page 13] The forward physical model connecting the desired object pattern with the field measured by the camera at each speckle illumination is obtained by propagating the object field, On(r) = O(r)Sn, through the scattering medium.
[page 14] we have adapted the well-established image variance metric (representing the contrast of the reconstructed image)… Once the image quality of It(r) is calculated, it can be easily differentiated with respect to φt(r) using automatic differentiation
compare the captured image with the simulated image; [page 6] Fig.2 …uncorrected reconstructed image is a low contrast diffusive blur (Fig.2A), the computationally corrected image is a high-contrast diffraction-limited image (Fig.2B) that reveals all the details of the target…verified by comparison to the reconstructed image of the target when the wavefront correction is recorded in an invasive reference measurement prior to placing the target (Fig.2C)
estimate the target, the aberration, and a phase delay based on back-propagation of the comparison; [page 6] Fig.2 show that while the uncorrected reconstructed image is a low contrast diffusive blur (Fig.2A) (aberration), the computationally corrected image is a high-contrast diffraction-limited image (Fig.2B) that reveals all the details of the target (target). This is verified by comparison to the reconstructed image of the target when the wavefront correction is recorded in an invasive reference measurement prior to placing the target (Fig.2C)… The optimization successfully recovers the object even when a different back propagation distance, zprop, is used. It compensates for the propagation difference by an additional spherical phase mask correction (a phase delay based on back-propagation of the comparison)
[Fig.1] …a computational reconstruction step (bottom) where the holographically-recorded fields are computationally traced-back to the estimated object plane, using a correction phase mask φt(r), forming a ’virtual SLM’ and computational back-propagation
[page 5] at each iteration of the algorithm, t, the incoherently-compounded object image (target) is reconstructed by tracing back each of the measured fields through a correction mask, e−iφt(r), and digital back propagation of a distance −zprop to the estimated plane where the object lies
[page 13] The incoherently compounded image is formed by averaging the intensity patterns of the back-propagated fields
and correct for the aberration based on at least one of the estimated target, the aberration, or the phase delay. [page 4] The end result is both the complex correction phase mask and the corrected object image
Claim 5. Haim teaches wherein the optical aberration includes a dynamic aberration. [page 5] …it allows for a fast acquisition process that is adapted to dynamically varying media
Claim 6. Haim teaches wherein the modulation patterns on the SLM or the DMA are optimized to maximize imaging performance. [page 5] … In image-guided wavefront-shaping a physical wavefront correction is found by varying an SLM phase-pattern to optimize the spatially-incoherent image quality metric [25].
Claim 7. Haim teaches wherein the optimized modulation patterns are obtained by using a neural network. [Abstract] … to a digital, naturally-parallelizable computation, leveraging state-of-the-art automatic-differentiation optimization tools used for the training of neural-networks.
Claim 8. Haim teaches wherein the instructions, when executed by the processor, further cause the system to: optimize the modulation pattern based on a model parameterized by the neural network. [Application in acousto-optic tomography] Acousto-optical modulation allows to noninvasively isolate small portions of the object that can be reconstructed with high fidelity with our approach… For each of the m = 1..M ultrasound focus positions, the N measured fields (Fig.5C) are fed as the input to the computational image-guided wavefront-shaping algorithm. Examiner interprets “with high fidelity” to be optimization and “m” is the parameter.
[page 3, para 2] The optimization is performed in a highly-parallel fashion using an automatic-differentiation based gradient-descent algorithm commonly used for the training of deep neural networks.
[page 12] … the optimization can be extended to additional parameters, such as the distances between correction planes, object position, and experimental imperfections.
Claim 9. Haim teaches wherein the modulation includes a series of known and stochastically generated patterns. [Methods, page 13] Each field is obtained when a different random (stochastically) and unknown spatially-coherent speckle field illumination of the object, Sn(r).
[page 5] the computationally reconstructed image quality depends in a well-defined fashion on the virtual SLM phase pattern…Examiner interprets well-defined on the virtual SLM phase pattern to be a known pattern.
Claim 10. Haim teaches wherein the modulation includes a random generation of patterns. [page 4] A hidden object, placed behind or inside a highly scattering complex medium, is illuminated by random unknown speckle patterns.
Claim 11. Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 15. Reviewed and analyzed in the same way as claim 5. See the above analysis and rationale.
Claim 16. Reviewed and analyzed in the same way as claim 9. See the above analysis and rationale.
Claim 17. Reviewed and analyzed in the same way as claim 7. See the above analysis and rationale.
Claim 18. Reviewed and analyzed in the same way as claim 8. See the above analysis and rationale.
Claim 19. Reviewed and analyzed in the same way as claim 10. See the above analysis and rationale.
Claim 20. Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Image-guided Computational Holographic Wavefront Shaping to Haim et al., hereinafter, “Haim” in view of Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network to Bostan et al., hereinafter, “Bostan”.
Claim 2. Haim teaches wherein the differential model includes: Haim [page 12] by leveraging state-of-the-art automatic-differentiation optimization tools used for the training of neural-networks, our physical-model based method allows an efficient parallel multi-conjugate correction
Haim [page 13] The forward physical model connecting the desired object pattern with the field measured by the camera at each speckle illumination is obtained by propagating the object field, On(r) = O(r)Sn, through the scattering medium.
a neural object representation configured to predict an intensity of the object; [page 3, para 2] The optimization is performed in a highly-parallel fashion using an automatic-differentiation based gradient-descent algorithm commonly used for the training of deep neural networks.
Haim fails to explicitly teach a neural aberration representation configured to predict the aberration. Bostan, in the field of phase microscopy imaging using neural networks, teaches and a neural aberration representation configured to predict the aberration. Figure 1: Deep Phase Decoder (DPD) Network…Zernike polynomials > aberrations
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Haim with the teachings of Bostan [Abstract] and [Introduction] for phase microscopy by using an untrained deep neural network for measurement formation.
Claim 12. Reviewed and analyzed in the same way as claim 2. See the above analysis and rationale.
Allowable Subject Matter
Claims 3-4 and 13-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Prior art fails to explicitly teaches predict the intensity of the object by the neural object representation by: predicting a displacement vector (Δx, Δy) by a first multilayer perceptron network based on a time-dependent observation (x, y, t); projecting a canonical space feature (x + Δx, y + Δy) on a neural texture map based on the displacement vector; sampling the neural texture map at (x + Δx, y + Δy) to obtain a multi-dimensional vector representing the spatial feature of the canonical coordinate (x + Δx, y + Δy); and predicting, by a second multilayer perceptron network, the intensity based on the multi-dimensional vector for each coordinate as recited in claims 3 and 13.
Claims 4 and 14 would be allowable because they are dependents of claims 3adn 13, respectively.
Claim 13. Reviewed and analyzed in the same way as claim 3. See the above analysis and rationale.
Claim 14. Reviewed and analyzed in the same way as claim 4. See the above analysis and rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DELOMIA L GILLIARD whose telephone number is (571)272-1681. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DELOMIA L GILLIARD/Primary Examiner, Art Unit 2661
Prosecution Insights