DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/07/2023 has been considered by the examiner.
Claim Objections
The following claims are objected to because of the following informalities:Claim 25 reads "a phase profile considered to be ideal with no phase error, and rather than using an input target image" should read “a phase profile considered to be ideal with no phase error rather than using an input target image” for clarity.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-10, 14-16, 18 and 21-25 are rejected under 35 U.S.C. 103 as being unpatentable over Sobieranski et al. (US PUB 2019/0340736; herein after “Sobieranski”; in related embodiments) in view of Goulanian et al. (US PUB 2005/0122549; herein after “Goulanian”).
Sobieranski and Goulanian disclose method and system for generating holographic images. Therefore, they are analogous art.
Regarding claim 1, Sobieranski teaches a method for generating holographic images (see Abstract) comprising: projecting (e.g., project, para. [0053]) a diffraction pattern image (e.g., diffracted image of frame (A), FIG. 19, para. [0070]) displayed at a spatial light modulator (SLM) (i.e., to effectuate controlled spatial displacements (shifts) over the sensor cells (pixels), para. [0050]) using diffraction pattern data and at multiple focal lengths (i.e., the squared error determined form the image data of (d) and (e) as |d-e|.sup.2 is illustrated in (g), see para. [0014], FIG. 6 … focal length(s) at different height positions [0114]); generating model images (LR and HR images of frames (B) and (C), FIG. 19, para. [0070], also see para. [0058], where area-based methods (of LR images) require some error metric (e.g., phase error) of the multiple focal lengths comprising using the diffraction pattern data (i.e., the obtained results display no significant differences for the HR image when compared to the former LR set, and a result of square-error-based SNR estimation and due to the nature of the registration method additional holographic fringes designed for holographic fringe propagation may be introduced (non-ideal model image), see para. [0071]); and generating a phase error map (i.e., obtained HR hologram is then decoded by a phase-retrieval method (phase error map) into an HR shadow image of the specimen at different height positions (e.g., different focal length), see para. [0034], also see para. [0017] and [0109]) comprising applying a gradient descent-type of operation (e.g., Optimization is performed with a gradient descent method, see para. [0067], FIGS. 5A-5C) that considers both a version of the model images (e.g., frames (B) and (C), FIG. 19 and HR-LR images of 5A and 5B) and a version of captured images (e.g., frame (A), FIG. 19 and optimized hologram of 5C) capturing the projection of the diffraction pattern image to the multiple focal lengths (i.e., FIGS. 5A, 5B, and 5C provide illustrations of implementation of algorithms for image registration (FIGS. 5A, 5B) and optimization (FIG. 5C), respectively. Image registration is performed by tracking the corresponding key-points between a candidate and a reference image as shown in FIGS. 5A and 5B, see para. [0013], and area-based methods require some error metric (phase error map) to measure the quality of matching of individual images with one another, para. [0058] also see para. [0067] to [0069], [0071], [0073] - [0074] and [0116]).
Sobieranski teaches all limitations except for explicit teaching of projecting a diffraction pattern image displayed at a spatial light modulator (SLM) using diffraction pattern data, and generating a phase error map.
However, in a related field of endeavor Goulanian teaches performing a large amount of intermediate computations for previously obtaining diffraction pattern data at each of small areas on a recording surface with respect to every selected object point when calculating an intensity distribution of diffraction light in true Computer Generated Holography (see para. [0094]) … a large amount of time for computing and processing 2-D images and time for updating screens, LCLVs, SLMs, displays or other means for projecting or displaying these images (see para. [0097]) … In the interference computation type Computer Generated Holography, where phase information relating to an entire object image is recorded in the interference fringe form, phase errors can be minimized to lead to an enhancement of image quality (para. [0070]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Sobieranski such that calculating an intensity distribution of diffraction light and processing 2-D images and updating screens, LCLVs, SLMs for projecting or displaying these images, and phase information relating to an entire object image is recorded in the interference fringe form as taught by Goulanian such that phase errors can be minimized to lead to an enhancement of image, namely, for improving conditions of the observation and perception of a 3-D optical image (an holographic image) to be produced and obtaining a high degree of image resolution or its higher quality as a whole, or for transmitting (communicating) selected data to remote users for such purposes.
Regarding claim 3, Sobieranski fails to teach the model images are non-ideal model images that include a phase error (i.e., and a result of square-error-based SNR estimation and due to the nature of the registration method additional holographic fringes designed for holographic fringe propagation may be introduced (non-ideal model image), see para. [0071]).
Sobieranski teaches all limitations except for explicit teaching of the model images are non-ideal model images.
However, in a related field of endeavor Goulanian teaches in the interference computation type Computer Generated Holography, where phase information relating to an entire object image is recorded in the interference fringe form, phase errors can be minimized to lead to an enhancement of image quality (para. [0070]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Sobieranski such that phase information relating to an entire object image is recorded in the interference fringe form, phase errors can be minimized as taught by Goulanian to lead to an enhancement of image quality.
Regarding claim 4, Sobieranski according to claim 1 further teaches generating the model images comprises inputting a phase profile of the diffraction pattern data into an optical field propagation model (i.e., angular diffraction calculation, or Fresnel convolution method (e.g., an optical field propagation model), for example, may be used to convert the HR holographic image of block (d) to a real, imaginary, phase and amplitude signals, para. [0048], as shown in FIG. 2 as block (e)).
Regarding claim 5, Sobieranski according to claims 1 and 4 further teaches generating the model images comprises inputting the diffraction pattern data into an optical field propagation model that is based on a convolutional Fresnel diffraction algorithm using complex numbers (see para. [0048], and presenting complex holographic patterns (e.g., complex numbers) as a function of time, para. [0109]).
Regarding claim 6, Sobieranski according to claim 1 further teaches generating the model images comprises operating a propagation model to generate model sensor optical fields that are each a model of an image captured at the sensor array (individual sensor cells 310, FIG. 3A, para. [0011]) is able to capture frames in monochromatic mode and that is convertible to the model images (i.e., numerical diffraction calculations used in implementation can be performed, where a technique for modeling the propagation of a wave-field by expanding a complex wave-filed into a summation of infinite number of plane waves, para. [0112], used CMOS imaging sensor 150 is able to capture frames in monochromatic mode, para. [0096] and [0097] …angular diffraction calculation, or Fresnel convolution method, for example, may be used to convert the HR holographic image of block (d) to a real, imaginary, phase and amplitude signals, para. [0048]).
Regarding claim 7, Sobieranski according to claim 6 further teaches converting the model sensor optical field into an intensity image to form the model image (i.e., angular diffraction calculation, or Fresnel convolution method, for example, may be used to convert the HR holographic image of block (d) to a real, imaginary, phase and amplitude signals, para. [0048] … during the sequential acquisition of LR images of the object, advantage is taken of flickering of intensity of the beam of light L1 over time and filtering the multiple LR image (e.g., model image) data sets to reduce noise and increase spatial resolution, para. [0073]).
Regarding claim 8, Sobieranski according to claim 1 further teaches the gradient descent-type of operation considers both pixel values of the model images and pixel values of the captured images, and generates a phase error map as an output at individual iterations of the gradient descent (i.e., Optimization is performed with a gradient descent method, where a search-space of variable decimation (admissible translation in x and y positions, rotation) is populated by a Continuous Ant Colony Optimization (ACO) approach, designed to find the global optimum solution after a number of iterations is completed, para. [0067]).
Regarding claim 9, Sobieranski according to claim 1 further teaches pre-processing the captured images before performing gradient descent with the captured images comprising performing vibration compensation and denoising (i.e., procedure of hybrid registration of individual LR-images to a reference frame, an HR hologram shadow image is formed, in which low- and high-frequency waves are recovered (FIG. 2, block (d)). The hybrid registration approach is configured to produce the effect of a noise-filter (for denoising): and noise and other undesirable artifacts (such as flickering/vibration) are suppressed over time and space, para. [0047]).
Regarding claim 10, Sobieranski teaches system for generating holographic images (see Abstract) comprising: memory to store (non-transitory storage medium) holographic image data and phase errors (see para. [0038] and [0058], FIGS. 1 and 6); and processor circuitry (Electronic circuitry and data-acquisition platform 160, FIG. 1) coupled to the memory (see para. [0038]) and forming at least one processor to operate (see para. [0115]) by: receiving holographic captured images of multiple different focal lengths extending from a spatial light modulator (SLM) (i.e., to effectuate controlled spatial displacements (shifts) over the sensor cells (pixels), para. [0050]) displaying diffraction pattern images (e.g., diffracted image of frame (A), FIG. 19, para. [0070]) projected to be captured in the captured images (i.e., the squared error determined form the image data of (d) and (e) as |d-e|.sup.2 is illustrated in (g), see para. [0014], FIG. 6 … focal length(s) at different height positions [0114]); generating pre-processed holographic image data of the captured images comprising: applying vibration compensation to the captured images (i.e., a high-resolution (HR) image or hologram is then obtained by resolving such displacements based on feature registration and sub-pixel optimization. LR images are first spatially aligned and registered on the same planar domain, followed by optimization of sub-pixel information based on fast-convergence approach used to find the global optimum solution. The obtained HR hologram is then decoded by a phase-retrieval method into an HR shadow image of the specimen at different height positions. A set of empirical results evidenced that the proposed methodology allows to obtain, staring with captured shadow images via a lensless platform, para. [0034]), and denoising the image data comprising subtracting a noise estimation from image data of the captured images (i.e., procedure of hybrid registration of individual LR-images to a reference frame, an HR hologram shadow image is formed, in which low- and high-frequency waves are recovered (FIG. 2, block (d)). The hybrid registration approach is configured to produce the effect of a noise-filter (for denoising): and noises are suppressed over time and space, para. [0047]); and generating a phase error map (i.e., obtained HR hologram is then decoded by a phase-retrieval method (phase error map) into an HR shadow image of the specimen at different height positions (e.g., different focal length), see para. [0034], also see para. [0017] and [0109]) comprising considering both the pre-processed holographic image data of the captured images (FIGS. 5A and 5B) and model images (FIG. 5C) generated by using data used to form the diffraction pattern images (i.e., FIGS. 5A, 5B, and 5C provide illustrations of implementation of algorithms for image registration (FIGS. 5A, 5B) and optimization (FIG. 5C), respectively. Image registration is performed by tracking the corresponding key-points between a candidate and a reference image as shown in FIGS. 5A and 5B, see para. [0013], and area-based methods require some error metric (phase error map) to measure the quality of matching of individual images with one another, para. [0058] also see para. [0067] to [0069], [0071], [0073] - [0074] and [0116]).
Sobieranski teaches all limitations except for explicit teaching of projecting a diffraction pattern image displayed at a spatial light modulator (SLM) using diffraction pattern data, and generating a phase error map.
However, in a related field of endeavor Goulanian teaches performing a large amount (number) of intermediate computations for previously obtaining diffraction pattern data at each of small areas on a recording surface with respect to every selected object point when calculating an intensity distribution of diffraction light in true Computer-Generated Holography (see para. [0094]) … a large amount of time for computing and processing 2-D images and time for updating screens, LCLVs, SLMs, displays or other means for projecting or displaying these images (see para. [0097]) … In the interference computation type Computer Generated Holography, where phase information relating to an entire object image is recorded in the interference fringe form, phase errors can be minimized to lead to an enhancement of image quality (para. [0070]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Sobieranski such that calculating an intensity distribution of diffraction light and processing 2-D images and updating screens, LCLVs, SLMs for projecting or displaying these images, and phase information relating to an entire object image is recorded in the interference fringe form as taught by Goulanian such that phase errors can be minimized to lead to an enhancement of image, namely, for improving conditions of the observation and perception of a 3-D optical image (an holographic image) to be produced and obtaining a high degree of image resolution or its higher quality as a whole, or for transmitting (communicating) selected data to remote users for such purposes.
Regarding claim 14, Sobieranski according to claim 10 further teaches the denoising comprises averaging the image data of multiple captured images (LR-HR-images) at the same focal length (para. [0114]) to form an average captured image to be used to generate the phase error map (i.e., averaging light intensities into such new LR image, para. [0050], FIG. 3A … (i.e., obtained HR hologram is then decoded by a phase-retrieval method (phase error map), para. [0034]).
Regarding claim 15, Sobieranski according to claim 14 further teaches the denoising comprises subtracting an average noise estimate from image data of pixel locations on the average captured image (i.e., noise suppression estimation and sharpness level of LR and HR images based on Laplace-operator(s), para. [0072]).
Regarding claim 16, Sobieranski according to claim 10 further teaches generating the phase error map comprises using a gradient descent-type of operation (e.g., Optimization is performed with a gradient descent method, see para. [0067], FIGS. 5A, 5B, and 5C) considering both the pre-processed holographic image data of the captured images and the model images (i.e., FIGS. 5A, 5B, and 5C provide illustrations of implementation of algorithms for image registration (FIGS. 5A, 5B) and optimization (FIG. 5C), respectively. Image registration is performed by tracking the corresponding key-points between a candidate and a reference image as shown in FIGS. 5A and 5B, see para. [0013], also see para. [0067] to [0069], [0071], [0073] - [0074] and [0116]).
Regarding claim 18, Sobieranski teaches at least one non-transitory machine-readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to operate (see para. [0115]) by: projecting a diffraction pattern image displayed at a spatial light modulator (SLM) (i.e., to effectuate controlled spatial displacements (shifts) over the sensor cells (pixels), para. [0050]) using diffraction pattern data and at multiple focal lengths; generating model images of the multiple focal lengths comprising using the diffraction pattern data (i.e., the squared error determined form the image data of (d) and (e) as |d-e|.sup.2 is illustrated in (g), see para. [0014], FIG. 6 … focal length(s) at different height positions [0114]); generating holographic captured images (see Abstract) comprising capturing the projection of the diffraction pattern image to the multiple focal lengths (e.g., diffracted image of frame (A), FIG. 19, para. [0070], also see para. [0053]); and generating a phase error map comprising applying a gradient descent-type of operation ((i.e., obtained HR hologram is then decoded by a phase-retrieval method (phase error map) into an HR shadow image of the specimen at different height positions (e.g., different focal length), see para. [0034], and Optimization is performed with a gradient descent method, see para. [0067], FIGS. 5A-5C) that considers both a version of the model images and a version of the captured images (i.e., FIGS. 5A, 5B, and 5C provide illustrations of implementation of algorithms for image registration (FIGS. 5A, 5B) and optimization (FIG. 5C), respectively. Image registration is performed by tracking the corresponding key-points between a candidate and a reference image as shown in FIGS. 5A and 5B, see para. [0013], and area-based methods require some error metric (phase error map) to measure the quality of matching of individual images with one another, para. [0058], also see para. [0067] to [0069], [0071], [0073] - [0074] and [0116]).
Sobieranski teaches all limitations except for explicit teaching of projecting a diffraction pattern image displayed at a spatial light modulator (SLM) using diffraction pattern data, and generating a phase error map.
However, in a related field of endeavor Goulanian teaches performing a large amount (number) of intermediate computations for previously obtaining diffraction pattern data at each of small areas on a recording surface with respect to every selected object point when calculating an intensity distribution of diffraction light in true Computer-Generated Holography (see para. [0094]) … a large amount of time for computing and processing 2-D images and time for updating screens, LCLVs, SLMs, displays or other means for projecting or displaying these images (see para. [0097]) … In the interference computation type Computer Generated Holography, where phase information relating to an entire object image is recorded in the interference fringe form, phase errors can be minimized to lead to an enhancement of image quality (para. [0070]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Sobieranski such that calculating an intensity distribution of diffraction light and processing 2-D images and updating screens, LCLVs, SLMs for projecting or displaying these images, and phase information relating to an entire object image is recorded in the interference fringe form as taught by Goulanian such that phase errors can be minimized to lead to an enhancement of image, namely, for improving conditions of the observation and perception of a 3-D optical image (an holographic image) to be produced and obtaining a high degree of image resolution or its higher quality as a whole, or for transmitting (communicating) selected data to remote users for such purposes.
21. The medium of claim 18 wherein the gradient descent-type of operation determines phase errors for the phase error map as the phase errors resulting in the average total minimum mean square error (MSE) (e.g., the squared error determined form the image data of (d) and (e) as |d-e|.sup.2 is illustrated in (g), para. [0014], FIG. 6) between the captured images and the model images at the multiple focal lengths (i.e., For the three sharpness estimators, the HR image has a better level of focus degree, involving noise suppression and sharpness level, when compared to the LR set, para. [0072], also see para. [0067]).
22. The medium of claim 18, wherein the instructions cause the computing device to operate by performing pre-processing on the captured images comprising vibration compensation and denoising before using the captured images for gradient descent (i.e., procedure of hybrid registration of individual LR-images to a reference frame, an HR hologram shadow image is formed, in which low- and high-frequency waves are recovered (FIG. 2, block (d)). The hybrid registration approach is configured to produce the effect of a noise-filter (for denoising): and noise and other undesirable artifacts (such as flickering/vibration) are suppressed over time and space, para. [0047], also see para. [0067]).
23. The medium of claim 18, wherein the gradient descent-type of operation is performed by starting with an initial phase error guess that is based on a phase profile of an image with a focal length longer than all of the multiple focal lengths (i.e., the holography matrix is used as an initial guess solution for the second step, where a fine adjustment in a sub-pixel level is performed, para. [0067]).
24. The medium of claim 18 wherein the gradient descent-type of operation modifies either the captured images or the model images of each focal length and contributing to the same phase error minimum value and modified with at least one scaling energy factor (i.e., the holography matrix is used as an initial guess solution for the second step, where a fine adjustment (modify) in a sub-pixel level is performed, para. [0067], and Area matching approaches seek to minimize energy E (cost function) (e.g., scaling energy factor) that represents the estimation of a registration error (phase error), para. [0065]).
25. The medium of claim 18 wherein the model images are generated by using SLM parameters to generate a phase profile considered to be ideal with no phase error, rather than using an input target image (i.e., optimization procedure based on area-matching approaches (minimization error) (e.g., no phase error), para. [0056]).
Allowable Subject Matter
Claims 2, 11-13, 17 and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the prior art does not teach, or renders obvious, regarding the generating of the phase error map occurs during a calibration stage, and the method comprising using the phase error map during a run-time to adjust phase values to be used to generate diffraction pattern image data of one or more diffraction pattern images to be displayed at the SLM.
Regarding claim 11, the prior art does not teach, or renders obvious, regarding the vibration compensation comprises aligning image content of multiple images of the same focal length to image content of a single anchor image.
Claims 12 and 13 depend upon allowable claim 11.
Regarding claim 17, the prior art does not teach, or renders obvious, regarding the at least one processor is arranged to operate by using the phase error map to adjust phase values to be used to generate diffraction pattern image data of one or more diffraction pattern images to be displayed at the SLM.
Regarding claim 19, the prior art does not teach, or renders obvious, regarding generating the model image comprises inputting at least one latest phase error guess into a propagation model that generates a model sensor optical field convertible into the model image, wherein the phase error guess is obtained at iterations from the gradient descent-type operation.
Claim 20 depend upon allowable claim 19.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Saito et al. (US 5668648) teaches “the interference computation types computer-generated holography, unlike the aforementioned diffraction computation type (i.e., Lohmann type) computer-generated holography, the phase information of a holographic image is recorded in the interference fringe form. The phase error can thus be minimized, which leads to enhancement of the image quality,” column 2, lines 44-50.
Supikov et al. (US PUB 20200117139) teaches “System 100 may also include a spatial light modulator 104 (SLM) such as a phase only SLM 104. In other embodiments, SLM 104 may be implemented separately from system 100 and resultant final phase only diffraction pattern 114 may be stored and/or transmitted to a remote SLM 104 for display,” paragraph 0030, and as shown in Figure 1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MUSTAK CHOUDHURY whose telephone number is (571)272-5247. The examiner can normally be reached on M-F 8AM-5PM EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ricky Mack can be reached on (571)272-2333. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MUSTAK CHOUDHURY/Primary Examiner, Art Unit 2872
December 3, 2025