DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-4, 6-17 and 19-26 are pending.
Claims 5 and 18 are canceled.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1, 3-4, 6-12, 14, 16-17, 19-24 and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lam et al (Ensemble CNN for Classifying Holograms, 2019) in view of Rivenson et al (Phase recovery, 2018) and further in view of He et al (Automated Fourier space, 2016).
Regarding claims 1, 14 and 26, Lam teaches a computer-implemented method for object identification and/or classification, comprising:
receiving digital hologram data of a digital hologram of an object, the digital hologram data comprising phase information and magnitude information; and
processing the digital hologram data based on a neural-network-based ensemble model to identify and/or classify the object,
(Lam, Fig. 2, digital histogram generation followed by magnitude CNN and phase CNN; Fig. 4, a CNN based ensemble model for hologram object classification; inputs of the CNN ensemble model are magnitude component and phase component of the hologram; “Through an ensemble decision maker, the CNN that outputs a class identity with a higher matching score will be selected as the identity of the input hologram”, p5; “obtain the identity of an object in a hologram”, p2)
wherein the digital hologram data is associated with wavefront from the object, the wavefront being complex-valued,
(Lam, Fig. 2, “digital hologram” data is represented by magnitude and phase, meaning that the digital hologram data is complex data; the complex hologram data is the wavefront data as compared with the conventional light signals based only on intensity)
wherein the neural-network-based ensemble model comprises a first neural network arranged to process the magnitude information, and a second neural network arranged to process the phase information,
(Lam, Figs. 2 and 4, magnitude CNN and phase CNN)
wherein the computer-implemented method further comprises training, testing and/or validating the neural-network-based ensemble model based on data for training, testing and/or validating, the data for training, testing and/or validating comprising digital wavefront data of multiple digital wavefronts of each object, and
(Lam, Fig. 2, “a large set of augmented holograms is generated, and applied to train a deep-learning network that is implemented with a pair of CNNs. One of the CNNs receive the magnitude component of the holograms as the input data, while the other accepts the phase component”, “holograms of handwritten characters are employed to train, and to test the CNN”, p3)
Lam does not expressly disclose but Rivenson teaches:
wherein the digital wavefront data includes defective, flawed or incomplete data, and
(Rivenson, “These results highlight that challenging problems in imaging science can be overcome through machine learning, providing new avenues to design powerful computational imaging systems”, [abstract]; “the twin-image artifact of the in-line holography, which is a result of the lost phase information, is strong and severely obstructs the spatial features of the sample in both the amplitude and phase channels, as illustrated in Figures 1 and 2”, p2:c2; Fig. 2, “These images are contaminated with twin-image and self-interference-related spatial artifacts due to the missing phase information in the hologram detection process”, “The yellow arrows point to artifacts in f, g, n, o (due to out-of-focus dust particles or other unwanted objects) ...”; using digital hologram data that is "contaminated" with artifacts and "dust particles" (defective/flawed) for training a neural network. It would be obvious to train the classifier of Lam on the "flawed" biological data of Rivenson to achieve robust classification of real-world samples)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Rivenson into the system or method of Lam in order to train the classifier of Lam on the "flawed" biological data of Rivenson to achieve robust classification of real-world samples. The combination of Lam and Rivenson also teaches other enhanced capabilities.
The combination of Lam and Rivenson further teaches:
wherein the data for training. testing and/or validating is obtained by capturing from biological samples using a laser-based optical system,
(Rivenson, “We validated this method by reconstructing the phase and amplitude images of various samples, including blood and Pap smears and tissue sections”, [abstract]; “In this work, we chose to demonstrate the proposed framework using lens-free digital in-line holography of transmissive samples, including human tissue sections and blood and Pap smears”, p2:c2; “Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography”, [abstract]; “Under plane wave illumination, we can assume that A has zero phase at the detection plane, without loss of generality, that is, A= |A|”, p5:c2; capturing data from biological samples (blood, Pap smears, tissue) using a coherent optical system (plane wave illumination/holography), which implies a laser or similar coherent source; practically all conventional holography requires lasers as light sources to ensure the light is coherent, meaning the waves are in phase and have a consistent frequency to produce clear interference patterns)
wherein the computer-implemented method further comprises obtaining the digital hologram data of the digital hologram of the object, the obtaining comprises:
receiving the digital hologram of the object obtained using the laser-based optical system: and
(Lam, " Fig. 4, preprocessing stage; “A hologram acquisition system, such as one based on optical scanning holography [2] or phase shifting holography [3], is used to capture digital holograms of physical objects”, p2", p2; Rivenson, "lens-free digital in-line holography", p2:c2; Lam and Rivenson teach receiving hologams obtained from an optical holography system)
processing the digital hologram by performing a digital signal processing operation to obtain the digital hologram data,
(Lam, Sec 2.1; "After a hologram Hp;q(m, n)is generated, it is decomposed into the magnitude component Mp;q(m, n)= |Hp;q(m, n)|, and the phase component Pp;q(m, n)= arg[Hp;q(m, n)]", p4; mathematically processing the digital hologram to obtain magnitude and phase data)
The combination of Lam and Rivenson does not expressly disclose but He teaches:
wherein the digital signal processing operation comprises:
performing a Fourier transform operation on the digital hologram to transform the digital hologram from a spatial domain to a frequency domain in a discrete manner: and
(He, "The numerical steps used in off-axis DHM technique include: discrete 2D Fourier transform, spatial frequency filtering", p3113; the digital signal processing operations is required for spatial filtering; "After fast Fourier transform (FFT) of the recorded hologram [Fig. 2(a)], the phase shift is recorded in the spatial frequency on two symmetrical areas", p3115; performing a discrete Fourier transform to transform the digital hologram into the spatial frequency domain)
after the Fourier transform operation, extracting hologram data associated with the object in the frequency domain.
(He, " Use ... the binary image form first step to get the right frequency component boundary and use it as a filtering window ", "Once the appropriate spatial frequency is selected and centered [Fig. 2(g)], an inverse FFT (iFFT) is performed to retrieve the complex amplitude of the sample", p3116; extracting the object's appropriate spatial frequency component data in the frequency domain)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of He into the modified system or method of Lam and Rivenson in order to accurately and automatically extract the object's spatial frequency data from scattering biological backgrounds, thereby providing high-quality complex magnitude and phase inputs to the low-complexity ensemble CNN for robust identification. The combination of Lam, Rivenson and He also teaches other enhanced capabilities.
Regarding claims 3 and 16, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein the neural-network-based ensemble model comprises a convolutional-neural-network-based ensemble model, model, and the first neural network and the second neural network are a first convolutional neural network and a second convolutional neural network respectively.
(Lam, Fig. 4; "One of the CNNs receive the magnitude component of the holograms as the input data, while the other accepts the phase component."; p3; "The structure of the CNN, which is identical for learning the magnitude and the phase components", p4; an ensemble model comprises a first CNN for magnitude and a second CNN for phase)
Regarding claims 4 and 17, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein the neural-network-based ensemble model further comprises a concatenate unit arranged to combine magnitude features extracted by the first neural network and phase features extracted by the second neural network for identification and/or classification of the object.
(Lam, Fig. 4; “Through an ensemble decision maker, the CNN that outputs a class identity with a higher matching score will be selected as the identity of the input hologram”, p5; obviously, it can be considered that the ensemble decision block acts like a concatenated unit in a way that it combines/groups the outputs of the magnitude CNN and the phase CNN together, and selects one of these two outputs as the ensemble classification output based on which output has the higher matching score)
Regarding claims 6 and 19, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein the digital signal processing operation further comprises:
after the extraction, performing an inverse Fourier transform operation on the extracted hologram data to obtain the digital hologram data.
(He, "Once the appropriate spatial frequency is selected and centered [Fig. 2(g)], an inverse FFT (iFFT) is performed to retrieve the complex amplitude of the sample", p3116; performing an inverse Fourier transform after frequency extraction to obtain the complex amplitude data)
Regarding claims 7 and 20, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, the laser-based optical system comprises a camera associated with an interferometer.
(He, Fig. 1; “Spectra Physics Model 127-3502 Stabilite Polarized Helium-Neon Laser”, p3114; "Interferometry measures the phase", p3113; " recombined with the sample beam by a non-polarizing beam splitter (BS, THORLABS CM-BS013) onto a charge coupled device (CCD) camera", p3115; an optical system utilizing interferometry, laser and a CCD camera)
Regarding claims 8 and 21, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 7, wherein the interferometer is an off-axis interferometer and the hologram is an off-axis hologram.
(He, “Digital holographic microscopy (DHM) ... off-axis DHM", [abstract]; "In off-axis system, the sample and reference beams interfere at a slightly different angle", p3114; an off-axis interferometer produces off-axis holograms)
Regarding claims 9 and 22, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, further comprising outputting or displaying the identification and/or classification result.
(Lam, Fig. 4, obviously, the output of class identity of the hologram object may be displayed in a screen for human monitoring; "The output of each CNN is a class identity", p4; outputting the determined classification identity result)
Regarding claims 10 and 23, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein the object comprises a biological tissue sample.
(Rivenson, "we used three different types of samples: blood smears, Pap smears and breast tissue sections", p3; imaging biological tissue samples)
Regarding claim 11, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 10,
wherein the biological tissue sample is sized for microscopy; and/or
wherein the biological tissue sample is incomplete, damaged, or flawed.
(Rivenson, Materials and Methods; " Formalin-fixed paraffin-embedded (FFPE) breast tissue is sectioned into 2 μm slices", p8; biological tissue is sectioned into thin slices, explicitly sizing it for microscopy)
Regarding claims 12 and 24, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein the digital hologram data is associated with an electromagnetic wavefront from the object.
(He, Sec 2; "continuous wave visible laser λ = 632.8nm", p3114; using a visible laser, which inherently produces an electromagnetic wavefront)
Claim(s) 2 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lam et al (Ensemble CNN for Classifying Holograms, 2019) in view of Rivenson et al (Phase recovery, 2018) and further in view of He et al (Automated Fourier space, 2016) and Cella et al (US2022/0187847).
Regarding claims 2 and 15, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination does not expressly disclose but Cella teaches the computer-implemented method of claim 1, wherein the neural-network-based ensemble model comprises an attention based transformer model.
(Cella, “transformer-based, encoder-decoder architectures using attention mechanisms may be used in conjunction with or in place of convolutional neural networks”, [1801])
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Cella into the modified system or method of Lam, Rivenson and He in order to use an attention based transformer model in place of a CNN for utilizing self-attention mechanisms to capture global dependencies and relations between image patches directly. The combination of Lam, Rivenson, He and Cella also teaches other enhanced capabilities.
Claim(s) 13 and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lam et al (Ensemble CNN for Classifying Holograms, 2019) in view of Rivenson et al (Phase recovery, 2018) and further in view of He et al (Automated Fourier space, 2016) and Cuche et al (US6262818).
Regarding claims 13 and 25, the combination of Lam, Rivenson and He teaches its/their respective base claim(s).
The combination does not expressly disclose but Cuche teaches the computer-implemented method of claim 1, wherein the digital hologram data is associated with an acoustic wavefront from the object.
(Cuche, “The present invention is not restricted to the optical domain and can be applied for the numerical reconstruction of holograms recorded with any kind of electromagnetic (e.g. X-ray) or non-electromagnetic (e.g. acoustics or heats) waves”; c6:45-55)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Cuche into the modified system or method of Lam, Rivenson and He in order to expand the applicability of the automated holographic reconstruction and classification methods beyond the optical domain, thereby enabling the system to process and classify objects using non-electromagnetic waves, such as acoustic wavefronts, as explicitly suggested by Cuche to be a compatible alternative for numerical hologram reconstruction. It allows for retrieving both amplitude and phase information of an object wave. The combination of Lam, Rivenson, He and Cuche also teaches other enhanced capabilities.
Response to Arguments
Applicant's arguments filed on 5/5/2026 with respect to one or more of the pending claims have been fully considered but are moot in view of the new ground(s) of rejection.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center. for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 6/27/2026
Prosecution Insights