DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-17 are pending.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 16 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim(s) 16 is/are directed to "A computer-readable recording medium". Applicant, in the specification, does not explicitly or deliberately define the recited “computer-readable recording medium” to include only a non-transitory medium. Moreover, the term "A computer-readable recording medium" can encompass carrier waves. Thus, claim(s) 16 is/are rejected under 35 U.S.C. 101 because, giving the claim(s) their broadest reasonable interpretation, the claimed "A computer-readable recording medium" encompasses non-statutory subject matter. It is suggested to amend the recited “A computer-readable recording medium” to be "A non-transitory computer-readable recording medium" for overcoming the rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-5, 8 and 13-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ghosh et al (3D block-based restoration, 2016) in view of Li-Ling et al (US20230069794A1).
Regarding claims 1, 15 and 16, Ghosh teaches a sample image generation device comprising a memory and a processor, wherein
a first image is an image obtained by capturing an image of a sample,
(Ghosh, "Experimental images of 6 μm beads were acquired from a sample", [Sec. 4.1], p7:c2; obtaining an initial image captured directly from a physical sample)
a predetermined direction in the first image is a direction in which a virtual observation optical system is present among optical axis directions of the virtual observation optical system,
(Ghosh, "K sections along Z", [Sec. 2.1], p3:c1; "the optical axis z", Fig. 3 caption; a predetermined Z-axis direction aligns with the optical axis of the observation system depth)
the processor
performs a first acquisition process of acquiring the first image from the memory,
(Ghosh, "3-D data were acquired from this region to evaluate the BB forward model", [Sec. 4.1], p8:c1; Li-Ling, Description [0035]; "receiving an acquired image", [0035]; Ghosh teaches acquiring the previously captured 3-D image data into the processing pipeline, while Li-Ling teaches explicitly receiving an acquired image at an input layer).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Li-Ling into the system or method of Ghosh in order to feed the acquired sensor data from memory into a processing layer for aberration evaluation. The combination of Ghosh and Li-Ling also teaches other enhanced capabilities.
The combination of Ghosh and Li-Ling further teaches:
performs a division process of dividing the acquired first image into a plurality of areas,
(Ghosh, "the object space is conceptually sectioned into M sections along X, N sections along Y, and K sections along Z based on the object’s RI distribution map, which results in the formation of blocks", [Sec. 2.1], p3:c1; in the division process the spatial image data is sectioned into a plurality of distinct block areas)
performs a second acquisition process of acquiring a refractive index distribution of the sample from the memory,
(Ghosh, "information about the RI of the specimen were used to infer an approximate 3-D RI map", [Sec. 4.1], p8:c1-c2; acquiring and utilizing a refractive index (RI) distribution map of the sample)
performs a calculation process of calculating respective point spread functions for the divided areas, using the acquired refractive index distribution,
(Ghosh, "Eight SV-PSFs associated with each block (one for each block vertex location) are computed using the N-interface PSF model", [Sec. 2.1], p3:c2-p4:c1; "using imaging conditions including thickness and RI of the sample at these unique locations.", [Sec. 2.1], p4:c1; calculating block-specific point spread functions (PSFs) by explicitly using the acquired refractive index parameters of those distinct areas)
performs a first generation process of generating respective second images corresponding to the areas, using the respective point spread functions calculated for the areas, and
(Ghosh, "The SV image, g(x̲i), of an object,f(x_o), can be written using the PCA-represented SV-PSFs, by the kernel KSV-PCA", eq. (5), [Sec. 2.1], p4:c1, the process of generating images utilizes the respective calculated point spread functions corresponding to the local areas; Li-Ling, "to generate and to output a restored image (401) from the image (400) acquired in the second imaging mode ", [claim 1]; Incorporating Li-Ling into Ghosh would lead to generating restored output sub-images that comprise fewer optical aberrations stemming from refractive mismatches)
combines the respective second images corresponding to the areas and generates a third image corresponding to the first image,
(Ghosh, "The intensity in the sample can be expressed by the sum of all the nonoverlapping blocks as ... eq. (3)", [Sec. 2.1], p3:c2; Li-Ling, "fusing the acquired images, wherein the multi-view fusion image has a higher spatial resolution", [0079]; Ghosh teaches combining the individual block images by summing them to reconstruct the complete final image, while Li-Ling teaches generating a multi-view fusion image. Incorporating Li-Ling into Ghosh would lead to fusing processed regional images together to produce a final sample image with a higher overall spatial resolution)
in the calculation process, the point spread function of a first area is calculated using a refractive index distribution of each of areas included in an area group,
(Ghosh, "models light propagation through N stratified layers within a block. SV-PSFs are calculated at discrete locations... marking block vertices, using imaging conditions including thickness and RI of the sample at these unique locations.", [Sec. 2.1], p4:c1; calculating a spatially variant point spread function (SV-PSF) for a specific target location, i.e., a block vertex representing the "first area", by evaluating the optical path through N stratified layers, representing the "area group", utilizing the distinct refractive index (RI) distribution across each of those individual layers)
the first area is an area for which the point spread function is to be calculated, and
(Ghosh, "SV-PSFs are calculated at discrete locations, (xo,yo,zo)=(Xm,Yn,Zk), marking block vertices", [Sec. 2.1], p4:c1; defining the discrete location (first area) for which the PSF is actively being evaluated)
the area group is constituted of a plurality of areas inside a range in which light rays originating from the first area radiate in the predetermined direction, and includes an area outside a range defined by extending the first area in the predetermined direction.
(Ghosh, "changes in the spherical wave-front of the emitted light", [Sec. 1], p1:c2; "models light propagation through N stratified layers", [Sec. 2.1], p4:c1; modeling the point spread functions dynamically as light wavefronts spread spherically (radiating) across boundaries into adjacent surrounding stratified layers, reaching areas outside the direct 1D axial extension of the origin point)
Regarding claim 2, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 1, wherein in the calculation process, the processor sets a point light source in the first area and calculates a point spread function of the first area using a first wavefront from a wave source of which is the set point light source.
(Ghosh, "The image of every point in the specimen function is associated with a unique SV-PSF", [Sec. 2.1], p3:c1; "changes in the spherical wave-front of the emitted light", [Sec. 1], p1:c2; calculating the point spread function corresponding to individual origin points (point light sources) in the specimen using the spherical wavefront emitted from that source point)
Regarding claim 3, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 2, wherein in the calculation process, the processor
calculates a second wavefront, using the first wavefront and the refractive index distribution corresponding to each of the areas included in the area group, the second wavefront being a wavefront propagating through the sample in the predetermined direction,
(Ghosh, "models light propagation through N stratified layers within a block.", [Sec. 2.1], p4:c1; modeling the propagation of the light wavefront as it travels through the different sequential refractive index layers of the sample in the direction of the observation system)
calculates an intensity distribution corresponding to a third wavefront, using the calculated second wavefront, the third wavefront being a wavefront at a position of a focal plane of the virtual observation optical system, and
calculates a point spread function of the first area, using the calculated intensity distribution.
(Ghosh, "SA introduces asymmetry, as well as attenuation and spreading of intensities in the point spread function (PSF)", [Sec. 1], p1:c2; tracking the wavefront's propagation dynamically through the sample directly determines the altered intensity distribution, which dictates the calculation of the final point spread function (PSF) at the optical focal plane)
Regarding claim 4, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 3, wherein in the calculation process,
the processor determines whether a wavefront propagating through the sample has reached an outer edge of the sample, in the predetermined direction, and
the second wavefront is a wavefront at a position where the wavefront is determined to have reached the outer edge.
(Ghosh, "RI mismatch between the different layers that make up the imaging system, namely the immersion medium of the lens, coverslip, and specimen.", [Sec. 1], p1:c2; tracking the light propagation out of the specimen layers up to the outer boundary/edge where the sample interfaces with the coverslip)
Regarding claim 5, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 3, wherein the second wavefront is a wavefront after passing through the sample and before reaching the virtual observation optical system.
(Ghosh, "immersion medium of the lens, coverslip, and specimen.", [Sec. 1], p1:c2; tracking the wavefront state as it exits the specimen boundary and passes into the coverslip/immersion medium, physically located prior to reaching the objective lens)
Regarding claim 8, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 1, wherein
the processor performs an estimation process of estimating the sample, and
(Ghosh, "A block is not necessarily cubic, and the number of blocks needed is adjusted based on the variability of the specimen’s RI; that is, a sample with a rapidly varying RI is approximated by more blocks than a sample with a slowly varying RI.", [Sec. 2.1], p3:c1-c2; estimating the physical extent and internal variance of the sample via a 3-D RI map to dictate the block division layout)
in the division process, in a direction orthogonal to the predetermined direction, a size of an area that is an area of the estimated sample and to which an outer edge of the estimated sample does not belong is set to be smaller than a size of an area that is not an area of the estimated sample and to which the outer edge of the estimated sample does not belong.
(Ghosh, " a sample with a rapidly varying RI is approximated by more blocks than a sample with a slowly varying RI ... The minimum size of a block is defined by the smallest volume of uniform RI distribution determined based on the 3-D RI map of the sample.", [Sec. 2.1], p3:c1-c2; adaptively dividing the image space such that the interior bounds of the estimated sample (which exhibits rapidly varying RI) are divided into more blocks and assigned a smaller area size, whereas regions outside the sample (uniform background medium with slowly varying/constant RI) are set to a larger block area size)
Regarding claim 13, the combination of Ghosh and Li-Ling teaches a sample image generation system comprising:
(Ghosh, Fig. 4(e), " A schematic of the imaging system", [Sec. 4.1], p8:c2; a sample imaging system.)
an observation optical system configured to form an optical image of a sample;
(Ghosh, "Zeiss AxioImager", "20×/0.5 NA air lens", [Sec. 4.1], p7:c2; Li-Ling, "sample imaging objective arranged to provide the light from the illumination module to a sample space of the microscope system and to collect light", [0013], an observation optical system utilizes objective lenses configured to form images; Incorporating Li-Ling into Ghosh would lead to providing a versatile sample imaging objective capable of highly aligned optical illumination and collection)
an imager configured to capture the optical image; and
(Ghosh, "AxioCam MRm camera", [Sec. 4.1], p7:c2, a camera imager; Li-Ling, "detection module comprises an array detector such as a camera for recording the light", [0032]; Incorporating Li-Ling into Ghosh would lead to utilizing an array detector for efficient and high-resolution optical image recording)
the sample image generation device according to claim 1.
(Ghosh, Li-Ling, see comments on claim 1
Regarding claim 14, the combination of Ghosh and Li-Ling teaches a sample image generation system comprising a memory and a processor, wherein
a first image is an image obtained by capturing an image of a sample,
a predetermined direction in the first image is a direction in which a virtual observation optical system is present among optical axis directions of the virtual observation optical system,
the processor
performs a first acquisition process of acquiring the first image from the memory,
performs a division process of dividing the acquired first image into a plurality of areas,
performs a second acquisition process of acquiring a refractive index distribution of the sample from the memory,
performs a calculation process of calculating respective point spread functions for the divided areas, using the acquired refractive index distribution,
performs a first generation process of generating respective second images corresponding to the areas, using the respective point spread functions calculated for the areas, and
combines the respective second images corresponding to the areas and generates a third image corresponding to the first image,
in the calculation process, the point spread function of a first area is calculated using a refractive index distribution of each of areas included in an area group,
the first area is an area for which the point spread function is to be calculated,
the area group is constituted of a plurality of areas inside a range in which light rays originating from the first area radiate in the predetermined direction, and includes an area outside a range defined by extending the first area in the predetermined direction,
(Ghosh, Li-Ling, see comments on claim 1)
the processor performs a machine learning process to train an AI model,
(Li-Ling, "An evaluation module, such as a computer, comprising a trained machine learning method", [0013]; an evaluation processor executes a process to train a machine learning method/AI model. Incorporating Li-Ling into Ghosh would lead to implementing a trained machine learning model to further automate aberration compensation, increasing computational speed, and improve the fidelity of resolving spatially variant aberrations across complex tissue samples)
in the machine learning process, the AI model is trained with a plurality of data sets,
(Li-Ling, "trained with a first set of images of a sample and a second set of images of the same sample", [0013]; training the machine learning model using pluralities of paired data sets)
the data sets include the first image and training data corresponding to the first image, and
(Li-Ling, "the first set of images acquired in the first imaging mode is a ground truth for the the machine learning method, particularly the deep learning network training and the second set of images acquired in the second imaging mode is a source for the the machine learning method", [0092]; the datasets include the initial/source images alongside the corresponding ground truth training images)
the training data corresponding to the first image is the second images corresponding to the first image.
(Li-Ling, "the first set of images acquired in the first imaging mode is a ground truth for the the machine learning method, particularly the deep learning network training and the second set of images acquired in the second imaging mode is a source for the the machine learning method", [0092]; the training data may directly correlate the secondary (ground truth/restored) images back to the primary source images)
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ghosh et al (3D block-based restoration, 2016) in view of Li-Ling et al (US20230069794A1) and further in view of Kam et al (WO2000033250A2).
Regarding claim 6, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination further teaches the sample image generation device according to claim 1, wherein
in the second acquisition process, the processor acquires the refractive index distribution for each of small areas obtained by further dividing the divided area, and
(Ghosh, "the object space is conceptually sectioned into M sections along X, N sections along Y, and K sections along Z based on the object’s RI distribution map, which results in the formation of blocks", [Sec. 2.1], p3:c1; "The minimum size of a block is defined by the smallest volume of uniform RI distribution determined based on the 3-D RI map of the sample.", [Sec. 2.1], p3:c2; "which models light propagation through N stratified layers within a block.", [Sec. 2.1], p4:c1; partitioning the target image space into blocks and further subdividing those blocks into stratified layers (small areas) for which the refractive index distribution is utilized to model light propagation)
in the calculation process, the processor calculates point spread functions of the small areas, using the refractive index distribution of the small areas, and calculates a point spread function of the area, using the point spread functions of the small areas.
(Ghosh, "Eight SV-PSFs associated with each block (one for each block vertex location) are computed using the N-interface PSF model, which models light propagation through N stratified layers within a block.", [Sec. 2.1], p3:c2-p4:c1; "These SV-PSFs can be represented using a few principal components (PCs), thereby reducing not only the memory required but also the number of convolutions in the forward SV imaging model.", [Sec. 2.1], p4:c1; Kam, "an isotropic fan of rays is drawn from every point in an imaged volume, e g the sample These rays are ray traced such that their paths and phases are modified in accordance with variations in the refractive index of the sample The result of the ray tracing is an aberrated wavefront for every point in the imaged volume", p14; "for every point in the vicinity of a focus RO, an interference integral is taken over the wavefront The integrations for all of the points in the vicinity of the focus RO define the three-dimensional point spread function", p14; Ghosh and Kam teach computing intermediate point spread functions/wavefronts at localized discrete points or layers (small areas) using the refractive index, and mathematically combining or integrating them to calculate the effective overall point spread function for the defined three-dimensional volume/area)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Kam’s ray tracing and wavefront integration into Ghosh’s block-based imaging model in order to accurately calculate and aggregate the local PSFs of Ghosh's sub-divided areas, predictably yielding a highly accurate overall PSF for improved 3D image restoration. The combination of Ghosh, Li-Ling and Kam also teaches other enhanced capabilities.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ghosh et al (3D block-based restoration, 2016) in view of Li-Ling et al (US20230069794A1) and further in view of Chen et al (enhancing the lateral resolution, 2017).
Regarding claim 17, the combination of Ghosh and Li-Ling teaches its/their respective base claim(s).
The combination does not expressly disclose but Chen teaches the sample image generation device according to claim 2, wherein in the calculation process, the processor
calculates an excitation light intensity at a position of the set point light source,
(Chen, "Classic CLSM works by illuminating a specimen with a single spot and detecting the resulting fluorescence.", p184:c1; an imaging model utilizes spot illumination, corresponding to calculating the excitation light intensity localized at the targeted point light source)
calculates a fluorescence intensity distribution, using the calculated intensity distribution and the calculated excitation light intensity, and
(Chen, "𝑃𝑆𝐹eff(𝑥,𝑦) is the effective PSF of CLSM, which is the product of the excitation and detection PSFs.", p187:c1; determining the resulting effective fluorescence intensity distribution mathematically as a product of the calculated excitation intensity profile (excitation PSF) and the detection distribution profile (detection PSF))
calculates a point spread function of the first area, using the calculated fluorescence intensity distribution.
(Chen, "𝑃𝑆𝐹eff(𝑥,𝑦) is the effective PSF of CLSM, which is the product of the excitation and detection PSFs.", p187:c1; defining the overall point spread function of the region using the aforementioned resulting combined fluorescence intensity distribution)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Chen’s effective point spread function (PSF) calculation for fluorescence microscopy into the image generation device of Ghosh and Li-Ling in order to accurately simulate fluorescence systems, predictably yielding a more precise effective PSF and improving overall image fidelity. The combination of Ghosh, Li-Ling and Chen also teaches other enhanced capabilities.
Allowable Subject Matter
Claim(s) 7 and 9-12 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening Claim(s).
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 7, 9 and 12 recite(s) limitation(s) related to adjusting area sizes by comparing a provisionally calculated point spread function's intensity peak to a reference; calculating the area's point spread function using a fluorescence intensity distribution derived from excitation light intensity; and determining the second wavefront during the calculation process specifically by using a beam propagation method. There are no explicit teachings to the above limitation(s) found in the prior art cited in this office action and from the prior art search.
Claim(s) 10-11 depend on claim 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent:center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 7/25/2026
Prosecution Insights