DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Claim(s) 1-20 do not use “means for “(or “step for”) language, or generic placeholders for “means” coupled with functional language without recitation of sufficient structure for carrying out the claimed functions and therefore do not invoke 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph).
Claim Objections
Claim 16 objected to because of the following informalities: Claim 16 should end with "." and not ";". Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
[Claim 8]
The term “high speed camera” in claim 8 is a relative term which renders the claim indefinite. The term “ high speed camera” is not defined by the claim. Paragraph 0025 of the specification describes an example of a high speed camera as, “a camera configured to capture images with short exposures, such as 1/30th of a second (or smaller), and/or frame rates exceedingly at least 30 frames per second).” However, this example is not an explicit definition for the term “high speed camera” as recited in claim 8. Therefore, one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
[Claim 12]
Claim 12 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form. Claim 12 recites," The system of claim 12," and thus does not contain reference to claim previously set forth. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Allowable Subject Matter
Claims 1-7, 9-11 and 13-20 are allowed.
Claims 8 and 12 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(d) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
[Claims 1-14]
Prior art does not teach or reasonably suggest a system comprising: a grating configured to include a plurality of openings, wherein each of the plurality of openings is subpixel in size, wherein the subpixel size is smaller than an image sensor pixel of a camera; a slider coupled to the grating and configured to move the grating laterally along an image plane of the camera; an illumination source, wherein the illumination source generates light that passes through the grating to form structured illumination configured to provide sub-pixel sized structured illumination on a subject as the grating moves laterally along the image plane to enable the camera to capture a plurality of low resolution image, at least one processor; and at least one memory including instructions, which when executed by the at least one processor, causes super resolution image reconstruction operations comprising: receiving the plurality of low resolution images; reconstructing a super resolution image using the plurality of low-resolution images, wherein the reconstructed super resolution image is noise filtered based on phase differences to remove the noise caused in part by reconstructing using the plurality of low resolution images; and outputting the reconstructed super resolution image as a representation of the subject, wherein the subpixel size is sized as a quarter of the image sensor pixel of the camera, wherein the subpixel size is sized to be smaller than the quarter, wherein the grating is configured as a hexagonal lattice grating, wherein a distance between centers among the openings is 10 to 500 micrometers, wherein the grating is configured as a hexagonal lattice grating, wherein a distance between centers among the openings is 10 to 500 micrometers, wherein slider is synchronized with the camera, such that a trigger signal is sent to the camera to capture at least one low resolution image at each of the first position, the second position, the third position, and the fourth position, further comprising the camera including the plurality of image sensor pixels, wherein the camera comprises a high speed camera, wherein the grating includes one or more registration landmarks captured in the plurality of low resolution images, wherein the super resolution image reconstruction operations further comprise: correcting the plurality of low-resolution images for flat field to compensate the plurality of low-resolution images for non-uniformities; and forming, using the plurality of low-resolution images, a plurality of high resolution images by upscaling each low resolution image, wherein the super resolution image reconstruction operations further comprises: registering the plurality of high resolution images; combining the plurality of high resolution images to generate a first high resolution image; and performing a Fourier Transform on the first high resolution image to form a Fourier domain representation of the first high resolution image, wherein the super resolution image reconstruction operations further comprise: generating a second high resolution image by at least recombing the plurality of low-resolution images based on a high-resolution grid; and performing a Fourier Transform on the second high resolution image to form a Fourier domain representation of the second high resolution image, wherein the super resolution image reconstruction operations further comprise: generating a second high resolution image by at least recombing the plurality of low-resolution images based on a high-resolution grid; and performing a Fourier Transform on the second high resolution image to form a Fourier domain representation of the second high resolution image, wherein the noise being filtered is determined based on phase differences between the Fourier domain representation of the first high resolution image and the Fourier Transform on the second high resolution image, wherein the subject comprises a dynamic flow.
Baker, Thomas (US-2019/0212266-A1) teaches system comprising (Baker, Figure 1A, a structured illumination imaging system 100): a grating (Baker, Figure 1A, Light structuring optical assembly 155) configured to include a plurality of openings (Baker, Figure 1A, a one-dimensional transmissive diffraction grating 155a has plurality of openings), a slider coupled to the grating and configured to move the grating laterally along an image plane of the camera (Baker teaches in Figure 1A, During each imaging cycle, imaging system 100 utilizes light structuring optical assembly 155 to acquire a plurality of images at various phases, displaced laterally along the sample plane (e.g., along x-y plane), with this procedure repeated one or more times by rotating the pattern orientation about the optical axis (i.e., with respect to the x-y plane of the sample), paragraph 0079; an illumination source ( Baker, Figure 1A,light emitter 150), wherein the illumination source generates light that passes through the grating to form structured illumination (Baker, in Figure 1A teaches During each image reading, light emitted by emitter 150 is structured by structuring optical assembly 155 to project fringes 160 having a pitch P (center to center spacing between fringes) and width w such that one of features 111 corresponding to each pixel is at least substantially illuminated, paragraph 0070); configured to provide sub-pixel sized structured illumination on a subject as the grating moves laterally along the image plane to enable the camera to capture a plurality of low resolution images at least one processor (Baker, although not illustrated, a controller can be provided to control the operation of structured illumination imaging system 100, including synchronizing the various optical components of system 100, paragraph 0082); and at least one memory including instructions, which when executed by the at least one processor (Baker, a memory, storage unit, and media. These and other various forms of computer program media or computer usable media may be involved in carrying one or more sequences of one or more instructions to a processing device for execution. Such instructions embodied on the medium, are generally referred to as “computer program code” or a “computer program product” (which may be grouped in the form of computer programs or other groupings, paragraph 0153), causes super resolution image reconstruction operations comprising: receiving the plurality of low resolution images (Baker, in Figure 10 teaches at operation 1020, a first image of the sample is captured. At operation 1040, a second image of the sample is captured, paragraph 0125); reconstructing a super resolution image using the plurality of low-resolution images (Baker further teaches in Figure 10, at operation 1050, the two captured images may be used to generate a sub pixel resolution or super resolution image, paragraph 0127); and outputting the reconstructed super resolution image as a representation of the subject (Baker further teaches the captured images (e.g., nine images) may be assembled into a single image having an extended spatial frequency bandwidth, which may be retransformed into real space to generate an image having a higher resolution that one captured by a conventional microscope. In these traditional systems, detection of molecules by structured illumination microscopy relies on recollecting the excitation light (typically with the same objective used to excite) and reimaging the emission signal onto a CCD camera, paragraph 0003).
However, Baker fails to teach or reasonably suggest, wherein each of the plurality of openings
subpixel in size, wherein the subpixel size is smaller than an image sensor pixel of a camera; wherein the reconstructed super resolution image is noise filtered based on phase differences to remove the noise caused in part by reconstructing using the plurality of low resolution images.
Langlois et al. (US-2021/0118110-A1) teaches a structured illumination microscopy imaging system in Figure 2, that includes a light emitter 150 that is configured to output a light beam that is collimated by collimation lens 151. The collimated light is structured (patterned) by light structuring optical assembly 155 and directed by dichroic mirror 160 through objective lens 142 onto a sample of a sample container 110, which is positioned on a motion stage 170, paragraph 0094. Light structuring optical assembly 155 includes one or more optical diffraction gratings or other beam splitting elements (e.g., a beam splitter cube or plate) to generate a pattern of light (e.g., fringes, typically sinusoidal) that is projected onto samples of a sample container 110. The diffraction gratings may be one-dimensional or two-dimensional transmissive or reflective gratings. The diffraction gratings may be sinusoidal amplitude gratings or sinusoidal phase gratings, paragraph 0095. Langlois further teaches a controller 195 can be provided to control the operation of structured illumination imaging system 100, including synchronizing the various optical components of system 100. The controller can be implemented to control aspects of system operation such as, for example, configuration of light structuring optical assembly 155 (e.g., selection and/or linear translation of diffraction gratings), paragraph 0108. Langlois further teaches The captured images may then be computationally reconstructed to generate a higher resolution image (e.g., an image having about twice the lateral spatial resolution of individual images, paragraph 0098. Langlois further teaches estimating phase displacement of tiles relative to the full field of view (FOV), so that measurement of phase in one subtile can be extrapolated to other subtiles across the tile. The illumination peak angle and illumination peak spacing for the full FOV can be estimated from the illumination peak angle and illumination peak spacing of the subtile using the quadratic models presented above. The phase displacement is less regular because it depends on pixel geometry of subtiles, which can produce an irregular step function, instead of a smooth function. Each phase estimate has a “frame of reference” anchored to the top-left corner of the image being used to estimate the phase. As a result, when we want to correct for phase differences geometrically across the image, the phases estimated from each subtile need to be compared to a reference phase (from the center estimation subwindow or subtile) in a common frame of reference, paragraph 0197. Langlois further teaches In various implementations, the controller 195 can be implemented using hardware, algorithms (e.g., machine executable instructions), or a combination of the foregoing. For example, in some implementations the controller can include one or more CPUs, GPUs, or processors with associated memory. As another example, the controller can comprise hardware or other circuitry to control the operation, such as a computer processor and a non-transitory computer readable medium with machine-readable instructions stored thereon. For example, this circuitry can include one or more of the following: field programmable gate array (FPGA), application specific integrated circuit (ASIC), programmable logic device (PLD), complex programmable logic device (CPLD), a programmable logic array (PLA), programmable array logic (PAL) and other similar processing device or circuitry. As yet another example, the controller can comprise a combination of this circuitry with one or more processors, paragraph 0109.
Langlois fails to teach or reasonably suggest wherein the subpixel size is smaller than an image sensor pixel of a camera.
[Claims 15-20]
Prior art do not teach or reasonably suggest a method comprising: generating, by an illumination source, light; passing the light through a grating, wherein the grating forms structured illumination configured to provide sub-pixel sized structured illumination on a subject, wherein the grating is configured to move laterally along an image plane of a camera to capture a plurality of low resolution images, wherein the grating is configured to include a plurality of openings, wherein each of the openings is subpixel in size, wherein the subpixel size is smaller than an image sensor pixel of the camera; receiving the plurality of low resolution images; reconstructing a super resolution image using the plurality of low- resolution images, wherein the reconstructed super resolution image is noise filtered based on phase differences to remove the noise caused in part by reconstructing using the plurality of low resolution images; and outputting the reconstructed super resolution image as a representation of the subject, further comprising moving, by a slider, the grating laterally along the image plane of the camera; wherein the slider is coupled to the grating and is configured to move, using at least a stepper motor, the grating laterally from at 4 least a first position, a second position, a third position, and a fourth position of the image plane of the camera, wherein slider is synchronized with the camera, such that a trigger signal is sent to the camera to capture at least one low resolution images at each of the first position, the second position, the third position, and the fourth position, further comprising: correcting the plurality of low-resolution images for flat field to compensate the plurality of low-resolution images for non-uniformities; forming, using the plurality of low-resolution images, a plurality of high resolution images by upscaling each low resolution image; registering the plurality of high resolution images; combining the plurality of high resolution images to generate a first high resolution image; and performing a Fourier Transform on the first high resolution image to form a Fourier domain representation of the first high resolution image, herein the noise being filtered is determined based on phase differences between the Fourier domain representation of the first high resolution image and the Fourier Transform on the second high resolution image.
Claims 15-20 are method claims of the apparatus claims of 1-14. Prior art does not teach or reasonably suggest a method related to the apparatus claims described in claims 1-14.
Salvador et al. (US-2013/0301933-A1) teaches a method of creating super resolution images based on plurality of low resolution images. Salvador further teaches super-resolving a single image comprises three stages. First, an interpolation-based up-scaling of the input image is performed, followed by an equivalent low-pass filtering operation on the low-resolution (LR) image, which results in a low-frequency (LF) band of a high-resolution (HR) image, paragraph 0012. Salvadore further teaches a de-noising mask is generated that indicates potentially noisy or disturbed areas, and local de-noising is applied separately to pixels of at least one of the HF band of the HR image and the LF band of the HR image, as defined by the de-noising mask, paragraph 0012. Salvador further teaches in Figure 3, usage and positioning of a search window within the low-resolution low-frequency data structure L.sub.0. For a first patch P.sub.11,L1 in L.sub.1, a first best matching block P.sub.11,L0 is searched in L.sub.0 within a first search window W.sub.11. Both patches have the same size. The search window is larger than the patch by at least one value in each direction (except on edges, as for the first patch). In this example, the first best matching block P.sub.11,L0 is found in L.sub.0 in the upper left corner of the first search window W.sub.11. The further process for this patch and block is as described above. Then, subsequent patches are shifted horizontally and/or vertically, wherein each patch overlaps a previous patch, paragraph 0046.
Salvador does not teach or reasonably suggest wherein the grating to form structured illumination configured to provide sub-pixel sized structured illumination on a subject as the grating moves laterally along the image plane to enable the camera capture low resolution images.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. The following show additional prior art systems/methods for imaging apparatus:
Baker et al. US 2019/0212266-A1
Langlois et al. US 2021/0118110-A1
Salvador et al. US 2013/0301933-A1
Ren et al. US 12,411,327 B2
Liu et al. US 12,281,888 B2
Xu et al. US 2024/0361812-A1
Leung et al. US 2023/0280271-A1
Rich et al. US 2022/0276501-A1
Marks et al. US 11,423,853 B1
Tsujio, Shoichi US 2021/0134868-A1
Pacala et al. US 2020/0116558-A1
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHEDI NMN HASSAN whose telephone number is (571)272-7173. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 5712727564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHEDI NMN HASSAN/Examiner, Art Unit 2637
/SINH TRAN/
Supervisory Primary Examiner, Art Unit 2637
Prosecution Insights