Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/08/2025 is being considered by the examiner.
Drawings
The drawings are objected to under 37 CFR 1.83(a) because they fail to show details in Fig. 1 and Fig. 2 as described in the specification. Applicant is advised to amend the drawings to include labels on at least the elements of Fig. 1 and Fig. 2: 12, 14, 16, 30, 32, and 34 to correspond to the specification. Any structural detail that is essential for a proper understanding of the disclosed invention should be shown in the drawing. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The disclosure is objected to because of the following informalities: page 8, line 4 recites “SytleGAN”; Applicant is advised to amend the line to “StyleGAN.”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “a capture device for capturing light” and “a reconstruction device for reconstructing first areas” in claim 14.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 recites the limitation "the second areas" in line 8. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the first light sheet" in line 3 and “the light sheet” in line 9. There is insufficient antecedent basis for these limitations in the claim. Previously the claim recites “a first grid-shaped light sheet,” but it is not clear if “the first light sheet” and “the light sheet” is the same element.
Additionally, claims 2-4, 7-12, 14, 16, 18, and 19 recite both “first grid-shaped light sheet” and “first light sheet”; particularly in Claim 4 it is not clear if the elements are the same. Claim 7 recites “a second grid-shaped light sheet and “the second light sheet,” but it is not clear if the “second light sheet” and “a second grid-shaped light sheet are the same elements. Claim 8 recites “a third grid-shaped light sheet,” “the third light sheet,” and “the light sheet” but it is not clear if the “third light sheet,” “the light sheet,” and “a third grid-shaped light sheet are the same elements. Applicant is advised to amend the claims such that the first, second, and third grid-shaped light sheets are recited in a manner consistent across the claims to improve clarity. Thus, Claims 1-4, 7-12, 14, 16, 18, and 19 are rejected under 35 U.S.C. 112(b) (Claim 19 depends from Claim 18 and thus contains all subject matter of Claim 18).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, 8, 10, 11, and 13-22are rejected under 35 U.S.C. 103 as being unpatentable over Haase (US 2021/0239952 A1) in view of Chen B. (Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution, published 2014).
Regarding Claim 1, Haase teaches “A method for generating an image of a sample, wherein the method comprises:
radiating (Haase, [0136] discloses “In the case of the wide-field microscope, partial regions 50-55 are in each case illuminated or irradiated and the respective partial region 50-55 is captured by the wide-field microscope. However, typically only partial regions 50-55 of the sample 1 are illuminated or irradiated, rather than the sample 1 as a whole”; where partial regions 50-55 are grid-shaped light sheets; sample 1 of Haase is inhomogeneously illuminated);
“capturing light emitted from the sample due to the radiating of the first light sheet of the first wavelength range onto the sample” (Haase, [0136] discloses “The partial regions 50-55 are captured by means of a wide-field microscope”) “and
reconstructing first areas of the sample, which are not illuminated or are more weakly illuminated using the first light sheet of the first wavelength range, on the basis of light captured from the second areas of the sample, which are more strongly illuminated using the light sheet of the first wavelength range, by means of a machine learning system” (Haase, [0134] discloses “The system 60 comprises a trained machine learning system 30 configured for determining the partial regions 50-55 of the sample 1 which are captured. Moreover, it is possible for the system 60 additionally to reconstruct an overall image 40 of the sample 1 from the data 65 of the captured partial regions 50-55.”)
PNG
media_image1.png
272
443
media_image1.png
Greyscale
Fig. 10 of Haase
Although Haase discloses inhomogenously illuminating a sample, as seen in Fig. 10, Haase does not explicitly teach “radiating a first grid-shaped light sheet of a first wavelength range onto the sample.”
However, in an analogous field of endeavor, Chen B. teaches “radiating a first grid-shaped light sheet of a first wavelength range onto the sample” (Chen B., Fig. 1, Section D, column 2 and Fig. 1 caption discloses “the hexagonal lattice in (D)optimizes the axial resolution as defined by the overall PSF of the microscope”; see lattice, or grid-shaped light sheet in the bottom row, second column of Fig. 1; where hexagonal lattice light-sheet is radiating a first grid-shaped light sheet in such a way that the sample is inhomogeneously illuminated by the first light sheet. Chen B. also discloses “At each step, we applied an 8-ms pulse of 405-nm light and then imaged the molecules thereby activated for 95 ms under 560-nm excitation”; where 403-nm light is a first wavelength range of light).
PNG
media_image2.png
591
885
media_image2.png
Greyscale
Fig. 1 of Chen B.
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified Haase to incorporate the teachings of Chen B. by illuminating a sample with a hexagonal lattice at a particular wavelength. The prior art Haase contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Haase discloses a method of scanning only partial regions of a sample and reconstructing microscope images from partial regions. The claimed invention recites specifically a grid shaped light sheet, which can be seen as an ‘improvement’ over the illuminated partial regions determined by a machine learning system in Haase. The prior art contained a ‘comparable’ method that has been improved in the same way as the claimed invention. Chen B. teaches illuminating a sample using a hexagonal lattice light sheet, which forms a grid as seen above in Fig. 1 of Chen B. One of ordinary skill in the art could have applied the known ‘improvement’ technique in the same way to the ‘base’ method and the results would have been predictable to one of ordinary skill in the art. That is, one of ordinary skill in the art could have applied the grid illumination technique to the reconstruction from partial illumination and scanning method of Haase. Although Haase is directed to scanning microscopes and Chen B. is directed to light-sheet microscopy, Chen X. (Deep-learning on-chip light-sheet microscopy enabling video-rate volumetric imaging of dynamic biological specimens, published 2021), incorporated as an evidentiary reference, discloses a method of deep-learning applied to light-sheet microscopy, thus it would have been obvious to one of ordinary skill in the art that machine learning methods of Haase may be applied to samples illuminated by the microscope of Chen B., with predictable results. Accordingly, the combination of Haase and Chen B. discloses the invention of Claim 1.
Regarding Claim 2, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein the first areas of the sample, which are not illuminated or are more weakly illuminated using the light sheet of the first wavelength range, lie in the same plane as the second areas of the sample, which are more strongly illuminated using the first wavelength range” (Chen B. , Fig. 1 caption discloses “The columns in(A) to (D) show the intensity pattern at the rear pupil plane of the excitation objective; the cross-sectional intensity of the pattern in the xz plane at the focus of the excitation objective (scale bar, 1.0 mm)”). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 1, apply to Claim 2 and are incorporated herein by reference. Thus, the apparatus recited in Claim 2 is met by Haase and Chen B.
Regarding Claim 3, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein at least one first plane exists, wherein first areas of the sample, which are not illuminated or are more weakly illuminated using the first light sheet of the first wavelength range, lie, and at least one second plane exists, in which only second areas of the sample, which are more strongly illuminated using the first light sheet of the first wavelength range, lie” (Chen B., page 3, column 1 discloses “In the SIM mode, we recorded F images at every z plane as we stepped the lattice light sheet in the x direction in F equal fractions of the lattice period”; where stepping lattice light sheet in x direction with illumination being a hexagonal lattice in the x-z plane as shown in Fig. 1(D), is illuminating both a first and second plane with more weakly and more strongly illuminated areas, respectively). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference. Thus, the apparatus recited in Claim 3 is met by Haase and Chen B.
Regarding Claim 4, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein the method further comprises: changing a phase and/or a position of the first grid-shaped light sheet together with a shift of the sample relative to the first light sheet in such a way that the first areas of the sample of a first plane, which are not illuminated or are more weakly illuminated, lie offset to the first areas of a second plane of the sample parallel to the first plane, which are not illuminated or are more weakly illuminated using the first wavelength range” (Chen B., page 3, column 1 discloses “In the SIM mode, we recorded F images at every z plane as we stepped the lattice light sheet in the x direction in F equal fractions of the lattice period”; where stepping lattice light sheet in the x direction is changing a position of first grid-shaped light sheet together with a shift of the sample relative to the first light sheet). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 1, apply to Claim 4 and are incorporated herein by reference. Thus, the apparatus recited in Claim 4 is met by Haase and Chen B.
Regarding Claim 5, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein in reconstructing the first areas of a first plane of the sample, which are not illuminated or are illuminated more weakly, light captured from more strongly illuminated second areas of a second plane of the sample, which is different from the first plane, is taken into consideration” (Haase, [0134] discloses “Moreover, it is possible for the system 60 additionally to reconstruct an overall image 40 of the sample 1 from the data 65 of the captured partial regions 50-55”).
Regarding Claim 8, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein the method further comprises: radiating a third grid-shaped light sheet onto the sample and changing a position and/or a phase of the third light sheet in such a way that the sample is substantially homogeneously illuminated by the light sheet, and capturing light emitted by the sample due to the radiating of the third light sheet onto the sample to form an image of the sample” (Chen B., page 3, paragraph 3 recites “In the dithered mode, we used a galvanometer (fig. S4) to oscillate the lattice pattern back and forth in x at an amplitude larger than the lattice period and a speed fast compared with the camera exposure time, pro-viding time-averaged uniform illumination across the xy plane. In this mode, we needed to capture only one 2D image at each z plane, but the resolution remained diffraction-limited”; where oscillating the lattice pattern is changing a position of the third light sheet; where uniform illumination is substantially homogeneously illuminated sample; where a 2D image is an image of the sample). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 1, apply to Claim 8 and are incorporated herein by reference. Thus, the apparatus recited in Claim 8 is met by Haase and Chen B.
Regarding Claim 10, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein a shape and/or a position and/or a wavelength range of the first light sheet and/or the second light sheet is adapted to the sample” (Haase, [0140] discloses “In this case, only partial regions 50-55 of the sample 1 are captured by means of the wide-field microscope. The machine learning system 30 determines which partial regions 50-55 of the sample 1 are captured and the order in which they are captured”; where capturing partial regions determined by a machine learning system is a position of a first light sheet being adapted to the sample).
Regarding Claim 11, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein the first areas of the sample and the second areas of the sample are illuminated for different lengths of time by the first light sheet” (Haase, [0131] and Fig. 5 discloses “FIG. 5 shows a third scan pattern of the sample 1 from FIG. 1 in accordance with one embodiment of the method according to the invention. FIG. 4 shows an adaptive scan pattern, wherein the adaptive scan pattern is linear and, upon a changeover from the background to an element 5-8, examines the element 5-8, in particular the edges thereof, more closely by virtue of the line of the scan pattern moving back and forth a number of times over the edge of the element 5-8. Such a scan pattern for example is determined by the trained machine learning system 30”; where moving back and forth a number of times over the edge of elements is illuminating a second area (edges) for a different length of time than first areas (background) ).
PNG
media_image3.png
191
244
media_image3.png
Greyscale
Fig 5 of Haase
Regarding Claim 13, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein the first areas which are more weakly illuminated are not non-illuminated” (Chen B., Fig. 1; see intensity scale in top left corner; see also in row D, the gradient in intensities. Thus, weakly illuminated areas are not non-illuminated in Chen B.). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 1, apply to Claim 13 and are incorporated herein by reference. Thus, the apparatus recited in Claim 13 is met by Haase and Chen B.
Regarding Claim 14, Haase teaches “A device for generating an image of a sample, comprising:
a beam device for radiating (Haase, [0136] discloses “In the case of the wide-field microscope, partial regions 50-55 are in each case illuminated or irradiated and the respective partial region 50-55 is captured by the wide-field microscope. However, typically only partial regions 50-55 of the sample 1 are illuminated or irradiated, rather than the sample 1 as a whole”; where partial regions 50-55 are grid-shaped light sheets; sample 1 of Haase is inhomogeneously illuminated),
“a capture device for capturing light emitted by the sample due to the radiating of the first light sheet of the first wavelength range onto the sample” (Haase, [0136] discloses “The partial regions 50-55 are captured by means of a wide-field microscope”); “and
“a reconstruction device for reconstructing first areas of the sample, which are not illuminated or are illuminated more weakly using the first light sheet of the first wavelength range, on the basis of light captured from the second areas of the sample, which are more strongly illuminated using the first light sheet of the first wavelength range, by means of a machine learning system” (Haase, [0134] discloses “The system 60 comprises a trained machine learning system 30 configured for determining the partial regions 50-55 of the sample 1 which are captured. Moreover, it is possible for the system 60 additionally to reconstruct an overall image 40 of the sample 1 from the data 65 of the captured partial regions 50-55”).
Although Haase discloses inhomogenously illuminating a sample, as seen in Fig. 10, Haase does not explicitly teach “radiating a first grid-shaped light sheet of a first wavelength range onto the sample.”
However, in an analogous field of endeavor, Chen B. teaches “radiating a first grid-shaped light sheet of a first wavelength range onto the sample” (Chen B., Fig. 1, Section D, column 2 and Fig. 1 caption discloses “the hexagonal lattice in (D)optimizes the axial resolution as defined by the overall PSF of the microscope”; see lattice, or grid-shaped light sheet in the bottom row, second column of Fig. 1; where hexagonal lattice light-sheet is radiating a first grid-shaped light sheet in such a way that the sample is inhomogeneously illuminated by the first light sheet. Chen B. also discloses “At each step, we applied an 8-ms pulse of 405-nm light and then imaged the molecules thereby activated for 95 ms under 560-nm excitation”; where 403-nm light is a first wavelength range of light).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified Haase to incorporate the teachings of Chen B. by illuminating a sample with a hexagonal lattice at a particular wavelength. The prior art Haase contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Haase discloses a method of scanning only partial regions of a sample and reconstructing microscope images from partial regions. The claimed invention recites specifically a grid shaped light sheet, which can be seen as an ‘improvement’ over the illuminated partial regions determined by a machine learning system in Haase. The prior art contained a ‘comparable’ method that has been improved in the same way as the claimed invention. Chen B. teaches illuminating a sample using a hexagonal lattice light sheet, which forms a grid as seen above in Fig. 1 of Chen B. . One of ordinary skill in the art could have applied the known ‘improvement’ technique in the same way to the ‘base’ method and the results would have been predictable to one of ordinary skill in the art. That is, one of ordinary skill in the art could have applied the grid illumination technique to the reconstruction from partial illumination and scanning method of Haase. Although Haase is directed to scanning microscopes and Chen B. is directed to light-sheet microscopy, Chen X. (Deep-learning on-chip light-sheet microscopy enabling video-rate volumetric imaging of dynamic biological specimens, published 2021), incorporated as an evidentiary reference, discloses a method of deep-learning applied to light-sheet microscopy, thus it would have been obvious to one of ordinary skill in the art that machine learning methods of Haase may be applied to samples illuminated by the microscope of Chen B., with predictable results. Accordingly, the combination of Haase and Chen B. discloses the invention of Claim 14.
Regarding Claim 15, the combination of Haase and Chen B. discloses “The device according to claim 14, wherein the first areas which are illuminated more weakly are not non-illuminated” (Chen B., Fig. 1; see intensity scale in top left corner; see also in row D, the gradient in intensities. Thus, weakly illuminated areas are not non-illuminated in Chen B.). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 14, apply to Claim 15 and are incorporated herein by reference. Thus, the apparatus recited in Claim 15 is met by Haase and Chen B.
Regarding Claim 16, Haase teaches “A method for training a machine learning system for reconstructing first areas of a sample, which are not illuminated or are illuminated more weakly (Haase, [0134] discloses “The system 60 comprises a trained machine learning system 30 configured for determining the partial regions 50-55 of the sample 1 which are captured. Moreover, it is possible for the system 60 additionally to reconstruct an overall image 40 of the sample 1 from the data 65 of the captured partial regions 50-55”), “wherein the method comprises:
(Haase, [0136] discloses “In the case of the wide-field microscope, partial regions 50-55 are in each case illuminated or irradiated and the respective partial region 50-55 is captured by the wide-field microscope. However, typically only partial regions 50-55 of the sample 1 are illuminated or irradiated, rather than the sample 1 as a whole”; where partial regions 50-55 are grid-shaped light sheets; sample 1 of Haase is inhomogeneously illuminated) “in order to generate a partial image of the sample” (Haase, [0119] discloses “During supervised learning for improved reconstruction of the overall image 40 of the sample 1 from scanned partial regions 10-15 of the sample 1, training data in the form of data 25 from the scanned partial regions 10-15 of the sample 1 are input into the machine learning system 30. The scanned partial regions 10-15 can be simulated data 25 generated on the basis of a non-reconstructed overall image 40, or real recording data of a scanning microscope”);
“generating a complete image of the sample” (Haase, [0119] discloses “In addition, a complete image or an overall image 40 is input into the machine learning system 30”); “and
inputting the complete image and the partial image into the machine learning system to train the machine learning system to reconstruct the first areas of the sample” (Haase, [0119] discloses “On the basis of the training data, the machine learning system 30 learns how as optimum an overall image 40 as possible of the sample 1 can be reconstructed from the data 25 of the partial regions 10-15 of the sample 1, since the non-reconstructed overall image 40 is likewise input as target or ideal into the machine learning system 30.”)
Although Haase discloses inhomogenously illuminating a sample, as seen in Fig. 10, Haase does not explicitly teach “radiating a first grid-shaped light sheet of a first wavelength range onto the sample.”
However, in an analogous field of endeavor, Chen B. teaches “radiating a first grid-shaped light sheet of a first wavelength range onto the sample” (Chen B., Fig. 1, Section D, column 2 and Fig. 1 caption discloses “the hexagonal lattice in (D)optimizes the axial resolution as defined by the overall PSF of the microscope”; see lattice, or grid-shaped light sheet in the bottom row, second column of Fig. 1; where hexagonal lattice light-sheet is radiating a first grid-shaped light sheet in such a way that the sample is inhomogeneously illuminated by the first light sheet. Chen B. also discloses “At each step, we applied an 8-ms pulse of 405-nm light and then imaged the molecules thereby activated for 95 ms under 560-nm excitation”; where 403-nm light is a first wavelength range of light).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified Haase to incorporate the teachings of Chen B. by illuminating a sample with a hexagonal lattice at a particular wavelength. The prior art Haase contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Haase discloses a method of scanning only partial regions of a sample and reconstructing microscope images from partial regions. The claimed invention recites specifically a grid shaped light sheet, which can be seen as an ‘improvement’ over the illuminated partial regions determined by a machine learning system in Haase. The prior art contained a ‘comparable’ method that has been improved in the same way as the claimed invention. Chen B. teaches illuminating a sample using a hexagonal lattice light sheet, which forms a grid as seen above in Fig. 1 of Chen B. . One of ordinary skill in the art could have applied the known ‘improvement’ technique in the same way to the ‘base’ method and the results would have been predictable to one of ordinary skill in the art. That is, one of ordinary skill in the art could have applied the grid illumination technique to the reconstruction from partial illumination and scanning method of Haase. Although Haase is directed to scanning microscopes and Chen B. is directed to light-sheet microscopy, Chen X. (Deep-learning on-chip light-sheet microscopy enabling video-rate volumetric imaging of dynamic biological specimens, published 2021), incorporated as an evidentiary reference, discloses a method of deep-learning applied to light-sheet microscopy, thus it would have been obvious to one of ordinary skill in the art that machine learning methods of Haase may be applied to samples illuminated by the microscope of Chen B., with predictable results. Accordingly, the combination of Haase and Chen B. discloses the invention of Claim 16.
Regarding Claim 17, the combination of Haase and Chen B. discloses “The method according to claim 16, wherein the first areas which are illuminated more weakly are not non-illuminated” (Chen B., Fig. 1; see intensity scale in top left corner; see also in row D, the gradient in intensities. Thus, weakly illuminated areas are not non-illuminated in Chen B.). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 16, apply to Claim 17 and are incorporated herein by reference. Thus, the apparatus recited in Claim 17 is met by Haase and Chen B.
Regarding Claim 18, the combination of Haase and Chen B. discloses “A method for training a machine learning system for reconstructing first areas of a sample, which are not illuminated or are illuminated more weakly using a first grid-shaped light sheet of a first wavelength range, on the basis of light captured from second areas of the sample, which are more strongly illuminated using the first light sheet of the first wavelength range” (Haase, [0136] discloses “In the case of the wide-field microscope, partial regions 50-55 are in each case illuminated or irradiated and the respective partial region 50-55 is captured by the wide-field microscope. However, typically only partial regions 50-55 of the sample 1 are illuminated or irradiated, rather than the sample 1 as a whole”; where partial regions 50-55 are grid-shaped light sheets; sample 1 of Haase is inhomogeneously illuminated. Chen B., Fig. 1, Section D, column 2 and Fig. 1 caption discloses “the hexagonal lattice in (D)optimizes the axial resolution as defined by the overall PSF of the microscope”; see lattice, or grid-shaped light sheet in the bottom row, second column of Fig. 1; where hexagonal lattice light-sheet is radiating a first grid-shaped light sheet in such a way that the sample is inhomogeneously illuminated by the first light sheet. Chen B. also discloses “At each step, we applied an 8-ms pulse of 405-nm light and then imaged the molecules thereby activated for 95 ms under 560-nm excitation”; where 403-nm light is a first wavelength range of light), “wherein the method comprises:
generating training data according to the method as claimed in claim 16” (see above, Regarding Claim 16); “and
inputting the complete images and the generated partial images respectively corresponding thereto into a machine learning system to train the machine learning system for reconstruction” (Haase, [0119] discloses “On the basis of the training data, the machine learning system 30 learns how as optimum an overall image 40 as possible of the sample 1 can be reconstructed from the data 25 of the partial regions 10-15 of the sample 1, since the non-reconstructed overall image 40 is likewise input as target or ideal into the machine learning system 30”). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 16, apply to Claim 18 and are incorporated herein by reference. Thus, the apparatus recited in Claim 18 is met by Haase and Chen B.
Regarding Claim 19, the combination of Haase and Chen B. discloses “The method according to claim 18, wherein the first areas which are illuminated more weakly are not non-illuminated” (Chen B., Fig. 1; see intensity scale in top left corner; see also in row D, the gradient in intensities. Thus, weakly illuminated areas are not non-illuminated in Chen B.). The proposed combination as well as the motivation for combining the Haase and Chen B. references presented in the rejection of Claim 16, apply to Claim 19 and are incorporated herein by reference. Thus, the apparatus recited in Claim 19 is met by Haase and Chen B.
Regarding Claim 20, the combination of Haase and Chen B. teaches “A machine learning system, which was trained by means of the method as claimed in claim 16” (Haase, [0119] discloses “On the basis of the training data, the machine learning system 30 learns how as optimum an overall image 40 as possible of the sample 1”).
Regarding Claim 21, the combination of Haase and Chen B. teaches “A computer program product, comprising: instructions readable by a processor of a computer which, when they are executed by the processor, prompt the processor to carry out the method as claimed in claim 16” (Haase, [0014] discloses “In particular, the object is also achieved by means of a computer program product having instructions which are readable by a processor of a computer and which, when they are executed by the processor, cause the processor to carry out the method mentioned above”).
Regarding Claim 22, the combination of Haase and Chen B. teaches “A computer-readable medium, on which the computer program product as claimed in claim 21 is stored” (Haase, [0015] discloses “In particular, the object is also achieved by means of a computer-readable medium, on which the computer program product program is stored”).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Haase (US 2021/0239952 A1) in view of Chen B. (Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution, published 2014), further in view of Gao et al. (US 12,663,630 B2).
Regarding Claim 7, the combination of Haase and Chen B. does not explicitly teach the method of Claim 7.
However, in an analogous field of endeavor, Gao teaches “The method as claimed in claim 1, wherein the method further comprises:
radiating a second grid-shaped light sheet of a second wavelength range, which is different from the first wavelength range, onto the sample in such a way that the first areas of the sample, which are not illuminated or are more weakly illuminated using the first wavelength range, are at least partially more strongly illuminated using the second light sheet of the second wavelength range than the second areas of the sample more strongly illuminated using the first light sheet of the first wavelength range” (Gao, column 6, lines 4-14 discloses “FIG. 3 shows a schematic structural diagram of the tiling light sheet microscope according to the embodiment of the present disclosure. As shown in FIG. 3, the tiling light sheet microscope comprises a first laser and a second laser (not shown). The first laser is configured to generate a first laser beam of a first wavelength range (exemplified as 561 nm for exciting red fluorescence), and the second laser is configured to generate a second laser beam of a second wavelength range (exemplified as 488 nm for exciting green fluorescence)”); “and
capturing light emitted by the sample due to the radiating of the second light sheet of the second wavelength range onto the sample, wherein in reconstructing, the captured light of the second light sheet of the second wavelength range is taken into consideration” (Gao, column 7, lines 38-44 discloses “In some embodiments, the first SLM assembly 301, the second SLM assembly 302, the first detection camera 309b and the second detection camera 310b may operate in a synchronized manner. In this way, fluorescence imaging in different colors may operate in a synchronized manner, such that the composite image obtained after superimposition will present information with less distortion and richer details”).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified the combination of Haase and Chen B. to incorporate the teachings of Gao by tiling multiple light sheets of two wavelengths to produce a composite image. One of ordinary skill in the art would be motivated to combine the Haase, Chen B., and Gao references in order to produce a composite image “with less distortion and richer details” (Gao, column 7, lines 38-44). Accordingly, the combination of Haase, Chen B., and Gao discloses the invention of Claim 7.
Allowable Subject Matter
Claim 6 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 9 and 12 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 6, the combination of Haase and Chen B. does not explicitly teach “The method as claimed in claim 1, wherein in reconstructing the first areas, which are not illuminated or are illuminated more weakly, a transmitted light image of the sample is taken into consideration.”
Siebenmorgen et al. (US 2017/0269345 A1), [0112] discloses “Further, the control unit 13 may be embodied to evaluate the captured overview images and/or the images of the light sheet 6. The control unit 13 may be connected to a display for graphically illustrating the captured overview images and/or the images of the light sheet 6.” Thus, Siebenmorgen teaches detecting both overview (transmitted light) images and light sheet images. However, Siebenmorgen does not explicitly teach taking “into consideration” a transmitted light image in reconstructing the first areas. That is, although employing both light sheet and transmitted light imaging in the same method or apparatus is known in the art, the prior art does not explicitly teach using a transmitted light image to reconstruct first areas of the sample which are not illuminated or are more weakly illuminated.
Thus, none of the previously cited prior art provides a motivation to teach the ordered combination of “The method as claimed in claim 1, wherein in reconstructing the first areas, which are not illuminated or are illuminated more weakly, a transmitted light image of the sample is taken into consideration.”
Regarding Claim 9, the cited prior art does not explicitly teach the method of Claim 9.
Chen X. (Deep-learning on-chip light-sheet microscopy enabling video-rate volumetric imaging of dynamic biological specimens, published 2021), discloses using training data with different amounts of noise to train a light sheet microscopy model (see Fig. 1 of Chen X.) However, Chen X. does not explicitly teach “a noise level of the second areas of the sample.” That is, although the prior art teaches the varying noise in light sheet microscope images, the prior art does not explicitly teach determining a noise level of more strongly illuminated areas and then generating noise based on this noise level in less strongly illuminated areas.
PNG
media_image4.png
680
763
media_image4.png
Greyscale
Fig. 1 of Chen X.
Thus, none of the previously cited references, alone or in combination, provide a motivation to teach the ordered combination of “The method as claimed in claim 1, wherein the method further comprises:
determining a noise level of the second areas of the sample; and
generating a noise in the reconstructed first areas of the sample, which are not illuminated or are more weakly illuminated using the first light sheet of the first wavelength range, on the basis of the determined noise level.”
Regarding Claim 12, the combination of Haase and Chen B. teaches “The method as claimed in claim 1, wherein radiating the first grid-shaped light sheet of the first wavelength range onto the sample and capturing the light emitted in this way are each carried out at multiple different points in time, wherein the points in time are chronologically spaced apart from one another, in particular equidistantly” (Chen B., page 3, paragraph 3 discloses “In the SIM mode, we recorded F images at every z plane as we stepped the lattice light sheet in the x direction in F equal fractions of the lattice period”),(Chen B., page 3, paragraph 3 discloses “These data were then used to reconstruct a 3D image(9) with resolution extended beyond the diffraction limit in x and z”).
Although Chen B. teaches reconstructing a 3D image from periodically recorded images from lattice light sheet illumination, Chen B. does not explicitly teach omitting a first grid shaped light sheet at one point in time and thus reconstructing an image at the omitted point in time based on other points in time. Likewise, Haase teaches a machine learning method but does not explicitly teach using a machine learning system to reconstruct an image of an omitted point in time based on light captured at other points in time. Rather, Haase is directed to using machine learning to reconstruct overall images, but does not explicitly teach machine learning to reconstruct images in a stack or series.
Thus, none of the previously cited prior art, alone or in combination, provides a motivation to teach the ordered combination of “The method as claimed in claim 1, wherein radiating the first grid-shaped light sheet of the first wavelength range onto the sample and capturing the light emitted in this way are each carried out at multiple different points in time, wherein the points in time are chronologically spaced apart from one another, in particular equidistantly, wherein at least one point in time is omitted in such a way that at the omitted point in time, radiating the first grid-shaped light sheet is omitted, and wherein an image of the sample at the omitted point in time is reconstructed on the basis of light captured at other points in time by a machine learning system.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Gao et al. (Lattice light sheet microscopy using tiling lattice light sheets, published 2019) discloses a method of tiling lattice light sheets in lattice light sheet microscopy, as in Gao (US 12663630 B2), above.
Betzig et al. (US 2019/0113731 A1) discloses a method of imaging with adaptive optics and lattice light sheet microscopy.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE TABANCAY DUFFY whose telephone number is (703)756-1859. The examiner can normally be reached Monday - Friday 8:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 5712723382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE TABANCAY DUFFY/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights