Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
ETAILED ACTION
This action is in response to the applicant's communication filed on 06/28/2024. In virtue of this communication, claims 1, 4, 10, 17-33 filed on 06/28/2024 are currently pending in the instant application.
Claims 2-3, 5-9, 11-16, and 34-48 have been cancelled in preliminary amendment filed on 06/08/2024.
Information Disclosure Statement
The information Disclosure statement (IDS) form PTO-1449, filed on 06/28/2024 are in compliance with the provisions of CFR 1.97. Accordingly, the information disclosed therein was considered by the examiner.
Drawings
The drawings were received on 06/28/2024 have been reviewed by Examiner and they are acceptable.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4, 10, 17-21, 23-33 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent Claims 1, 17 and 33 recites receiving first imaging data associated with a sample, the first imaging data acquired using a first imaging modality; receiving second imaging data associated with the sample, the second imaging data acquired using a second imaging modality that is different than the first imaging modality; aligning the first imaging data and the second imaging data; identifying a portion of the first imaging data overlapping with the second imaging data; training a neural network based at least in part on the portion of the first imaging data and the second imaging data; and generating output imaging data by applying the trained neural network to the first imaging data or a third imaging data acquired using the first imaging modality.
Step 1:
With regard to Step 1, the instant claims are directed to an apparatus, a method, and a non-transitory computer-readable medium, all among the statutory categories of invention.
Step 2A — Prong 1:
With regard to Step 2A — Prong 1, for example in method Claim 17, the limitations “receiving first imaging data associated with a sample, the first imaging data acquired using a first imaging modality; receiving second imaging data associated with the sample, the second imaging data acquired using a second imaging modality that is different than the first imaging modality; aligning the first imaging data and the second imaging data; identifying a portion of the first imaging data overlapping with the second imaging data; training a neural network based at least in part on the portion of the first imaging data and the second imaging data; and generating output imaging data by applying the trained neural network to the first imaging data or a third imaging data acquired using the first imaging modality.”, as recited, is a method that, under its broadest reasonable interpretation, covers performance of the limitation in the mind/observation of a person inspecting two different images/pictures of an environment and identifying the similar portions of the second one to the first one. That is, other than reciting “by a computer" nothing in the claim steps preclude the limitations from practically being performed in the mind or through observation of a person inspecting an image of an environment. The recited computer is simply a generic device. If a claim limitation, under its broadest reasonably interpretation covers performance of the limitation in the mind but for the recitation of a generic components, then it falls within the "Mental processes" grouping of the abstract idea, which include concepts performed in the human mind, including an observation, evaluation, judgement, opinion. Accordingly, the claim recites an abstract idea. In addition, the additional components recited in independent Claims 1 and 33, i.e., a memory, a processor, and a non-transitory computer-readable medium are simply generic computing components, accordingly, these independent claims include the above- described abstract idea.
Step 2A — Prong 2:
The 2019 PEG defines the phrase “integration into a practical application’ to require an additional element or a combination of additional elements in the claim
to apply, rely on, or use the judicial exception. In the instant case, the additional elements in the claims do not apply, rely on, or use the judicial exception.
This judicial exception is not integrated into a practical application because the claims only recite additional elements using a computer, a memory, a processor, or a non-transitory Computer-readable medium, for instance, that includes to perform the recited elements/functions/steps. The additional recitation of a neural network merely limit the abstract idea to a particular technological environment and do not recite a specific improvement to neural network architecture and training procedure. These computing components in all are recited at high-level of generality and there are no other recited additional limitations in the claims. Accordingly, these additional steps/element do not integrate the abstract idea into a practical application because it is a field-of-use limitation that does not impose any meaningful limits on practicing the abstract idea. Therefore, independent Claims 1, 17, and 33 recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as discussed above with respect to integration of the abstract idea into practical application, the additional element of using a computer, a memory, a processor, or a non-transitory computer- readable medium to execute programming instructions to perform the step amounts to no more than mere instructions to apply the exception using a generic apparatus component. The additional recitation of a neural network merely limit the abstract idea to a particular technological environment and do not recite a specific improvement to neural network architecture and training procedure. These computing components in all are recited at high-level of generality. Mere instructions to apply an exception using generic apparatus component cannot provide an inventive concept. The claim is not patent eligible.
Further, with regard to dependent Claims 4, 10, 18-21, 23-32 viewed individually, these additional elements are under their broadest reasonable interpretation, cover performance of the limitation in the mind and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Accordingly, Claims 1, 4, 10, 17-21, 23-33 are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 17, 22, 25, 27, and 31-33 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson et al; “Multimodal imaging and machine learning to enhance microscope images of shale”; Computers & Geosciences 145 (2020), further in view of Ozcan et al. (US 2021/0264214), hereinafter Ozcan (214).
As per claim 1, A system, comprising: one or more data processors; and a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:“ receiving first imaging data associated with a sample, the first imaging data acquired using a first imaging modality;” (Anderson, page 1, abstract, Col. 2 discloses Vaza Muerta shale sample using Transmission X-ray Microscopy (TXM), TXM is an acronym used within the synchrotron X-ray community and the technique is also referred to as ‘‘nano-CT’’. further page 3, Col. 1 section 3.1 discloses TXM acquisition, then reconstruction resulting in a stack of 419 images with isometric 3D resolution of 31.2 nm/px.)
“receiving second imaging data associated with the sample, the second imaging data acquired using a second imaging modality that is different than the first imaging modality;”(Anderson page 1, abstract, Col. 2, discloses imaging the same sample focused on Ion Beam-Scanning Electron Microscopy (FIB-SEM). Further page 3, section 3.1 discloses a total of 149 FIB-SEM images were acquired to create a sub-volume of the TXM image stack. )
“aligning the first imaging data and the second imaging data;”(Anderson, page 1, Abstract discloses aligning cross-sectional images form each modality (TXM and FEB-SEM) using manual and automated registration. Page 4, Col. 1, description of fig. 4, last paragraph discloses the cross-sections show (a) the registered and aligned TXM image (b) the original FIB-SEM image. see section 3.4disclsoes using image matching, similarity metrics, TXM plane-rotation and registration to final TXM images )
“identifying a portion of the first imaging data matching with the second imaging data;”(Anderson, page3, Col. 1 discloses FIB-SEM images create a sub-volume of TXM image stack, and matching process of TXM images for each FIB-SEM image and in Step, 3.4 discloses after correction, the SEM images were registered to final matched 149 pairs of TXM images.)
“training a neural network based at least in part on the portion of the first imaging data and the second imaging data;”(Anderson, Page 5, Col. 1, section 4.1 discloses The 149 contiguous image stack slices are split into 5 validation, 119 training, and 20 test set slices. Then next paragraph discloses training sampled image patches. Further page 6, Col. 2, discloses image-to-image CNNs mapping input image domain to predicated target output image domain and using pix2pix cGAN model for the TXM to SEM problem using image pairs. Then page 7, Col. 1 discloses cGAN training with TXM/SEM image pair and generation of Synthetic SEM images. )
“and generating output imaging data by applying the trained neural network to the first imaging data or a third imaging data that is acquired using the first imaging modality.” (Anderson, abstract discloses prediction of image cross sections with SEM-like resolution from nondestructive TXM data, page 6, Col. 2 discloses CNN mapping from input image domain to predicated target output image domain, page 7, Col. 1 discloses the cGAN generator creating synthetic SEM images form TXM input. Page 9, Figure 8, discloses Image translation results on test set images (a) and (b) that were not used for training or validation. Page 12, Col. 1, section 7. discloses enhancing images from nondestructive TXM to SEM-like quality. using aligned dataset to train image-to-image translation models.)
However Anderson does not explicitly disclose the following which would have been obvious in view of Ozcan from similar filed of endeavor “ identifying a portion of the first imaging data overlapping with second image data” (Ozcan(214), ¶[0046] discloses performing an accurate image registration, between the two imaging modalities (QPI and brightfield), which involves both global matching and local alignment steps. it is crucial to accurately align the FOVs for each input and target image pair in the dataset.¶ [0047] The first step is to find a roughly matched FOV between QPI and the corresponding brightfield image. then a correlation score matrix is calculated. the image with the highest correlation score indicates a match between the two images. the corresponding brightfield image is cropped out from the WS. ¶[0048] discloses performing rotation, aberrations, or small perturbations t between these coarsely matched image pairs. The result of this digital procedure is an affine transformation matrix, which is applied to the brightfield microscope image patch, to match it with the quantitative phase image of the same sample. further cropping of 64 pixels on each side to the aligned image pairs. ¶[0051] discloses these accurately aligned fields-of-view were partitioned to overlapping patches, which were then used to train the GAN-based deep neural network.¶[0058] discloses For the patch generation, data augmentation was applied by using 50% patch overlap for the liver and skin tissue images, and 25% patch overlap for the kidney tissue images.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(214) technique of digital staining microscope images using deep learning into Anderson technique to provide the known and expected uses and benefits of Ozcan(214) technique over multimodal imaging and machine learning technique of Anderson. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(214) to Anderson in order to reveal accurate cellular and sub-cellular morphological information of a sample under brightfield microscopy. (Refer to Ozcan(214) paragraph [0003].)
Claims 17 and 33 have been analyzed and are rejected for the reasons indicated in claim 1 above.
As per claim 22, The method of claim 17, “wherein the first imaging modality includes a first major imaging modality that is X-ray microscopy, and wherein the second imaging modality includes a second major imaging modality that is not X-ray microscopy.” (Anderson, Abstract and page 1, Col. 2, discloses X-ray Microscopy (TXM) is nondestructive and preserves the sample for further tests and characterization.
TXM is an acronym used within the synchrotron X-ray community and the technique is also referred to as ‘‘nano-CT’’. further discloses destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM).)
As per claim 30, The method of claim 17, “further comprising generating a two-dimensional image or three-dimensional volume using the output imaging data.” (Anderson, page 12, Col.1, discloses an image-processing workflow that is capable of enhancing images from nondestructive TXM to SEM-like quality. Col.2 discloses The aligned dataset was then used to train 2D image-to-image translation models. )
As per claim 31, The method of claim 30, “wherein the generated two-dimensional image or three- dimensional volume is generated using the output imaging data and the first imaging data such that at least a region of the two-dimensional image or three-dimensional volume is based on the output imaging data and at least a different region of the two-dimensional image or three- dimensional volume is based on the first imaging data.”(Anderson, page 9, Col. 2 and page 10, Col.1, discloses using a charge mask to windowing the generated repaired image and adding the result to the uncharged portion of the original image. sample TXM and SEM image patches and a charge mask used for repairing charged regions. )
As per claim 32, The method of claim 17, “wherein generating the output imaging data includes applying the trained neural network to the first imaging data.” (Ozcan(214), ¶[0006] discloses a microscopy method for label-free samples (e.g., tissue sections) includes the operations of providing a trained deep neural network that is executed by image processing software using one or more processors. A quantitative phase microscopy image of the label-free tissue section is obtained and input to the trained deep neural network. The trained deep neural network outputs a digitally stained output image (e.g., a virtually stained image) of the sample (e.g., tissue section) that is substantially equivalent to a corresponding brightfield image of the same sample that has been chemically stained or otherwise labeled. ¶[0032] discloses the newly acquired quantitative phase images are blindly fed to the pre-trained deep neural network to output a digitally-stained output image.)
As per claim 25, The method of claim 17, “wherein training the neural network includes: providing the portion of the first imaging data associated with the second imaging data to the neural network to generate the output data; and adjusting the neural network to reduce a difference between the generated output data and the second imaging data.” (Anderson, page 6, Col. 2, discloses training image to image CNNs to map an image form the input TXM images to predicted output SEM images in target image domain, and minimizing the L1 norm between the prediction and target image. )
As per claim 27, The method of claim 17, “wherein generating the output imaging data includes: segmenting the first imaging data into a plurality of chunks, wherein each chunk includes at least one overlapping region with an adjacent chunk;”(Ozcan(214), ¶[0051] discloses registered aligned fields-of-view were partitioned to overlapping patches of 256×256 pixels to train the GAN-based deep neural network. ¶[0058] discloses 50% patch overlap for the liver and skin tissue images, and 25% patch overlap for the kidney tissue images.)
“applying each of the plurality of chunks of the first imaging data to the trained neural network to generate a corresponding chunk of the output imaging data; and stitching together the overlapping regions of the corresponding chunks of the output imaging data.”( Ozcan(214), ¶[0061] discloses deep neural network 10 was tested with four image patches of 1792×1792 pixels with an overlap of ˜7%. The outputs of the network were then stitched to form the final network output image.)
Claim(s) 4, 10, 18-21, 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson et al; “Multimodal imaging and machine learning to enhance microscope images of shale”; Computers & Geosciences 145 (2020), in view of Ozcan et al. (US 2021/0264214), hereinafter Ozcan (214), further in view of Ozcan et al. (US 2019/0333199), hereinafter Ozcan (199).
As per claim 4, The system of claim 1, “wherein the first imaging data is acquired at a first resolution, wherein the second imaging data is acquired at a second resolution”(Anderson, page2, Col. 2 discloses we take the approach of predicting SEM images directly from TXM data to combine the non-destructive nature of TXM imaging with the image resolution and enhanced contrast of FIB-SEM. Predicting shale SEM images from TXM data can be described as an image translation and super-resolution problem. Image super resolution algorithms create a high resolution image from low resolution data.)
However OweverHAnderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Ozcan (199) from similar field of endeavor “wherein the first imaging data is acquired at a first resolution and first field of view, wherein the second imaging data is acquired at a second resolution that is higher than the first resolution and a second field of view that is smaller than the first field of view, and wherein the output imaging data has a higher resolution than the first resolution.” (Ozcan (199), ¶[0080] discloses The lower resolution images 20′ were acquired with a 40×/0.95 NA objective lens providing a FOV of 150 μm×150 μm per image, while the higher resolution training images 50 were acquired with a 100×/1.4 NA oil-immersion objective lens providing a FOV of 60 μm×60 μm per image, i.e., 6.25-fold smaller in area. Both the low-resolution 20′ and high-resolution images 50 were acquired with 0.55-NA condenser illumination leading to a diffraction limited resolution of ˜0.36 μm and 0.28 μm, respectively, both of which were adequately sampled by the image sensor chip, with an ‘effective’ pixel size of ˜0.18 μm and ˜0.07 μm. ¶[0082] discloses After this training procedure the deep neural network output high resolution images 40 of samples 22 of any type from a low resolution input image 20. Further ¶[0086] discloses a deep neural network 10 was used to transform lower resolution images of these tissue samples 22 into higher resolution ones 40, also showing significant enhancement in FOV and DOF of the output images.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(199) technique of deep learning microscopy into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Ozcan(199) technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(199) to Anderson as modified by Ozcan(214) in order to generate more accurate models with lower computational cost. (Refer to Ozcan(199) paragraph [0003].)
Claim 20 has been analyzed and is rejected for the reasons indicated in claim 4 above.
As per claim 10, The system of claim 1, Anderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Ozcan (199) from similar field of endeavor “wherein generating the output imaging data includes: applying a zoom transform to the first imaging data to achieve an effective pixel size equal to the pixel size of the second imaging data;” (Ozcan (199), ¶[0065] discloses up sampled network input images, ¶[0066-0067] discloses Pixel size per image is 7.1 nm; the input image is up-sampled by a factor of 2. ¶[0164] disclose The LR images are then linearly interpolated two times to match the effective pixel size of the HR images. ¶[0198] discloses the low-resolution images 20′ were taken at a magnification of 10000× (14.2 nm pixel size), while the high-resolution images 50 were taken at 20000× magnification (7.1 nm pixel size.).¶[0199] discloses the SEM low-resolution images are up-sampled using a Lanczos filter. Up-sampling and interpolation(zoom transfer) )
“applying the transformed first imaging data to the trained neural network to generate the output imaging data.” (Ozcan (199), ¶[0205] discloses the technique enhances the resolution of lower magnification SEM images 20 such that the network's output images 40 accurately matches the resolution given by the higher resolution SEM images ¶ [0208], discloses the electron microscopy input image 20 is obtained using an electron microscope 110. The trained deep neural network 10 outputs or generates an output image 40 that has improved resolution as compared to the input image. )
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(199) technique of deep learning microscopy into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Ozcan(199) technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(199) to Anderson as modified by Ozcan(214) in order to generate more accurate models with lower computational cost. (Refer to Ozcan(199) paragraph [0003].)
Claim 26 has been analyzed and is rejected for the reasons indicated in claim 10 above.
As per claim 18, The method of claim 17, Anderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Ozcan (199) from similar field of endeavor “wherein the first imaging data is associated with a first image quality,”( Ozcan (199), ¶[0073] discloses “lower” quality training images (or patches of images) 20′ that are obtained with the microscopy device.)
wherein the second imaging data is associated with a second image quality,”( Ozcan (199), ¶[0073] discloses their corresponding high-resolution “gold standard” images 50. These gold standard or label images 50 are used to train the deep neural network 10 and may be obtained using the same microscopy device 110 but at a higher resolution or setting.)
“and wherein the output imaging data is associated with a third image quality that is improved with respect to the first image quality.”( Ozcan (199), ¶[0071] discloses outputs or generates an “improved” output image that has improved one or more of resolution, depth-of-field, signal-to-noise ratio, and/or contrast .)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(199) technique of deep learning microscopy into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Ozcan(199) technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(199) to Anderson as modified by Ozcan(214) in order to generate more accurate models with lower computational cost. (Refer to Ozcan(199) paragraph [0003].)
As per claim 19, The method of claim 18, “wherein the first image quality is associated with a first objective measurement,” (Ozcan(199), Ozcan(199), ¶[0073] discloses the “lower” quality training images (or patches of images) 20′ that are obtained with the microscopy device , then ¶[0109], table 1 discloses bicubic up-sampling SSIM value of0.672 and 0.685 corresponding to the lower-resolution first imaging data). wherein the second image quality is associated with a second objective measurement,” (Ozcan(199), ¶[0073] discloses their corresponding high-resolution “gold standard” images 50. These gold standard or label images 50 are used to train the deep neural network 10 and may be obtained using the same microscopy device 110 but at a higher resolution or setting. ¶[0109], discloses SSIM is defined as 1 for an image that is identical to the gold standard image. so the high resolution gold standard second imaging data has SSIM=1 when measured against itself. )
“and wherein the third image quality is associated with a third objective measurement that is i) between the first objective measurement and the second objective measurement, or ii) the same as the second objective measurement.”(Ozcan (199), discloses ¶[0109], and table 1, discloses deep neural network output SSIM values of 0.796 and 0.806, which are between the first bicubic up-sampling SSIM values of 0.672 and 0.685 and second gold standard SSIM value of 1.)
As per claim 21, The method of claim 17, Anderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Ozcan (199) from similar field of endeavor “wherein the first imaging modality and the second imaging modality share a common major imaging modality.” (Ozcan (199), ¶[0079] discloses the same deep learning framework as brightfield microscopy images is applicable to other microscopy modalities, including e.g., holography, dark-field, fluorescence, multi-photon, optical coherence tomography, coherent microscopy, confocal microscopy ¶[0080] discloses The lower resolution images 20′ were acquired with a 40×/0.95 NA objective lens.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(199) technique of deep learning microscopy into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Ozcan(199) technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(199) to Anderson as modified by Ozcan(214) in order to generate more accurate models with lower computational cost. (Refer to Ozcan(199) paragraph [0003].)
As per claim 24, The method of claim 17, “further comprising resampling at least a portion of the first imaging data.” (Ozcan(214), ¶[0047] discloses bicubic down-sampling 49 the whole slide image (WSI) to match the pixel size of the phase retrieved image. ¶[0048] discloses an affine transformation matrix, which is applied to the brightfield microscope image patch, to match it with the quantitative phase image of the same sample.)
Anderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Ozcan (199) from similar field of endeavor “resampling into a grid of the second imaging data”(Ozcan(199), ¶[0100] discloses resampling of these 75 (M×M) pixels channels to three channels with (M×5)×(M×5) pixels grid.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Ozcan(199) technique of deep learning microscopy into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Ozcan(199) technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Ozcan(199) to Anderson as modified by Ozcan(214) in order to generate more accurate models with lower computational cost. (Refer to Ozcan(199) paragraph [0003].)
Claim(s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson et al; “Multimodal imaging and machine learning to enhance microscope images of shale”; Computers & Geosciences 145 (2020), in view of Ozcan et al. (US 2021/0264214), hereinafter Ozcan (214), further in view of Siewerdsen et al.( US 2017/0238897)
As per claim 23, The method of claim 17, “wherein aligning the first imaging data and the second imaging data” (Anderson, page3, The matching process started with the FIB-SEM images being filtered, rotated, padded (black pixels added at the edges of the images), scaled, and resampled at TXM resolution. page 4, Col. 1 , fig. 4, discloses registered and aligned TXM image and the original FIBSEM image.)
“includes automatically sub-pixel alignment to at least one of the first imaging data and the second imaging data to align the first imaging data and the second imaging data.” (Ozcan (214), ¶[0043] discloses laterally shift on a 6×6 grid with sub-pixel spacing at each sample-to-sensor distance. ¶[0046] An important step in the training process of the deep neural network 10 is to perform an accurate image registration, between the two imaging modalities (QPI and brightfield), which involves both global matching and local alignment steps as illustrated in FIG. 7. Since the network 10 aims to learn the transformation from a label-free phase retrieved image 21 to a histochemically-stained brightfield image 48, it is crucial to accurately align the FOVs for each input and target image pair in the dataset .)
Anderson as modified by Ozcan (214) does not explicitly disclose the following which would have been obvious sin view of Siewerdsen from similar field of endeavor “applying a 3, 6, or 9 degree of freedom sub-pixel alignment ” (Siewerdsen, applying a 9-degree-of-freedom geometric calibration of the selected projection. Additionally the method includes generating a projection matrix from the 9-degree-of-freedom geometric calibration of the selected projection. ¶[0049] discloses Each projection was independently self-calibrated using both 6-DOF and 9-DOF 3D-2D registration. )
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Siewerdsen technique of volumetric image reconstruction into Anderson as modified by Ozcan(214) technique to provide the known and expected uses and benefits of Siewerdsen technique over multimodal imaging and machine learning technique of Anderson as modified by Ozcan(214). The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Siewerdsen to Anderson as modified by Ozcan(214) in order to improve image quality without workflow complexity and interruption. (Refer to Siewerdsen paragraph [0007].)
Allowable Subject Matter
Claims 28-29 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and on the pending conditions of the rejected and objected matter set forth in this action.
The following is a statement of reasons for the indication of allowable subject matter: the prior art of record, alone or in combination, fails to teach or suggest the limitations set forth by each of claims 28-29.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAGHAYEGH AZIMA whose telephone number is (571)272-1459. The examiner can normally be reached Monday-Friday, 9:30-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAGHAYEGH AZIMA/Examiner, Art Unit 2671
Prosecution Insights