DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filed on 03/07/2025.
3. Claims 1-20 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statement (IDS) submitted on 08/05/2025 has been considered by the examiner.
Drawings
6. Figure 7 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g). See also par. [0062] of specification, “The detected cores may be registered using available libraries as shown in Fig. 7”, which indicates it is prior art, and Gatenbee (see 103 citations), pg. 4, Fig. 2. Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
8. Claim 1, 8, and 15 are objected to because of the following informalities:
In ln. 5, claim 1 recites “…bounding shapes surrounding to a continuous area…”, consider correction to “…bounding shapes surrounding a continuous area…”.
Claims 8 and 15 recite analogous limitations to claim 1 in ln. 7 and 6 respectively. Appropriate correction is required.
Claim Rejections - 35 USC § 103
9. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
10. Claims 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Virtual alignment of pathology image series for multi-gigapixel whole slide images” to Gatenbee et al. (hereinafter Gatenbee), and further in view of U.S. Publication No. 2024/0119746 to Liu et al. (hereinafter Liu).
11. Regarding Claim 1, Gatenbee discloses a computer-implemented method for analyzing electronic medical images, comprising ([pg. 1, Abstract, par. 1, ln. 1-15] “Interest in spatial omics is on the rise, but generation of highly multiplexed images remains challenging, due to cost, expertise, methodical constraints, and access to technology. An alternative approach is to register collections of whole slide images (WSI), generating spatially aligned datasets. WSI registration is a two-part problem, the first being the alignment itself and the second the application of transformations to huge multi-gigapixel images. To address both challenges, we developed Virtual Alignment of pathoLogy Image Series (VALIS), software which enables generation of highly multiplexed images by aligning any number of brightfield and/or immunofluorescent WSI, the results of which can be saved in the ome.tiff format. Benchmarking using publicly available datasets indicates VALIS provides state-of-the-art accuracy in WSI registration and 3D reconstruction. Leveraging existing open-source software tools, VALIS is written in Python, providing a free, fast, scalable, robust, and easy-to-use pipeline for registering multi-gigapixel WSI, facilitating down stream spatial analyses.”):
receiving a plurality of medical images {associated with a patient} ([pg. 1, Abstract, par. 1, ln. 1-15], [pg. 2, Fig. 1, see example of medical images provided to VALIS], [pg. 4, Fig. 2, see Convert and Mask input], [pg. 6, col. 2, Validation, par. 1, ln. 1-16] “To test the robustness and generalizability of VALIS, we performed image registration on an additional 613 samples, with images captured under a wide variety of conditions (Fig. 4). These images were collected for routine analysis, and so were not curated in any sort of way. That is, these are “real world” images that reflect the regular challenges associated with WSI registration. Each sample had between 2 and 69 images; 273 were stained using immunohistochemistry (IHC),and 340 using immunofluorescence (IF); 333 were regions of interest (ROI) or cores from tumor microarrays (TMA), while 280 were whole slide images (WSI); the original image dimensions ranged from 2656×2656 to 104,568×234,042 pixels in width and height; 162 underwent stain/ wash cycles, 451 were serial slices; 49 came from breast tumors, 109 from colorectal tumors, 156 from head and neck squamous cell carcinomas (HNSCC), and 299 from ovarian tumors. In total, this validation dataset involved registering and calculating the error for 4099 image pairs.”);
determining one or more bounding shapes for each of the plurality of medical images, each of the bounding shapes surrounding to a continuous tissue sample depicted in the plurality of medical images ([pg. 4, Fig. 2, see Convert and Mask, Serial rigid registration with neon green outline] “VALIS uses Bio-Formats and OpenSlide to read the slides and convert them to images for use in the pipeline, meaning it is compatible with 322 image formats. Once converted from slides, masks are created and applied to each image, which will focus the registration on the tissue (mask outline highlighted in green). Next, images are processed and normalized to look as similar as possible. Features are then detected in each image and then matched between all possible pairwise combinations. Feature distances are used to construct a distance matrix, which is then clustered and sorted, ordering the images such that each image should be adjacent to its most similar image. Once ordered, the matches undergo another round of filtering, wherein features used to rigidly register an image to the next image must also have been matched in the previous image. Unique matches to previous and next images are shown as purple or blue lines, while those remaining after neighbor filtering are shown as green lines. Images are then aligned serially towards the center of the image stack (or reference image, if specified),going from the inside out. Serial rigid transformations are found first, using the neighbor-filtered matches. The bounding box around the region where masks overlap or touch is used to slice out a higher resolution image for use in non-rigid registration, again performed serially from the inside out. The non-rigid transformations are accumulated as one moves towards the edges of the stack, which can bring distant features together. A final optional “micro-registration” step can be performed, which is accomplished by performing a second non-rigid registration on higher resolution non-rigidly warped images. Once registration is complete, the slides can be warped and saved in their native resolution as ome.tiff images for downstream analyses.”, [pg. 10, col. 2, Mask creation, par. 1, ln. 1 to pg. 11, col. 1, par. 1, ln. 15] “To help focus registration on the tissue, and thus avoid attempting to align background noise, VALIS generates tissue masks for each image. The underlying idea is to separate background (slide) from foreground (tissue) by calculating how dissimilar each pixel’s color is from the background color. The first step converts the image to the CAM16-UCS color space, yielding L (luminosity), A, and B channels. In the case of brightfield images, it is assumed that the background will be bright, and so the background color is the average LAB value of the pixels that have luminosities greater than 99% of all pixels. The difference to background color is then calculated as the Euclidean distance between each LAB color and the background LAB, yielding a new image, D, where larger values indicate how different in color each pixel is from the background. Otsu thresholding is then applied to D, and pixels greater than that threshold are considered foreground, yielding a binary mask. The final mask is created by using OpenCV to find, and then fill, all contours, yielding a mask that covers the tissue area. The mask can then be applied during feature detection and non-rigid registration to focus registration on the tissue.”);
based on the one or more bounding shapes, determining a plurality of levels of the tissue sample ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6] “If the order of images is unknown, they will be optimally ordered based on their feature similarity, such that the most similar images are adjacent to one another in the z-stack. This increases the chances of successful registration because each image will be aligned to one that looks very similar. This step can be skipped if the order is known (such as with 3D tissue reconstruction).”, [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10] “In order to increase the chances of successful registration, VALIS orders the images such that each image is surrounded by the two most similar images. This is accomplished by using matched features to construct a similarity matrix S, where the default similarity metric is simply the number of good matches between each pair of images. S is then standardized such the maximum similarity is 1, creating the matrix S’, which is used to create the distance matrix, D=1-S’. Hierarchical clustering is then performed on D, generating a dendrogram T. The order of images can then be inferred by optimally ordering the leaves of T, such that most similar images (i.e. leaves) are adjacent to one another in the series. This approach was validated by reordering a shuffled list of 40 serially sliced H+E images from, where the original ordering of images is known. All 40 images were correctly ordered (Supplementary Fig. 2e), indicating that this approach is capable of sorting images such that each image is neighbored by similar looking images. While VALIS can sort images based on similarity, it is also possible to align the images in a specified order, based on the filename. This option is particularly useful if the aim is to construct a 3D image by registering serial sections, as shown in Fig.6e. Once the order of images has been determined, VALIS finds the transformation matrices that will serially rigidly warp each image to an adjacent image in the stack. Specifically, images are aligned towards (not directly to) the image at the center of the series (or a reference image, if one is specified). For example, given N images, the center image is
I
(
N
2
)
. Therefore,
I
N
2
-
1
is aligned to
I
(
N
2
)
, then
I
N
2
-
2
is aligned to the rigidly warped version of
I
N
2
-
1
, and so on. While the combination of RANSAC and Tukey’s outlier detection methods remove most poor matches, VALIS performs a third match filtering step, which we refer to as neighbor match filtering. In this step, only features that are found in the image and its neighbors are considered inliers, the idea being that matches found in both neighbors reflect good tissue feature matches (Supplementary Fig.3a). That is, the features used to align image
I
i
and
I
i
-
1
are the features that
I
i
also has in common with
I
i
+
1
, and thus consequently that
I
i
-
1
also has in common with
I
i
+
1
. This approach may be thought of as using sliding window to filter out poor matches by using only features shared within an image’s neighborhood. The coordinates of the filtered matches are then used to find the trans formation matrix (
M
i
) that rigidly aligns
I
i
to
I
i
-
1
or
I
i
+
1
(depending on the position in the stack). After warping all images using their respective rigid transformation matrices, the group of images has been registered. However, one can optionally use an intensity-based method to improve the alignment between
I
i
and its neighbor. One option is to maximize Mattes mutual information between the images, while also minimizing the distance between matched features. Once optimization is complete,
M
i
will be updated to be the matrix found in this optional step. This step is optional because the improvement (if any) may be marginal (distance between features being improved by fractions of a pixel),and it is time consuming.”); and
determining a three-dimensional association of the plurality of levels ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6], [pg. 9, Fig. 6e, see original vs. registered 3D image stacks], [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10], [pg. 7, col. 2, par. 2, ln. 1-13] “Cell segmentation can be a difficult task, and it may not be desirable to re-perform that analysis on newly registered images. For this reason, VALIS also provides the functionality to warp coordinates, meaning that the registration parameters can be used to warp existing cell segmentation data (Fig. 6c). These methods could also be used to warp annotations and regions of interest coordinates. Since VALIS can register different modalities (Fig. 6d), this also makes it possible to transfer annotations from H&E images to a corresponding IF dataset. As discussed above, VALIS can also be used for 3D tissue reconstruction (Fig. 6e). Other promising areas where VALIS, and image registration in general, could be useful are, among others, aiding generation of training data for virtual staining, or aligning spatial transcriptomics images to brightfield and/or IF images.”).
Gatenbee does not specifically disclose wherein the medical images are associated with a patient.
However, Liu specifically discloses wherein the medical images are associate with a patient ([par. 0103, ln. 1-20] “To investigate the value of a computational 3D pathology workflow vs. a computational 2D pathology workflow, 300 ex vivo biopsies were extracted from archived radical prostatectomy (RP) specimens obtained from 50 patients who underwent surgery over a decade ago. The biopsies were prepared with a first labelling technique using an inexpensive small-molecule (i.e., rapidly diffusing) fluorescent analog of H&E. The first labelling technique also include optically clearing the biopsies with a dehydration and solvent-immersion protocol to render them transparent to light, and then using an open-top light-sheet (OTLS) microscopy platform to obtain whole-biopsy 3D pathology datasets. The prostate glandular network segmented using ITAS3D, from which 3D glandular features (i.e., histomorphometric parameters) and corresponding 2D features were extracted from the 118 biopsies that contained prostate cancer (PCa). These 3D and 2D features were evaluated for their ability to stratify patients based on clinical biochemical recurrence (BCR) outcomes, which serve as a proxy endpoint for aggressive vs. indolent PCa.”, [par. 0107, ln. 1-5] “Patient-level glandular features were obtained by averaging the biopsy-level features from all cancer-containing biopsies from a single patient. Patients who experienced BCR within 5 years post-RP are denoted as the “BCR” group, and all other patients are denoted as “non-BCR”.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Gatenbee and Liu as within the same field of pathology segmentation using stacked images, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, in that by associating the medical images with a patient as disclosed in Liu, you allow for the real-world application on the method of Gatenbee, (e.g., by determining cancer is present in a patient biopsy, tracking patient outcomes, determine treatment plans, etc.). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Gatenbee with the medical images associated with a patient as taught in Liu through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of Gatenbee with the medical images associated with a patient as taught in Liu such that the method of Gatenbee was performed on images associated with a patient to determine irregular biopsies (e.g., cancer present in a biopsy of a patient).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient of Liu to obtain the invention as specified in claim 1.
12. Regarding Claim 2, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee further discloses wherein the bounding shapes are determined by determining the images of a tissue sample and identifying a contour region of each tissue ([pg. 4, Fig. 2, see Convert and Mask, Serial rigid registration with neon green outline], [pg. 10, col. 2, Mask creation, par. 1, ln. 1 to pg. 11, col. 1, par. 1, ln. 15]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient of Liu to obtain the invention as specified in claim 2.
13. Regarding Claim 3, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee further discloses determining a three-dimensional association comprises determining an order of the plurality of levels of the tissue sample based on a similarity index ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6], [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10] see specifically “…S is then standardized such the maximum similarity is 1, creating the matrix S’, which is used to create the distance matrix D=1-S’. Hierarchical clustering is then performed on D, generating a dendrogram T. The order of images can then be inferred by optimally ordering the leaves of T, such that most similar images (i.e. leaves) are adjacent to one another in the series…”). Specifically, the examiner notes that this is effectively describes a similarity matrix comprising a similarity index 0-1 between each image of the stack of images, which is then used to determine the ordering. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient of Liu to obtain the invention as specified in claim 3.
14. Regarding Claim 4, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee further discloses stacking {and displaying}, based on the three-dimensional association, the plurality of levels into at least one multipanel view ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6], [pg. 9, Fig. 6c and 6e, see original vs. registered], [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10], [pg. 7, col. 2, par. 2, ln. 1-13]). Gatenbee does not specifically disclose displaying the mutlipanel view.
However, Liu specifically discloses displaying the stack to a user ([par. 0059, ln. 1-10] “…the synthetic images may be presented to a user (e.g., via display 136). In some embodiments, the synthetic images may be presented as an overlay over the captured images 164. In some embodiments, a user may be able to select how the images are displayed, and whether the synthetic or captured images are displayed. In some embodiments, a clinician or other user may receive the synthetic images and may use them as is. In some embodiments, the computing system may segment features of interest in the synthetic images.”). The motivation to combine would have been obvious to one of ordinary skill in the art, and is analogous to claim 1, in that by displaying the image you allow for the real-world application of the method of Gatenbee (e.g., determination of irregular biopsies in registered image by clinician, etc.). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Gatenbee with the medical images associated with a patient, and displaying of images as taught in Liu through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of Gatenbee with the medical images associated with a patient and displaying of images as taught in Liu such that the method of Gatenbee was performed on images associated with a patient to register the images and subsequently display the images to determine irregular biopsies (e.g., by displaying register images to a clinician).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and displaying of images of Liu to obtain the invention as specified in claim 4.
15. Regarding Claim 5, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee discloses determining a plurality of {benign} core levels ([pg. 6, col. 2, Validation, par. 1, ln. 1-16] see “…333 were regions of interest (ROI) or cores from tumor microarrays (TMA)…”); and rendering the plurality of {benign} core levels into a single image view ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6], [pg. 9, Fig. 6c and 6e, see original vs. registered], [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10], [pg. 7, col. 2, par. 2, ln. 1-13]). Gatenbee does not specifically disclose wherein the levels are related to a benign biopsy.
However, Liu specifically teaches benign core levels and an analogous stacking operation to the method of Gatenbee ([Fig. 6], [par. 0088, ln. 1-14] “The images 600 show a set of example images of benign glands 602 and example images of cancerous glands 604. The images of the cancerous glands 604 also include insets showing detailed cellular structure. Each set of images 602 and 604 includes a representative 2D image from a 3D depth stack of captured images 610 which are imaged with a first labelling technique (e.g., an H&E analog), a representative 2D synthetic image from a 3D depth stack of synthetic images 620 which predicts the appearance of the tissue sample as if it was prepared with a second labelling technique, a representative 2D image from a 3D depth stack of a segmentation mask 630 overlaid on the synthetic image, and a 3D rendering 640 based on the segmentation mask and the images.”). The motivation to combine is disclosed in Liu, wherein it allows for easier determination of different tissue within the 3D stack ([par. 0089, ln. 1-12] “As may be seen, while features such as glands are present in the original captured images 610, it may be relatively hard to determine features such as the lumen and epithelial walls, especially for an automated segmentation process. However, the segmentation masks 630 (and 3D renderings 640) generated from the synthetic images 620 clearly show such features and allow for more straightforward automated processing. In addition, clinicians may be used to the appearance of images in the format of the synthetic images 620, and may be more comfortable analyzing such images than if the segmentation mask 630 was developed directly from the captured images 610.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Gatenbee with the medical images associated with a patient as taught and benign core levels in Liu through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of Gatenbee with the medical images associated with a patient as taught and benign core levels in Liu such that the method of Gatenbee was performed on images associated with a patient that contain benign tumors to determine irregular biopsies.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and benign core levels of Liu to obtain the invention as specified in claim 5.
16. Regarding Claim 6, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee discloses, wherein the plurality of medical images come from a single tissue that has been restained ([pg. 8, Fig 5c], [pg. 8, col. 1, par. 2, ln. 1 to pg. 9, col. 1, par. 1, ln. 11] “In addition to other applications that can benefit from image registration (e.g.,using annotations across multiple images, retrieving regions of interest in multiple images, warping cell coordinates, etc…), VALIS also makes it possible to construct highly multiplexed images from collections multi-gigapixel IF and/or IHC WSI. However, a challenge of constructing multiplexed images by combining stain/wash cycling and image registration is that multiple cycles eventually degrade the tissue and staining quality can decrease because antibody binding weakens as the number of cycles increases. It has been estimated that t-CyCIF images can reliable undergo 8-10 stain/wash cycles, and possibly up to 20 in some cases. In our examples, we use four markers per cycle (including DAPI), suggesting one could construct a 24-60 plex image using a similar protocol. We suspect this number will increase as technological advances staining protocols are made. To maximize the number of markers used to generate multiplexed images within image registration, one can conduct experiments wherein each individual antibody undergoes multiple stain/wash cycles, measuring antibody sensitivity after each repeat. One can then use the results of theses experiments to determine how many cycles an antibody can undergo while maintaining sensitivity. With this data in hand, one can order the staining sequence such that weaker stains are used first, and more robust stains are used in the later cycles. Such an approach should help maximize the number of markers one can stain for. Using this approach, we were able to find staining sequences that allows between 13-20 IHC stain/wash cycles per tissue slice (HNSCC data in Fig. 5c). The impacts of stain/wash cycles on stain quality does not appear to affect registration accuracy. The CyCIF images, which underwent 11 cycles, had very low registration error, the maximum being estimated at 6μm.Likewise, most samples in the HNSCC panels had similar error distributions, despite each undergoing differing numbers of IHC staining rounds, being between 13 and 20 rounds depending on the staining panel (HNSCC data in Fig. 5c). These results did not include the micro-registration step, which should bring the error even lower. These experiments suggest stain/wash cycling has little impact on registration accuracy, and that VALIS is able to “fix” tissue deformations that can occur during repeated washing.”), and wherein the plurality of medical images are stacked to identify the same region across the plurality of medical images ([pg. 4, Fig. 2, see Sort step], [pg. 4, col. 1, Overview, par. 1, listing 5, ln. 1-6], [pg. 9, Fig. 6a, b, and d], [pg. 11, col. 1, Rigid Registration, par. 3, ln. 1 to col. 2, par. 2, ln. 10], [pg. 7, col. 2, par. 2, ln. 1-13]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient of Liu to obtain the invention as specified in claim 6.
17. Regarding Claim 7, a combination of Gatenbee and Liu teaches the method of claim 1. Gatenbee further discloses wherein the bounding shapes comprise boxes ([pg. 4, Fig. 2, see Higher resolution serial non-rigid registration], [pg. 4, col. 2, par. 1, listing 7, ln. 1-9] “The masks are rigidly warped and combined to create a non-rigid registration mask. The bounding box of this mask is then used to extract higher resolution versions of the tissue from each slide. The higher resolution images are then re-processed and used for non-rigid registration, which is performed either by: aligning each image towards (or to) the reference image following the same sequence used during rigid registration (the default); using groupwise registration that non-rigidly aligns the images to a common frame of reference.”, [pg. 11, col. 2, Non-Rigid Registration, par. 1, ln. 1 to par. 2, ln. 18] “Non-rigid registration involves finding 2D displacement fields, X and Y, that warp a “moving” image to align with a “fixed” image by optimizing a metric. As the displacement fields are non-uniform, they can warp the image such that local features align better than they would with a single global rigid transformation. However, these methods require that the images provided are already somewhat aligned. Therefore, once VALIS has rigidly registered the images, they can be passed onto one to a non-rigid registration method. Recall that rigid registration is performed on low resolution copies of the full image. However, it may be that the tissue to be aligned makes up only a small part of this image, and thus the tissue is at an overly low resolution (Fig. 2). However, the lack of detailed alignment can be overcome during the non-rigid registration step, which can be performed using higher resolution images. This is accomplished by creating a non-rigid registration mask, which is constructed by combining all rigidly aligned image masks, keeping only the areas where all masks overlap and/or where a mask touches at least one other mask. The bounding box of this non-rigid mask can then be used to slice out the tissue at a higher resolution from the original image (Fig. 2, additional examples in Supplementary Fig. 3c). These higher resolution images are then processed and normalized as before, warped with the rigid transformation parameters, and then used for non-rigid registration. This approach makes it possible to non rigidly register the images at higher resolution without loading the entire high-resolution image into memory, increasing accuracy with low additional computational cost (due to re-processing the image).”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient of Liu to obtain the invention as specified in claim 7.
18. Regarding Claim 8, the claim language is analogous to claim 1, with the exception of “A system for analyzing electronic medical images, the system comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising…” wherein the remainder of the claim is analogous to claim 1. Gatenbee does not specifically disclose a system, memory, or processor. However, Liu specifically teaches a system for analyzing electronic medical images, the system comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations of their method ([par. 0030, ln. 1-12] “The computing system 130 may include one or more of a processor 132 which executes various operations in the computing system 130, a controller 134 which may send and receive signals to operate the microscope 102 and/or any other devices based on instructions from the processor 132, a display 136 which presents information to a user, an interface 138 which allows a user to operate the computing system 130, and a communications module 139 which may send and receive data (e.g., images from the detector 112). The computing system 130 includes a memory 140, which includes various instructions 150 which may be executed by the processor 132.”). The motivation to combine remains analogous to claim 1. Specifically, by incorporating a system analogous to Liu to implement the method of Gatenbee, it allows or the real-world implementation of the method of Gatenbee for biopsies. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Gatenebee with the medical images associated with patients and system of Liu through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have combined the method of Gatenbee with the medical images associated with a patient as taught in Liu such that the method of Gatenbee was performed on a system analogous to the system of Liu and on images associated with a patient to determine irregular biopsies (e.g., cancer present in a biopsy of a patient).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and system of Liu to obtain the invention as specified in claim 8.
19. Regarding Claims 9-14, a combination of Gatenbee and Liu teaches the system of claim 8. The claim language of claims 9-14 is analogous to claims 2-7 respectively, and rejections analogous to claims 2-7 are further applicable to claims 9-14 in view of the system of the combination of Gatenbee and Liu. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and system of Liu to obtain the invention as specified in claims 9-14.
20. Regarding Claim 15, the claim language is analogous to claim 1, with the exception of “A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations analyzing electronic medical images, the operations comprising…” wherein the remainder of the claim is analogous to claim 1. Gatenbee does not specifically disclose a non-transitory computer-readable storage medium. However, Liu teaches a non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations analyzing electronic medical images, the operations comprising their method ([par. 0030, ln. 1-12]). Specifically, given that the system of Liu describes a hardware implementation, one of ordinary skill in the art would recognize the memory 140 to be non-transitory in nature (e.g., RAM, ROM, hard drive, SSD, etc.). The motivation to combine remains analogous to claim 8. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and non-transitory computer-readable medium of Liu to obtain the invention as specified in claim 15.
21. Regarding Claims 16-20, a combination of Gatenbee and Liu teaches the non-transitory medium of claim 15. The claim language of claims 16-20 is analogous to claims 2-6 respectively, and rejections analogous to claims 2-7 are further applicable to claims 16-20 in view of the non-transitory medium of the combination of Gatenbee and Liu. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Gatenbee with the medical images associated with a patient and non-transitory computer-readable medium of Liu to obtain the invention as specified in claims 16-20.
Conclusion
22. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights