DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to communications filed on 03/30/2026. Claims 1-26 are pending and have been examined.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Information Disclosure Statement
The information disclosure statement (IDS) submitted was filed on 01/06/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted was filed on 02/19/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted was filed on 02/24/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted was filed on 05/19/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1 and 15 are objected to because of the following informalities:
As per claim 1, it appears that the semicolon in “…obtained;” in line 8 should be replaced with a comma (this similarly applies to claim 15). It appears that “display…the one or more additional objects characteristic comprises one of…” in the 8th/7th to last line should be replaced with “display…the one or more additional objects, wherein the characteristic comprises one of…” (see claim 15 and previous claim 1 dated 08/12/2024).
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-26 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 is amended to recite “selecting pixels of a merged image for each tile based on the tile of plane values, (d) tiling the merged images for each tile to generate a plurality of respective swaths”. However, the specification does not support the above features. For example, the specification describes “the stack is then converted into a single plane merged focus image by comparing each pixel with adjacent pixels in the same plane, and selecting the pixel of the plane with the best focus quality using an algorithm for determining pixel focus having a focus metric” (e.g. in paragraph 150), and “obtain a merged image of the entire specimen… Once all of the sub-region images (e.g., 14 sub-regions) [referring to the 14 focal planes of figure 25C] for a tile have been captured, the merged image is calculated… The pixels of the merged image are then selected based on the associated plane values. The merged images are then tiled together to generate a swath” (e.g. in paragraph 152), but it does not describe “the merged images for each tile”. It is noted that “The merged images” in the specification does not actually have antecedent basis (note antecedent basis issues with the claim as well; see 112(b) rejection below; as understood by examiner given paragraphs 150-152 in light of figure 25C, applicant may have intended “The merged images” to refer to images that are merged to form “a merged image of the entire specimen”, such as images corresponding to focal planes of figure 25C, but this is confusing as “a merged image of the entire specimen” implies that “merged image” refers to an image after merging). Moreover, as noted above, the specification states “for a tile have been captured, the merged image is calculated”. Hence, at best, there is a merged image for each tile, not “merged images for each tile”. As such, the claim lacks written description. At best, this is misleading. Independent claim 15 includes the same limitations, and therefore also lack written description. Dependent claims 2-14 and 16-26 incorporate features of respective independent claims and thus also lack written description.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per claim 1, it is unclear what “its” is referring to in line 17. There is lack of antecedent basis for “the pixels” in line 19. There is lack of antecedent basis for “the merged images for each tile” in line 22. It is unclear whether “the swaths” in line 23 is referring to the “plurality of respective swaths”, or is different. As such, the claim is indefinite. Independent claim 15 includes the same limitations, and therefore is also indefinite. Dependent claims 2-14 and 16-26 incorporate features of respective independent claims and thus are also indefinite.
Response to Arguments
Previous objections to the claims not included in this action have been withdrawn in view of amendments.
Previous rejections under 35 USC 112 not included in this action have been withdrawn in view of amendments.
Applicant’s arguments with respect to newly amended features have been considered but are moot in view of new grounds of rejection. See combination including Kuwabara (US 20050057577 A1) below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 8-15, and 21-25 are rejected under 35 U.S.C. 103 as being unpatentable over Takayama (US 20140184778 A1) in view of Gahm et al. (US 20030210262 A1), Damaskinos et al. (US 20140125776 A1), Maddison (US 20060038144 A1), Zhang (US 6518554 B1), Kuwabara (US 20050057577 A1) and Kincaid et al. (US 20100128988 A1).
As per independent claim 1, Takayama teaches a system for reviewing a whole specimen image, the system comprising:
an automated slide imaging system, the automated slide imaging system comprising an imager configured for acquiring images of a specimen affixed to a surface of a slide (e.g. in paragraphs 8-9, 57-58, and 67-69, “an image processing apparatus configured to generate a display image used to display on a display apparatus a captured image captured by imaging a slide on which a specimen is placed by an imaging apparatus… acquisition unit configured to acquire an overall image generated from the captured image for displaying the entirety of the slide and a magnified image generated from the captured image for displaying a portion of the specimen in a magnified manner… digital camera… imaging sensor… converts an analog signal output by the image pickup sensor 208 into a digital signal”), the specimen comprising a plurality of objects distributed within a three-dimensional volume, and for generating a whole specimen image from the acquired images (e.g. in paragraphs 11, 65, 110, and 273, “generate a display image containing the overall image and the magnified image… slide 206 is a piece prepared by placing a slice of tissue or a smear of cells to be observed on a slide glass and fixing it under a cover glass with mounting agent… the same specimen 505 is illustrated in different sizes in the respective layers of images to facilitate the understanding of the layered structure… three dimensional specimen”), the whole specimen image comprising a plurality of objects depicted in focus regardless of individual locations of the respective objects within an original three-dimensional specimen volume from which the whole specimen image was obtained (e.g. in paragraphs 225 and 228, “a plurality of images captured by imaging the specimen with the same imaging area at…a plurality of focusing positions”, each image having “having a high focusing quality in the region”, i.e. in focus regardless of location);
wherein the imager comprises a digital camera (e.g. in paragraphs 58, 67-69, and 82, “digital camera… converts an analog signal output by the image pickup sensor 208 into a digital signal… processing of stitching divisional images captured by divisional stitching by the imaging apparatus 101 to generate a single image”); and
a review station including a display monitor, a user interface, and a processor operatively coupled with the respective display monitor and user interface (e.g. in paragraphs 57, 59, and 227, “a display apparatus 103… a keyboard and/or mouse or the like, which is used by the user to input various commands to the image processing apparatus… CPU” and figure 24A), wherein the processor is configured to display the whole specimen image on the display monitor (e.g. in paragraph 116, “generates display data for image presentation, using the multi-layer image data and GUI data…and outputs the display data to the display apparatus 103”, and figure 24A); and
wherein the system is configured to allow a system user to review the whole specimen image and, based on the review, use the user interface to characterize the whole specimen image using the user interface (e.g. in paragraphs 162 and 182, “nuclear enlargement and disordered cell arrangement in an area suspected to be cancerous… shows such characteristic features”),
but does not specifically teach along with separate images of individual objects within the specimen image, the imager comprising a digital camera having an optical axis at a non-orthogonal angle with the slide and configured to acquire a plurality of micro images of the specimen at a same z-axis of the three-dimensional volume; an image processor configured to generate the whole specimen image from the micro images by a process comprising: (a) dividing the micro images into a plurality of sub-regions representing different focal planes across a specimen depth for each tile captured by the digital camera; (b) vertically reassembling the sub-regions along the z-axis to make a through- focus stack for each tile, (c) converting the through-focus stack for each tile into a single plane merged image by calculating a difference between each pixel and pixels in its neighboring region, weighting the calculated difference based on each pixel's value to generate a tile of plane values for the pixels, applying a moving average to the tile of plane values, and selecting pixels of a merged image for each tile based on the tile of plane values, (d) tiling the merged images for each tile to generate a plurality of respective swaths; and (e) stitching the swaths together to form the whole specimen image (as a whole), wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, Gahm teaches displaying separate images of individual objects within a specimen image (e.g. in figure 1 and paragraphs 6, 8, and 76, “objects of interest are identified, relocated and scanned at higher resolution… scans of different parts of the tissue. These series of scans represent individual tiles of the overall tissue sample… [displays] series of tiles 14… separate window 16 on the display provides the user with higher magnified images of the selected area… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of Takayama to include the teachings of Gahm because one of ordinary skill in the art would have recognized the benefit of allowing the user to easily select and focus on particular objects of interest and/or providing relevant features to the user,
but does not specifically teach the imager having an optical axis at a non-orthogonal angle with the slide and configured to acquire a plurality of micro images of the specimen at a same z-axis of the three-dimensional volume, and an image processor configured to generate the whole specimen image from the micro images by a process comprising: (a) dividing the micro images into a plurality of sub-regions representing different focal planes across a specimen depth for each tile captured by the digital camera; (b) vertically reassembling the sub-regions along the z-axis to make a through- focus stack for each tile, (c) converting the through-focus stack for each tile into a single plane merged image by calculating a difference between each pixel and pixels in its neighboring region, weighting the calculated difference based on each pixel's value to generate a tile of plane values for the pixels, applying a moving average to the tile of plane values, and selecting pixels of a merged image for each tile based on the tile of plane values, (d) tiling the merged images for each tile to generate a plurality of respective swaths; and (e) stitching the swaths together to form the whole specimen image (as a whole), wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, Damaskinos teaches an imager including a digital camera having an optical axis at a non-orthogonal angle with a slide (e.g. in paragraphs 6, 24-25 and 36, “With tiling microscopes, the image of a small area of the specimen is recorded with a digital camera… detector array that is tilted [i.e. non-orthogonal] in the scan direction… such that a series of image frames tilted with respect to the surface of microscope slide are acquired as the stage scans… Optical tilt of the detector… detector array is mounted to tilt about an axis”) and configured to acquire a plurality of micro images of a specimen at a same z-axis of a three-dimensional volume (e.g. in paragraphs 6, 9, 31, 57, and 84, “combining a sequence of equally-spaced line images from the array to construct an image of one strip across the specimen. Strips are then assembled to form a complete image… the array detector 410 is triggered to collect a series of image frames of a tilted object plane 450 as it moves through the specimen… where the scan direction is the Y direction, and the vertical (focus) direction is the Z direction… resulting in collection of a digital 3D image stack of one strip of the specimen. Adjacent strips are then scanned and the 3D stack images of all strips are combined to assemble a 3D image of the entire specimen… automatically capture image planes above and below the specimen as well as planes inside the specimen, and a single, in-focus image plane can be assembled [i.e. configured to acquire along a same z-axis]”), and an image processor configured to generate a whole specimen image from the micro images by a process comprising: (a) dividing the micro images into a plurality of sub-regions representing different focal planes across a specimen depth for each tile captured by the digital camera (e.g. in paragraphs 6, 9, 31, 57, and 84, “computer stitching… line images [i.e. tiles] from the array to construct an image of one strip across the specimen. Strips [i.e. tiles] are then assembled to form a complete image… automatically capture image planes above and below the specimen as well as planes inside the specimen [i.e. different focal planes across a specimen depth]”); (b) vertically reassembling the sub-regions along the z-axis to make a through-focus stack for each tile (e.g. in paragraphs 6, 9, 31, 57, and 84, “the vertical (focus) direction is the Z direction… automatically capture image planes above and below the specimen as well as planes inside the specimen, and a single, in-focus image plane can be assembled”, i.e. a through-focus stack), (c) converting the through-focus stack for each tile into a single plane merged image (e.g. in paragraphs 6, 9, 29, 31, 57, and 84, “the vertical (focus) direction is the Z direction… produce a maximum-spatial-frequency projection image… automatically capture image planes above and below the specimen as well as planes inside the specimen, and a single, in-focus image plane can be assembled”) (d) tiling the merged images for each tile to generate a plurality of respective swaths (e.g. in paragraphs 6, 9, and 57, “combining a sequence of equally-spaced line images from the array to construct an image of one strip across the specimen”); and (e) stitching the swaths together to form the whole specimen image (e.g. in paragraphs 9 and 57, “Strips are then assembled to form a complete image”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Damaskinos because one of ordinary skill in the art would have recognized the benefit of creating an in-focus image,
but does not specifically teach by calculating a difference between each pixel and pixels in its neighboring region, weighting the calculated difference based on each pixel's value to generate a tile of plane values for the pixels, applying a moving average to the tile of plane values, and selecting pixels of a merged image for each tile based on the tile of plane values and wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, Maddison teaches converting a through-focus stack into a single plane merged image by selecting pixels of a merged image for each tile based on a tile of plane values (e.g. in paragraphs 8 and 34-36, “merging at least two of said N digital images… selecting, from two or more of said N digital images, those pixels having the highest variance in intensity relative to the mean intensity of pixels within the image… selects, in each case, the pixel with the highest intensity variance… mathematical algorithms for determining the highest intensity variance pixel, which corresponds to the best focus…for a feature [i.e. individual object] within an image… creates an "optimal" image, which combines the best-focussed features [i.e. individual objects]”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Maddison because one of ordinary skill in the art would have recognized the benefit of creating an optimal image,
but does not specifically teach calculating a difference between each pixel and pixels in its neighboring region, weighting the calculated difference based on each pixel's value to generate a tile of plane values for the pixels, applying a moving average to the tile of plane values and wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, Zhang teaches calculating a difference between each pixel and pixels in its neighboring region (e.g. in column 5 line 59 – column 6 line 7, “In order to assess focus quality of a particular image [i.e. plane],… determines focus power is by calculating the amount of pixel variance occurring within a particular image or at the edge of the image. Pixel variance is the difference in intensity of a pixel in comparison to a nearby or adjacent pixel(s)”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Zhang because one of ordinary skill in the art would have recognized the benefit of facilitating determination of pixel variance,
but does not specifically teach weighting the calculated difference based on each pixel's value to generate a tile of plane values for the pixels, applying a moving average to the tile of plane values and wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, Kuawbara teaches weighting a calculated difference based on each pixel's value to generate a tile of plane values for the pixels and applying a moving average to the tile of plane values (e.g. in paragraph 74, “variance among the pixel data values…in the sub scanning direction… it is desirable to obtain moving averages of the correction coefficients, which were obtained for each single linear region [i.e. moving average of tile of plane values], in the sub scanning direction, which is the direction that the composite images of the linear regions are synthesized. Thereby, high frequency components related to the variance in the values of the correction coefficients are reduced. Accordingly, correction coefficients, in which variance in short period components of the original correction coefficients in the sub scanning direction are reduced, are obtained [i.e. weighting calculated difference]. These correction coefficients are utilized to correct the image data sets that represent each single linear region, then the image data set that represents the single composite image is generated”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Kuawbara because one of ordinary skill in the art would have recognized the benefit of correcting image data,
but does not specifically teach wherein the system is configured to: (a) use a segmentation process to determine a characteristic associated with a respective object in the whole specimen image, (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object, and (c) in response to selection of the system user prompt displayed on the review station, display, on the review station, the one or more additional objects, the one or more additional objects characteristics comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness, which are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal.
However, the combination teaches one or more characteristics that are capable of being used to classify cells as being pre-cancerous, cancerous, normal, and/or abnormal (e.g. Takayama, in paragraphs 162 and 182, “nuclear enlargement and disordered cell arrangement in an area suspected to be cancerous”) and Kincaid teaches (a) use a segmentation process to determine a characteristic associated with a respective object in an image (e.g. in paragraphs 79-81, “finds all cells (or sub-cellular components) with a similar attribute profile. In this example, the attribute profile is defined by the absence or presence of each attribute in the cell”), (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object (e.g. in paragraph 80, “provide a function where not only do identified similar cells need to have the same combination of attributes present and absent, but, where present, the attributes each need to be within a range of intensity defined by the corresponding intensity of that attribute in the selected cell 514 plus or minus a threshold value”), and (c) in response to selection of a system user prompt displayed on the review station, display, on the review station, the one or more additional objects (e.g. in paragraphs 20, 66, and 79-80, “selecting, by a user, one of the cells in the image or the table; and selecting, by a user, a feature for finding all other cells in the image having similar attributes, wherein upon the selecting a feature, all other cells in the image having similar attributes to the selected cell are identified in at least one of the image and the table… selection of the "find similar attributes" function”, and figure 3) characteristic comprises one of cell size, nucleus to cytoplasm area ratio, nuclear corrected integrated optical density, cytoplasmic vacuolization, and/or darkness (e.g. in paragraph 68, “Attributes can include, but are not limited to, size (e.g., cell size or size of a particular type of sub-cellular component), shape (e.g., cell shape or shape of a sub-cellular component), number of nuclei, etc.”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Kincaid because one of ordinary skill in the art would have recognized the benefit of allowing the user to easily analyze relevant information in the context of the image/objects.
As per claim 2, the rejection of claim 1 is incorporated and the combination further teaches wherein the specimen is a cytological cellular specimen and the objects are cells (e.g. Takayama, in paragraphs 237 and 243, “specimen for cytodiagnosis… cell”).
As per claim 3, the rejection of claim 1 is incorporated and the combination further teaches wherein the specimen is a pathologic tissue sample and the objects are tissue structures (e.g. Takayama, in paragraph 236, “pathological tissue diagnosis”; Gahm, in paragraph 4, “cancer areas in the tissue section”).
As per claim 4, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to allow the system user to choose the whole specimen image using the user interface from a listing of stored whole specimen images using the user interface that have not yet been finally characterized or otherwise forwarded for secondary review (e.g. Takayama, in paragraph 119, “user can select one specimen (individual specimen) from among the plurality of specimen”; Gahm, in paragraphs 7, 31, 33, 37-38, 70, and 75-76, and claim 4, user can perform scans and make annotations to scans prior to [i.e. not yet] saving [i.e. finally characterized or otherwise forwarded] snapshots for subsequent review which forms a list).
As per claim 8, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to allow annotations to be added by a system user to at least one of the whole specimen image and a data file associated with a whole specimen image, wherein the annotations are available to subsequent system users to review along with the whole specimen image (e.g. Gahm, in paragraph 33, 38, and 75, “allow the user to enter and store notes associated with a scan. These annotations can be associated with a complete scan or with individual selected locations within a scan… the user may add textual comment to the snapshots. These comments are also stored with date, time and user identification… select to display her/his own snapshots only or the snapshots of all users”).
As per claim 9, the rejection of claim 8 is incorporated and the combination further teaches wherein the annotations are associated with individual objects in the whole specimen image (e.g. Gahm, in paragraphs 33, 38, and 75-76, “allow the user to enter and store notes associated with a scan. These annotations can be associated with a complete scan or with individual selected locations within a scan… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”).
As per claim 10, the rejection of claim 8 is incorporated and the combination further teaches wherein the annotations are in the form of electronic markings made on the whole specimen image or on images of a portion of the whole specimen image (e.g. Gahm, in paragraphs 33, 38, and 75, “allow the user to enter and store notes associated with a scan. These annotations can be associated with a complete scan or with individual selected locations within a scan… annotations can be in text form or they may be graphic information, such as lines, circles, etc., that hi-light parts of the scan”).
As per claim 11, the rejection of claim 1 is incorporated, but Takayama does not specifically teach wherein the system user prompt is a user selectable command displayed on one of the separate images showing the respective object. However, the combination teaches displaying separate images showing a respective object (e.g. Gahm, in figure 1 and paragraphs 6, 8, and 76, “These series of scans represent individual tiles of the overall tissue sample… [displays] series of tiles 14… separate window 16 on the display provides the user with higher magnified images of the selected area… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”) and Kincaid teaches a system user prompt being a user selectable command displayed on one of items showing a respective object (e.g. in paragraphs 20, 66, and 79, “selecting, by a user, one of the cells in the image or the table; and selecting, by a user, a feature for finding all other cells in the image having similar attributes, wherein upon the selecting a feature, all other cells in the image having similar attributes to the selected cell are identified in at least one of the image and the table”, and figure 6 showing “find similar attributes" function 602 on item/image showing a respective object). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Kincaid because one of ordinary skill in the art would have recognized the benefit of allowing the user to easily analyze information in the context of objects.
As per claim 12, the rejection of claim 11 is incorporated and the combination further teaches wherein the one or more additional objects are from the whole specimen image (e.g. Gahm, in paragraphs 6, 8, 36, and 71, “allows the user access…to scans representing different magnifications”; Kincaid, in paragraphs 20, 66, and 79, “wherein upon the selecting a feature, all other cells in the image having similar attributes to the selected cell are identified in at least one of the image and the table”).
As per claim 13, the rejection of claim 11 is incorporated and the combination further teaches wherein the one or more additional objects are from a library containing previously classified objects (e.g. Gahm, in paragraph 8, 36, and 71, “show views of additional scans which are positionally unrelated to the displayed initial scan, but are related in a sense of complementary information display, such as for example views of scans out of a reference database or a histology or cytology image atlas”; Kincaid, in paragraphs 20, 29, 66-67, and 79, “proper assignment of attributes to the cells from which the attributes were derived… wherein upon the selecting a feature, all other cells in the image having similar attributes to the selected cell are identified in at least one of the image and the table”).
As per claim 14, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to allow data regarding the whole specimen image or individual objects therein to be input through the user interface and stored in a data file associated with the whole specimen image (e.g. Gahm, in paragraph 33, 38, 64, and 75-76, “allow the user to enter and store notes associated with a scan. These annotations can be associated with a complete scan or with individual selected locations within a scan… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”).
Claim 15 corresponds to claims 1 and 11 and is rejected under the same reasons set forth, and the combination further teaches display, on one of the separate images of individual objects having a respective object, a show similar command associated with the respective object and in response to selection of the show similar command, display the one or more additional objects (e.g. Gahm, in figure 1 and paragraphs 6, 8, and 76, “These series of scans represent individual tiles of the overall tissue sample… [displays] series of tiles 14… separate window 16 on the display provides the user with higher magnified images of the selected area… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”; Kincaid, in paragraphs 20, 66, and 79-80, “selecting, by a user, one of the cells in the image or the table; and selecting, by a user, a feature for finding all other cells in the image having similar attributes, wherein upon the selecting a feature, all other cells in the image having similar attributes to the selected cell are identified in at least one of the image and the table… selection of the "find similar attributes" function”, and figure 6 showing “find similar attributes" function 602 on item/image showing a respective object).
As per claim 21, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to display a review screen on the display monitor, the review screen including a main image panel in which whole specimen image is displayed, and an object panel in which separate images of individual objects within the specimen image are displayed (e.g. Takayama, in paragraph 11, “generate a display image containing the overall image”; Gahm, in figure 1 and paragraphs 6, 8, and 76, “objects of interest are identified, relocated and scanned at higher resolution… scans of different parts of the tissue. These series of scans represent individual tiles of the overall tissue sample… [displays] series of tiles 14… separate window 16 on the display provides the user with higher magnified images of the selected area… tissue layers, large cell clusters etc., or of smaller compounds such as individual cells”).
As per claim 22, the rejection of claim 21 is incorporated and the combination further teaches wherein the system is configured to allow a system user, via the user interface, to zoom in/out and pan around the whole specimen image within the main image panel (e.g. Takayama, in paragraph 100, “user input information obtaining unit 401 obtains command information input by the user through the keyboard 311 and/or the mouse 312 via the operation I/F 310. Examples of the command information include start and termination of image display, and scroll, reduction, and magnification of the display image”).
As per claim 23, the rejection of claim 21 is incorporated and the combination further teaches wherein the system is configured to display an area of the whole specimen image containing a respective object within the main image panel upon a system user selecting, via the user interface, a separate image of the object in the object panel (e.g. Gahm, in paragraphs 6, 8, 36, and 71, “objects of interest are identified, relocated and scanned at higher resolution… Using a mouse or keyboard commands, the user selects an area of interest in the slide. A separate window 16 on the display provides the user with higher magnified images of the selected area”).
As per claim 24, the rejection of claim 21 is incorporated and the combination further teaches wherein the system is configured to allow a system user to select, via the user interface, an object in the displayed whole specimen image, wherein the system displays an image of the selected object in a system user selected image panel (e.g. Gahm, in paragraphs 6, 8, 36, and 71, “objects of interest are identified, relocated and scanned at higher resolution… Using a mouse or keyboard commands, the user selects an area of interest in the slide. A separate window 16 on the display provides the user with higher magnified images of the selected area”).
As per claim 25, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to allow the system user to characterize the whole specimen image as one or more of normal, abnormal, diseased, healthy, pre-cancerous, and cancerous (e.g. Takayama, in paragraphs 162 and 182, “nuclear enlargement and disordered cell arrangement in an area suspected to be cancerous”).
Claims 5 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Takayama (US 20140184778 A1) in view of Gahm et al. (US 20030210262 A1), Damaskinos et al. (US 20140125776 A1), Maddison (US 20060038144 A1), Zhang (US 6518554 B1), Kuwabara (US 20050057577 A1) and Kincaid et al. (US 20100128988 A1) as applied above, and further in view of Zhang et al. (US 20070274585 A1, hereinafter Zhang2).
As per claim 5, the rejection of claim 1 is incorporated and the combination further teaches wherein the system is configured to display a magnified view of at least a portion of the whole specimen image (e.g. Takayama, in paragraph 10, “a magnified image generated from the captured image for displaying a portion of the specimen in a magnified manner”), but does not specifically teach to automatically scan said displayed at least a portion of the whole specimen image. However, Zhang2 teaches automatically scanning a displayed at least a portion of an image (e.g. in paragraphs 88-91, “provides the "electronic magnifying glass" that pans, that is, scans across, the image automatically, following a selected scan pattern 80, such as that traced in a view 62 of FIG. 9… scan pattern 80 may change the relative position of magnification outline window 66 in discrete increments, so that, for example, each successive redefined magnified image portion is displaced 8 mm from the previous magnified image portion” and figure 9). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Zhang2 because one of ordinary skill in the art would have recognized the benefit of allowing the user to easily scan the image in a magnified manner.
As per claim 16, the rejection of claim 15 is incorporated and the combination further teaches wherein the system is configured to display a magnified view of at least a portion of the whole specimen image (e.g. Takayama, in paragraph 10, “a magnified image generated from the captured image for displaying a portion of the specimen in a magnified manner”), but does not specifically teach to automatically scan said displayed at least a portion of the whole specimen image, wherein the system automatically scans in a scanning pattern selected from the group consisting of a serpentine pattern, a row by row pattern, and a column by column pattern. However, Zhang2 teaches automatically scanning a displayed at least a portion of an image including automatically scanning in a scanning pattern selected from the group consisting of a serpentine pattern, a row by row pattern, and a column by column pattern (e.g. in paragraphs 87-89, “moving from left to right and from the top of the display downward… an ever-widening circular pattern” and figure 9 showing row by row in item 80). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Zhang2 because one of ordinary skill in the art would have recognized the benefit of allowing the user to easily scan the image in a magnified manner.
As per claim 17, the rejection of claim 16 is incorporated and the combination further teaches wherein the system is configured to allow a system user, via the user interface, to set a magnification level at which the system displays said scanned at least a portion of the whole specimen image (e.g. Zhang2, in paragraphs 88-91, “provides the "electronic magnifying glass" that pans, that is, scans across, the image automatically, following a selected scan pattern 80, such as that traced in a view 62 of FIG. 9… the operator has the option to…change magnification…of the magnification display window” and figure 9).
As per claim 18, the rejection of claim 16 is incorporated and the combination further teaches wherein the system is configured to allow a system user, via the user interface, to stop and start the scanning at a displayed location of the scan (e.g. Zhang2, in paragraphs 88-91, “During the scan display, the operator has the option to pause, adjust scanning speed or direction, specify movement increments, pan to change the displayed area… change pattern, reverse direction, stop, restart”).
As per claim 19, the rejection of claim 16 is incorporated and the combination further teaches wherein the system is configured to allow a system user, via the user interface, to set a speed of the scan (e.g. Zhang2, in paragraphs 88-91, “During the scan display, the operator has the option to…adjust scanning speed”).
As per claim 20, the rejection of claim 16 is incorporated and the combination further teaches wherein the system is configured to allow a system user to pause the scan at each object as the respective objects are displayed during the scan (e.g. Zhang2, in paragraphs 88-91, “During the scan display, the operator has the option to pause… stop, restart”).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Takayama (US 20140184778 A1) in view of Gahm et al. (US 20030210262 A1), Damaskinos et al. (US 20140125776 A1), Maddison (US 20060038144 A1), Zhang (US 6518554 B1), Kuwabara (US 20050057577 A1) and Kincaid et al. (US 20100128988 A1) as applied above, and further in view of Lareau et al. (US 20130231957 A1).
As per claim 6, the rejection of claim 4 is incorporated, but the combination does not specifically teach wherein the listing of stored whole specimen images that have not yet been finally characterized or otherwise forwarded for secondary review are organized and displayed in a plurality of differing formats based on input received through the user interface. However, the combination teaches a listing of stored whole specimen images that have not yet been finally characterized or otherwise forwarded for secondary review based on input received through a user interface (e.g. e.g. Takayama, in paragraph 119, “user can select one specimen (individual specimen) from among the plurality of specimen”; Gahm, in paragraphs 7, 31, 33, 37-38, 70, and 75-76, and claim 4, user can perform scans and make annotations to scans prior to [i.e. not yet] saving [i.e. finally characterized or otherwise forwarded] snapshots for subsequent review which forms a list) and Lareau teaches information that have not yet been finally characterized or otherwise forwarded being organized and displayed in a plurality of differing formats (e.g. in paragraphs 74-75, “information 138 includes data from a variety of sources [including different system users]… information 138 can be in a variety of different formats… extract the items from the…information 138 and save the items in a common format as [related] information 280”; i.e. prior to saving, the information may be organized/displayed in various formats and, after saving, the information will be organized/displayed in a common format). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Lareau because one of ordinary skill in the art would have recognized the benefit of allowing users to not have to worry about format and/or allowing information from multiple disparate sources to be used together.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Takayama (US 20140184778 A1) in view of Gahm et al. (US 20030210262 A1), Damaskinos et al. (US 20140125776 A1), Maddison (US 20060038144 A1), Zhang (US 6518554 B1), Kuwabara (US 20050057577 A1) and Kincaid et al. (US 20100128988 A1) as applied above, and further in view of Shnitzer et al. (US 20170034659 A1).
As per claim 7, the rejection of claim 4 is incorporated, but the combination does not specifically teach wherein the system is configured to allow an authorized third party to populate, in whole or part, the listing of stored whole specimen images for a particular system user to review. However, the combination teaches the listing of items including stored whole specimen images (e.g. Takayama, in paragraph 119, whole specimen images; Gahm, in paragraph 31 and claim 4, list) and Shnitzer teaches allowing an authorized third party to populate, in whole or part, a listing of items for a particular system user to review (e.g. in paragraph 206, “device connects to… third party servers in order to populate a list of [items] for the user to select”; note: the device connecting to the third party implies that the third party is authorized). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Shnitzer because one of ordinary skill in the art would have recognized the benefit of expanding the source of data to include additional devices.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Takayama (US 20140184778 A1) in view of Gahm et al. (US 20030210262 A1), Damaskinos et al. (US 20140125776 A1), Maddison (US 20060038144 A1), Zhang (US 6518554 B1), Kuwabara (US 20050057577 A1) and Kincaid et al. (US 20100128988 A1) as applied above, and further in view of Doerrer et al. (US 5677966 A).
As per claim 26, the rejection of claim 1 is incorporated, but the combination does not specifically teach wherein the system is configured to allow the system user to forward the whole specimen image for secondary review using the user interface. However, Doerrer teaches allowing a system user to forward an image for secondary review using a user interface (e.g. in column 6 lines 14-42, “cytologist…believe the slide to show only normal cells, the slide receives no further processing, and the diagnosis "normal" is reported on the patient. If…the cytologist…find abnormalities, the slide is forwarded to another cytologist for a full review of the entire slide… If the cytologist supervisor or the cytologist finds the gallery to be normal, the slide receives no further processing and a diagnosis of "normal" is reported. However, if the rereview finds any abnormal or questionable cells, the slide is subjected to full review”). It would have been obvious to one of ordinary skill in the art before the effective date of the claimed invention to modify the teachings of the combination to include the teachings of Doerrer because one of ordinary skill in the art would have recognized the benefit of allowing images to be classified for review.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For example,
Kaufman et al. (US 20140314300 A1) at least teaches (a) use a segmentation process to determine a characteristic associated with a respective object in an image (e.g. in paragraph 11, “navigating through and reviewing cytological image data, the image data comprising images of a cytological specimen including individual images of objects of interest… a cytological object having a characteristic similar to a characteristic of the respective object of interest in the selected image from the first plurality”) and (b) use a matching algorithm to compare the characteristic to identify one or more additional objects sharing the characteristic within a predetermined range of the respective object (e.g. in paragraphs 11 and 48, “The second plurality of images depict additional cytological objects in the specimen… one or more calculated metric 324 (in this case cell size greater than 50 microns)… a second plurality of thumbnail images (e.g., 310) of similar objects (e.g., 320) having the selected calculated metrics in the bottom window 306 of the display monitor 300. Other calculated metrics include nucleus/cytoplasm ratio, optical density, and regularity of cell contour. In some embodiments, the system is configured to generate cell size ranges dynamically based on review of a slide and properties of the OOI discovered with the slide and/or digital image. In further embodiments, each selection of calculated metrics can be configured to display a number of matching OOI”).
Sung et al. (US 20080079839 A1) teaches converting a through-focus stack into a single plane merged image by comparing pixels and selecting a portion of the plane with a best focus quality for individual objects in merged images of each individual object (e.g. in paragraphs 8, 50 “extracting image portions [i.e. individual objects as seen in figure 9] with optimum sharpness from the view-angle corrected images and merging the extracted image portions [to form an image with a best focus quality for each object in contrast to paragraph 2 where only one object is in focus]… detect image portions with optimum sharpness from the images obtained by the view-angle correction performed in operation S330 on a pixel-by-pixel basis”).
Dobson (US 20150310652 A1) teaches “FIG. 10 illustrates the GUI 1000 including a main image panel 1002 for displaying a field of view of a biological specimen, and a biomarker panel 1004 for displaying thumbnail images of biomarker expression associated with a region in the field of view of the biological specimen” (e.g. in paragraph 117 and figure 10).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM WONG whose telephone number is (571)270-1399. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TAMARA KYLE can be reached at (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.W/Examiner, Art Unit 2144 06/14/2026
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
Prosecution Insights