Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of the present application being a continuation of and claiming priority to PCT International Application No. PCT/JP2023/026383 filed 07/19/2023. Acknowledgement is also made of applicant’s claim of the present application claiming priority and benefit under 35 U.S.C. 119(a-d) to Japanese application No. JP2022-119115 filed 07/26/2022.
Information Disclosure Statement
The information disclosure statements (“IDS”) filed 5/19/2025 and 9/30/2025 have been reviewed and the listed references were noted.
Drawings
The 30-page drawings have been considered and placed in the file.
Status of Claims
Claims 1-13 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS).
Regarding claim 1, Jain teaches, “An image processing apparatus comprising: a processor configured to: acquire a first specimen image depicting a tissue specimen of a subject;” (Jain, Abstract discloses; “In one embodiment, the image slides from patient tissue samples are divided into patches”) “extract, using a machine learning model, first feature amounts from respective first patch images into which the first specimen image is subdivided;” (Jain, Para. [0075] discloses; “In other words, the models of the vectorization module 302 learn morphological patterns and features corresponding to each label 210 and can distinguish and categorize the image patches based on those morphological features and patterns.”) “(Jain, Para. [0009] discloses; “clustering the labeled patch vectors, with an unsupervised artificial intelligence network, wherein each cluster corresponds to a morphological subtype expressed in the patch corresponding to the labeled patch vector and a patient outcome; generating a patch-level score for each patch based at least partly on the cluster to which the patch vector of the patch belongs;” Examiner interprets this disclosure of Jain to teach the soft clustering option.) Jain does not explicitly teach, “determine, based on each of the first feature amounts, whether a morphological abnormality is present in the tissue specimen depicted in a corresponding first patch image of the first patch images”. Since Jain does not explicitly disclose these limitations, Examiner relies on the teachings of Pati in an analogous field of endeavor. Specifically, Pati teaches, “determine, based on each of the first feature amounts, whether a morphological abnormality is present in the tissue specimen depicted in a corresponding first patch image of the first patch images” (Pati, Section 3.4 discloses; “The patch-wise feature representations are processed by the trained GMM clusters which assign a cluster label with a confidence score (probability) to each patch.”)
Jain and Pati are considered to be analogous to the claimed invention because they are in the same field of using machine learning models to detect abnormalities in tissue samples. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Jain to incorporate the teachings of Pati in order to decide whether an abnormality is present based on feature amounts. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain with the teachings of Pati to assess all features of the patches for abnormalities. Accordingly, it would have been obvious to combine Jain and Pati to obtain claim 1.
Claim 12 recites a method with steps corresponding to the elements of the system recited in Claim 1. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Jain and Pati references, presented in rejection of Claim 1, apply to this claim.
Claim 13 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Jain and Pati references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of Jain and Pati references discloses a machine-readable storage medium (Jain, Figure 12, item 1224).
Claims 2-5 are rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS), in further view of Barnes et al. (US 2017/0262984 A1).
Regarding claim 2, the combination of Jain and Pati does not explicitly teach, “The image processing apparatus according to claim 1, wherein the processor is configured to perform control to display a result of the manual clustering processing or the soft clustering processing.” Since the combination of Jain and Pati does not explicitly disclose this limitation, Examiner relies on the teachings of Barnes in an analogous field of endeavor. Specifically, Barnes teaches, “The image processing apparatus according to claim 1, wherein the processor is configured to perform control to display a result of the manual clustering processing or the soft clustering processing.” (Barnes, Para. [0228] discloses; “The result of the clustering can be visualized such as by entry of a user's selection of one of the single channel images 104-112, e.g. single channel channel image 108 which is rendered on a display device 117.” It would be obvious to use the clustering of Jain and Pati in the clustering display of Barnes.)
Jain, Pati, and Barnes are considered to be analogous to the claimed invention because they are in the same field of using clustering techniques to analyze tissue results in an image. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Jain and Pati to incorporate the teachings of Barnes in order to display the clustering results for the user. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain and Pati with the teachings of Barnes to display the results for ease-of-use for the user. Accordingly, it would have been obvious to combine Jain, Pati, and Barnes to obtain claim 2.
Regarding claim 3, the combination of Jain, Pati, and Barnes teaches, “he image processing apparatus according to claim 2, wherein the result is displayed by a plurality of cluster images generated by processing the first specimen image, and the plurality of cluster images are images that enable the plurality of clusters to be identified based on a display format preset for each of the plurality of clusters.” (Barnes, Para. [0152] discloses; “A cluster map may also be created by assigning a color to each cluster in the cluster output array and displaying on a computer screen. In this manner, pixels associated with cluster ‘0’ may appear as a white area, pixels associated with cluster ‘1’ are colored red, pixels associated with cluster ‘2’ yellow, and so forth. The cluster map and the color overlay of the analyte images may be placed in alternate windows of a specialized image viewer to compare the spatial areas of various clusters with the relative expression level of different markers.”) The proposed combination as well as the motivation for combining the Jain, Pati, and Barnes references presented in the rejection of claim 2, apply to claim 3 and are incorporated herein by reference. Thus, the apparatus recited in claim 3 is met by Jain, Pati, and Barnes.
Regarding claim 4, the combination of Jain, Pati, and Barnes teaches, “The image processing apparatus according to claim 3, wherein the processor is configured to display at least one cluster image among the plurality of cluster images to be superimposed on the first specimen image.” (Barnes, Para. [0231] discloses; “The result of the clustering, i.e. the delimitations 114 and 116, are then overlaid on the full resolution image, e.g. single channel image 108, as depicted in FIG. 21.”) The proposed combination as well as the motivation for combining the Jain, Pati, and Barnes references presented in the rejection of claim 2, apply to claim 4 and are incorporated herein by reference. Thus, the apparatus recited in claim 4 is met by Jain, Pati, and Barnes.
Regarding claim 5, the combination of Jain, Pati, and Barnes teaches, “The image processing apparatus according to claim 4, wherein the processor is configured to receive, from the user, a designation of the at least one cluster image displayed to be superimposed on the first specimen image.” (Barnes, Para. [0228] discloses; “The result of the clustering can be visualized such as by entry of a user's selection of one of the single channel images 104-112, e.g. single channel channel image 108 which is rendered on a display device 117.”) The proposed combination as well as the motivation for combining the Jain, Pati, and Barnes references presented in the rejection of claim 2, apply to claim 5 and are incorporated herein by reference. Thus, the apparatus recited in claim 5 is met by Jain, Pati, and Barnes.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS), in further view of Barnes et al. (US 2017/0262984 A1), and still in view of Arcot Desai et al. (US 2020/0272857 A1).
Regarding claim 6, the combination of Jain, Pati, and Barnes does not explicitly teach, “The image processing apparatus according to claim 2, wherein the processor is configured to display statistical information based on the result.” Since the combination of Jain, Pati, and Barnes does not explicitly disclose this limitation, Examiner relies on the teachings of Arcot Desai in an analogous field of endeavor. Specifically, Arcot Desai teaches, “The image processing apparatus according to claim 2, wherein the processor is configured to display statistical information based on the result.” (Arcot Desai, Para. [0008] discloses; “The method also includes providing an output comprising information, e.g., graphical display and image data, that enables a display of one or more of the plurality of clusters; enabling a mechanism, such a graphical user interface, for selecting at least one feature vector within a selected cluster of the one or more of the plurality of clusters; and providing an output comprising information e.g., graphical display and image data, that enables a display of the record of physiological information corresponding to the at least one selected feature vector. The display of the record of physiological information may be, for example, a time series waveform of electrical activity of the brain.” Examiner interprets “a time series waveform of electrical activity of the brain” to be statistical information.)
Jain, Pati, Barnes, and Arcot Desai are considered to be analogous to the claimed invention because they are in the same field of using clustering techniques to analyze tissue results in an image. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Jain, Pati, and Barnes to incorporate the teachings of Arcot Desai in order to display the statistical information for the user. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain, Pati, and Barnes with the teachings of Arcot Desai to display the results for ease-of-use for the user. Accordingly, it would have been obvious to combine Jain, Pati, Barnes, and Arcot Desai to obtain claim 6.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS), in further view of Bazakos et al. (US 2006/0053342 A1).
Regarding claim 7, the combination of Jain and Pati does not explicitly teach, “The image processing apparatus according to claim 1, wherein the processor is configured to, in the manual clustering processing: reduce a number of dimensions of the first feature amounts to two or three dimensions; display a graph in which the first feature amounts with a reduced number of dimensions are plotted in a two-dimensional space or a three-dimensional space; and receive the designation on the graph.” Since the combination of Jain and Pati does not explicitly disclose these limitations, Examiner relies on the teachings of Bazakos in an analogous field of endeavor. Specifically, Bazakos teaches, “The image processing apparatus according to claim 1, wherein the processor is configured to, in the manual clustering processing: reduce a number of dimensions of the first feature amounts to two or three dimensions;” (Bazakos, Para. [0061] discloses; “Although a 3D FEATURE SPACE section 112 is specifically depicted in the illustrative embodiment, it should be understood that the feature space is a super space of features, and therefore can be displayed in more or less than three dimensions, as desired. In certain embodiments, the graphical user interface 94 may include an icon button and/or pull-down menu that permits the user to vary the manner in which the feature vector is represented and/or to select those features the user desires to view on the display screen 96.”) “display a graph in which the first feature amounts with a reduced number of dimensions are plotted in a two-dimensional space or a three-dimensional space;” (Bazakos Para. [0061] discloses; “Although a 3D FEATURE SPACE section 112 is specifically depicted in the illustrative embodiment, it should be understood that the feature space is a super space of features, and therefore can be displayed in more or less than three dimensions, as desired. In certain embodiments, the graphical user interface 94 may include an icon button and/or pull-down menu that permits the user to vary the manner in which the feature vector is represented and/or to select those features the user desires to view on the display screen 96.”) “and receive the designation on the graph.” (Bazakos, Para. [0007] discloses; “In some embodiments, the groups of cluster points representing each feature vector can be displayed graphically on a graphical user interface, allowing a user to visually confirm possible event candidates by selecting the appropriate cluster of points on a display screen.”)
Jain, Pati, and Bazakos are considered to be analogous to the claimed invention because they are in the same field of using clustering techniques to analyze results in an image. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Jain and Pati to incorporate the teachings of Bazakos in order to display the clustering results for the user in a 2- or 3-dimensional graph. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain and Pati with the teachings of Bazakos to display the results for ease-of-use for the user, as desired by the user. Accordingly, it would have been obvious to combine Jain, Pati, and Bazakos to obtain claim 7.
Claims 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS), in further view of Drews et al. (US 2023/0253070 A1).
Regarding claim 8, the combination of Jain and Pati teaches, “The image processing apparatus according to claim 1, wherein the machine learning model is a model trained using, as labeled training data, second patch images into which second specimen images are subdivided, (Jain, Para. [0019] discloses; “train the artificial intelligence network, based on the plurality of labels to identify the morphological types associated with each label in the image slides; generate labeled patch vectors, wherein the label of a patch vector is assigned based, at least partly, on the plurality of the labels and the morphological type expressed in the patch”). The combination of Jain and Pati does not explicitly teach, “the second specimen images depicting tissue specimens of a plurality of subjects constituting a control group to which a candidate substance for a medicine is not administered in a past evaluation test of the candidate substance”. Since the combination of Jain and Pati does not disclose this limitation, Examiner relies on the teachings of Drews in an analogous field of endeavor. Specifically, Drews teaches, “the second specimen images depicting tissue specimens of a plurality of subjects constituting a control group to which a candidate substance for a medicine is not administered in a past evaluation test of the candidate substance.” (Drews, Para. [0006] discloses; “In some embodiments, the method includes training a machine learning model using the positive control group and the negative control group to determine a correlation of the at least one genetic variation condition to a pathway dysregulation score.” And Drews Para. [0287] discloses using specimen images. It would be obvious to combine the training patch images of Jain with the method of using control groups of Drews.)
Jain, Pati, and Drews are considered to be analogous to the claimed invention because they are in the same field of training machine learning models in image analysis. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Jain and Pati to incorporate the teachings of Drews in order to train the model on a control group which had no medicine administered to it. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain and Pati with the teachings of Drews to ensure the model would be able to tell the difference between treated and untreated tissue patch images. Accordingly, it would have been obvious to combine Jain, Pati, and Drews to obtain claim 8.
Regarding claim 9, the combination of Jain, Pati, and Drews teaches, “The image processing apparatus according to claim 8, wherein the labeled training data further includes a patch image depicting the tissue specimen in which the morphological abnormality is present.” (Jain, Para. [0080] discloses; “The ground truth for the model 402 can be provided by a group of experienced pathologists, who discuss, agree and then identify known morphology features such as benign glands, cancer precursors, low/medium/high-grade cancers, immune cells, stroma, etc. These labels are based on known morphological patterns that pathologists use for cancer detection and grading.”)
Regarding claim 10, the combination of Jain, Pati, and Drews teaches, “The image processing apparatus according to claim 9, wherein the machine learning model is a model that performs a task of identifying a type of the morphological abnormality.” (Jain, Para. [0080] discloses; “Artificial intelligence models 402 can be trained to identify different morphological subtypes in H&E slides (or other input images if used).”)
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US 2021/0193323 A1 - IDS), in view of Pati et al. (“Deep Positive-Unlabeled Learning for Region of Interest Localization in Breast Tissue Images” – IDS), in further view of Drews et al. (US 2023/0253070 A1), and still in view of Okada et al. (US 2014/0240486 A1).
Regarding claim 11, the combination of Jain, Pati, and Drews does not explicitly teach, “The image processing apparatus according to claim 8, wherein the processor is configured to: acquire information on a distribution of second feature amounts extracted using the machine learning model from the second patch images into which the second specimen images are subdivided; calculate a distance between the distribution and each of the first feature amounts; and perform the determination based on the distance.” Since the combination of Jain, Pati, and Drews does not explicitly disclose these limitations, Examiner relies on the teachings of Okada in an analogous field of endeavor. Specifically, Okada teaches, “The image processing apparatus according to claim 8, wherein the processor is configured to: acquire information on a distribution of second feature amounts extracted using the machine learning model from the second patch images into which the second specimen images are subdivided;” (Okada, Para. [0044] discloses; “The learning abnormality determination unit 105 generates a spatial distribution of the feature amounts of a plurality of frames of the same time (learning data) from the moving pictures of the normal group obtained by the group operation unit 103. In addition, the learning abnormality determination unit 105 compares the generated spatial distribution of the feature amounts (learning data) and a feature amount of a frame of a new inspection object and calculates how far the feature amount (point) of the frame of the inspection object is spaced apart from the learning data(distribution) as a distance.” It would be obvious to use the distribution information acquisition of Okada with the second patch images of Jain.) “calculate a distance between the distribution and each of the first feature amounts;” (Okada, Para. [0044] discloses; “The learning abnormality determination unit 105 generates a spatial distribution of the feature amounts of a plurality of frames of the same time (learning data) from the moving pictures of the normal group obtained by the group operation unit 103. In addition, the learning abnormality determination unit 105 compares the generated spatial distribution of the feature amounts (learning data) and a feature amount of a frame of a new inspection object and calculates how far the feature amount (point) of the frame of the inspection object is spaced apart from the learning data(distribution) as a distance.”) “and perform the determination based on the distance.” (Okada, Para. [0045] discloses; “When inspecting already recorded moving picture data, the learning abnormality determination unit 105 calculates how far feature amounts of frames of the recorded moving picture data are spaced apart from the spatial distribution of the feature amounts of the normal frames and compares the calculated distances and a threshold so as to extract (detect) an abnormal frame. Specifically, among the recorded moving picture data, a frame that has a distance exceeding the threshold is extracted (detected) as an abnormal frame.” Okada decides based on a distance threshold. Thus, it would be obvious to combine the determination of Okada with the second patch images of Jain.)
Jain, Pati, Drews, and Okada are considered to be analogous to the claimed invention because they are in the same field of biological image processing. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Jain, Pati, and Drews to incorporate the teachings of Okada in order to decide based on distribution information. One of ordinary skill in the art would have been motivated to combine the previously described device of Jain, Pati, and Drews with the teachings of Okada to ensure the labels are close to the ground-truth labels. Accordingly, it would have been obvious to combine Jain, Pati, Drews, and Okada to obtain claim 11.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN M. OAKES whose telephone number is (571)272-9379. The examiner can normally be reached 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUSTIN M OAKES/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights