Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED OFFICE ACTION
Status of Claims
Claims 1-21 are pending examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b) (2) (C) for any potential 35 U.S.C. 102(a) (2) prior art against the later invention.
1. Claims 1,2,3,4,11,12,13,14,15,18,19 and 21 are rejected under 35 U.S.C 103(a) as being unpatentable over ERRICO et al. ( USPUB 20210113190 ) in view of Caner Mercan et al.(NPL Doc: " Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology," 9th November 2020, IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 25, NO. 6, JUNE 2021, 2021,Pages 2041-2049.).
As per claim 1, ERRICO et al. teaches A computer-based apparatus ( Fig. 1 -100 and Paragraph [0024]- “…The system 100 is used for scanning an area or volume of a patient's body. The system 100 includes an ultrasound imaging probe 110 in communication with a host 130 over a communication interface or link 120. …”) comprising: at least one computer readable non-transitory memory storing software instructions; and at least one processor coupled with the at least one computer readable non- transitory memory and that performs the following operations upon execution of the software instructions ( Processing component within FIG. 1 and Paragraphs [0028-0032]- “…The processing component 116 is coupled to the beamformer 114. The processing component 116 may include a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processing component 134 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processing component 116 is configured to process the beamformed image signals. …”) and :
partitioning a series of images of a tissue of an individual taken at different times into sets of areas ( Series of biological object ( liver lesion/surrounding tissue) images with different time period broken down into segment taught within FIG. 5 and Paragraphs [0058-0060]- “…The images 504 are shown as 504.sub.t(1) to 504.sub.t(N), where t(1) to t(N) may correspond to a time period including an AP, a PVP, and a late phase after a microbubble injection. The images 504 are 2D images (e.g., in an x-y dimensions) across a time period t. The images 504 may be time-stamped. Thus, the images 504 can be represented by I(x,y,t). The images 504 may be passed through each layer 532 and 536 in succession for feature extraction, analysis, and/or classification….”) ; identifying a tissue type of each area of each image in the series of images ( Paragraph [0033]- “…The processing component 134 can apply deep-learning-based techniques to determine whether a lesion is malignant or benign and/or the type of malignant/benign lesions based on the image data or image frames, as described in greater detail herein….” And Paragraphs [0055-0056]- “…The malignant lesion class 540 may further classify a malignant lesion into L1 plurality of malignant lesion sub-classes 542 (e.g., shown as 542.sub.M(0) to 542.sub.M(L1), where L1 is a positive integer). The benign lesion class 550 may further classify a benign lesion into L2 plurality of benign lesion sub-classes 552 (e.g., shown as 552.sub.B(0) to 552.sub.B(L2), where L2 is a positive integer). In an embodiment, the malignant lesion subclasses 542 may include HCC, ICC, metastasis and the benign lesion subclasses 552 may include haemangioma…”) ;
ERRICO et al. does not explicitly teach determining a region of interest in the series of images by executing a classifier that determines regions of interest based on changes in the tissue types of the sets of areas over time; determining a change in the region of interest over time; and outputting an indication of the change in the region of interest over time.
However, within analogous art, Carner Mercan et al. teaches determining a region of interest in the series of images by executing a classifier that determines regions of interest based on changes in the tissue types of the sets of areas over time ( Page 2047-0248- TABLE VI teaching ROI level classification, , Fig. 7 AND Page 2048- Col.1 – “….The classifier that used the Hypercolumn-Weighted representation was able to classify more cases of INV and Benign correctly. We observed that inclusion of pixel-level information in the patch-level representation extracted by the hypercolumn features led to an improvement where the classifier learned the characteristics of Benign and INV better, with a small cost of misclassifying more ADH cases as Benign. Overall, the classifiers trained using the proposed feature representations outperformed the other approaches in comparison. The challenges regarding the categorization of the preinvasive lesions such as ADH and DCIS…”) ; determining a change in the region of interest over time ( Page 2042- Fig. 1 and Col. 1 – “…design a representation that can capture the variations in local details of variable-sized ROIs. For example, Mehta et al. [19] propose the Y-Net framework that is jointly trained for segmentation and classification where the classification output is used with a threshold to obtain a tissue level discriminative segmentation mask. Then, the frequencies of the selected tissue components are used with a multi-layer perceptron to obtain the ROI-level diagnosis” AND Col.2 – “… Our whole slide analysis pipeline involves three stages illustrated in Fig. 1: 1) detection of ROIs; 2) modeling of these ROIs using a variable number of fixed-sized patches; 3) modeling of slides using these ROIs. This model can be considered with the weakly supervised learning paradigm where ROI-level class labels are missing when only slide-level diagnoses are available, and the contributions of individual patches to an ROI are also not known. We proposed both traditional [21] and deep learning-based [22] solutions to the first stage. We also proposed a multi-instance multi-label learning formulation [3] for the third stage. In this paper, we focus on the second stage of modeling the individual ROIs by designing an ROI-level representation via weighted aggregation of patch-level representations….”) ; and outputting an indication of the change in the region of interest over time (the output from the classification process of ROI changes/diagnosis over time taught within Page 2042-Col. 1- “…jointly trained for segmentation and classification where the classification output is used with a threshold to obtain a tissue level discriminative segmentation mask. Then, the frequencies of the selected tissue components are used with a multi-layer perceptron to obtain the ROI-level diagnosis…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Carner Mercan et al. within the modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. because the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. provides a method and system for implementation of Deep convolutional neural learning for region of interest processing within tissue images.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. within the modified teaching of the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of Deep convolutional neural learning for region of interest processing within tissue images.
As per claim 2, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. teaches wherein the operations further include generating a measurement of the region of interest in the series of images of the tissue over time ( FIG. 6 and Paragraph [0064]- “…illustrating a ROI detection scheme 600 for lesion assessment, according to aspects of the present disclosure. The scheme 600 can be applied in the ROI selection step 420 of the method 400. In the scheme 600, the image sequence 502 may be input into a recurrent-CNN (R-CNN) 610. …” AND Paragraph [0008]- “… through the subject's tissue during different sub-periods of the time period based on the spatiotemporal correlation; and displaying, by the display, the text report. In some embodiments, the method further comprises identifying a region of interest (ROI) from the subject's tissue based on the sequence of image frames; and applying the predictive network based on the identified ROI….”) .
As per claim 3, Combination of ERRICO et al. and Caner Mercan et al. teach claim 2,
Within analogous art, Caner Mercan et al. teaches wherein the measurement comprises one or more of: a change in size, a change in shape, a change in symmetry, a change in color, a change in density, or a change in moment of the region of interest ( Page 2044- Col. 2- “…activations in the earlier layers provide low-level information such as color, texture, and shape, while the activations in later layers encode contextual information about the input image [31].The hypercolumn feature representation of a pixel combines these low-level and high-level features and is obtained by concatenating all activations at that pixel location through the layers in the network when pixel-level representations are needed for tasks such as semantic segmentation…”) .
As per claim 4, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. teaches wherein the operations further include determining a diagnosis for the individual based on the region of interest in the series of images of the tissue over time ( FIG.8 and Paragraph [0069] and Paragraph [0061]- “…use of a deep reinforcement learning framework that assimilates clinical diagnosis from clinical reports to produce free-text. For instance, when the ground truth is available, clinical diagnosis can be learnt and the deep learning model may generate a text report with the candidate diagnosis and meaningful features associated with the output 534….”) .
As per claim 11, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. teaches wherein the operations further include determining a spatial relationship between areas of a first tissue type and areas of a second tissue type in each image of the series of images ( Paragraphs [0063-0064]- “…The CNN 530 may perform the refined classification into the malignant lesion subclasses 542 and the benign lesion subclasses 552 based on the spatiotemporal correlation between microbubbles retrieved in a cluster of neighboring pixels (e.g., the clusters 632 shown in FIG. 6) in the images 504….FIG. 6 is a schematic diagram illustrating a ROI detection scheme 600 for lesion assessment, according to aspects of the present disclosure. The scheme 600 can be applied in the ROI selection step 420 of the method 400. In the scheme 600, the image sequence 502 may be input into a recurrent-CNN (R-CNN) 610. …”) .
As per claim 12, Combination of ERRICO et al. and Caner Mercan et al. teach claim 11,
ERRICO et al. teaches wherein the operations further include determining a change in the spatial relationship over time ( Paragraphs [0079-0080]- “…the method 900 may include generating, by the predictive network, a text report (e.g., the text report 562) including information associated with perfusion of the contrast agent through the subject's tissue during different sub-periods (e.g., an AP, a PVP, and/or a late phase) of the time period based on the spatiotemporal correlation….”) .
As per claim 13, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Within analogous art, Caner Mercan et al. teaches wherein the operations further include generating at least one mask indicating the areas of one tissue type among the sets of areas for each image in the series ( series of image taught within Fig. 4 for processing and masks and ROI level taught within Page 2046- Col. 2- “…This approach extends the U-Net [36] model for joint training for segmentation and classification using a multi-task loss with 8-class tissue segmentation masks and ROI-level labels. The network produces a tissue-level discriminative segmentation mask after applying a threshold to the local patch probabilities. Histograms of patch class assignments are used as ROI features….”) .
As per claim 14, Combination of ERRICO et al. and Caner Mercan et al. teach claim 13,
Within analogous art, Caner Mercan et al. teaches wherein the operations further including presenting the series of images including the at least one mask for each image ( Page 2046- Col. 2- “…Then, for each ROI, we cropped the largest square region that could fit into that ROI’s mask and resized it to 1120 × 1120 pixels. We used the resulting regions to fine-tune a ResNet-50 network for ROI-level prediction. This procedure is denoted as Crop/resize in the results. We also evaluated replacing the average pooling layer with spatial pyramid pooling [18] using three scales….”) .
As per claim 15, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. teaches wherein the classifier is a convolutional neural network including a convolutional classification layer that has been trained to determine regions of interest in images based on the tissue types of the sets of areas of the image (Classification layer based on CNN and the training based on ROI assessment from images taught within Paragraphs [0055-0056]- “… CNN 530 is a three-dimensional (3D) network. The CNN 530 may operate on a sequence 502 of CEUS images 504 across a time period. For example, the CEUS images 504 may correspond to the motion compensated images produced at the step 430 of the method 400. The CNN 530 may include a set of N convolutional layers 532 followed by a set of K fully connected layers 536, …The classifications at the output 538 may include a binary classification between a malignant lesion class 540 and benign lesion class 550. The malignant lesion class 540 may further classify a malignant lesion into L1 plurality of malignant lesion sub-classes 542 (e.g., shown as 542.sub.M(0) to 542.sub.M(L1), where L1 is a positive integer). The benign lesion class 550 may further classify a benign lesion into L2 plurality of benign lesion sub-classes 552 (e.g., shown as 552.sub.B(0) to 552.sub.B(L2), where L2 is a positive integer).
As per claim 18, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. wherein the operations further include outputting a confidence level for the determination of the region of interest ( Paragraph [0083]- “…he disclosed embodiments may also be used in conjunction with current clinical practices and scanning protocols in lesion assessment to increase the confidence level of a diagnosis or assessment by a clinician. Thus, the disclosed embodiments can also add values to current clinical practices in lesion assessment. While the disclosed embodiments are described in the context of applying deep learning based techniques to classify tissues from CEUS images,…”) .
As per claim 19, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
ERRICO et al. teaches wherein the series of images comprise x-rays or CT scans ( Paragraph [0083]- “…the disclosed embodiments can be applied to classify tissues from images of other modes such as MRI, CT, and/or X-ray. For example, the images 504 can be MRI images, CT images, or X-ray images….”).
As per claim 21, ERRICO et al. teaches A computer-based method( Fig. 1 -100 and Paragraph [0024]- “…The system 100 is used for scanning an area or volume of a patient's body. The system 100 includes an ultrasound imaging probe 110 in communication with a host 130 over a communication interface or link 120. …”) comprising: partitioning a series of images of a tissue of an individual taken at different times into sets of areas( Series of biological object ( liver lesion/surrounding tissue) images with different time period broken down into segment taught within FIG. 5 and Paragraphs [0058-0060]- “…The images 504 are shown as 504.sub.t(1) to 504.sub.t(N), where t(1) to t(N) may correspond to a time period including an AP, a PVP, and a late phase after a microbubble injection. The images 504 are 2D images (e.g., in an x-y dimensions) across a time period t. The images 504 may be time-stamped. Thus, the images 504 can be represented by I(x,y,t). The images 504 may be passed through each layer 532 and 536 in succession for feature extraction, analysis, and/or classification….”); identifying a tissue type of each area of each image in the series of images( Paragraph [0033]- “…The processing component 134 can apply deep-learning-based techniques to determine whether a lesion is malignant or benign and/or the type of malignant/benign lesions based on the image data or image frames, as described in greater detail herein….” And Paragraphs [0055-0056]- “…The malignant lesion class 540 may further classify a malignant lesion into L1 plurality of malignant lesion sub-classes 542 (e.g., shown as 542.sub.M(0) to 542.sub.M(L1), where L1 is a positive integer). The benign lesion class 550 may further classify a benign lesion into L2 plurality of benign lesion sub-classes 552 (e.g., shown as 552.sub.B(0) to 552.sub.B(L2), where L2 is a positive integer). In an embodiment, the malignant lesion subclasses 542 may include HCC, ICC, metastasis and the benign lesion subclasses 552 may include haemangioma…”);
ERRICO et al. does not explicitly teach determining a region of interest in the series of images by executing a classifier that determines regions of interest based on changes in the tissue types of the sets of areas over time; determining a change in the region of interest over time; and outputting an indication of the change in the region of interest over time. However, within analogous art, Carner Mercan teaches determining a region of interest in the series of images by executing a classifier that determines regions of interest based on changes in the tissue types of the sets of areas over time ( Page 2047-0248- TABLE VI teaching ROI level classification, , Fig. 7 AND Page 2048- Col.1 – “….The classifier that used the Hypercolumn-Weighted representation was able to classify more cases of INV and Benign correctly. We observed that inclusion of pixel-level information in the patch-level representation extracted by the hypercolumn features led to an improvement where the classifier learned the characteristics of Benign and INV better, with a small cost of misclassifying more ADH cases as Benign. Overall, the classifiers trained using the proposed feature representations outperformed the other approaches in comparison. The challenges regarding the categorization of the preinvasive lesions such as ADH and DCIS…”) ; determining a change in the region of interest over time ( Page 2042- Fig. 1 and Col. 1 – “…design a representation that can capture the variations in local details of variable-sized ROIs. For example, Mehta et al. [19] propose the Y-Net framework that is jointly trained for segmentation and classification where the classification output is used with a threshold to obtain a tissue level discriminative segmentation mask. Then, the frequencies of the selected tissue components are used with a multi-layer perceptron to obtain the ROI-level diagnosis” AND Col.2 – “… Our whole slide analysis pipeline involves three stages illustrated in Fig. 1: 1) detection of ROIs; 2) modeling of these ROIs using a variable number of fixed-sized patches; 3) modeling of slides using these ROIs. This model can be considered with the weakly supervised learning paradigm where ROI-level class labels are missing when only slide-level diagnoses are available, and the contributions of individual patches to an ROI are also not known. We proposed both traditional [21] and deep learning-based [22] solutions to the first stage. We also proposed a multi-instance multi-label learning formulation [3] for the third stage. In this paper, we focus on the second stage of modeling the individual ROIs by designing an ROI-level representation via weighted aggregation of patch-level representations….”) ; and outputting an indication of the change in the region of interest over time (the output from the classification process of ROI changes/diagnosis over time taught within Page 2042-Col. 1- “…jointly trained for segmentation and classification where the classification output is used with a threshold to obtain a tissue level discriminative segmentation mask. Then, the frequencies of the selected tissue components are used with a multi-layer perceptron to obtain the ROI-level diagnosis…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Carner Mercan et al. within the modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. because the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. provides a method and system for implementation of Deep convolutional neural learning for region of interest processing within tissue images.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. within the modified teaching of the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of Deep convolutional neural learning for region of interest processing within tissue images.
2. Claims 5 and 6 are rejected under 35 U.S.C 103(a) as being unpatentable over ERRICO et al. ( USPUB 20210113190 ) in view of Caner Mercan et al.(NPL Doc: " Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology," 9th November 2020, IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 25, NO. 6, JUNE 2021, 2021,Pages 2041-2049.) in further view of FAUST et al. ( USPUB 20200272864).
As per claim 5, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include determining a prognosis for the individual based on the region of interest in the series of images of the tissue over time.
Within analogous art, FAUST et al. teaches wherein the operations further include determining a prognosis for the individual based on the region of interest in the series of images of the tissue over time ( Paragraph [0134]- “… Digital pathology platform 110 can use one or more trained convolutional neural networks (for example, a hierarchy of convolutional neural networks), classification models, clustering, and/or other data manipulation techniques to annotate the slide. The data can be further processed and results of same can be sent to external system 150 in a remote community. This may help guide the doctor in the remote community on treatment options, likely prognosis or clinical significance…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of FAUST et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. because the Platform, device and process for annotation and classification of tissue specimens using convolutional neural network mentioned by FAUST et al. provides a method and system for implementation of processing region of interest within medical images with neural network algorithms.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the Platform, device and process for annotation and classification of tissue specimens using convolutional neural network mentioned by FAUST et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of processing region of interest within medical images with neural network algorithms.
As per claim 6, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include determining a treatment recommendation for the individual based on the region of interest in the series of images of the tissue over time.
Within analogous art, FAUST et al. teaches wherein the operations further include determining a treatment recommendation for the individual based on the region of interest in the series of images of the tissue over time ( Paragraph [0104]- “…This may allow digital pathology platform 110 to resolve distinct classes (such as tumor classes) that are indistinguishable to the human observer, with applications, for example, in diagnosis, ordering tests or courses of treatment, in providing information such as prognostic correlates, or for identifying appropriate pharmacological or clinical intervention….”) .
3. Claims 7,8,9,10,17 and 20 are rejected under 35 U.S.C 103(a) as being unpatentable over ERRICO et al. ( USPUB 20210113190 ) in view of Caner Mercan et al.(NPL Doc: " Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology," 9th November 2020, IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 25, NO. 6, JUNE 2021, 2021,Pages 2041-2049.) in further view of Reeves et al. ( USPUB 20080205717).
As per claim 7, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include determining a similarity measurement between the region of interest and a stored description of a region of interest.
Within analogous art, Reeves et al. teaches wherein the operations further include determining a similarity measurement between the region of interest and a stored description of a region of interest ( Paragraph [0399]- “…Two additional measures of nodule shape may be defined that capture both the compactness and the major: minor aspect ratio the nodule. Sphericity is a measure of similarity of the nodule to a sphere of the same volume….”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. because the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. provides a method and system for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
As per claim 8, Combination of ERRICO et al. and Caner Mercan et al. and Reeves et al. teach claim 7,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include determining a change in the similarity measurement over time.
Within analogous art, Reeves et al. teaches wherein the operations further include determining a change in the similarity measurement over time ( Paragraph [0236]- “…This method works quite well for bimodal distributions where the modes encompass similar area and have minimal overlap. One factor affecting the relative area in each of the modes, of course, is ROI selection. If the nodule is surrounded by a large amount of lung parenchyma (i.e. the ROI is large compared to the nodule itself), the high-attenuation mode may not have sufficient area to be distinguished above the contributions of image noise and other high-attenuation, non-nodule structures. …”) .
As per claim 9, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the change in the region of interest over time comprises a growth rate of the region of interest.
Within analogous art, Reeves et al. teaches wherein the change in the region of interest over time comprises a growth rate of the region of interest ( Paragraph [0488]- “…weighted nodule ROIs as a surrogate for the nodule volume, it is possible to estimate the growth rate of a nodule in sequential CT scans without performing explicit segmentation. Given two 3D nodule ROIs, ROI.sub.1 and ROI.sub.2, acquired at t.sub.1 and t.sub.2, respectively, we may use the first to compute the appropriate weighting function for the pair,…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. because the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. provides a method and system for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
As per claim 10, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the change in the region of interest over time comprises a change in a growth rate of the region of interest.
Within analogous art, Reeves et al. teaches wherein the change in the region of interest over time comprises a change in a growth rate of the region of interest ( Paragraphs [0308-0309]- “…a stage prior to appreciable growth retardation; it has been reported that reductions from a constant growth rate may not be observable until the lesion is 2-3 cm in diameter [55]. These facts have led to the choice of a simple exponential growth model for our characterization of nodule growth….”) .
As per claim 17, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include comparing the change in the region of interest over time to changes in other regions of interest over time.
Within analogous art, Reeves et al. teaches wherein the operations further include comparing the change in the region of interest over time to changes in other regions of interest over time ( Paragraph [0112]- “…method and system of determining volumetric change in size over a period of time. In this method a doubling time of change is first determined, followed by establishing a relationship based on an exponential change model of said period of time between volumetric change time and a relative change in volume, followed by determining the volumetric change over such period of time…”) .
As per claim 20, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the region of interest comprises a tumor, and wherein the change in the region of interest over time indicates tumor progression.
Within analogous art, Reeves et al. teaches wherein the region of interest comprises a tumor, and wherein the change in the region of interest over time indicates tumor progression ( Paragraphs [0593-0594]- “…the non-nodule structures in the ROI correspond to vasculature or small airways. In each case, the size of such structures is likely to remain constant between scans, unless the nodule (as in the case of some malignant tumors) is responsible for angiogenesis in the region. In this case, the additional density measured in the ROI would contribute to a more aggressive estimate of growth, appropriate in such cases…”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. because the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. provides a method and system for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the System And Method For Three-dimensional Image Rendering And Analysis mentioned by Reeves et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of analyzing of three-dimensional images and diagnosis and treatment based on medical image processing.
4. Claim 16 is rejected under 35 U.S.C 103(a) as being unpatentable over ERRICO et al. ( USPUB 20210113190 ) in view of Caner Mercan et al.(NPL Doc: " Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology," 9th November 2020, IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 25, NO. 6, JUNE 2021, 2021,Pages 2041-2049.) in further view of Larsen et al. ( USPUB 20210172931).
As per claim 16, Combination of ERRICO et al. and Caner Mercan et al. teach claim 1,
Combination of ERRICO et al. and Caner Mercan et al. does not explicitly teach wherein the operations further include determining that the change in the region of interest over time indicates resistance to treatment.
Within analogous art, Larsen et al. teaches wherein the operations further include determining that the change in the region of interest over time indicates resistance to treatment ( Multiple treatment regimes based on ROI ( tumor) assessment indicates the change in treatment/based on effectiveness of treatment taught within Paragraph [0147-0148]- “… classifier is a multinomial classifier that is trained to provide a plurality of likelihoods, wherein each respective likelihood in the plurality of likelihoods is a likelihood that a different respective treatment regime, in a plurality of treatment regimes, for the cancer in the subject will be more effective than the other treatment regimes in the plurality of treatment regimes.”) .
One of ordinary skill in the art would have been motivated to combine the teaching of Larsen et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. because the Large scale organoid analysis mentioned by Larsen et al. provides a method and system for implementation of predicating patient response to treatment related to specific disease.
Therefore, it would have been obvious for one in the ordinary skills in the art before the effective filing date of the claimed invention to the Large scale organoid analysis mentioned by Larsen et al. within the combined modified teaching of the Ultrasound lesion assessment and associated devices, systems, and methods mentioned by ERRICO et al. and the Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology mentioned by Carner Mercan et al. for implementation of predicating patient response to treatment related to specific disease.
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2123.
Examiner’s Notes
5. The Examiner acknowledges the following prior arts below as pertinent to the current applications claim limitations and inventive concept, although the following prior arts shown below were not relied upon to address the limitations within the claim , they are analogous art mentioning the inventive concept key points on (Tissue/lesion image processing, deep neural network, segmentation/partitioning of tissue/medical images, ROI- region of interest detection and processing , classification etc.).
1) Hameedur Rahman et al.,"A Deep Learning Approach for Liver and Tumor Segmentation in CT Images Using ResUNet," 5th August 2022, Bioengineering 2022, 9, 368,Pages 1-16.
2) Brendon Lutnick et al. , “An integrated iterative annotation technique for easing neural network training in medical image analysis," February 2019, Nature Machine Intelligence ,VOL 1 112 , FEBRUARY 2019,Pages 112-118.
3) SONG-TOAN TRAN et al.," A Multiple Layer U-Net, Un-Net, for Liver and Liver Tumor Segmentation in CT," 7th January 2021, IEEEAccess,Volume 9,2021,Pages 3752-3761.
4) Fawad Salam Khan et al.," Breast Cancer Histological Images Nuclei Segmentation using Mask Regional Convolutional Neural Network ,” 26th March 2021, 2020 IEEE Student Conference on Research and Development (SCOReD),Pages 1-5.
5) Zaneta Swiderska-Chadaj et al., “Convolutional Neural Networks for Lymphocyte Detection in Immunohistochemically Stained Whole-Slide Images,” 11th April ,2018, 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands,Pages 1-8.
6) YUE ZHANG et al., “Deep Learning Initialized and Gradient Enhanced Level-Set Based Segmentation for Liver Tumor From CT Images,” 6th May 2020, IEEEAccess, Volume 8,2020, Pages 76056-76065.
7) Lyndon Chan et al., “HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images,” October 2019, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 10662-10667.
8) Simon Graham et al., “ Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images,” 18th September 2019, Medical Image Analysis 58 (2019) 101563, Pages 1-12.
9) Sultan Almotairi et al., “ Liver Tumor Segmentation in CT Scans Using Modified SegNet,” 10th March 2020, Sensors 2020, 20, Pages 1-10.
10) Tahsin Kurc et al.,” Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches,” 21st February 2020, Frontiers in Neuroscience,February 2020, Volume 14,Article 27, Pages 1-10.
11) Noah F. Greenwald et al.,” Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning,” 18th November 2021, Nature Biotechnology , VOL 40, April 2022,Pages 555-560, 570-575.
12) Asiedu et al. (USPUB 20210374953 )
13) NIAZI et al. ( USPUB 20210364496 )
14) Papagiannakis et al. ( USPUB 20210249118 )
15) Weisenfeld et al. ( USPUB 20210150707)
16) BERNAT et al. (USPUB 20210150707)
17) BANERJEE et al. (USPUB 20210027436)
18) TSUYUKI ( USPUB 20200219252)
19) Caballero et al. ( USPAT 10701394)
20) HAYASHI et al. ( USPUB 20200074224)
21) Mehanian et al. ( USPUB 20200054306 )
22) Forman et al. ( USPUB 20200029853)
23) Meyer et al. ( USPUB 20190279361)
24) Han et al. ( USPUB 20190251694)
25) Wang et al. (USPUB 20170347110)
26) Inazawa et al.( USPUB 20050037345)
27) Reeves et al. ( USPUB 20040252870)
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to OMAR S ISMAIL whose telephone number is (571)272-9799 and Fax # is (571)273-9799. The examiner can normally be reached on M-F 9:00am-6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David C. Payne can be reached on (571) 272-3024. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)? If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OMAR S ISMAIL/
Primary Examiner, Art Unit 2635
Prosecution Insights