DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on April 21, 2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
Color photographs and color drawings are not accepted in utility applications unless a petition filed under 37 CFR 1.84(a)(2) is granted. Any such petition must be accompanied by the appropriate fee set forth in 37 CFR 1.17(h), one set of color drawings or color photographs, as appropriate, if submitted via the USPTO patent electronic filing system or three sets of color drawings or color photographs, as appropriate, if not submitted via the via USPTO patent electronic filing system, and, unless already present, an amendment to include the following language as the first paragraph of the brief description of the drawings section of the specification:
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
Color photographs will be accepted if the conditions for accepting color drawings and black and white photographs have been satisfied. See 37 CFR 1.84(b)(2).
Examiner notes it appears a petition was filed April 21, 2023 and the petition was discussed on June 15, 2023 due to the fact that there was now showing of necessity.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-6, 8-10, 12-13, 15-16 and 18-20 (and claims 10, 14 and 17 due to their dependency on a rejected base claim and not curing the deficiency) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Relative terminology
The term “powerful” in claim 10 is a relative term which renders the claim indefinite. The term “powerful” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. For the sake of examination, the examiner will interpret the “powerful tumor detection models” as “tumor detection models.”
Antecedent Basis:
Claim 5 recites the limitation "the brightness range" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the deepnet capacity" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the dimension" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the organ region" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation "the hybrid" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation "the image format" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 12 recites the limitation "the nervous system" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 12 recites the limitation "the gastrointestinal system" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 13 recites the limitation "the deep-net based approach" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 13 recites the limitation "the analytics-based approach" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 15 recites the limitation "the brightness range" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 16 recites the limitation "the dimension" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 16 recites the limitation "the dimension" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 18 recites the limitation "the organ region" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 19 recites the limitation "the hybrid" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 19 recites the limitation "the image format" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 20 recites the limitation "the model creation stage " in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 6-7 and 10-12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Garcia, Sara I. "Meta-learning for skin cancer detection using deep learning techniques." arXiv preprint arXiv:2104.10775 (2021) (hereinafter Garcia).
Regarding independent claim 1, Garcia discloses A method for diagnosing tumors on a medical image (abstract, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images.”),
wherein the diagnosis is made through analyzing the medical image by a diagnosis model (abstract, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images;” page 3, “For this study, a ResNet50 model was used for all the experiments. This is a variation of the ResNet architecture that consists of 50 convolutional layers. This architecture was used for its high performance on ImageNet and availability in the Keras library. All the samples of the combined dataset were normalized and reduced by 254 x 254 pixels to match the input of ResNet.”),
wherein the diagnosis model comprises meta-image-based deepnets developed by cooperating with experts' knowledge for accurate tumor recognition in medical images (page 3, “For this study, only a subset from the training set was used.;” page 3, “The images from the three datasets were chosen randomly, which corresponds to 1.2% of the total size of the datasets. The labels of the three datasets were combined into three groups: melanoma, malignant and benign. The malignant class contains labels of skin cancer moles, such as basal cell carcinoma, squamous cell carcinoma and actinic keratosis;” the experts knowledge is read as the labels used; page 5, “• Pre-trained ResNet model fine-tuned with medical data.”).
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Garcia further discloses wherein the method comprises creating a diagnosis model with integrating adopted knowledge to design appropriate loss functions by counting loss values occurring in meta-images during training a tumor detection deepnet (page 6, Figure 3, “Figure 3: Left image: training and validation loss of the meta-learning experiments using the Class balancer algorithm. Right image: training and validation accuracy;” the class balancer algorithm is used to balance the loss values (see Figure 2); adopted knowledge is read as the pre-training; abstract, “The results show an increase of performance on detecting melanoma, malignant (skin cancer) and benign moles with the prior knowledge obtained from images of everyday objects from ImageNet dataset by 20 points.”).
Regarding dependent claim 3, the rejection of claim 2 is incorporated herein. Additionally, Garcia further discloses wherein the meta-images are generated from transforming knowledge rules by a deepnet-based approach (abstract, “The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase of performance on detecting melanoma, malignant (skin cancer) and benign moles with the prior knowledge obtained from images of everyday objects from ImageNet dataset by 20 points;” the meta-image is read as the output classification applied to the image) and/or an analytics-based approach.
Regarding dependent claim 4, the rejection of claim 3 is incorporated herein. Additionally, Garcia further discloses wherein the deepnet-based approach comprises using deepnets to represent knowledge rules (abstract, “The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data;” RestNet is read as a type of deepnet which is trained to learn the knowledge rules) and constructing a meta-image (abstract, “The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase of performance on detecting melanoma, malignant (skin cancer) and benign moles with the prior knowledge obtained from images of everyday objects from ImageNet dataset by 20 points;” the meta-image is read as the output classification applied to the image).
Regarding dependent claim 6, the rejection of claim 3 is incorporated herein. Additionally, Garcia further discloses wherein meta-image is created by uniformly transforming medical images to knowledge-embedded tensors for a deepnet and improving the deepnet capacity by increasing the dimension of feature space from exotic domain knowledge (NOTE: applicants PGPub paragraph 0050 states, “On the other hand, knowledge-annotated image A can be seen as a tensor-based KR representation for accommodating the original CT image C and all n specified knowledge rules. Each knowledge-rule layer consisting of M(k) is a view of explaining the base layer C. The tensor formatted A is the required knowledge structure as it can be processed by most deepnets.” Exemplifying the tensor structure is well known; abstract, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images. The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase of performance on detecting melanoma, malignant (skin cancer) and benign moles with the prior knowledge obtained from images of everyday objects from ImageNet dataset by 20 points”).
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, Garcia further discloses wherein the medical image is obtained from an imaging technology selected from X-ray radiography, magnetic resonance imaging, ultrasound, endoscopy, elastography, tactile imaging, thermography, medical photography (abstract, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images.”), positron emission tomography (PET) and computed tomography (CT).
Regarding dependent claim 10, the rejection of claim 2 is incorporated herein. Additionally, Garcia further discloses wherein the loss functions are knowledge-derived loss functions that aid a deepnet optimizer to create powerful tumor detection models (comparison of figure 2 and figure 3; abstract, “In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model;” fine tuning is known to adjust parameters based on loss; the loss is based on the output/expected as such is inherently based on the knowledge, in that what is output is utilizing the knowledge; page 4, “Pre-trained ResNet model fine-tuned with medical data.”).
Regarding dependent claim 11, the rejection of claim 10 is incorporated herein. Additionally, Garcia further discloses wherein the optimizer is used to tune parameters that do not meet exotic knowledge via loss function of knowledge during the model creation stage (page 4, “Pre-trained ResNet model fine-tuned with medical data.”).
Regarding dependent claim 12, the rejection of claim 1 is incorporated herein. Additionally, Garcia further discloses wherein the tumor is selected from bladder tumors, breast tumors, cervical tumors, colon or rectal tumors, endometrial tumors, kidney tumors, lip or oral tumors, liver tumors, skin tumors (abstract, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images;” page 3, “The labels of the three datasets were combined into three groups: melanoma, malignant and benign. The malignant class contains labels of skin cancer moles, such as basal cell carcinoma, squamous cell carcinoma and actinic keratosis.”), lung tumors, ovarian tumors, pancreatic tumors, prostate tumors, thyroid tumors, brain tumors, bone tumors, muscle or tendon tumor, tumors of the nervous system, and tumors of the gastrointestinal system.
Claim(s) 13-14, 16 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Z. Cao et al., "Meta-Seg: A Generalized Meta-Learning Framework for Multi-Class Few-Shot Semantic Segmentation," in IEEE Access, vol. 7, pp. 166109-166121, 2019, doi: 10.1109/ACCESS.2019.2953465. (hereinafter Cao).
Regarding independent claim 13, Cao discloses A method for processing a medical image (abstract, “Semantic segmentation performs pixel-wise classification for given images, which can be widely used in autonomous driving, robotics, medical diagnostics and etc.”… “In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named MetaSeg. It consists of a meta-learner and a base-learner.”) , comprising generating meta-images from transforming knowledge rules by the deepnet-based approach (NOTE: in applicant’s PG Pub paragraph 0041-0043 describes the deepnet-based approach and further at paragraph 0043, “The output of the deepnet is a meta-image, where pixels inside the organ region are set to 255 (maximum value) and ones outside the organ are set to 0;” Figure 5, “FIGURE 5. Schematic diagram of semantic segmentation results on split-1.The first row shows the original input images. The second row shows the ground truths. The third and fourth rows show the predicted results of Seg-JT and Seg-FT, respectively. The last row shows the predicted results of Meta-seg.”) and/or the analytics-based approach (NOTE: in applicant’s PG Pub paragraph 0044-0046 describes the analytics-based approach, and paragraph 0046 states, “The following steps are used to generate a meta-image for KR2: (1) creating a two-dimensional matrix (i.e., meta-image) whose weight and height align the original CT image, and (2) sequentially setting the pixel value to 255 (maximum value) if brightness of the associated pixel is in the range [b⊥, bT]; otherwise, set to 0. FIG. 2(b) is a visual representation of the meta-image for KR2;”).
Regarding dependent claim 14, the rejection of claim 13 is incorporated herein. Additionally, Cao further discloses wherein the deepnet-based approach comprises using deepnets to represent knowledge rules and constructing a meta-image (abstract, “In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named MetaSeg. It consists of a meta-learner and a base-learner. Specifically, the meta-learner learns a good initialization and a parameter update strategy from a distribution of few-shot semantic segmentation tasks.;” Figure 5, “FIGURE 5. Schematic diagram of semantic segmentation results on split-1.The first row shows the original input images. The second row shows the ground truths. The third and fourth rows show the predicted results of Seg-JT and Seg-FT, respectively. The last row shows the predicted results of Meta-seg.”).
Regarding dependent claim 16, the rejection of claim 13 is incorporated herein Additionally, Cao discloses wherein meta-image is created by uniformly transforming medical images to knowledge-embedded tensors for deepnet and improving the deepnet capacity by increasing the dimension of feature space from exotic domain knowledge (NOTE: applicants PGPub paragraph 0050 states, “On the other hand, knowledge-annotated image A can be seen as a tensor-based KR representation for accommodating the original CT image C and all n specified knowledge rules. Each knowledge-rule layer consisting of M(k) is a view of explaining the base layer C. The tensor formatted A is the required knowledge structure as it can be processed by most deepnets.” Exemplifying the tensor structure is well known ; abstract, “Semantic segmentation performs pixel-wise classification for given images, which can be widely used in autonomous driving, robotics, medical diagnostics and etc;” page 166112, left column “Meta Learning [56], also known as learning to learn, is the science of systematically learning the experience or meta-data which is learned by different machine learning approaches in a wide range of learning tasks” page 166112, right column, “Meta-Training: In meta-training phase, the meta-learning model is trained on meta-training set Dtr. We sample categories and training images to build a task T . In each K-way N-shot task T , for T tr, there are K categories and N training images per category, and for T te, there are same K categories but different N training images per category from T tr. Within each task, the base-learner is trained on T tr, then update its parameters under the guidance of meta-learner by leveraging the feedback from T tr. Hence, we can get a temporary semantic segmentation model for the current task;” the knowledge representations per tasks are read as tensors ).
Regarding dependent claim 20, the rejection of claim 16 is incorporated herein. Additionally, Cao in the combination further discloses wherein a deepnet optimizer is used to tune parameters that do not meet exotic knowledge via loss function of knowledge during the model creation stage (page 166116, left column, “fine-tune it on meta-test classes with few examples.”… “For Meta-Seg and two baselines, we fine-tune the model weights of the Imagenet-pretrained VGG16 network to adapt them for the segmentation task.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Garcia as applied to claim 3 above, and further in view of U.S. Publication No. 2019/0392563 to Manhart (hereinafter Manhart).
Regarding dependent claim 5, the rejection of claim 3 is incorporated herein. Additionally, Garcia fails to explicitly disclose wherein the analytics-based approach comprising using analytic models to find pixels of medical images that fit the brightness range and constructing a meta-image.
However, Manhart discloses wherein the analytics-based approach comprising using analytic models to find pixels of medical images that fit the brightness range and constructing a meta-image (paragraph 0007, “To generate the respective binary mask, a check is then made for each pixel of the respective single image as to whether the pixel's intensity or brightness value lies above or below the specified threshold value on the Hounsfield scale, in other words in Hounsfield units (HU). Depending on the result, one or the other of the two binary values, for example 1 or 0, is defined for a respective corresponding pixel of the binary mask. By way of the binary mask an initially rough distinction may therefore be made between the different components or types of material of the target object.”).
Garcia is directed toward, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images (abstract).” Manhart is directed toward “Binary masks are generated for single images of the image data set. The masks differentiate different components of an imaged target object from each other (abstract).” As can be easily seen by one of ordinary skill in the art at the time of filing the claimed invention, both Garcia and Manhart are directed toward similar methods of endeavor of image analysis and specifically determining components of the image. Further, one of ordinary skill in the art would easily understand that often doctors are provided with too much unnecessary information; thus, presenting binary options can be helpful to reduce analysis fatigue. Further, often there are not enough medical images for a learning data set, thus applying the meta-learning technique to the masking algorithm would allow for more efficient learning in that extra training data is not needed. Thus, it would have been obvious to a person having ordinary skill in the art at the time the claimed invention was filed to incorporate the teaching of Manhart to ensure only relevant information is output to a user, and the overall efficiency is increased.
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Garcia as applied to claim 3 above, and further in view of U.S. Patent No. 7,702,153 to Hong et al. (hereinafter Hong).
Regarding dependent claim 18, the rejection of claim 13 is incorporated herein. Additionally, Garcia fails to explicitly disclose wherein the knowledge rules include determining the organ region, identifying the tumors that reside in the organ region, and displaying the tumors in a specified brightness range. However, Hong discloses wherein the knowledge rules include determining the organ region (column 3, line 44, “The background model builder 210 estimates the intensity distribution of voxels in organ regions from the samples provided by user interactions The samples may be two or three small sampling blocks at, for example, liver regions. The background model builder 210 marks the liver regions in the input 3D liver CT image based on the estimated intensity distribution.”), identifying the tumors that reside in the organ region (column 4, line 60, “The tumor region locator 230 starts from the position initially provided by user interactions to identify substantially all the voxels that belong to the target liver tumor. It segments the target liver tumor using, for example, the likelihood field with a boundary-based constraint,”), and displaying the tumors in a specified brightness range (abstract, “the foreground model builder uses an intensity distribution estimate of voxels in a target organ tumor to build a first foreground model;” column 7, line 61, “The computer system 101 is generally coupled through the I/O interface 104 to a display 105 and various input devices 106 such as a mouse and keyboard. ”).
Garcia is directed toward, “This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images (abstract).” Hong is directed toward “A system for segmenting a target organ tumor from an image includes a background model builder, a foreground model builder and a tumor region locator (abstract).” As can be easily seen by one of ordinary skill in the art at the time of filing the claimed invention, both Garcia and Hong are directed toward similar methods of endeavor of image analysis and specifically determining components of the image. Further, one of ordinary skill in the art would easily understand that often doctors are provided with too much unnecessary information; thus, presenting clear data in an output can be advantageous for accurate diagnosis. Said differently, an output can be confusing to a user with no context; thus identifying an organ to allow a user to realize what region of the body the images are and the tumor is within can be helpful. Further, often there are not enough medical images for a learning data set, thus applying the meta-learning technique to the masking algorithm would allow for more efficient learning in that extra training data is not needed. Thus, it would have been obvious to a person having ordinary skill in the art at the time the claimed invention was filed to incorporate the teaching of Hong to ensure accurate and contextual data is output to a user while utilizing the efficiency of meta-learning
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over Cao as applied to claim 13 above, and further in view of Manhart.
Regarding dependent claim 15, the rejection of claim 13 is incorporated herein. Additionally, Cao fails to explicitly disclose wherein the analytics-based approach comprises using analytic models to find pixels of medical images that fit the brightness range and constructing a meta-image.
However, Manhart discloses wherein the analytics-based approach comprises using analytic models to find pixels of medical images that fit the brightness range and constructing a meta-image (paragraph 0007, “To generate the respective binary mask, a check is then made for each pixel of the respective single image as to whether the pixel's intensity or brightness value lies above or below the specified threshold value on the Hounsfield scale, in other words in Hounsfield units (HU). Depending on the result, one or the other of the two binary values, for example 1 or 0, is defined for a respective corresponding pixel of the binary mask. By way of the binary mask an initially rough distinction may therefore be made between the different components or types of material of the target object.”).
Cao is directed toward, “In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named Meta-Seg. It consists of a meta-learner and a base-learner (abstract).” Manhart is directed toward “Binary masks are generated for single images of the image data set. The masks differentiate different components of an imaged target object from each other (abstract).” As can be easily seen by one of ordinary skill in the art at the time of filing the claimed invention, both Cao and Manhart are directed toward similar methods of endeavor of image analysis and specifically determining components of the image. Further, one of ordinary skill in the art would easily understand that often doctors are provided with too much unnecessary information; thus, presenting binary options can be helpful to reduce analysis fatigue. Further, often there are not enough medical images for a learning data set, thus applying the meta-learning technique to the masking algorithm would allow for more efficient learning in that extra training data is not needed. Thus, it would have been obvious to a person having ordinary skill in the art at the time the claimed invention was filed to incorporate the teaching of Manhart to ensure only relevant information is output to a user, and the overall efficiency is increased.
Claim(s) 17 is rejected under 35 U.S.C. 103 as being unpatentable over Cao as applied to claim 13 above, and further in view of J. Li, C. Feng, X. Lin and X. Qian, "Utilizing GCN and Meta-Learning Strategy in Unsupervised Domain Adaptation for Pancreatic Cancer Segmentation," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 1, pp. 79-89, Jan. 2022, doi: 10.1109/JBHI.2021.3085092. (hereinafter Li).
Regarding dependent claim 17, the rejection of claim 13 is incorporated herein. Additionally, Cao fails to explicitly disclose wherein the medical image is selected from CT images and X- ray images. However, Li discloses wherein the medical image is selected from CT images (page 79, right column, “a few studies appeared on the automatic segmentation of pancreatic cancer [4]–[6], where these studies all focused on computed tomography (CT) images.”) and X- ray images.
Cao is directed toward “In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named MetaSeg. It consists of a meta-learner and a base-learner (abstract)” and the application of segmentations to images (See figure 1). Li is directed toward, “we propose an unsupervised domain adaptation segmentation framework for pancreatic cancer based on GCN and meta-learning strategy (abstract).” As can be easily seen by one of ordinary skill in the art both Co and Li are directed toward similar methods of endeavor of meta-learning for image processing. Further, Li allows for processing of additional image types such as CT (see above) and MRI (see page 80, left column). Cao also discloses “Semantic segmentation performs pixel-wise classification for given images, which can be widely used in autonomous driving, robotics, medical diagnostics and etc (abstract).” One of ordinary skill in the art at the time of filing the claimed invention would easily understand that there are a multitude of image types that can benefit from meta-learning. Thus, in order to ensure medical images can be processed using a meta-learning algorithm, it would have been obvious to a person having ordinary skill in the art a the time of filing the claimed invention to incorporate the teaching of Li.
Claim(s) 18 is rejected under 35 U.S.C. 103 as being unpatentable over Cao as applied to claim 13 above, and further in view of Hong.
Regarding dependent claim 18, the rejection of claim 13 is incorporated herein. Additionally, Cao fails to explicitly disclose wherein the knowledge rules include determining the organ region, identifying the tumors that reside in the organ region, and displaying the tumors in a specified brightness range. However, Hong discloses wherein the knowledge rules include determining the organ region (column 3, line 44, “The background model builder 210 estimates the intensity distribution of voxels in organ regions from the samples provided by user interactions The samples may be two or three small sampling blocks at, for example, liver regions. The background model builder 210 marks the liver regions in the input 3D liver CT image based on the estimated intensity distribution.”), identifying the tumors that reside in the organ region (column 4, line 66, “The tumor region locator 230 starts from the position initially provided by user interactions to identify substantially all the voxels that belong to the target liver tumor. It segments the target liver tumor using, for example, the likelihood field with a boundary-based constraint,”), and displaying the tumors in a specified brightness range (abstract, “the foreground model builder uses an intensity distribution estimate of voxels in a target organ tumor to build a first foreground model;” column 7, line 61, “The computer system 101 is generally coupled through the I/O interface 104 to a display 105 and various input devices 106 such as a mouse and keyboard”).
Cao is directed toward “In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named MetaSeg. It consists of a meta-learner and a base-learner (abstract)” and the application of segmentations to images (See figure 1). Hong is directed toward “A system for segmenting a target organ tumor from an image includes a background model builder, a foreground model builder and a tumor region locator (abstract).” As can be easily seen by one of ordinary skill in the art at the time of filing the claimed invention, both Cao and Hong are directed toward similar methods of endeavor of image analysis and specifically determining components of the image. Further, one of ordinary skill in the art would easily understand that often doctors are provided with too much unnecessary information; thus, presenting clear data in an output can be advantageous for accurate diagnosis. Said differently, an output can be confusing to a user with no context; thus identifying an organ to allow a user to realize what region of the body the images are and the tumor is within can be helpful. Further, often there are not enough medical images for a learning data set, thus applying the meta-learning technique to the masking algorithm would allow for more efficient learning in that extra training data is not needed. Thus, it would have been obvious to a person having ordinary skill in the art at the time the claimed invention was filed to incorporate the teaching of Hong to ensure accurate and contextual data is output to a user while utilizing the efficiency of meta-learning
Allowable Subject Matter
Claim 9 and 19 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of utilizing meta-learning to perform learning operations on medical image datasets. However, none of them alone or in any combination teaches translating knowledge rules using a combination of the deepnet and analytics approaches, so that human knowledge and image data are mixed in the image format.
The closest prior art being above cited Garcia discloses “The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase of performance on detecting melanoma, malignant (skin cancer) and benign moles with the prior knowledge obtained from images of everyday objects from ImageNet dataset by 20 points (abstract).” Garcia does not disclose a hybrid model of multiple methods being combined to then incorporate both human knowledge and image data in an image format.
Thus, Garcia fails to disclose translating knowledge rules using a combination of the deepnet and analytics approaches, so that human knowledge and image data are mixed in the image format.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
G. F. C. Campos, S. Barbon and R. G. Mantovani, "A Meta-Learning Approach for Recommendation of Image Segmentation Algorithms," 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Sao Paulo, Brazil, 2016, pp. 370-377, doi: 10.1109/SIBGRAPI.2016.058. discloses “This paper applies meta-learning to recommend segmentation algorithms based on meta-knowledge. We performed experiments in four different meta-databases representing various real world problems, recommending when three different segmentation techniques are adequate or not (abstract).”
U.S. Patent No. 11,620,582 to Chen et al. discloses, “Techniques regarding one or more automated machine learning processes that analyze time series data (abstract)”
U.S. Publication No. 2019/0147298 to Rabinovich et al. discloses, “Methods and systems for meta-learning are described for automating learning of child tasks with a single neural network (abstract).”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Nelson whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Examiner, Art Unit 2661