CTNF 18/719,316 CTNF 100743 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority Acknowledgement is made of Applicant’s claim of this application being a National Stage application of the PCT Application No. PCT/EP2022/086291, filed on December 16, 2022. As well as acknowledgement of priority to EP 22150655.3 with filing date of January 10, 2021, and priority to IN202141059703 with filing date of 12/21/2021. Information Disclosure Statement The information disclosure statement (“IDS”) filed on 06/13/2024 has been reviewed and the listed references have been considered. Drawings The 9-page drawings have been considered and placed on record in the file. 12-151 AIA 26-51 12-51 Status of Claims Claims 1-15 are pending. Claim Objections Claims 1-14 are objected to because of the following informalities: Claim 1 recites: “System for classifying lesions…” should be "A system for classifying lesions…" "the dual-modal image comprising a computed tomography (CT) image" should be "the dual-modal image comprising a computed tomography (CT) image" Claims 2 (similar claims 3-9) recites “System according to…" should be "The system according to…" Claim 7 recites “…to determine the reference image segment in the CT as an output” should be “…to determine the reference image segment in the CT image as an output” Claim 8 recites “…reference image segment in the computed tomography image…” should be “…reference image segment in the CT image…” Claim 10 recites “Method for classifying lesions…” should be “A method for classifying lesions…” Claim 11 recites “Method for choosing a set of image…” should be “A method for choosing a set of image…” Claims 12 (similarly claims 13-14) recites “Method according to…” should be “The method according to…” Appropriate corrections are required. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpretated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are, “an image providing unit for providing …” “an identifying unit for identifying …” “a normalization unit for normalizing …”, “an image feature extraction unit for extracting…” and “a classifying unit for classifying…” in claims 1-9. Because of these claim limitations being interpretated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009). Claim 6 recites “wherein the set of image features includes…” then lists the features for the CT image and PET image as “a total energy”, “a maximum two-dimensional diameter in row-direction”, “a sphericity”, “a surface area”, “a large area emphasis”, “a dependence entropy”, “a ninetieth percentile”, “a minimum”, “a flatness”, and “a sphericity”. The specification states on page 15 lines 27 and page 16 lines 1-3 “any combination of one or more of the CT features and one or more of the PET listed in Table 1 could be used”. Therefore, claim 6 is interpretated as the list being disjunctive for the list of features of the CT image and the PET image, any one of the elements found in the prior art is sufficient to reject the claim . While citations have been provided for completeness and rapid prosecution, only one element is required . Because, on balance, it appears the disjunctive interpretation enjoys the most specification support, and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Examiner suggests amending the claim to recite “…a large area emphasis or a dependence entropy…”. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history. Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. Claim 12 and 13 are rejected under 35 U.S.C 112(b). Claim 12 recites the limitation "…plurality of candidate ML architectures…". There is insufficient antecedent basis for this limitation in the claim because it is unclear if “…plurality of candidate ML architectures…".” is the same as “candidate ML architecture” in claim 11 or if it is something different. For examination purposes, they will be interpreted to be the same. Claim 12, dependent on claim 12 is similarly rejected. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim s 1-11 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Richter et al. (US 2020/0245960 A1) in view of Madabhushi et al. (US 2017/0352157 A1) . Regarding claim 1 , Richter teaches “System for classifying lesions (Richter paragraph [0014] "the full body segmentation approaches described herein allow for automated analysis of combinations of anatomical and functional images in order to accurately identify and grade cancerous lesions within a subject") , comprising: an image providing unit for providing a dual-modal image of an imaging region, the dual-modal image comprising a (CT) image and a positron emission tomography (PET) image registered to each other , wherein the imaging region (Richter paragraph [0202] "structural information and identified target VO Is from anatomical images are combined with and/or used to analyze images obtained via functional imaging modalities") includes a) a tissue region of interest comprising a lesion to be classified (Richter paragraph [0014] "The automated, machine learning-based segmentation approaches described herein are used to identify, within the CT image of the PET/CT composite, target volumes of interest (VOIs) representing target tissue regions where cancerous lesions may be found") and b) a reference tissue region comprising healthy tissue (Richter paragraph [0204] "In certain embodiments, reference regions are identified and used to compute normalization values and levels on a scale to which intensities in other regions are compared to evaluate a cancer status") , - an identifying unit for identifying, in each of the CT image and the PET image, a respective lesion image segment corresponding to the lesion in the tissue region of interest (Richter paragraph [0200] "The AI-based segmentation technologies described herein utilize machine learning techniques, such as Convolutional Neural Networks (CNNs) to automatically to identify a plurality of target 3D volumes of interest (VOis) each corresponding to a specific target tissue region, such as one or more organs, portions of organs, particular bone(s), a skeletal region etc. Each identified 3D VOI may be represented via a segmentation mask") and, in the PET image, a reference image segment corresponding to the reference tissue region (Richter paragraph [0204] "In certain embodiments, reference regions are identified and used to compute normalization values and levels on a scale to which intensities in other regions are compared to evaluate a cancer status") , a normalizing unit for normalizing the lesion image segment in the PET image with respect to the reference image segment in the PET image (Richter paragraph [0204] "In certain embodiments, reference regions are identified and used to compute normalization values and levels on a scale to which intensities in other regions are compared to evaluate a cancer status") , - an image feature extracting unit for extracting values of a set of image features associated with the lesion from the lesion image segment in the CT image and from the normalized lesion image segment in the PET image (Richter paragraph [0107] "the overall index value is determined as a weighted sum of at least a portion [ e.g., located within a 3D volume of the functional image that corresponds to a skeletal region of the subject; e.g., located within a 3D volume of the functional image that corresponds to a lymph region of the subject] of the individual hotspot index values [ e.g., wherein each hotspot index value in the sum is weighted by a measure of size ( e.g., 3D volume; e.g., average diameter) of the associated hotspot]") , and - a classifying unit for classifying the lesion according to its severity (Richter paragraph [0017] "The overall index can serve as an indicator of disease severity and/or risk, for example by reflecting a total lesion volume within a particular target tissue region, with the weighting based on the hotspot index values for the individual detected hotspots to account for radiopharmaceutical uptake within the lesions") based on the extracted image feature values (Richter paragraph [0241] "Hotspots may also be classified following their initial detection, e.g., as cancerous or not, and/or assigned likelihood values representing their likelihood of being a metastases. Hotspot classification may be performed by extracting hotspot features (e.g., metrics that describe characteristics of a particular hotspot) and using the extracted hotspot features as a basis for classification, e.g., via a machine learning module") .” However, Richter is not relied on to teach “an image feature extracting unit for extracting values of a set of image features associated with the lesion from the lesion image segment in the CT image” In an analogous field of endeavor, Madabhushi teaches “an image feature extracting unit for extracting values of a set of image features associated with the lesion from the lesion image segment (Madabhushi paragraph [0047] "Feature extraction circuit 553 extracts a set of radiomic features from the ROI. The set of radiomic features includes at least two texture features and at least one shape feature") in the CT image (Madabhushi paragraph [0014] "methods and apparatus non-invasively predict the extent or density of TILs in NSCLC tumors using computerized textural and shape features extracted from computed tomography (CT) images of a region of tissue demonstrating cancerous pathology") ”. It would have been obvious to a person having ordinary skill in the art before effective filing date of the claimed invention of the instant application to combine a system for classifying a lesion using CT and PET images and extracting features of a PET image as taught by Richter to include extracting features from a CT image as taught by Madabhushi. The suggestion/motivation for doing so would have been that there is a need in the field of cancer treatment, " Clinical trials with immune checkpoint inhibitors report significant increase in TILs in responders to treatment in follow up biopsies. However, since biopsies are invasive, time consuming, expensive, and may expose a patient to significant side effects, it would be beneficial to more accurately and non-invasively determine which patients are more likely to exhibit increased levels of TILs" as noted by the Madabhushi disclosure in paragraph 3. Therefore, it would have been obvious to combine the disclosure of Richter with the Madabhushi disclosure to obtain the invention as specified in claim 1 as there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results. Regarding claim 2 , the combination of Richter and Madabhushi teaches “System according to claim 1, wherein the identifying unit is adapted to identify, in each of the CT image and the PET image (Richter paragraph [0016] "an aorta and a liver VOI, corresponding to a representation of a portion of an aorta and liver, respectively, are identified within the anatomical image and mapped to the functional image to identify corresponding reference volumes therein") , a respective reference image segment corresponding to the reference tissue region, wherein the reference image segment in the PET image is identified as the image segment registered to the reference image segment identified in the CT image (Richter paragraph [0222] "the individual segmentation masks ( of the segmentation map) are mapped from the 3D anatomical image to the 3D functional image. The 3D volumes identified within the 3D functional image can be used for a variety of purposes in analyzing images for assessment of cancer status") .” Regarding claim 3 , the combination of Richter and Madabhushi teaches “System according to claim 1, wherein the tissue region of interest refers to a patient's prostate, the lesion is a prostate tumor region (Richter paragraph [0031] "the method comprises: (g) determining, by the processor, a cancer status [(e.g., a prostate cancer status; e.g., a metastatic cancer status ( e.g., metastatic cancer, including, e.g., metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.)] for the subject ( e.g., using intensities of voxels within the functional image and the one or more identified 3D volumes)(e.g., based on detected lesions) [e.g., a likelihood of the subject having and/or developing prostate cancer and/or a particular stage of prostate cancer ( e.g., metastatic prostate cancer)][e.g., a likelihood of the subject having and/or developing a metastatic cancer (including, e.g., metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.)]") and the PET image is generated using an imaging substance binding to prostate specific membrane antigen (Richter paragraph [0054] "the 3D PET image of the subject is obtained following administration to the subject of a radiopharmaceutical comprising a prostate-specific membrane antigen (PSMA) binding agent") .” Regarding claim 4 , the combination of Richter and Madabhushi teaches “System according to claim 3, wherein the lesion is classified according to a binary classification of severity based on a Gleason score (Richter paragraph [0243] Hotspots representing lesions may be used to determine risk indices that provide an indication of disease presence and/or state ( e.g., a cancer status, similar to a Gleason score) for a patient") , the binary classification distinguishing indolent from aggressive prostate tumor regions (Richter paragraph [0040] "step (i) comprises detecting an initial set of hotspots and, for at least a portion of the hotspots of the initial set, classifying each hotspot of at least a portion of the detected hotspots as either a cancerous lesion or not a cancerous lesion ( e.g., as noise) [ e.g., using a machine learning module; e.g., based on a shape and/or location of the hotspot ( e.g., in combination with anatomical knowledge; e.g., wherein the location includes an identification of a particular target tissue region corresponding to the 3D volume in which the hotspot is located and/or a relative position of the hotspot within the particular target tissue region); e.g., and removing hotspots classified as not a cancerous lesion from the initial set, thereby obtaining a final set of hotspots determined to represent lesions]") .” Regarding claim 5 , the combination of Richter and Madabhushi teaches “System according to claim 1, wherein the reference tissue region is a patient's liver, wherein the reference tissue region is the patient's liver (Richter paragraph [0016] "an aorta and a liver VOI, corresponding to a representation of a portion of an aorta and liver, respectively, are identified within the anatomical image and mapped to the functional image to identify corresponding reference volumes therein") .” Regarding claim 6 , the combination of Richter and Madabhushi teaches “System according to claim 1, wherein the set of image features includes, as features whose values are to be extracted from the lesion image segment in the CT image (Madabhushi paragraph [0014] "methods and apparatus non-invasively predict the extent or density of TILs in NSCLC tumors using computerized textural and shape features extracted from computed tomography (CT) images of a region of tissue demonstrating cancerous pathology") , a total energy, a maximum two-dimensional diameter in row-direction, a sphericity, a surface area, a large area emphasis and a dependence entropy (Madabhushi paragraph [0017] "The set of radiomic features includes textural and shape features. Textural analysis was performed on the entire tumor volume in this example. Performing textural analysis on the entire tumor volume, or on a threshold level of tumor volume, overcomes spatial heterogeneity in TIL distribution across the tumor volume. In this example, a total of 669 radiomic features, including radiomic texture features and radiomic shape features, were extracted from the tumor volume. The set of radiomic texture features includes Haralick features, gray level features, gradient features, Gabor features, local binary pattern (LBP) features, Law features, and Law-Laplacian features. The set of radiomic shape features includes location features, size features, perimeter features, eccentricity features, compactness features, roughness features, elongation features, convexity features, equivalent diameter features, extension features, and sphericity features") , and, as features whose values are to be extracted from the normalized lesion image segment in the PET image, a ninetieth percentile (Richter paragraph [0036] "automatically detecting, by the processor, one or more hotspots within the 3D functional image determined to represent lesions based on intensities of voxels within the 3D functional image [ e.g., based on a comparison of intensities within the 3D functional image with a threshold value (e.g., wherein the 3D functional image is a 3D PET image and the threshold is a particular standard uptake value (SUV) level)] (e.g., and also based on the one or more 3D volumes identified within the 3D functional image)") , a minimum, a flatness or a sphericity.” The proposed combination as well as the motivation for combining Richter and Madabhushi references presented in the rejection of claim 1, applies to claim 6. Finally the system recited in claim 6 is met Richter and Madabhushi. Regarding claim 7 , the combination of Richter and Madabhushi teaches “System according to claim 2, wherein the identifying unit comprises, for identifying the reference image segment in the CT image, a machine learning (ML) architecture adapted to receive a CT image of the imaging region including the reference tissue region as an input and to determine the reference image segment in the CT as an output (Richter paragraph [0200] " The AI-based segmentation technologies described herein utilize machine learning techniques, such as Convolutional Neural Networks (CNNs) to automatically to identify a plurality of target 3D volumes of interest (VO is) each corresponding to a specific target tissue region, such as one or more organs, portions of organs, particular bone(s), a skeletal region etc. Each identified 3D VOI may be represented via a segmentation mask") .” Regarding claim 8 , the combination of Richter and Madabhushi teaches “System according to claim 7, wherein the ML architecture comprised by the identifying unit comprises a convolutional neural network with convolutional, activation and pooling layers as a base network and, at a plurality of convolutional stages of the base network, a respective side set of convolutional layers connected to a respective end of the convolutional layers of the base network at the respective convolutional stage (Richter paragraph [0192] "for each of one or more filters applied by a convolution layer, response values computed by a given filter are stored in a corresponding output channel. Accordingly, a convolution layer that receives an input array having n+1 dimensions computes an output array also having n+1 dimensions, wherein the (n+1)th dimension represents the output channels corresponding to the one or more filters applied by the convolution layer. In this manner, an output array computed by a given convolution layer can be received as input by a subsequent convolution layer") , wherein the reference image segment in the computed tomography image is determined based on outputs of the plurality of side sets of convolutional layers (Richter paragraph [0200] "The AI-based segmentation technologies described herein utilize machine learning techniques, such as Convolutional Neural Networks (CNNs) to automatically to identify a plurality of target 3D volumes of interest (VO is) each corresponding to a specific target tissue region, such as one or more organs, portions of organs, particular bone(s), a skeletal region etc. Each identified 3D VOI may be represented via a segmentation mask") .” Regarding claim 9 , the combination of Richter and Madabhushi teaches “System according to claim 1, wherein the classifying unit comprises, for classifying the lesion, a ML architecture suitable for receiving image feature values as an input and determining an associated lesion severity class as an output (Richter paragraph [0241] "Hotspots may also be classified following their initial detection, e.g., as cancerous or not, and/or assigned likelihood values representing their likelihood of being a metastases. Hotspot classification may be performed by extracting hotspot features (e.g., metrics that describe characteristics of a particular hotspot) and using the extracted hotspot features as a basis for classification, e.g., via a machine learning module") .” Claim 10 recites a method with steps corresponding to the system with elements recited in claim 1. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements of system claim 10. Additionally, the rationale and motivation to combine the Richter and Madabhushi references, presented in rejection of claim 1 apply to this claim. Regarding claim 11 , the combination of Richter and Madabhushi teaches “Method for choosing a set of image features and a machine learning architecture (ML for use by a system according to claim 1 for classifying lesions, comprising: receiving a set of dual-modal training images (Richter paragraph [0202] "structural information and identified target VO Is from anatomical images are combined with and/or used to analyze images obtained via functional imaging modalities") , each training image comprising a CT image and a PET image registered to each other (Richter paragraph [0202] "structural information and identified target VO Is from anatomical images are combined with and/or used to analyze images obtained via functional imaging modalities") , in each of which a respective lesion image segment corresponding to a respective lesion in a tissue region of interest has been identified, wherein, for each of the lesions in the training images, an assigned severity class is provided (Richter paragraph [0284] "in order to train the machine learning modules used to perform the segmentations, numerous pre-labeled sample images, such as the three images (710, 720, 730) shown in FIG. 7, were used as a training data set. The machine learning modules were fine-tuned using l000's of manually annotated labels along with l00's of models trained on dozens of GPUs over l000's of hours before optimal configurations for the many components of the segmentation platform were discovered. Once trained, however, segmentation can be performed rapidly") , - choosing a candidate set of image features from a predefined collection of possible image features (Madabhushi paragraph [0040] "Method 200 further includes, at 290, training a machine learning classifier using the set of top-ranked features") , - generating a training dataset by extracting, from each of the lesion image segments identified in the training images, corresponding values of each of the candidate set of image features and assigning them to the respective severity class assigned to the lesion in the respective training image (Madabhushi paragraph [0035-0036] "accessing a set of pre-surgical CT images corresponding to the outliers. A member of the set of pre-surgical CT images represents a pre-surgical tumor volume associated with the tumor represented in the set of H&E stained slides. Accessing the set of pre-surgical CT images may include retrieving electronic data from a computer memory, receiving a computer file over a computer network, or other computer or electronic based action. Method 200 also includes, at 250 generating an annotated tumor volume by annotating the tumor volume represented in a member of the set of pre-surgical CT images") , - choosing a candidate ML architecture suitable for receiving image feature values as an input and determining an associated lesion severity class as an output (Richter paragraph [0207] "As shown in FIG.4, a machine learning module 402, such as a convolutional neural network (CNN), receives contextual anatomical information ( e.g., a CT image or other anatomical imaging modality) 404 along with functional information (e.g., a functional image, such as a SPECT, PET, or other functional image obtained using a particular imaging probe) 406. The machine learning module 402 then uses the functional 406 and anatomical 404 information to identify, classify, and/or quantify cancerous lesions in a subject 410") , - training the candidate ML architecture on the training dataset (Madabhushi paragraph [0040] "training the machine learning classifier may further include testing the machine learning classifier") , - receiving a set of dual-modal test images (Richter paragraph [0202] "structural information and identified target VO Is from anatomical images are combined with and/or used to analyze images obtained via functional imaging modalities") , each test image comprising a CT image and a PET image (Richter paragraph [0202] "structural information and identified target VO Is from anatomical images are combined with and/or used to analyze images obtained via functional imaging modalities") , in each of which a respective lesion image segment corresponding to a respective lesion in a tissue region of interest has been identified, wherein, for each of the lesions in the test images, an assigned severity class is provided (Richter paragraph [0284] "in order to train the machine learning modules used to perform the segmentations, numerous pre-labeled sample images, such as the three images (710, 720, 730) shown in FIG. 7, were used as a training data set. The machine learning modules were fine-tuned using l000's of manually annotated labels along with l00's of models trained on dozens of GPUs over l000's of hours before optimal configurations for the many components of the segmentation platform were discovered. Once trained, however, segmentation can be performed rapidly") , generating a test dataset by extracting, from each of the lesion image segments identified in the test images, corresponding values of each of the candidate set of image features and assigning them to the respective severity class assigned to the lesion in the respective test image (Madabhushi paragraph [0035-0036] "accessing a set of pre-surgical CT images corresponding to the outliers. A member of the set of pre-surgical CT images represents a pre-surgical tumor volume associated with the tumor represented in the set of H&E stained slides. Accessing the set of pre-surgical CT images may include retrieving electronic data from a computer memory, receiving a computer file over a computer network, or other computer or electronic based action. Method 200 also includes, at 250 generating an annotated tumor volume by annotating the tumor volume represented in a member of the set of pre-surgical CT images") , - determining a performance of the candidate ML architecture on the test dataset based on a relation between the assigned severity classes provided for the lesions in the test images and the corresponding lesion severity classes determined by the candidate ML architecture (Madabhushi paragraph [0040] "determining that the machine learning classifier trained on a first set of top ranked features does not achieve a threshold level of accu-racy in classifying an ROI as TIL positive or TIL negative") , - repeating the above steps with increasing numbers of image features in the candidate set of image features, wherein for each repetition, the candidate set of image features is enlarged by a further image feature from the predefined collection of possible image features, the further image feature being the one that results in the highest increase in performance, until an abort criterion is met (Madabhushi paragraph [0040] "determining that the machine learning classifier trained on a first set of top ranked features does not achieve a threshold level of accu-racy in classifying an ROI as TIL positive or TIL negative, method 200 may select a second different set of ranked features to train the machine learning classifier") , and - repeating the above steps with at least one candidate ML architecture (Madabhushi paragraph [0040] "determining that the machine learning classifier trained on a first set of top ranked features does not achieve a threshold level of accu-racy in classifying an ROI as TIL positive or TIL negative, method 200 may select a second different set of ranked features to train the machine learning classifier") , wherein the set of image features and the ML architecture to be used for classifying lesions are chosen from the respective candidates according to the determined performances (Madabhushi paragraph [0040] "Different radiomic features may have different discriminability performance levels based, in part, on the slice thickness. Other properties or combinations of properties of the CT image, including resolution, may affect the discriminability of a radiomic feature. In another embodiment, method 200 may extract the set of candidate set of features from a threshold fraction (e.g. ¼, ½, 2/2, '1/s) of the annotated tumor volume. Thus, example methods and apparatus may select different radiomic features or combinations of radiomic features and may provide different levels of discriminability based on a property (e.g. slice thickness) of the CT image") .” The proposed combination as well as the motivation for combining Richter and Madabhushi references presented in the rejection of claim 1, applies to claim 11. Finally the method recited in claim 11 is met Richter and Madabhushi. Regarding claim 14 , the combination of Richter and Madabhushi teaches “Method according to claim 11,wherein the test images include different subsets of the training images, thereby performing cross- validation (Richter paragraph [0276] "A deep learning algorithm based on cascaded deep learning convolutional neural networks for semantic segmentation of 12 skeletal regions was developed. In particular, the 12 skeletal regions were the thoracic and lumbar vertebrae, sinister (left)/dexter (right) ribs, sternum, sinister (left)/dexter (right) clavicle, sinister (left)/dexter (right) scapula, sinister (left)/dexter (right) ilium, and the sacrum. A training set (N=90) and validation set (N=22) of pairs of low-dose CT images and manually crafted segmentation maps were used to develop the deep learning algorithm. The algorithm's performance was assessed on a test set (N=l 0) of low-dose CT images obtained from a PyL™-PSMA study. In the test set of images, five representatively body parts: sinister (left) ilium, lumbar vertebrae, sinister (left) ribs, dexter (right) scapula, and sternum were manually segmented. These manual segmentations were used as ground truth for evaluation of the automated segmentation procedure") .” Claim 15 recites a computer readable medium including computer program corresponding to the elements of the system recited in claim 1. Therefore, the recited program of the computer readable medium of claim 15 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 1. Additionally, the rationale and motivation to combine Richter and Madabhushi presented in rejection of claim 1, apply to this claim . 07-21-aia AIA Claim s 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Richter and Madabhushi in view of Malato ("Hyperparameter tuning. Grid search and random search" - 2021) Regarding claim 12 , the combination of Richter and Madabhushi teaches the method of claim 11. However, the combination of Richter and Madabhushi is not relied on to teach “wherein the plurality of candidate ML architectures correspond to random forests with different hyperparameters”. Malato teaches “wherein the plurality of candidate ML architectures correspond to random forests with different hyperparameters (Malato page 2 paragraph 4 "Hyperparameters are model parameters whose values are set before training. For example, the number of neurons of a feedforward neural network is a hyperparameter, because we set it before training. Another example of hyperparameter is the number of trees in a random forest or the penalty intensity of a Lasso regression. They are all numbers that are set before the training phase and their values affect the behavior of the model") .” It would have been obvious to a person having ordinary skill in the art before effective filing date of the claimed invention of the instant application to combine a system for classifying a lesion by extracting and analyzing features from CT and PET images as taught by Richter and Madabhushi to include a random forest of machine learning architecture with different hyperparameters as taught by Malato. The suggestion/motivation for doing so would have been “Why should we tune the hyperparameters of a model? Because we don't really know their optimal values in advance. A model with different hyperparameters is, actually, a different model so it may have a lower performance" as noted by the Malato disclosure in page 2 paragraph 5. Therefore, it would have been obvious to combine the disclosure of Richter and Madabhushi with the Malato disclosure to obtain the invention as specified in claim 12 as there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results. Regarding claim 13 , the combination of Richter, Madabhushi, and Malato teaches “Method according to claim 12, wherein the hyperparameters are chosen successively according to a grid search (Malato page 3 paragraph 3 "Grid search is the simplest algorithm for hyperparameter tuning. Basically, we divide the domain of the hyperparameters into a discrete grid. Then, we try every combination of values of this grid, calculating some performance metrics using cross validation") .” The proposed combination as well as the motivation for combining Richter, Madabhushi, and Malato references presented in the rejection of claim 12, applies to claim 13. Finally the method recited in claim 13 is met Richter, Madabhushi, and Malato. Reference Cited 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. “Deep transfer learning-based prostate cancer classification using 3 Tesla multi-parametric MRI” to Zhong et al. discloses a deep learning approach to classify prostate cancel lesions using MRI imaging by identifying region of interest and extracting features within the region . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASPREET KAUR whose telephone number is (571)272-5534. The examiner can normally be reached Monday - Friday 7:30 am - 4:00 PST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice . If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov . Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JASPREET KAUR/Examiner, Art Unit 2662 /AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662 Application/Control Number: 18/719,316 Page 2 Art Unit: 2662 Application/Control Number: 18/719,316 Page 3 Art Unit: 2662 Application/Control Number: 18/719,316 Page 4 Art Unit: 2662 Application/Control Number: 18/719,316 Page 5 Art Unit: 2662 Application/Control Number: 18/719,316 Page 6 Art Unit: 2662 Application/Control Number: 18/719,316 Page 7 Art Unit: 2662 Application/Control Number: 18/719,316 Page 8 Art Unit: 2662 Application/Control Number: 18/719,316 Page 9 Art Unit: 2662 Application/Control Number: 18/719,316 Page 10 Art Unit: 2662 Application/Control Number: 18/719,316 Page 11 Art Unit: 2662 Application/Control Number: 18/719,316 Page 12 Art Unit: 2662 Application/Control Number: 18/719,316 Page 13 Art Unit: 2662 Application/Control Number: 18/719,316 Page 14 Art Unit: 2662 Application/Control Number: 18/719,316 Page 15 Art Unit: 2662 Application/Control Number: 18/719,316 Page 16 Art Unit: 2662 Application/Control Number: 18/719,316 Page 17 Art Unit: 2662 Application/Control Number: 18/719,316 Page 18 Art Unit: 2662 Application/Control Number: 18/719,316 Page 19 Art Unit: 2662 Application/Control Number: 18/719,316 Page 20 Art Unit: 2662 Application/Control Number: 18/719,316 Page 21 Art Unit: 2662 Application/Control Number: 18/719,316 Page 22 Art Unit: 2662 Application/Control Number: 18/719,316 Page 23 Art Unit: 2662 Application/Control Number: 18/719,316 Page 24 Art Unit: 2662 Application/Control Number: 18/719,316 Page 25 Art Unit: 2662 Application/Control Number: 18/719,316 Page 26 Art Unit: 2662 Application/Control Number: 18/719,316 Page 27 Art Unit: 2662 Application/Control Number: 18/719,316 Page 28 Art Unit: 2662
Prosecution Insights