DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The IDS(s) has/have been considered and placed in the application file.
Priority
Applicant's claim for foreign priority to European application No. 22305747.2 (filed May 19, 2022) is acknowledged, certified copies of papers required by 37 CFR 1.55 have been received. Claims 1-4, 7-13, 15, and 16 are entitled to that date. Claims 5 and 6 are not. The priority application discloses the feedback only as “a feedback linking the output of the network and the bottle-neck region” (claim 5) and, in the description, as concatenation “with the output of the building block of the layer of lowest resolution of the encoder.” It does not discloses concatenation to the lowest-resolution layer of the forward system, nor the recited “for at least one training phase” or its multi-phase training procedure. This matter was first added in the PCT (PCT/EP2023/063366, filed May 17, 2023). Therefore, Claims 5 and 6 are accorded an effective filing date of May 17, 2023.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8 (9 by dependence) and 10 (11 by dependence) rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 8 and 10 are rejected under 35 U.S.C. 112(b) as indefinite: “the lesion dissemination” (claim 8) and “the lesion burden” (claim 10) lack antecedent basis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 7, 10-13, 15, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable Blanc-Durand et al., “Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PETICT for total metabolic tumour volume prediction using a convolutional neural network,” (hereinafter “Blanc-Durand”), in view of Xiao et al., “Automated dendritic spine detection using convolutional neural networks on maximum intensity projected microscopic volumes,” (hereinafter “Xiao”).
Claim 1.
Blanc-Durand discloses a method of processing imaging data of a patient having cancer (Blanc-Durand: “DLBCL” (Abstract).), comprising:
providing three-dimensional imaging data of the patient (Blanc-Durand: “Both resampled/rescaled PET and CT image volumes served as inputs for the CNN” (p.1364)),
computing from said three-dimensional imaging data, at least one two-dimensional Maximum Intensity Projection (MIP) image (Blanc-Durand: masks shown “superimposed over the PET maximum intensity projection” (Fig. 2, p.1366)), corresponding to a projection of maximum intensity of the three-dimensional imaging data along one direction onto one plane (Blanc-Durand computes a MIP of the PET volume – masks are shown “superimposed over the PET maximum intensity projection” (Fig. 2, p.1366); see also Xiao: “maximal intensity projection (MIP) images are generated from each volume” (Section 2.1). A MIP is, by definition, the projection of the maximum voxel intensity of the volume along the projection direction onto the projection plane.),
extracting a mask (Blanc-Durand: "A 3D U-net architecture with 2 input channels for PET and CT was trained" to segment the lesions (Abstract)).
Blanc-Durand applies the model to the three-dimensional volume, not the MIP.
However, Xiao applies a trained model to a two-dimensional MIP to extract a mask (Xiao:
"maximal intensity projection (MIP) images are generated from each volume ... These
projected 2D images are input into the CNNs. The output of the CNNs leads to a probability map of the location of [structures]" (Section 2.1), trained against "binary masks" and binarized (Fig. 9)).
Xiao is analogous art, reasonably pertinent to the inventor's problem of efficiently segmenting lesions from three-dimensional biomedical volumes. It would have been obvious, before the effective filling date, to segment on the two-dimensional MIP, as Xiao teaches, rather than on the volume, for the predictable reduction in training and inference cost – a known technique applied to yield a predictable result (MPEP 2143(D)).
Claim 2.
The combination of Blanc-Durand and Xiao discloses, wherein the three-dimensional imaging data is PET scan data (Blanc-Durand: "FDG-PET/CT" (Abstract)).
Claim 3.
The combination of Blanc-Durand and Xiao discloses, comprising computing from the three-dimensional imaging data two MIP images corresponding to the projection of the maximum intensity of the three-dimensional imaging data onto two orthogonal planes (Blanc-Durand computes a PET MIP (Fig. 2)). Computing two MIPs onto orthogonal coronal and sagittal planes (i.e., the standard whole-body PET views) would have been obvious, reducing the lesion overlap lost in a single projection (MPEP 2143(A)).
Claim 4.
The combination of Blanc-Durand and Xiao discloses, wherein the trained model has been previously trained by supervised learning on a database comprising a plurality of MIP images corresponding to projections of three-dimensional imaging data according to a first plane, and a plurality of MIP images corresponding to projections of three-dimensional imaging data according to a second plane, orthogonal to the first, and, for each MIP image, a corresponding mask of the image corresponding to cancerous lesions (Xiao trains the network by supervised learning on projected MIP images and corresponding "binary masks" (Section 2)). Extending that training to the two orthogonal-plane MIPs of claim 3 is a predictable variation.
Claim 7.
The combination of Blanc-Durand and Xiao discloses a method for assisting with cancer prognosis comprising: performing the method according to claim 1 on three-dimensional imaging data of a patient to output a two-dimensional cancerous lesion mask of a MIP image computed from the three-dimensional imaging data, and- processing said cancerous lesion mask to compute at least one prognosis indicator (Blanc-Durand computes "TMTV ... as the sum of voxels in the binarized ... predicted masks ... multiplied by the voxel volume" (p.1364), an established DLBCL prognostic biomarker.).
Claim 10.
The combination of Blanc-Durand and Xiao discloses, wherein the at least one prognosis indicator comprises an indicator of the lesion burden (Blanc-Durand's TMTV quantifies the metabolically active tumor burden (p.1364).).
Claim 11.
The combination of Blanc-Durand and Xiao discloses, wherein processing the cancerous lesion mask comprises computing a number of pixels belonging to the lesion multiplied by the area represented by each pixel (Blanc-Durand computes TMTV as "the sum of voxels in the binarized ... predicted masks ... multiplied by the voxel volume" (p.1364) – a lesion-pixel count times per-pixel area).
Claim 12.
The combination of Blanc-Durand and Xiao discloses, wherein the cancer is a lymphoma (Blanc-Durand segments lesions in "diffuse large B cell lymphoma (DLBCL)" patients (Abstract)).
Claim 13.
The combination of Blanc-Durand and Xiao discloses, wherein the lymphoma is Diffuse Large B-cell Lymphoma (Blanc-Durand segments lesions in "diffuse large B cell lymphoma (DLBCL)" patients (Abstract)).
Claim 15.
The combination of Blanc-Durand and Xiao discloses a non-transitory computer readable storage having stored thereon code instructions for implementing the method according to claim7, when they are executed by a processor (Blanc-Durand: “models for inference” made freely available (p. 1364); implemented as software with freely available inference models; “Ubuntu 16.04 workstation equipped with two 11-Go GTX1080Ti graphical processing units” (p. 1364)).
Claim 16.
The combination of Blanc-Durand and Xiao discloses, a non-transitory computer readable storage having stored thereon code instructions for implementing the method according to claim 1, when they are executed by a processor (Blanc-Durand: “models for inference” made freely available (p.1364); implemented as software with freely available inference models; “Ubuntu 16.04 workstation equipped with two 11-Go GTX1080Ti graphical processing units” (p. 1364)).
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Blanc- Durand in view of Girum et al., “Learning With Context Feedback Loop for Robust Medical Image Segmentation,” (hereinafter “Girum”), and, as to claim 6, further in view of Jafari et al., “FU-net: Multi-class Image Segmentation Using Feedback Weighted U-net,” (hereinafter “Jafari”).
Claim 5.
The combination of Blanc-Durand and Xiao discloses, wherein the trained model is a Convolutional Neural Network comprising a forward system comprising: an encoder region comprising a succession of layers of decreasing resolutions, a decoder region comprising a succession of layers of increasing resolutions, wherein a layer of the decoder region concatenates the output of the layer of the encoder region of the same resolution with the output of the layer of the decoder region of the next lower resolution, a bottle-neck region between the encoder and decoder regions (Blanc-Durand: "A standard U-net architecture ... composed of an encoder and a decoder network with skipped connections between the two" (p.1364)), and
Blanc-Durand and Xiao teaches everything except for the feedback system. However, Girum teaches it (Girum: "The feedback system's output (i.e., hf … ) is concatenated with forward system encoder's output (i.e., hs)" (Fig. 1); the feedback system is divided "into two: the encoder Fe ... and the decoder Fd" (Section II-B); trained in phases – “Train ... the forward system ... zero feedback ... Train ... the feedback system ... While not converged, repeat” (Section II-D)).
The motivation to combine Girum's feedback loop would be to let the network "learn and extract more high-level image features and fix previous mistakes," improving accuracy; Applicant admits the model “may be the model disclosed by Kibrom Berihu Girum et al." (Spec. p. 7).
Claim 6.
The combination of Blanc-Durand and Xiao discloses, wherein the encoder, decoder and bottle-neck regions of the network comprise building blocks where each building block is a residual block comprising at least a convolutional layer and an activation layer, with a skip connection between the input of the block and the activation layer (Girum uses a "residual" convolutional building block (Fig. 1; Section II); Jafari, layers of "two 3x3 convolutions (Conv)" each “followed by a rectified linear unit (ReLU) as illustrated in Fig. 1(a)" with an added "residual block (RB) ... Fig. 1(b)" (Section 2.1).).
The motivation to combine Blanc-Durand, Xiao, and Girum's feedback loop would be to let the network "learn and extract more high-level image features and fix previous mistakes" (Girum Abstract) improving accuracy; Applicant admits the model "may be the model disclosed by Kibrom Berihu Girum et al." Jafari’s residual blocks "achieve faster convergence and train deeper networks" (Jafari § 2.1). Both are predictable improvements to Blanc-Durand's network (MPEP 2143(D)).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Blanc-Durand in view of Xiao, and further in view of Cottereau et al., “18F-FDG PET Dissemination Features in Diffuse Large B-Cell Lymphoma Are Predictive of Outcome,” (hereinafter “Cottereau”).
Claim 8.
The combination of Blanc-Durand and Xiao discloses, wherein the at least one prognosis indicator comprises an
Blanc-Durand and Xiao do not teach a dissemination indicator. Cottereau does (Cottereau: a dissemination feature, "the distance between the two lesions that were the furthest apart (Dmaxpatient)," predictive of outcome in DLBCL (Abstract; Methods)).
It would have been obvious to compute Cottereau's dissemination biomarker from the lesion mask for its recognized value in predicting progression-free and overall survival – a predictable addition to the burden indicator (MPEP 2143(D)).
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The following prior art made of record but not relied, yet considered pertinent to the applicant’s disclosure:
CN 111091530 A (Huazhong Univ. of Science and Technology) – teaches computing a maximum intensity projection of a three-dimensional volume and extracting a lesion mask therefrom by a semantic-segmentation neural network.
US 2020/0349712 A1 (Kardiolytics Inc.) – teaches a trained convolutional neural network that receives a maximum intensity projection image as input and outputs a segmentation mask of the imaged structure.
US 2022/0122729 A1 (JLK Inc.) – teaches inputting a maximum intensity projection of three-dimensional angiographic data to a neural network to identify lesions.
US 2025/0000473 A1 (Genentech, Inc. ) – teaches convolutional-neural-network segmentation of lesions in PET imaging data and computation of total metabolic tumor volume from the resulting mask.
US 2023/0351586 A1 (EXINI Diagnostics AB) – teaches machine-learning detection and segmentation of lesions in PET /SPECT images and aggregation of the results into a tumor-burden and risk index.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights