Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of CN202311674971X, filed in China on 12/08/2023, and PCT/CN2024/089581, filed in China on 04/24/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/03/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
In paragraph [0005], page 2, line 11, “is susceptibility” is suggested to read “is susceptible.”
In paragraph [0014], page 4, line 24, “ranges from 0 to 1” is suggested to read “ranging from 0 to 1.”
In paragraph [0031], page 8, line 1, “a data processing” is suggested to read “data processing.”
In paragraph [0034], page 9, line 7, “ranges from 0 to 1” is suggested to read “ranging from 0 to 1.”
Appropriate correction is required.
Claim Objections
Claims 1 and 3 are objected to because of the following informalities:
In claim 1, on lines 3, 23, and 24-25, “internal organ generation model of an/the imaging target” is suggested to read “internal organ generation model for generating an/the internal structure of an/the imaging target.” As it is currently written, the claim language is unclear. The Examiner suggests adjusting the language of the limitation to improve clarity.
In claim 3, on line 5, “ranges from 0 to 1” is suggested to read “ranging from 0 to 1.”
In claim 3, on lines 5-7, the claim states, “t represents a Gaussian noise” and later recites “time point t” It is unclear as to whether “ t ” represents a Gaussian noise or a time point in the equation.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5 and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope.
With respect to Claim 5, the claim recites the following, each of which renders the claim indefinite:
“ a binary mask image ” on line 6 (unclear antecedent basis); it is unclear as to whether “a binary mask” recited on line 6 of claim 5 is the same or different from “a binary mask” recited on lines 14-15 of claim 1. The Examiner suggests replacing “a” with “the”: “the binary mask image” on line 6 in claim 5.
With respect to Claim 7, the claim recites the following, each of which renders the claim indefinite:
“ an original three-dimensional body surface contour image ” on lines 3-4 (unclear antecedent basis); it is unclear as to whether “an original three-dimensional body surface contour image” recited on lines 3-4 of claim 7 is the same or different from “an original three-dimensional body surface contour image” recited on line 12 of claim 1. The Examiner suggests replacing “an” with “the”: “ the original three-dimensional body surface contour image ” on lines 3-4 in claim 7.
“ an organ mask image ” on line 4 of claim 7 (unclear antecedent basis); it is unclear as to whether “an organ mask image” recited on line 4 of claim 7 is the same or different from “an organ mask image” on line 16 in claim 1. The Examiner suggests replacing “an” with “the”: “the organ mask image ” on line 4 of claim 7.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 2 are rejected under 35 U.S.C. 103(a) as being unpatentable over Singh et al. (U.S. Patent Pub. No. 20180228460 A1, hereafter referred to as Singh) in view of Sjöstrand et al. (U.S. Patent Pub. No. 20250191752 A1, hereafter referred to as Sjöstrand) in further view of Huy et al. (NPL “Denoising Diffusion Medical Models,” April 2023, hereafter referred to as Huy).
[AltContent: arrow]Regarding Claim 1, Singh teaches a deep learning-based method for generating an internal structure of an organism (Paragraphs [0043], [0024], [0026], Fig. 2, Singh teaches a method using a deep neural network for predicting the internal anatomy of the patient based on patient surface measurements. The predicted internal anatomy may include, but is not limited to, internal body markers such as a lung center, thyroid, organ surfaces such as lung, heart, brain, etc.),
PNG
media_image1.png
216
488
media_image1.png
Greyscale
the deep learning-based method comprising the following steps: S1, constructing (Paragraphs [0037], [0039-40], Fig. 3A, Singh teaches using machine learning to predict an internal surface of an organ or body tissue. The prediction block (119) can be implemented by the processor (111). The prediction is performed by the prediction block based on regression, using machine learning.) and training an internal organ generation model of an imaging target (Paragraph [0040], Figs. 2 & 3A, Singh teaches training the prediction block (119) of the system (100) of Fig. 2. Fig. 3A (below) shows the data flow during the training phase. A regressor (203) determines a function f for predicting the compact representation (104’) of the organ surface (104) based on the compact representation (102’) of the body surface (102). A second autoencoder (202) constructs the predicted organ surface data (104) based on the compact representation (104’) of the organ surface (104).);
PNG
media_image2.png
513
682
media_image2.png
Greyscale
S2, obtaining a three-dimensional body surface contour image of the imaging target (Paragraph [0032], Fig. 1, Singh teaches a depth sensor (101) for sensing an outer surface (102) of a body of a subject to collect body surface data. The 3D depth sensor (101) captures a depth image of the outer body surface (102) of the patient.);
PNG
media_image3.png
516
694
media_image3.png
Greyscale
S3, inputting the obtained three-dimensional body surface contour image of the imaging target into the internal organ generation model (Paragraphs [0040-41], [0047], Fig. 3B, Singh teaches inputting the body surface data/depth image (102) captured by the 3D camera (101) into the prediction block (119) of the system. See Fig. 3B below.)
[AltContent: arrow][AltContent: arrow]
PNG
media_image4.png
497
634
media_image4.png
Greyscale
to obtain a three-dimensional internal organ distribution image (Paragraph [0032], Fig. 2, Singh teaches using machine learning to predict a surface of an internal organ (104) of the subject based on the body surface data. The organs can be lungs, heart, liver, kidneys, or brain, for example. The Examiner interprets the surface of the internal organ to be a “3D internal organ distribution image” since it shows the arrangement/position of the organ within the patient’s body.);
PNG
media_image5.png
215
488
media_image5.png
Greyscale
[AltContent: arrow](Paragraph [0037], Fig. 1, Singh teaches the 3D sensor captures a depth image of outer body surface (102) of the patient.) and an internal structure image of the imaging target (Paragraph [0085], Figs. 9A-9C, Singh teaches an input CT scan of a patient representing the “ground truth” CT volume with all anatomical details. The CT scan included a plurality of slices parallel to the frontal or coronal plane. The Examiner interprets the “internal structure image” to be a medical image such as a CT/MRI that shows the internal structure(s) of the body since the claim is silent to the meaning of “internal structure image.” The image below show the patient’s lungs.);
PNG
media_image6.png
277
664
media_image6.png
Greyscale
S102, performing segmentation on the internal structure image (Paragraphs [0112], [0119], Singh teaches applying a deep network for segmenting the topogram image by labelling each pixel to belong to one of the following classes: background, body (inside patient’s body), lung or bone.) normalizing the original three-dimensional body surface contour image (Paragraph [0105-106], Singh teaches the volumes may be normalized based on multiple body markers, such that the normalized space for the number of voxels from neck to ischium is fixed across all patients.), taking the normalized three-dimensional body surface contour image as an input image of a deep learning neural network (Paragraph [0041-42], Singh teaches during training, a large number (e.g., 1,000 to 10,000) of body surface images (102) are input into the decoder (201e) of the first autoencoder (201). The training dataset of organ surface data (104) can correspond to the same or different set of patients from the training dataset of body surface data (102).), taking the organ mask image as an output result of the deep learning neural network (Paragraph [0101], Fig. 11, Singh teaches generating a volumetric mask which represents the organ volume. The Examiner interprets the volumetric mask to be a 3D organ mask image. The claim does not specify the dimensionality of the mask image (i.e., that the mask is two-dimensional (2D).).), and taking the three-dimensional body surface contour image (Paragraphs [0050-51], Fig. 4, Fig. 3A, Singh teaches in training stage 1, the body surface images (102) are input to an encoder (201). The encoder (201e) encodes the body surface images (102) and provides body latent variables or principal coordinates (102’). The decoder (201d) decodes the body latent variables or principal coordinates 102’ and generates body surface images (102).) and the organ mask image (Paragraphs [0052-53], [0029], Fig. 5, Fig. 3A, Singh teaches in training stage 2, the prediction block (119) uses a second autoencoder (202) to learn the manifold for points on an internal surface of an organ (e.g., lungs, heart, liver, kidneys). As shown in Fig. 5, organ surface images (104) care input into the autoencoder (202). The encoder (202e) encodes the organ surface images (104) and provides latent variables of principal components (104’) and generates organ surface images (104). The internal structures may be represented as volumetric masks.
as a training data sample pair (Paragraph [0062], Singh teaches training the regressor with a third dataset. For each body surface input image data, the internal organ surface image data corresponding to the same subject are included. Body surface images without corresponding organ images are excluded from the third dataset.); and S105, training the internal organ generation model of the imaging target using the training data sample pair (Paragraph [0043], Singh teaches for training the regressor (203), the training dataset should include a large number (e.g., 600 to 10,000) of sets, where each set includes body latent variables or principal component 102’ and respective organ latent variables or principal component 104’ corresponding to the same person.), wherein the internal organ generation model of the imaging target is based on a neural network (Paragraph [0043], Singh teaches the regressor is a deep neural network (DNN).).
Singh does not explicitly disclose S4, performing superimposition and merging on the three-dimensional internal organ distribution image and the three-dimensional body surface contour image, and displaying a result obtained based on the superimposition and the merging; obtain(ing) a binary mask image of an organ; and S103, obtaining an organ mask image registered with the original three-dimensional body surface contour image.
Sjöstrand is in the same field of art of segmenting medical images to identify internal structures of interest in the body such as organs. Further, Sjöstrand teaches S4, performing superimposition (Paragraphs [0105], [0100], Sjöstrand teaches overlaying the boundaries of particular regions (e.g., segmentation masks), such as particular organs, identified in a segmentation map (210) (e.g., a 3D segmentation map) upon functional image (306) to identify volumes within functional image (306). The Examiner interprets “overlaying” to be synonymous to “superimposition.” The Examiner interprets the 3D segmentation map to be a “three-dimensional internal organ distribution image” since it identifies the boundaries and locations of the organs within the body. In addition, the Examiner interprets the functional image to be a “three-dimensional body surface contour image” since the functional image is three-dimensional and shows the contour/outline/shape of the patient’s body surface.) and merging on the three-dimensional internal organ distribution image and the three-dimensional body surface contour image (Paragraph [0105], Sjöstrand teaches transferring the segmentation map (e.g., a 3D segmentation map) upon a functional image such as a 3D functional image to identify volumes within the functional image for purposes of classifying useful indices. The Examiner interprets “transferring upon” to be synonymous to “merging.” In addition, the Examiner interprets the (3D) segmentation map to be a “three-dimensional internal organ distribution image” since it identifies one or more organs of interest and depicts their distribution (arrangement within the body). In addition, under BRI, the Examiner interprets the 3D functional image to be a “three-dimensional body surface contour image” since the functional image is three-dimensional and shows the contour/outline/shape of the patient’s body surface.), and displaying a result obtained based on the superimposition and the merging (Paragraph [0105], Sjöstrand teaches segmentation maps and masks may be displayed, for example as a graphical representation overlaid on a medical image to guide physicians.); S103, obtaining an organ mask image registered with the original three-dimensional body surface contour image (Paragraph [0136], Sjöstrand teaches segmenting a 3D anatomical image, such as a CT image, that is co-aligned with the 3D functional image to determine a segmentation mask representing a region of the CT image determined to correspond to the prostate. The segmentation mask can then be mapped to the 3D functional image to identify, as the prostate volume, a corresponding region within the 3D functional image. The Examiner interprets “mapping” the prostate segmentation mask to the 3D functional image to be synonymous to “registering” the organ mask to the 3D body surface contour image since registration is the process of transforming the moving image (mask) to align with the target image (the functional image). Further, as mentioned above, under BRI, the Examiner interprets the 3D functional image to be a “three-dimensional body surface contour image” since the functional image is a three-dimensional and shows the contour/outline/shape of the patient’s body surface.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh by overlaying and transferring the three-dimensional segmentation map, which outlines/identifies one or more organs in the 3D functional image and maps specific segmentation masks such as an organ segmentation mask to the 3D functional image and registers the organ segmentation mask to the functional image that is taught by Sjöstrand, to make the invention that identifies and segments 3D boundaries of particular tissue regions of interest (such as organs, organ sub-regions, etc.); thus, one of ordinary skilled in the art would be motivated to combine the references since it distinguishably and accurately identifies one or more tissue regions and/or sub-regions of interest, such as one or more particular organs, and can be displayed to help guide physicians and medical practitioners identify volumes for purposes of classifying hotspots, and determining useful indices that serve as measures and/or predictions of cancer status, progression, and response to treatment (Sjöstrand , Paragraphs [0102], [0105]). Further, identifying segmentation masks for organs in the functional image can help localize one or more regions corresponding to lesions or potential lesions in the organs, such as a lesion in the prostate (Sjöstrand, Paragraph [0040]).
Singh in view of Sjöstrand does not explicitly disclose obtain(ing) a binary mask image of an organ.
Huy is in the same field of art of performing segmentation on medical images to identify an internal structure of interest such as an organ (lungs). Further, Huy discloses obtain(ing) a binary mask image of an organ (4. Data, Fig. 1, Huy teaches lung region segmentations. See Fig. 1 (cropped portion included below), binary mask/segmentation of lungs.).
PNG
media_image7.png
153
117
media_image7.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh by performing binary segmentation to identify an outline and isolate an anatomical structure of interest, such as an organ (lungs) that is taught by Huy, to make the invention that isolates the organ of interest in an image by assigning pixel values of 1 and 0; thus, one of ordinary skilled in the art would be motivated to combine the references since there is a need to identify the hidden location of an internal organ or region of interest in a patient and identifying the location of an internal organ from low-dose topogram images can result in extra radiation to the patients. Further, by providing the model with the image/label (radiograph/segmentation) pairs during training, the model learns to improve downstream tasks such as performing segmentation (Huy, 6. Conclusion).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 2, Singh in view of Sjöstrand in further view of Huy discloses the deep learning-based method for generating the internal structure of the organism according to claim 1, wherein said training the internal organ generation model (Paragraphs [0039-41], Fig. 3A, Singh teaches training the prediction block for predicting organ surface data.) in step S105 comprises: adopting a diffusion model for the deep learning neural network (Abstract, Huy teaches a Denoising Diffusion Medical Model (DDMM), a diffusion-based multi-branch model that can jointly produce realistic XR medical images and their associated segmentation masks.), performing a forward process by the diffusion model (2.2 Denoising diffusion probabilistic model, Huy teaches a forward process step (q) which adds Gaussian noise.), and then performing an inverse diffusion process by the diffusion model (2.2 Denoising diffusion probabilistic model, Huy teaches a reverse process that gradually removes the noise in the inputs.), wherein: the forward process is a process of gradually adding a Gaussian noise (2.2 Denoising diffusion probabilistic model, Huy teaches with an input data sample x0 ~ q(x0), the forward process q adds the Gaussian noise at each time-step to the given input.), wherein the forward process comprises adding a random noise to the organ mask image of the imaging target (Fig. 1 caption, Fig. 1, 2.2 Denoising diffusion probabilistic model, Huy teaches random Gaussian noisy input. See left pointing arrows (red arrows) in Fig. 1 below. Fig. 1 depicts forward process q of adding Gaussian noise to the organ (lungs) mask.);
PNG
media_image8.png
445
1241
media_image8.png
Greyscale
and the inverse diffusion process is a process of learning a random noise component on the organ mask image of the imaging target under guidance (2.2 Denoising diffusion probabilistic model, Fig. 1, Fig. 1 caption, Huy teaches a reverse process can be defined as a routing that gradually removes the noise in the inputs. The upper branch of the DDMM model attempts to denoise the random Gaussian noisy input and produce the XR-like image while the lower branch tries to generate the corresponding segmentation.) of the three-dimensional body surface contour image (6. Conclusion, Huy teaches scaling DDMM to CT or MRI images (i.e., 3D images/volumes).), and denoising the organ mask image of the imaging target (Fig. 1, Huy teaches denoising the random Gaussian noisy input and generating the corresponding segmentation.).
PNG
media_image9.png
334
1230
media_image9.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh in view of Sjöstrand in further view of Huy by adopting a denoising diffusion model to jointly produce realistic XR images (or 3-D CT/MRI images) and their associated segmentation masks that is taught by Huy, to make the invention that during a forward process, adds noise at each time-step and a reverse process that gradually removes the noise in the input images; thus, one of ordinary skilled in the art would be motivated to combine the references since diffusion-based models are beneficial for downstream tasks, such as segmentation in medical images (6. Conclusion).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Singh et al. (U.S. Patent Pub. No. 20180228460 A1, hereafter referred to as Singh) in view of Sjöstrand et al. (U.S. Patent Pub. No. 20250191752 A1, hereafter referred to as Sjöstrand) in further view of Huy et al. (NPL “Denoising Diffusion Medical Models,” April 2023, hereafter referred to as Huy) in further view of Azam et al. (NPL, “Automated Detection of Broncho-Arterial Pairs Using CT Scans Employing Different Approaches to Classify Lung Diseases”, Jan. 2023, hereafter referred to as Azam).
Regarding Claim 5, Singh in view of Sjöstrand in further view of Huy discloses teaches the deep learning-based method for generating the internal structure of the organism according to claim 1, wherein said obtaining the binary mask image of the organ in step S102 comprises: performing segmentation to obtain an organ contour of the imaging target in the internal structure image (Paragraphs [0112], [0119], Singh teaches applying a deep network for segmenting the topogram image by labelling each pixel to belong to one of the following classes: background, body (inside patient’s body), lung or bone.),
Singh in view of Sjöstrand in further view of Huy does not explicitly disclose assigning a value of 1 to a region outside the organ contour and assigning a value of 0 to a region within the organ contour, to obtain a binary mask image of a target organ.
Azam is in the same field of art of identifying internal structures (organs) in medical images. Further, Azam teaches assigning a value of 1 to a region outside the organ contour and assigning a value of 0 to a region within the organ contour, to obtain a binary mask image of a target organ (2.5.1., Lung Segmentation, Otsu Threshold Algorithm, Figs. 12 & 13, Azam teaches an output binarized image according to intensity level of the input image based on the following conditions. If intensity [pixel] > particular threshold, the resultant pixel = 1 (white); Else if intensity [pixel] <= particular threshold, the equivalent output pixel = 0 (black). As shown in Figs. 12 and 13, the region outside the lung contour is white (assigned a value of 1) and the region within the lung contour is black (assigned a value of 0).).
PNG
media_image10.png
471
826
media_image10.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh in view of Sjöstrand in further view of Huy by generating a binary image where the region corresponding to the organ is assigned “0” (black) and the region outside of the organ outline is assigned a value of “1” (white) that is taught by Azam, to make the invention that identifies and extracts lung (organ) portions from the medical images in order to train the deep learning model to extract the organ regions from body surface contour images; thus, one of ordinary skilled in the art would be motivated to combine the references since it would have been an obvious to swap/invert the standard binary mask convention (inside organ = 1 (white); outside organ = 0 (black)) in order to extract only the lung (organ) portion of the image by first filling the inner areas of the lung with white and everything else in the image is assigned 0 (black), then inverting the mask to achieve the final binary mask (Azam, 2.5.1, Lung Segmentation).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Singh et al. (U.S. Patent Pub. No. 20180228460 A1, hereafter referred to as Singh) in view of Sjöstrand et al. (U.S. Patent Pub. No. 20250191752 A1, hereafter referred to as Sjöstrand) in further view of Huy et al. (NPL “Denoising Diffusion Medical Models,” April 2023, hereafter referred to as Huy) in further view of Piper (U.S. Patent Pub. No. 0110268330 A1, hereafter referred to as Piper).
Regarding Claim 6, Singh in view of Sjöstrand in further view of Huy discloses the deep learning-based method for generating the internal structure of the organism according to claim 1, wherein step S103 comprises: registering the binary mask image with the original three-dimensional body surface contour image (Paragraph [0136], Sjöstrand teaches segmenting a 3D anatomical image, such as a CT image, that is co-aligned with the 3D functional image to determine a segmentation mask representing a region of the CT image determined to correspond to the prostate. The segmentation mask can then be mapped to the 3D functional image to identify, as the prostate volume, a corresponding region within the 3D functional image. The Examiner interprets “mapping” the prostate segmentation mask to the 3D functional image to be synonymous to “registering” the organ mask to the 3D body surface contour image. Further, as mentioned above, under BRI, the Examiner interprets the 3D functional image to be a “three-dimensional body surface contour image” since the functional image is a three-dimensional and shows the contour/outline/shape of the body surface of the patient.)
Singh in view of Sjöstrand in further view of Huy does not explicitly disclose to obtain a contour registration displacement field, and applying the contour registration displacement field to the binary mask image to obtain the organ mask image registered with the three-dimensional body surface contour image.
Piper is in the same field of art of identifying one or more objects, such as organs (bladder), within 3D medical images. Further, Piper teaches to obtain a contour registration displacement field (Paragraphs [0005], [0044], Piper teaches the deformation field indicative of changes between the one or more objects from the source image and the target image. The “objects” may include the patient’s left and right femur, bladder, rectum, and seminal vesicles.), and applying the contour registration displacement field to the binary mask image (Paragraphs [0014], [0017], Piper teaches the contour transformation engine applies the deformation field data to the source contour data to create target contour data. The source and target contour data may be in the form of a binary mask that identifies pixels within a contour as a binary “1” and identifies pixels outside of a contour as a binary “0.”) to obtain the organ mask image registered with the three-dimensional body surface contour image (Paragraphs [0015-17], [0013], [0044], Figs. 8 & 9, Piper teaches warping the contour data to match the changes from the source image to the target image. The source contour data, possibly including contours of multiple objects within the source image, may be transformed at the same time to generate corresponding contours for the target image. The source and target contour data may be in the form of a binary mask that identifies pixels within a contour as “1” and pixels outside the contour as “0”. The target contour data may be generated by interpolating the source contour image (binary mask) at positions defined by the deformation field to create the target contour. The source image may be 3D medical images, such as CT images of a patient’s pelvic region, such that the 2D slices together represent a 3D medical image. As shown in Fig. 8, contours (800) identify portions of the patient’s left and right femur, bladder (organ), rectum, and seminal vesicles. Fig. 9 shows an example of the target image that has been automatically contoured from the source image and contour data of Fig. 8.).
PNG
media_image11.png
449
690
media_image11.png
Greyscale
PNG
media_image12.png
483
745
media_image12.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh in view of Sjöstrand in further view of Huy by first obtaining deformation field data in order to register the contour data (binary masks) onto the target image that is taught by Piper, to make the invention that indicates how one or more objects within the image have changed from the source to the target; thus, one of ordinary skilled in the art would be motivated to combine the references since there is a need for improved automation techniques for contouring 2D image slices to generate a 3D contour (Piper, Paragraph [0003]) . The deformation field data and the source contour data (binary mask) can be used to automatically generate target contour data that identifies the one or more objects with the target image, thereby improving efficiency of contouring the 2D image slices (Piper, Paragraphs [0004], [0017]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Singh et al. (U.S. Patent Pub. No. 20180228460 A1, hereafter referred to as Singh) in view of Sjöstrand et al. (U.S. Patent Pub. No. 20250191752 A1, hereafter referred to as Sjöstrand) in further view of Huy et al. (NPL “Denoising Diffusion Medical Models,” April 2023, hereafter referred to as Huy) in further view of Stamoulou et al. (NPL “Harmonization Strategies in Multicenter MRI-Based Radiomics”, Nov. 2022, hereafter referred to as Stamoulou).
Regarding Claim 9, Singh in view of Sjöstrand in further view of Huy discloses the deep learning-based method for generating the internal structure of the organism according to claim 1.
Singh in view of Sjöstrand in further view of Huy does not explicitly disclose wherein said normalizing the original three-dimensional body surface contour image is performed through the following formula:
PNG
media_image13.png
92
216
media_image13.png
Greyscale
where Qj represents a j-th normalized three-dimensional body surface contour image, Pj represents a j-th original three-dimensional body surface contour image, Pminj represents a minimum value in the j-th original three-dimensional body surface contour image, and Pmaxj represents a maximum value in the j-th original three-dimensional body surface contour image.
Stamoulou is in the same field of art of performing detection/segmentation of regions of interest in a medical image. Further, Stamoulou discloses wherein said normalizing the original three-dimensional body surface contour image (2.2.3. Intensity Normalization, Stamoulou teaches performing signal intensity normalization to change the range of the signal intensity. Min-max standardizes the image by rescaling the range of values to [0, 1]. Under BRI, the Examiner interprets the MRI images to be “three-dimensional body surface contour images” since the images are 3-D (“voxels”) and the MRI images show the outline “contour” of the brain surface, which is a surface in the body.) is performed through the following formula: Qj = Pj-Pminj / Pmaxj-Pminj
PNG
media_image14.png
61
282
media_image14.png
Greyscale
, where Qj represents a j-th normalized three-dimensional body surface contour image (2.2.3. Intensity Normalization, Stamoulou teaches Inorm(x) is the intensity of the normalized MRI.), Pj represents a j-th original three-dimensional body surface contour image (2.2.2. Intensity Normalization, Stamoulou teaches I(x) is the intensity of the raw MRI.), Pminj represents a minimum value in the j-th original three-dimensional body surface contour image (2.2.3. Intensity Normalization, Stamoulou teaches min(x) is the minimum signal intensity value per patient.), and Pmaxj represents a maximum value in the j-th original three-dimensional body surface contour image (2.2.3. Intensity Normalization, Stamoulou teaches max(x) is the maximum signal intensity value per patient.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Singh in view of Sjöstrand in further view of Huy by performing min-max normalization on the 3D body surface contour image that is taught by Stamoulou, to make the invention that normalizes the 3D body surface contour images prior to providing them to the deep learning model for training; thus, one of ordinary skilled in the art would be motivated to combine the references since performing intensity normalization on the images helps to compensate for scanner-dependent and inter-subject variations as well as producing intensity values that (i) have common interpretation across regions with the same tissue type, (ii) are reproducible, (iii) maintain their rank, (iv) share similar distributions for the same ROI within and across subjects, (v) are not affected by biological abnormalities or population heterogeneity, (vi) are minimally sensitive to noise, and (vii) do not lead to loss of information related to pathology or other phenomena (Stamoulou, 2.2.3. Intensity Normalization).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allowable Subject Matter
Claims 3, 4, 7, and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if (1) rewritten in independent form including all of the limitations of the base claim and any intervening claims, and (2) Applicant addresses other rejections and objections if applicable.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 3, no prior arts teach wherein the forward process is expressed by the following equation:
PNG
media_image15.png
59
461
media_image15.png
Greyscale
where N represents the three-dimensional body surface contour image; φt represents a hyperparameter, ranges from 0 to 1, and satisfies φ1<φ2<⋯<φT; t represents a Gaussian noise at a predetermined time point; xt represents a data sample with a Gaussian noise at the time point t; and I represents a unit matrix.
The closest prior art, Huy teaches a denoising diffusion probabilistic model where the forward process q adds the noise with variance
β
t
∈
(
0,1
)
at each time-step to the given input
x
t
-
1
and produces T latents
x
t
where the subscription ranges from 1 to T:
PNG
media_image16.png
135
533
media_image16.png
Greyscale
Further, Huy teaches, supposing that the time steps T is long enough, and a good beta scheduler is properly designed, the latent
x
t
approximates a Gaussian distribution. If the Gaussian distribution is known,
x
T
~
N
(
0
,
I
)
can be sampled and fed to the forward process to get
q
(
x
0
)
. See Huy section 2.2. Denoising diffusion probabilistic model. However, in the instant application’s equation, the mean introduces a state- and time-dependent function, written as
1
-
φ
t
(
x
t
,
t
)
, which differs from Huy’s mean
1
-
β
t
x
t
-
1
, which directly scaled the previous latent state
x
t
-
1
by a variance that depends on the current timestep.
In regards to Claim 4, no prior arts teach wherein the inverse diffusion process is a Gaussian distribution process and is expressed by the following equation:
PNG
media_image17.png
51
344
media_image17.png
Greyscale
where N represents the three-dimensional body surface contour image, each of μθ and ∑θxt,t represents a learning parameter, and Q represents the normalized three-dimensional body surface contour image in step S104.
The closest prior art, Huy teaches a reverse process can be defined as a routing that gradually removes noise in the inputs, begins at the point
p
x
T
=
N
(
x
t
,
0
,
I
)
. The join distribution
p
θ
(
x
0
:
T
)
is calculated from the starting point by the following Markov chain:
PNG
media_image18.png
124
537
media_image18.png
Greyscale
In this case, p could be considered an approximation of q in each time step t. Therefore, p and q are components of a variational auto-encoder. However, the equation in the instant application differs from equation (4) in Huy in that it is missing “Q”, the three-dimensional body surface contour image.
In regards to Claim 7, no prior arts teach said registering the binary mask image with the original three-dimensional body surface contour image comprises: for an original three-dimensional body surface contour image and an organ mask image of a same cross section, a body surface contour curve being behind the cross section of the original three-dimensional body surface contour image, elastically registering an edge of the organ mask image with the body surface contour curve to obtain an elastic registration displacement field, and applying the elastic registration displacement field on an organ structure in the organ mask image.
The closest prior art, Piper teaches mapping pixel positions along an edge of an object (e.g., a piece of anatomy, such as a bladder) in the source image to corresponding pixel positions along the edge of the same object in the target object, showing how the object (e.g., anatomy) has changed from the source image to the target image.
In regards to Claim 8, no prior arts teach said registering the binary mask image with the original three-dimensional body surface contour image comprises: performing curved surface registration on the original three-dimensional body surface contour image and a three-dimensional contour of the binary mask image to obtain an elastic registration displacement field, and applying the elastic registration displacement field on an organ structure in the organ mask image.
The closest prior art, Piper teaches utilizing free-form intensity-based deformable registration algorithm that maps similar tissues from the source image to the target image by matching intensity values from one image to the next.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kaufman et al. (U.S. Patent Pub. No. 20220237801 A1) teaches a multiclass image segmentation model which can receive multiple single-class image datasets, receive a target mask of the single-class image datasets, receive a condition of an object associated with each of the single-class image datasets, and generate a multiclass segmentation model based on the single class image datasets, target masks, and the identification of the target objects. The models segment the individual organs in the image, and the separate segmentations are fused together to create final outlines.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7120. The examiner can normally be reached 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYDNEY L BLACKSTEN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights