DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The three information disclosure statements (IDS) respectively filed on September 5, 2025, January 26, 2025, and February 28,2025, have been considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a ‘segmentation component’ in Claims 6 and 7, and
a ‘ detection information determination component’ in Claims 6 and 7,
a ‘first preliminary component’ in Claims 33 and 34, and
a ‘second preliminary component’ in Claims 33 and 34.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
The ‘segmentation component’ and is being interpreted as a machine learning model capable of segmenting an image (see paragraph [0102] of the instant specification). The ‘detection information determination component’ is being interpreted as a machine learning model (see paragraph [0104] of the instant specification). The first and second ‘preliminary components’ are being interpreted as pre-trained machine learning models (see paragraphs [0135-0136] of the instant specification).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-5, 13-15, and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Us Pub No 20190133544), hereinafter Liu, in view of Gao et al. (US Pub No 20210374950), hereinafter Gao.
As to Claim 1, Liu teaches a method for determining information of a region of interest (ROI) of a target subject (see paragraph [0004-0007], “In an aspect of the present disclosure, a method is provided. The method may include obtaining first image data of a subject related to a first scan of the subject...the determining the dose plan of the second scan based on the first image may include determining at least one lesion in the first image”, where the ‘subject’ is the target subject, and the lesion is the region of interest),
implemented on a computing device having at least one processor and at least one storage device, the method comprising (see Fig. 2, computing device 200, comprising processor 210 and storage 220):
determining positioning information of the ROI of the target subject (see paragraph [0008], “In some embodiments, the determining the at least one ROI in the first image may include determining at least one candidate ROI in the first image based on the at least one lesion. ..The determining the at least one ROI in the first image may also include determine a coordinate range of the at least one candidate ROI along an axial direction, and determining the at least one ROI in the first image based on the coordinate range of the at least one candidate ROI along the axial direction”, where the ‘coordinate range’ is interpreted as the positioning information),
the first medical imaging data being acquired by performing a first scan with a first field of view (FOV) on the target subject (see paragraph [0090], “In 810, the acquisition module 410 may obtain first image data of a subject related to a first scan of the subject. The first scan may be of a first type of scan and have a first scan duration…. Merely by way of example, the first scan may be a whole-body PET scan”, where the ‘whole body’ is being interpreted the first FOV);
acquiring second medical imaging data of the target subject by performing one or more second scans with a second FOV on the target subject based on the positioning information of the ROI (see paragraph [0061], “In 530, the dose planning module 430 may generate a dose plan of a second scan based on the first image”, and see paragraph [0091], “The second scan may be of a second type of scan configured to acquire anatomical data of the subject. In 840, the acquisition module 410 may obtain second image data of the subject related to the second scan of the subject”, where the second scan must inherently have a field of view of the subject);
and determining, based on the second medical imaging data, detection information of the ROI (see paragraph [0100], “In 960, the image fusion module 440 may generate a fused image based on the corrected first image and the second image by fusing the corrected first image with the second image. The fused image may include both anatomical data and attenuation corrected functional data of the subject, and thereby can provide more detailed information for diseases diagnosed”).
Liu fails to teach that the positioning information is determined through a first determination model. However, in an analogous art, Gao teaches a method for determining information of a region of interest (ROI) of a target subject (see paragraph [0002], “The present disclosure relates to a device and system for medical image analysis, and more specifically, to a device and system for vessel plaque analysis”, where the ‘plaque’ is interpreted as the ROI),
determining positioning information of the ROI of the target subject using a first determination model (see paragraph [0011], “The first learning network includes an encoder configured to extract feature maps based on the sequence of image patches and a plaque range generator configured to generate a start position and an end position of each plaque based on the extracted feature maps” where the ‘first learning network’ is interpreted as the first determination model, where the ‘start and end positions’ are interpreted as the positioning information),
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the first determination model taught by Gao with the ROI determination method taught by Liu. The motivation for doing so would be to would be to quickly and automatically identify regions of interest. Gao teaches in paragraph [0008], “Systems and methods are disclosed for vessel plaque analysis, which can automatically and flexibly detect and locate plaque for any branch, path, segment of a vessel or the entire vascular tree accurately and quickly in an end-to-end manner”. Thus, it would have been obvious to combine the first determination model taught by Gao with the teachings of Liu in order to obtain the invention as claimed in Claim 1.
As to Claim 3, Liu in view of Gao teaches wherein a first resolution corresponding to the first medical imaging data is lower than a second resolution corresponding to the second medical imaging data (see Liu, paragraph [0090], “The first image may be used to generate a dose plan for a second scan as described in connection with FIG. 5. Compared to an image used in disease diagnose, the first image used to generate a dose plan may need a relatively lower image quality (measured by, for example, an image resolution, a signal-to-noise ratio, or image contrast)”).
As to Claim 4, Liu fails to teach inputting the first medical imaging data into the first determination model, the first determination model being a trained machine learning model. However, Gao teaches inputting the first medical imaging data into the first determination model (see paragraph [0009], “The method includes receiving a set of images along a vessel acquired by a medical imaging device…The method further includes detecting plaques based on the sequence of image patches using a first learning network”, where the ‘first learning network’ is the first determination model),
the first determination model being a trained machine learning model (see paragraph [0070], “In some embodiments, the model training device 910 may be configured to train learning networks (for example, the first learning network and the second learning network), and transmit the trained learning network to the plaque analysis device 930”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the first determination model taught by Gao with the method of identifying regions of interests taught by Liu. The motivation for doing so would be to automatically and quickly find regions of interests (see Gao, paragraph [0008]). Thus, it would have been obvious to combine the first determination model taught by Gao with the teachings of Liu in order to obtain the invention as claimed in Claim 4.
As to Claim 5, Liu fails to teach wherein the determining, based on the second medical imaging data, detection information of the ROI includes: determining the detection information of the ROI by inputting the second medical imaging data into a second determination model, the second determination model being a trained machine learning model. Liu teaches determining detection information from second medical imaging data (see paragraph [0100)], but fails to teach that this is done through a trained machine learning model.
However, Gao teaches determining the detection information of the ROI by inputting the medical imaging data into a second determination mode (see paragraph [0073], “In step 804, each detected plaque is classified (e.g., according to its type) using a second learning network,
the second determination model being a trained machine learning model (see paragraph [0076], “In some embodiments, the model training device 910 may be configured to train learning networks (for example, the first learning network and the second learning network), and transmit the trained learning network to the plaque analysis device 930”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the second determination model taught by Gao with the region of interest identification method taught by Liu. The motivation for doing so would be to would be to quickly identify and determine the severity of regions of interest in the body. Gao teaches in paragraph [00014], “The disclosed systems and methods for vessel plaque analysis according to various embodiments of the present disclosure may automatically and flexibly detect and locate a plaque for any branch, path, segment of a vessel or the entire vascular tree accurately and quickly in an end-to-end manner, and determine the type and the stenosis degree of each detected plaque. As a result, they effectively reduce the computational complexity (involving the detection phase and the training phase), and significantly improve the operating convenience and user-friendliness.”). Thus, it would have been obvious to combine the second determination model taught by Gao with the teachings of Liu in order to obtain the invention as claimed in Claim 5.
As to Claim 13, Liu in view of Gao teaches determining first feature information of the target subject based on the first medical imaging data (see Liu, paragraph [0008], “The determining the at least one ROI in the first image may also include determine a coordinate range of the at least one candidate ROI along an axial direction, and determining the at least one ROI in the first image based on the coordinate range of the at least one candidate ROI along the axial direction”);
determining second feature information of the ROI based on the detection information of the ROI (see paragraph [0085], “In 730, the image fusion module 440 may generate a fused image based on the corrected first image and the second image by fusing the corrected first image with the second image. For example, the first image may be a PET image and the second image may be a CT image. The fused image may be generated by fusing the corrected PET image and the CT image. The fused image may include both anatomical data and attenuation corrected functional data of the subject, and thereby can provide more detailed information for diseases diagnose”, where the ‘fused image’ is the detection information, and the ‘anatomical data’ is the second feature information.
Liu fails to explicitly evaluating a status of the ROI based on the detection information of the ROI, the first feature information, and the second feature information.
However, Gao teaches obtaining first feature information from imaging data (see paragraph [0028], “The encoder is configured to extract feature maps based on the sequence of image patches, and the plaque range generator is configured to generate the start position and the end position of each plaque based on the extracted feature maps”),
determining second feature information of the ROI based on the detection information of the ROI (see paragraph [0011], “each detected plaque is classified (e.g., according to its type) using a second learning network reusing at least part of parameters of the first learning network”),
evaluating a status of the ROI based on the detection information of the ROI, the first feature information, and the second feature information (see paragraph [0073], “In some embodiments, as part of step 804, the second learning network can be used to further determine a stenosis degree for each detected plaque along with its type”, where the ‘stenosis degree’ is the status of each ROI).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu and Gao in order to evaluate the status of an ROI taught by Gao. The motivation for doing so would be to improve treatment strategies. Gao teaches in paragraph [0031], “By comparing the analysis results (such as plaque position, plaque type, and plaque stenosis degree) of multiple channels and determining the final analysis result through, for example, a majority decision strategy, the accuracy of the analysis results can be further improved.” Thus, it would have been obvious to combine the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 14.
As to Claim 14, Liu fails to teach teat the first medical imaging data and the second medical imaging data are vascular imaging data of blood vessels of the target subject, and the ROI includes a plaque in the blood vessels.
However, Gao teaches that the medical imaging data are vascular imaging data of blood vessels of the target subject (see abstract, “The disclosure relates to systems and methods for vessel image analysis. The method includes receiving a set of images along a vessel acquired by a medical imaging device”),
and the ROI includes a plaque in the blood vessels (see abstract, “The method further includes detecting plaques based on the sequence of image”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu and Gao in order to identify plaques as taught by Gao. The motivation for doing so would be to improve the detection of plaques. Gao teaches in paragraph [0006-0008], “Therefore, the analysis and diagnosis of plaque is a difficult and time-consuming task. This is true even for experienced radiologists and cardiovascular specialists.. Therefore, there is still room for improving the prior vascular plaque analysis algorithms.” Thus, it would have been obvious to combine the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 14.
As to Claim 15, Liu in view of Gao teaches a system (see Liu, paragraph [0013], “In another aspect of the present disclosure, a system is provided”, for determining information of an ROI of a target subject and Liu, paragraph [0007], “The determining the dose plan of the second scan may also include determining at least one region of interest (ROI)”), comprising:
at least one storage device including a set of instructions (see Liu, computing device 200, storage 220); and
at least one processor configured to communicate with the at least one storage device (see Liu, computing device 200, containing processor 210),
wherein when executing the set of instructions (see Liu, paragraph [0013], “The system may include at least one storage device storing a set of instructions and at least one processor in communication with the at least one storage device”), the at least one processor is configured to direct the system to perform operations including the same steps disclosed in Claim 1. Therefore, the rejection and rationale are analogous to that of Claim 1.
As to Claim 31, Liu in view of Gao teaches that the second determination model is generated by: obtaining a plurality of second training samples, each of the plurality of second training samples including sample second medical imaging data of a sample subject and sample detection information of a sample ROI of the sample subject (see Gao, paragraph [0054], “Each training sample may include a sequence of image patches at a set of centerline points of the vessel, and the ground truth information of each plaque in that portion of the vessel, including, e.g., the start and end positions of each plaque, the plaque type label, and the stenosis degree label”, where the ‘stenosis degree label’ is interpreted as the sample detection information);
and generating the second determination model by training a second initial model using the plurality of second training samples (see Gao, paragraph [0054], “FIG. 7 shows a schematic flowchart for training a learning network for vessel plaque analysis according to an embodiment of the present disclosure”, where the untrained ‘learning network’ is interpreted as the initial model).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the second determination model and training taught by Gao with the teachings of Liu. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the first determination model taught by Gao with the ROI determination method taught by Liu. The motivation for doing so would be to would be to quickly and automatically identify regions of interest (see Gao, paragraph [0008]). Thus, it would have been obvious to combine the teachings of Liu with the second determination model taught by Gao in order to obtain the invention as claimed in Claim 31.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Us Pub No 20190133544), hereinafter Liu, in view of Gao et al. (US Pub No 20210374950), hereinafter Gao, and further in view of Lewin et al. (US Pub No 20040044279), hereinafter Lewin.
As to Claim 2, Liu in view of Gao fails to explicitly teach that the second FOV is smaller than the first FOV.
However, in an analogous art, Lewin teaches a method for obtaining medical data (see paragraph [0005], “This section presents a simplified summary of methods, systems, data packets, computer readable media and so on for automatically adjusting image acquisition parameters based on motion feedback derived during MRI imaging”),
In which second medical imaging data may have a second FOV smaller than FOV (see paragraph [0035], “If the doctor encounters an area that warrants closer inspection, the doctor can slow the catheter movement, or, after passing over an area that looked interesting, can back up to get a closer look. These changes in motion can be identified by device tracking logic and interpreted by parameter control and adjustment logic so that subsequent images are taken with a second set of image acquisition parameters (e.g., smaller FOV, higher resolution)”)
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed inventio to combine the second field of view taught by Lewin with the region of interest identification method taught by Liu in view of Gao. The motivation for doing so would be to allow a health provider a closer look at regions of interest, as taught by Lewin paragraph [0035]. Thus, it would have been obvious to combine the field of view taught by Lewin with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 2.
Claims 6-7 and 32-34 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Us Pub No 20190133544), hereinafter Liu, in view of Gao et al. (US Pub No 20210374950), hereinafter Gao, and further in view of Tan et al. (US Pub No 20220284570), hereinafter Tan.
As to Claim 6, Liu in view of Gao fails to teach wherein the second determination model includes a segmentation component and a detection information determination component, the segmentation component is configured to generate at least one first segmentation image of a reference region relating to the ROI based on the second medical imaging data, and the detection information determination component is configured to determine the detection information of the ROI by processing the second medical imaging data and the at least one first segmentation image.
Liu teaches that regions of interest may be segmented (see paragraph [0074], “Exemplary lesion identification technique may include a lesion identification technique based on image segmentation”), but fails to explicitly teach a determination model that does the segmentation. Gao teaches a second determination model, and teaches segments of blood vessels can be obtained, but fails to teach the detection information is obtained using the segmented image.
However, in an analogous art, Tan teaches a model for identifying regions of interest (see paragraph [0021], “The disclosure further includes systems and methods for determining a volume of the one or more object classes of interest and/or a pathology prediction based on the inferred thickness mask of the one or more object classes of interest”),
which comprises a segmentation component configured to generate at least one first segmentation image of a reference region relating to the ROI based on the medical imaging data (see paragraph [0025], “A first CNN 106 is configured to receive the features extracted from the 2D medical image 102 and segment one or more object classes of interest to produce a segmentation mask 110”, where the CNN is interpreted as the segmentation component),
and a detection information determination component configured to determine the detection information of the ROI by processing the medical imaging data and the at least one first segmentation image (see paragraph [0025], “ Optionally, a pathology prediction 132 may be determined by classifier 130, based on the features extracted by feature extractor 104, in addition to the segmentation mask 110 and the thickness mask 112”, where the ‘classifier’ is interpreted as the detection determination component, and the volume is interpreted as the detection information).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the segmentation component and the detection information determination component taught by Tan with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to obtain accurate detection information. Tan teaches in paragraph [0030], “By applying the segmentation mask 110 to the thickness mask 112 to suppress thickness values not classified as belonging to the object class of interest, a more accurate volume may be determined for the object class of interest.” Thus, it would have been obvious to combine the components taught by Tan with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 6.
As to Claim 7, Liu in view of Gao fails to teach wherein the detection information determination component is further configured to generate a second segmentation image of the ROI by processing the second medical imaging data and the at least one first segmentation image.
However, Tan teaches a detection information determination component (see Fig. 1, thickness prediction system 100 and classifier 130)
which is further configured to generate a second segmentation image of the ROI by processing the second medical imaging data and the at least one first segmentation image (see paragraph [0033], “Thickness prediction system 100 may optionally include a second feature extractor 170, and a third CNN 172, configured to determine a second thickness mask 174 from 2D medical image 102”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the segmentation component and the detection information determination component taught by Tan with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to obtain accurate detection information. Tan teaches in paragraph [0030], “By applying the segmentation mask 110 to the thickness mask 112 to suppress thickness values not classified as belonging to the object class of interest, a more accurate volume may be determined for the object class of interest.” Thus, it would have been obvious to combine the components taught by Tan with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 7.
As to Claim 32, Liu in view of Gao fails to explicitly teach that each of the plurality of second training samples further includes at least one sample first segmentation image of a sample reference region and/or a sample second segmentation image of the sample ROI.
However, Tan teaches a model for segmentation of regions of interest which is trained using at least one sample first segmentation image of a sample reference region (see paragraph [0022], “Training data, comprising 2D images and corresponding ground truth thickness masks, may be used to train a deep neural network to infer thickness of an object class of interest”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the segmentation model and training taught by Tan with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to obtain accurate detection information. Tan teaches in paragraph [0053], “In this way, the imaging system reduces noise in the thickness mask, and enables more accurate volume determination for the object class of interest.” Thus, it would have been obvious to combine the segmentation model taught by Tan with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 32.
As to Claim 33, Liu in view of Gao fails to explicitly teach the second initial model includes a first preliminary component and a second preliminary component downstream to the first preliminary component.
However, Tan teaches a second initial model which comprises a first preliminary component (see paragraph [0025], “A first CNN 106 is configured to receive the features extracted from the 2D medical image 102 and segment one or more object classes of interest to produce a segmentation mask 110”, and see paragraph [0022], “Training data, comprising 2D images and corresponding ground truth thickness masks, may be used to train a deep neural network to infer thickness of an object class of interest, such as the first CNN 106”, where the untrained first CNN is interpreted as the first preliminary component);
and a second preliminary component downstream to the first preliminary component (see paragraph [0036], “Further, thickness prediction system 100 may optionally include classifier 130… classifier 130 comprises a pre-trained deep neural network, trained to map a segmentation map and thickness map for at least a first object class of interest to a pathology prediction”, where the untrained ‘classifier’ is interpreted as the second preliminary component, and see Fig. 1, where the ‘classifier’ is located downstream from the first CNN),.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the first and second preliminary components taught by Tan with the teachings of Liu and Gao. The motivation for doing so would be to obtain accurate segmentation and detection information (see Tan paragraph [0053]). Thus, it would have been obvious to combine the first and second preliminary components taught by Tan with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 33.
As to Claim 34, Liu in view of Gao fails to explicitly wherein for a second training sample, the first preliminary component is configured to generate at least one predicted first segmentation image of the sample reference region relating to the sample ROI of the second training sample.
However, Tan teaches a first preliminary component is configured to generate at least one predicted first segmentation image (see paragraph [0025], “A first CNN 106 is configured to receive the features extracted from the 2D medical image 102 and segment one or more object classes of interest to produce a segmentation mask 110”, and see paragraph [0025], “], “Training data, comprising 2D images and corresponding ground truth thickness masks, may be used to train a deep neural network to infer thickness of an object class of interest, such as the first CNN 106”),
and the second preliminary component is configured to determine predicted detection information of the sample ROI based on the at least one predicted first segmentation image and the sample second medical imaging data of the second training sample (see paragraph [0036], “Further, thickness prediction system 100 may optionally include classifier 130…classifier 130 comprises a pre-trained deep neural network, trained to map a segmentation map and thickness map for at least a first object class of interest to a pathology prediction”, where the untrained classifier is interpreted as the second preliminary component).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the first and second preliminary components taught by Tan with the teachings of Liu and Gao. The motivation for doing so would be to obtain accurate segmentation and detection information (see Tan, paragraph [0053]). Thus, it would have been obvious to combine the first and second preliminary components taught by Tan with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 34.
Claims 8-11 and 38 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Us Pub No 20190133544), hereinafter Liu, in view of Gao et al. (US Pub No 20210374950), hereinafter Gao, and further in view of Chiu et al. (US Pub No 20230162353), hereinafter Chiu.
As to Claim 8, Liu in view of Gao fails to explicitly teach that the second medical imaging data at least includes a first subset acquired using a first scanning protocol and a second subset acquired using a second scanning protocol different from the first scanning protocol.
Liu teaches that the second medical imaging data may be captured using a different modality than the first medical imaging data (see paragraph [0037], “For example, the first image data may include PET data and the first image may be a PET image of the subject…The system may further acquire second image data (e.g., CT image data)”, but fails to explicitly teach the second medical imaging data contains a subset of data.
However, in an analogous art, Chiu teaches a method to identify and segment regions of interest in medical imaging data (see paragraph [0011], “the present invention provides an automatic method for performing lesion segmentation and classification on a set of MRI images registered with a plurality of MRI modalities”, where the ‘lesions’ are the regions of interest),
wherein the medical imaging data comprises a first subset acquired using a first scanning protocol and a second subset acquired using a second scanning protocol different from the first scanning protocol (see paragraph [0027], “FIG. 1 shows structure of a multistream fusion encoder 100 for encoding a set of MRI images registered with different MRI modalities according to one embodiment of the present invention…By way of example, the multistream fusion encoder 100 of FIG. 1 is depicted to have three streams corresponding to three MRI modalities: T2W, ADC and DWI,”, where T2W is interpreted as the first subset using a first scanning protocol, and ADC is interpreted as a second subset corresponding to a second scanning protocol).
Thus, it would have been obvious to one of ordinary skill in the art to combine the subsets of imaging data taught by Chiu with the region of interest identification method taught by Liu in view of Gao. The motivation for doing so would be to obtain precise segmentation by leveraging different imaging modalities. Chiu teaches in the abstract, “Adaptive weighting of fusion maps at each layer allows flexibility in highlighting different image modalities according to their relative influence on the segmentation/classification performance.” Thus, it would have been obvious to combine the multiple modality datasets taught by Chiu with the region of interest identification taught by Liu in view of Gao in order to obtain the invention as claimed in Claim 8.
As to Claim 9, Liu in view of Gao fails to teach the second determination model includes a first branch configured to process the first subset of the second medical imaging data and a second branch configured to process the second subset of the second medical imaging data.
However, Chiu teaches a first branch to process the first subset of the second medical imaging data and a second branch configured to process the second subset of the second medical imaging data (see paragraph [0027], “The multistream fusion encoder 100 may be structured to have different streams corresponding to the different MRI modalities respectively. By way of example, the multistream fusion encoder 100 of FIG. 1 is depicted to have three streams corresponding to three MRI modalities: T2W, ADC and DWI, denoted by T, A, and D, respectively.”, where each ‘stream’ is being interpreted as a branch).
Thus, it would have been obvious to one of ordinary skill in the art to combine the branches taught by Chiu with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to obtain precise segmentation by leveraging different imaging modalities (see abstract). Thus, it would have been obvious to combine the multiple modality datasets taught by Chiu with the region of interest identification taught by Liu in view of Gao in order to obtain the invention as claimed in Claim 9.
As to Claim 10, Liu in view of Gao fails to teach wherein the second medical imaging data includes a plurality of sets of second medical imaging data each of which has a weight value, and the determining, based on the second medical imaging data, detection information of the ROI includes: obtaining fusion imaging data by processing the plurality of sets of second medical imaging data based on the weight values of the plurality of sets of second medical imaging data; and obtaining the detection information of the ROI by inputting the fusion imaging data into the second determination model.
However, Chiu teaches obtaining a set of medical imaging data (see paragraph [0027], “FIG. 1 shows structure of a multistream fusion encoder 100 for encoding a set of MRI images registered with different MRI modalities”, where each modality is a different ‘set’),
assigning a weight to sets of medical imaging data, (see paragraph [0032], “The multistream fusion encoder 100 may further comprise a weighting operator 103 configured to generate, based on the fusion map Fmap, a plurality of weighted fusion maps for the plurality of MRI modalities respectively”),
obtaining fusion imaging data by processing the plurality of sets of second medical imaging data based on the weight values of the plurality of sets of second medical imaging data (see paragraph [0035], “Each of the fusion operators 104(T), 104(A) and 104(D) may be configured to generate a fusion-encoded feature map based on a corresponding extracted feature map and a corresponding weighted fusion map”);
and obtaining the detection information of the ROI by inputting the fusion imaging data into a model (see paragraph [0043], “At the end of the decoder paths, an intermediate classification layer comprising a plurality of intermediate classifiers 300 are arranged to form respective classification paths in the T2W, ADC and DWI streams to generate a plurality of intermediate lesion probability maps corresponding to the plurality of MRI modalities respectively”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the weighting and fusion taught by Chiu with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to would be to ensure that all regions of interest, including smaller less identifiable regions, are identified. Chiu teaches in paragraph [0013], “The multistream CNN embedded with the fusion encoder of the present invention has the ability to segment suspicious but lower grade lesions, which is important in planning the location to be sampled in MRI-targeted biopsies. Segmentation of suspicious lesions that are later found out to be benign is also important for accurate classification of the lesions from mpMRI.” Thus, it would have been obvious to combine the fusion and weighting taught by Liu with the region identification method taught by Liu in view of Gao in order to obtain the invention as claimed in Claim 10.
As to Claim 11, Liu in view of Gao fails to teach the second medical imaging data includes a plurality of sets of second medical imaging data, the determining, based on the second medical imaging data, detection information of the ROI includes: for each of the plurality of sets of second medical imaging data, obtaining preliminary detection information of the ROI by inputting the set of second medical imaging data into the second determination model; and obtaining the detection information of the ROI by fusing the preliminary detection information of the plurality of sets of second medical imaging data.
However, Chiu teaches obtaining a plurality of sets of medical imaging data (see paragraph [0027], “FIG. 1 shows structure of a multistream fusion encoder 100 for encoding a set of MRI images registered with different MRI modalities”, where each modality is a different ‘set’),
obtaining preliminary detection information of the ROI by inputting the set of second medical imaging data into a model (see paragraph [0028], “The multistream fusion encoder 100 may comprise a plurality of feature extractors 101(T), 101(A) and 101(D), each configured to extract a feature map, denoted by F(T), F(A) and F(D) respectively, for a corresponding MRI modality T, A, and D. In other words, the features maps F(T), F(A) and F(D), are independently generated in the three streams of the multistream fusion encoder 100”, where each ‘feature map’ is interpreted as preliminary detection information),
and obtaining the detection information of the ROI by fusing the preliminary detection information of the plurality of sets of second medical imaging data (see paragraph [0030], “The multistream fusion encoder 100 may further comprise a fusion map generator 102 configured to generate a fusion map Fmap based on the plurality of extracted feature maps F(T), F(A) and F(D),l”, and see paragraph [0039], “The multistream fusion encoder 100 can be embedded in various CNN segmentation and classification architectures to form a multistream lesion segmentation and/or classification network.”, where segmentation and classification is interpreted as detection information).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the primary detection taught and fusion taught by Chiu with the second determination model taught by Liu in view of Gao. The motivation for doing so would be to would be to ensure that all regions of interest, including smaller less identifiable regions, are identified, as taught by Chiu, in paragraph [0013]. Thus, it would have been obvious to combine the preliminary detection taught by Chui with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 11.
As to Claim 38, Liu in view of Gao fails to teach sample second medical imaging data at least includes a first subset acquired using a first scanning protocol and a second subset acquired using a second scanning protocol different from the first scanning protocol.
However, Chiu teaches sample medical imaging data including a first subset acquired using a first scanning protocol and a second subset acquired using a second scanning protocol different from the first scanning protocol (see paragraph [0013], “The multistream neural network embedded with the fusion encoder provided in the present invention has been trained and evaluated by incorporating information available in T2W, ADC and high b-value DW images in lesion segmentation and classification from multiparametric prostate MR imaging”, where T2W images are a first subset of images obtained through a first protocol, and ADC images are a second subset of images obtained through a second protocol).
Thus, it would have been obvious to one of ordinary skill in the art to combine the training subsets of imaging data taught by Chiu with the region of interest identification method taught by Liu in view of Gao. The motivation for doing so would be to obtain precise segmentation by leveraging different imaging modalities (see Chiu abstract). Thus, it would have been obvious to combine the multiple modality training datasets taught by Chiu with the region of interest identification taught by Liu in view of Gao in order to obtain the invention as claimed in Claim 38.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Us Pub No 2019/0133544), hereinafter Liu, in view of Gao et al. (US Pub No 20210374950), hereinafter Gao, and further in view of Fonte et al. (US Pub No 20160300349), hereinafter Fonte.
As to Claim 12, Liu in view of Gao fails to teach obtaining a first evaluation result of the second medical imaging data by analyzing image quality of the second medical imaging data; obtaining a second evaluation result of the second medical imaging data by analyzing the detection information of the ROI; and determining whether the one or more second scans need to be re-performed based on the first evaluation result and the second evaluation result.
However, in an analogous art, Fonte teaches a method of assessing medical imaging quality (see abstract) which comprises obtaining a first evaluation result of medical imaging data by analyzing image quality of imaging data (see paragraph [0013], “One method includes receiving one or more images of at least a portion of the patient's anatomy; determining, using a processor of the computer system, one or more image properties of the received images”, and see paragraph [0075], “A global level issue may involve detecting an image quality issue based on the entire image volume, and may in some cases be referred to as an ‘image property’”),
obtaining detection information relating to a ROI;(see paragraph [0013], “performing, using a processor of the computer system, anatomic localization or modeling of at least a portion of the patient's anatomy based on the received images”, where ‘modeling’ of patient data interpreted as detection information),
and obtaining a second evaluation result of the medical imaging data by analyzing the detection information of the ROI (see paragraph [0013], “obtaining an identification of one or more image characteristics associated with an anatomic feature of the patient's anatomy based on the anatomic localization or modeling”, and see paragraph [0074], “A local level issue may involve the detection space of a particular region, e.g., around some or all of the coronary arteries, coronary plaque, along one or more vessel centerlines, etc., and may in some cases be referred to as an ‘image characteristic’,” where the examiner has interpreted the ‘identification of charcteristics’ as obtaining a second evaluation result);
and determining whether the one or more second scans need to be re-performed based on the first evaluation result and the second evaluation result (see paragraph [0013], “and calculating, using a processor of the computer system, an image quality score based on the one or more image properties and the one or more image characteristics”, and see paragraph [0085], “For example, method 100 may include using the results of the image quality assessment to assess or score a single, various, or combination of features of image quality in a timeframe that allows feedback to be provided to the personnel providing the imaging data such that they could correct, redo, or update the imaging data”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the image quality assessment taught by Fonte with the method taught by Liu in view of Gao. The motivation for doing so would be to obtain high quality images that can be used to obtain accurate model results. Fonte teaches in paragraph [0011], “the characteristics and quality of the image data is important. During acquisition of medical imaging data, a variety of artifacts or limitations may exist that affect the quality of the image…Since these quality issues may affect the performance and accuracy of models and simulations based on the imaging data, there is a need to determine if image quality is suitable or to determine the effect of image quality on modeling and simulation results.” Thus, it would have been obvious to combine the quality assessment taught by Fonte with the teachings of Liu and Gao in order to obtain the invention as claimed in Claim 12.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wang, Yixin, et al., “Modality-Paring Learning for Brain Tumor Segmentation”, Arxiv, Cornell University, 2020), teaches a model comprising several parallel branches for processing different subsets of data captured using different image modalities.
Isgum et al. (US Pub No 20200394795) teaches a system for determining vessel obstruction comprising a first and second machine learning model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOUMYA THOMAS whose telephone number is (571)272-8639. The examiner can normally be reached M-F 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.T./Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights