DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 17 and 18 is/are rejected under 35 U.S.C. 102(a) as being anticipated by WANG (US 20210303930 A1).
Regarding claim 1, WANG teaches a system (WANG, fig. 5), comprising:
at least one processor; and at least one memory coupled to the at least one processor and having instructions stored thereon, wherein, in response to the at least one processor executing the instructions, the instructions facilitate performance of operations (WANG, para. 0027), comprising:
receiving a set of images (WANG discloses receiving a set of training images comprising a first plurality of images and a second plurality of images, para. 0012-15), wherein the set of images comprises:
a first image having a first region of interest (Rol), wherein the first image is associated with a first task; and a second image having a second Rol, wherein the second image is associated with a second task (WANG's disclosure of images obtained from "projects with different objectives" maps to the first and second tasks. Specifically, WANG discloses receiving a first image having an RoI associated with a first task (e.g., a chest CT image with labels for only lung structures produced for a study focusing on lung diseases) and receiving a second image having a second RoI associated with a second task (e.g., a different chest CT image with labels for only cardiac structures produced for a study on cardiac vascular diseases), para. 0008-11), and the first Rol and the second Rol are included in a common shared label group (WANG discloses merging different partially-labeled datasets (e.g., a first set labeling the heart and stomach and a second set labeling the heart and lungs). WANG teaches that in this situation, "the combination of all labels amongst all of the partially-labeled datasets can define the full label set". This "full label set" corresponds to the "common shared label group." para. 0039-44);
applying the set of images to a segmentation model, wherein the segmentation model is a computer-implemented segmentation model and comprises an output channel configured to output segmented images based the common shared label group (WANG discloses applying the training images to a neural network segmentation model, such as a 2D U-net architecture or a 3D convolutional neural network (CNN), where the last layer provides pixel-wise probabilities of each segmentation level. The model is trained to output predictions for the full label set (the common shared label group), para 0054-57); and
outputting, based on the common shared label group, at least one segmented image comprising the first Rol or the second Rol (WANG discloses that the trained model generates a label for each organic structure within the medical image and outputs segmentation results comprising the modeled organic structures (RoIs), para. 0034-44).
Regarding claim 17, WANG teaches a computer program product stored on a non-transitory computer-readable medium and comprising machine-executable instructions, wherein, in response to being executed, the machine-executable instructions cause a system to perform operations (WANG, para. 0027), comprising:
receiving a set of images (WANG discloses receiving a set of training images comprising a first plurality of images and a second plurality of images, para. 0012-15), wherein the set of images comprises:
a first image having a first region of interest (Rol), wherein the first image is associated with a first task; and a second image having a second Rol, wherein the second image is associated with a second task (WANG's disclosure of images obtained from "projects with different objectives" maps to the first and second tasks. Specifically, WANG discloses receiving a first image having an RoI associated with a first task (e.g., a chest CT image with labels for only lung structures produced for a study focusing on lung diseases) and receiving a second image having a second RoI associated with a second task (e.g., a different chest CT image with labels for only cardiac structures produced for a study on cardiac vascular diseases), para. 0008-11), and the first Rol and the second Rol are included in a common shared label group (WANG discloses merging different partially-labeled datasets (e.g., a first set labeling the heart and stomach and a second set labeling the heart and lungs). WANG teaches that in this situation, "the combination of all labels amongst all of the partially-labeled datasets can define the full label set". This "full label set" corresponds to the "common shared label group." para. 0039-44);
applying the set of images to a segmentation model, wherein the segmentation model is a computer-implemented segmentation model and comprises an output channel configured to output segmented images based the common shared label group (WANG discloses applying the training images to a neural network segmentation model, such as a 2D U-net architecture or a 3D convolutional neural network (CNN), where the last layer provides pixel-wise probabilities of each segmentation level. The model is trained to output predictions for the full label set (the common shared label group), para 0054-57); and
outputting, based on the common shared label group, at least one segmented image comprising the first Rol or the second Rol (WANG discloses that the trained model generates a label for each organic structure within the medical image and outputs segmentation results comprising the modeled organic structures (RoIs), para. 0034-44).
Regarding claim 18, WANG teaches the computer program product of claim 17, wherein the set of images in the task comprise medical images (WANG discloses that the training images comprise medical images, such as chest CT volumes).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 11-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over WANG in view of HENRY (US 20220392048 A1).
Regarding claim 11, WANG teaches a computer-implemented method (WANG discloses a computer-implemented method of training a model, para. 0012), comprising:
receiving, by a device comprising at least one processor, a first image comprising a first segmentation label associated with a first region of interest (Rol), wherein the first segmentation label is included in a first shared label group (WANG discloses a server (device) receiving a first partially-labeled dataset including labels for specific organic structures (e.g., heart and stomach) which are part of a merged "full label set" (first shared label group), para. 0039-44);
receiving, by the device, a second image, wherein the second image comprises a second Rol and a second segmentation label associated with the second Rol (WANG discloses receiving a second set of partially-labeled images comprising its own annotations/labels (e.g., heart and lungs), para. 0039-44).
WANG is silent to teaching that comprising determining, by the device, whether the second Rol relates to the first Rol; and in response to determining, by the device, the second Rol is related to the first Rol, applying, by the device, the second segmentation label to the first shared label group.
In the same field of endeavor, HENRY teaches a method comprising determining, by the device, whether the second Rol relates to the first Rol; and in response to determining, by the device, the second Rol is related to the first Rol, applying, by the device, the second segmentation label to the first shared label group (HENRY discloses determining similarities and relationships between different tasks or classes (RoIs) by computing the overlap of task intensities. Specifically, HENRY teaches comparing the intensity frequency histograms of different anatomical structures or pathologies (such as the oesophagus, lungs, and consolidation) to infer how similar a pair of tasks/classes are, para. 0153-163).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the invention to modify the method of WANG to include determining, by the device, whether the second RoI relates to the first RoI, as taught by HENRY, by analyzing the overlap of their image intensity distributions. The motivation to combine these references would be to automate the discovery of relationships between unaligned anatomical structures across different partially-annotated datasets. By enabling the device to dynamically determine these relationships (rather than relying on predefined human knowledge), the system could automatically apply the appropriate label merging function—thus applying the second segmentation label to the first shared label group in response—to seamlessly integrate heterogeneous medical datasets during training.
Regarding claim 12, the combination of WANG and HENRY teaches the computer-implemented method of claim 11, wherein the first shared label group is associated with a first channel assigned to a segmentation model implemented by the device, wherein the segmentation model is implemented by the device to determine the second Rol is related to the first Rol (HENRY teaches a multihead architecture where "each task has its own output units" or channels. The first task's label group is associated with a first channel. Furthermore, HENRY teaches that the device implements a model (such as a Siamese net or triplet net) to compute distance metrics and learn embeddings that separate or group different data, or determines similarity through overlap of task intensities. This maps to the segmentation model (or an associated feature extractor) being implemented by the device to determine if the second RoI is related to the first RoI).
Regarding claim 13, the combination of WANG and HENRY teaches the computer-implemented method of claim 12, wherein, in response to determining by the device, the second Rol is unrelated to the first Rol, the computer-implemented method further comprising:
identifying, by the device, a third image comprising a third segmentation label, wherein the third segmentation label is associated with a third Rol and the third segmentation label is included in a second shared label group; determining, by the device, the second Rol relates to the third Rol (HENRY teaches evaluating the relationship of a new task/class against multiple previous classes. For example, the device compares the intensity distribution of a new class (e.g., consolidation; second RoI) and determines it is unrelated to a first class (e.g., lung; first RoI), but identifies a third class (e.g., oesophagus; third RoI) and determines that the new consolidation task relates to it because they have the "highest HU intensity overlap"); and
applying the second segmentation label to the second shared label group, wherein the second shared label group is associated with a second channel assigned to the segmentation model implemented by the device, wherein the segmentation model is implemented by the device to determine the second Rol is related to the third Rol (WANG teaches applying label merging functions to map labels to their correlated structures. Combining this with HENRY, if the second RoI is determined to be related to the third RoI, it is merged/mapped to that group. HENRY teaches that different tasks/domains utilize their own separate output channels (second channel) assigned to the model).
Regarding claim 14, the combination of WANG and HENRY teaches the computer-implemented method of claim 13, wherein, in response to determining by the device, the second Rol is unrelated to the first Rol and is also unrelated to the third Rol, the computer-implemented method further comprising:
updating, by the device, the segmentation model to include a third channel, wherein the third channel is assigned with a third shared label group; and assigning the second segmentation label to the third shared label group (HENRY teaches incremental class learning where a model sequentially learns to recognize entirely new, unrelated classes (e.g., sequentially adding a lung task, a spinal cord task, and a ground glass task). To accommodate tasks that do not relate to previous tasks, HENRY utilizes a multihead output layer, adding new output units (channels) for each distinct task. Updating the model for a completely new, unrelated task maps to adding a third channel and assigning the new, unrelated label to its own third shared label group).
Regarding claim 15, the combination of WANG and HENRY teaches the computer-implemented method of claim 12, wherein the segmentation model is a U- net model (WANG explicitly teaches that the neural network segmentation model may be a "2D U-net architecture").
Regarding claim 16, the combination of WANG and HENRY teaches the computer-implemented method of claim 11, wherein the determination, by the device, of the second Rol relates to the first Rol is based on at least one of similarity of shape of the first Rol to the second Rol, pixel level between the first Rol and the second Rol, or patch level between a first region of interest (Rol) and a secondRoI (HENRY teaches determining similarity based on the "overlap of task intensities", specifically by analyzing histograms of Hounsfield Units (HU) intensity distributions across images. Analyzing and comparing the raw intensity values (Hounsfield units) of the images constitutes determining a relationship based on the "pixel level" (or voxel level) between the RoIs).
Claim(s) 2-5, 7-10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over WANG as applied to claims 1 and 17 above, and further in view of HENRY.
Regarding claim 2, WANG teaches the system of claim 1, wherein the common label group is a first label group, the output channel is a first output channel, and at least one segmented image is included in a first set of segmented images (WANG teaches combining all labels into a full label set (common label group).
WANG is silent to teaching that wherein the operations further comprise:
receiving a second set of images sharing a second shared label group, wherein the first shared label group and the second shared label group are disparate, and the second set of images comprises:
a third image having a third Rol, wherein the third image is associated with the first task; and
a fourth image having a fourth Rol, wherein the fourth image is associated with the second task, and the third Rol and the fourth Rol are included in a second shared label group;
applying the second set of images to the segmentation model, wherein the segmentation model further comprises a second output channel configured to output segmented images based on the second shared label group; and
outputting, based on the second shared label group, at least one segmented image comprising the third Rol or the fourth Rol.
In the same field of endeavor, HENRY teaches a system wherein the operations further comprise:
receiving a second set of images sharing a second shared label group, wherein the first shared label group and the second shared label group are disparate (HENRY teaches sequentially receiving new training data for different incremental tasks (e.g., segmenting ground glass or consolidation, which are distinct from prior lung tasks) or from disparate domains (e.g., images from Japan vs. Glasgow). The unique labels/classes associated with this second task or domain can be mapped to a disparate "second shared label group."), and the second set of images comprises:
a third image having a third Rol, wherein the third image is associated with the first task; and a fourth image having a fourth Rol, wherein the fourth image is associated with the second task, and the third Rol and the fourth Rol are included in a second shared label group (HENRY, new image data, para. 0124);
applying the second set of images to the segmentation model, wherein the segmentation model further comprises a second output channel configured to output segmented images based on the second shared label group (HENRY explicitly discloses an architecture retaining "two separate output channels... one per domain" or multi-head units where "each task has its own output units"); and
outputting, based on the second shared label group, at least one segmented image comprising the third Rol or the fourth Rol. (HENRY discloses taking the outputs of the multiple channels at prediction time to obtain segmentation maps, new task, para. 0131).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the invention to modify the method of WANG to include determining, by the device, whether the second RoI relates to the first RoI, as taught by HENRY, by analyzing the overlap of their image intensity distributions. The motivation to combine these references would be to automate the discovery of relationships between unaligned anatomical structures across different partially-annotated datasets. By enabling the device to dynamically determine these relationships (rather than relying on predefined human knowledge), the system could automatically apply the appropriate label merging function—thus applying the second segmentation label to the first shared label group in response—to seamlessly integrate heterogeneous medical datasets during training.
Regarding claim 3, the combination of WANG and HENRY teaches the system of claim 2, wherein an architecture of the segmentation model is common to application of two or more segmentation tasks and generation of segmented images based on the first shared label group and the second shared label group (WANG utilizes a common CNN architecture (e.g., U-net). HENRY further teaches that in multi-task incremental learning, while the output channels may be split, "the rest of the network may be shared between tasks". A single distil model (common architecture) is trained to perform two or more tasks.)
Regarding claim 4, the combination of WANG and HENRY teaches the system of claim 2, wherein a set of common weightings are applied during creation of both the first shared label group and the second shared label group during application of two or more segmentation tasks (HENRY teaches applying a common weighting parameter w in the loss function during incremental multi-task training to balance the model's performance between past and current tasks).
Regarding claim 5, the combination of WANG and HENRY teaches the system of claim 2, wherein labels in the first shared label group are disparate to the labels in the second shared label group (HENRY teaches models learning totally distinct, disparate new classes incrementally (e.g., learning a calcium segmentation vs. a nodule detection vs. an oesophagus segmentation).
Regarding claim 7, WANG teaches the system of claim 1.
WANG is silent to teaching that wherein the operations further comprise generating the common shared label group as a function of at least one similarity between the at least two images.
In the same field of endeavor, HENRY taches a system wherein the operations further comprise generating the common shared label group as a function of at least one similarity between the at least two images (HENRY teaches determining "the similarity between the domains or the difficulty of discrimination between tasks" by computing the "overlap of task intensities" (e.g., evaluating similarities between HU histograms of different organs)).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the invention to modify the method of WANG to include determining, by the device, whether the second RoI relates to the first RoI, as taught by HENRY, by analyzing the overlap of their image intensity distributions. The motivation to combine these references would be to automate the discovery of relationships between unaligned anatomical structures across different partially-annotated datasets. By enabling the device to dynamically determine these relationships (rather than relying on predefined human knowledge), the system could automatically apply the appropriate label merging function—thus applying the second segmentation label to the first shared label group in response—to seamlessly integrate heterogeneous medical datasets during training.
Regarding claim 8, WANG teaches the system of claim 1.
WANG is silent to teaching that wherein the operations further comprise:
applying a third task to the segmentation model; generating at least one shared label group from the third task, wherein the at least one shared label group includes the common shared label group; and training the segmentation model across multiple tasks incrementally utilizing any of the at least two images or the common shared label group.
In the same field of endeavor, HENRY teaches a system wherein the operations further comprise:
applying a third task to the segmentation model; generating at least one shared label group from the third task, wherein the at least one shared label group includes the common shared label group; and training the segmentation model across multiple tasks incrementally utilizing any of the at least two images or the common shared label group (HENRY explicitly teaches incremental learning applied sequentially to multiple tasks: a first task (left lung), a second task (right lung), a third task (spinal cord), a fourth task (trachea), etc. Models are trained incrementally utilizing distil models to aggregate tasks without access to older raw training data).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the invention to modify the method of WANG to include determining, by the device, whether the second RoI relates to the first RoI, as taught by HENRY, by analyzing the overlap of their image intensity distributions. The motivation to combine these references would be to automate the discovery of relationships between unaligned anatomical structures across different partially-annotated datasets. By enabling the device to dynamically determine these relationships (rather than relying on predefined human knowledge), the system could automatically apply the appropriate label merging function—thus applying the second segmentation label to the first shared label group in response—to seamlessly integrate heterogeneous medical datasets during training.
Regarding claim 9, the combination of WANG and HENRY teaches the system of claim 8, wherein the operations further comprise:
subsequently applying a subsequent task to the segmentation model to facilitate incremental training of the segmentation model over a series of iterations to adapt the segmentation model to the subsequent task; and maintaining incremental training until determining convergence has occurred (HENRY teaches repeatedly applying subsequent new tasks (e.g., 5th task: oesophagus, 6th task: ground glass, 7th task: consolidation) to a model to facilitate incremental training over iterations. Training neural networks with loss functions inherently relies on training via stochastic gradient descent until convergence).
Regarding claim 10, the combination of WANG and HENRY teaches the system of claim 9, wherein the operations further comprise:
re-applying a prior task to the converged segmentation model to prevent catastrophic forgetting of the prior task as a function of the segmentation model being converged with the subsequent task (HENRY explicitly does this using global distillation and regularization loss functions without needing to re-apply the actual data from the prior task. HENRY specifically states "there is no requirement for data from previous tasks to be used" and relies on teacher-student distillation).
Regarding claim 19, WANG teaches the computer program product of claim 17.
WANG is silent to teaching that wherein the first RoI being related to the second RoI is based on at least one of similarity of shape of the first RoI to the second RoI, pixel level between the first RoI and the second RoI, or patch level between a first RoI and a second RoI.
In the same field of endeavor, HENRY teaches a device wherein the first RoI being related to the second RoI is based on at least one of similarity of shape of the first RoI to the second RoI, pixel level between the first RoI and the second RoI, or patch level between a first RoI and a second RoI (Identical to the analysis for Claim 16. HENRY teaches analyzing similarity via intensity distributions (Hounsfield units)
, which maps to determining relations on a pixel level).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the invention to modify the method of WANG to include determining, by the device, whether the second RoI relates to the first RoI, as taught by HENRY, by analyzing the overlap of their image intensity distributions. The motivation to combine these references would be to automate the discovery of relationships between unaligned anatomical structures across different partially-annotated datasets. By enabling the device to dynamically determine these relationships (rather than relying on predefined human knowledge), the system could automatically apply the appropriate label merging function—thus applying the second segmentation label to the first shared label group in response—to seamlessly integrate heterogeneous medical datasets during training.
Claim(s) 6 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over WANG and HENRY as applied to claims 2 and 17 above, and further in view of HAN (US 20220284687 A1).
Regarding claim 6, the combination of WANG and HENRY teaches the system of claim 2.
The combination WANG and HENRY is silent to teaching that wherein the operations further comprise:
determining a first number of shared label groups in the first task and a second number of shared label groups in the second task; and
in response to a determination that the first number of shared label groups is different to the second number of shared label groups, configuring the number of output channels of the segmentation model to equal the greater of the first number of shared label groups or the second number of shared label groups.
In the same field of endeavor, HAN teaches a system wherein the operations further comprise:
determining a first number of shared label groups in the first task and a second number of shared label groups in the second task; and in response to a determination that the first number of shared label groups is different to the second number of shared label groups, configuring the number of output channels of the segmentation model to equal the greater of the first number of shared label groups or the second number of shared label groups (HAN teaches that different sets of images can be annotated with different types of ROIs to generate training images for different tasks. For example, HAN discloses a first task where images are annotated with the heart to train a heart segmentation model, and a second task where images are annotated with the lung to train a lung segmentation model. HAN also discloses tasks involving different numbers of ROIs, such as segmenting a single ROI (e.g., a liver) or segmenting multiple ROIs (e.g., left and right kidneys). Determining the different numbers of distinct annotated organs/classifications across these different medical tasks maps to determining a first number of shared label groups in a first task and a second number of shared label groups in a second task).
Therefore, a Person of Ordinary Skill in the Art (POSITA) would have been motivated to combine the teachings of WANG and HAN. A POSITA would be motivated to apply HAN’s multi-resolution segmentation framework to WANG’s system. By doing so, WANG's system could first coarsely locate the combined organic structures (e.g., lungs, heart, stomach) at a low resolution, and then apply WANG's high-resolution pixel-wise probability mapping only to those localized regions. This combination would drastically reduce the processing time required to generate WANG's multi-organ segmentations, making the system much faster and more efficient than conventional segmentation methods. WANG's system evaluates large image volumes to segment multiple organic structures simultaneously, the voxel-by-voxel prediction process can be computationally expensive. HAN teaches a multi-resolution, coarse-to-fine segmentation approach to improve processing speed. Specifically, HAN teaches using a first ROI segmentation model at a lower image resolution to quickly identify a coarse "preliminary region" (such as a bounding box enclosing the organ), and then using a second ROI segmentation model at a higher image resolution to refine the "target region" solely from that smaller preliminary area. By restricting the high-resolution segmentation to a smaller, localized bounding box rather than the entire original image, the processing time and computational cost are substantially reduced.
A POSITA would be motivated to apply HAN’s multi-resolution segmentation framework to WANG’s system. By doing so, WANG's system could first coarsely locate the combined organic structures (e.g., lungs, heart, stomach) at a low resolution, and then apply WANG's high-resolution pixel-wise probability mapping only to those localized regions. This combination would drastically reduce the processing time required to generate WANG's multi-organ segmentations, making the system much faster and more efficient than conventional segmentation methods.
Regarding claim 20, WANG teaches the computer program product of claim 17.
WANG is silent to teaching that wherein the operations further comprising defining the number of output channels prior to generating the set of shared label groups.
In the same field of endeavor, HAN teaches a device wherein the operations further comprising defining the number of output channels prior to generating the set of shared label groups (HAN teaches that different sets of images can be annotated with different types of ROIs to generate training images for different tasks. For example, HAN discloses a first task where images are annotated with the heart to train a heart segmentation model, and a second task where images are annotated with the lung to train a lung segmentation model. HAN also discloses tasks involving different numbers of ROIs, such as segmenting a single ROI (e.g., a liver) or segmenting multiple ROIs (e.g., left and right kidneys). Determining the different numbers of distinct annotated organs/classifications across these different medical tasks maps to determining a first number of shared label groups in a first task and a second number of shared label groups in a second task).
Therefore, a Person of Ordinary Skill in the Art (POSITA) would have been motivated to combine the teachings of WANG and HAN. A POSITA would be motivated to apply HAN’s multi-resolution segmentation framework to WANG’s system. By doing so, WANG's system could first coarsely locate the combined organic structures (e.g., lungs, heart, stomach) at a low resolution, and then apply WANG's high-resolution pixel-wise probability mapping only to those localized regions. This combination would drastically reduce the processing time required to generate WANG's multi-organ segmentations, making the system much faster and more efficient than conventional segmentation methods. WANG's system evaluates large image volumes to segment multiple organic structures simultaneously, the voxel-by-voxel prediction process can be computationally expensive. HAN teaches a multi-resolution, coarse-to-fine segmentation approach to improve processing speed. Specifically, HAN teaches using a first ROI segmentation model at a lower image resolution to quickly identify a coarse "preliminary region" (such as a bounding box enclosing the organ), and then using a second ROI segmentation model at a higher image resolution to refine the "target region" solely from that smaller preliminary area. By restricting the high-resolution segmentation to a smaller, localized bounding box rather than the entire original image, the processing time and computational cost are substantially reduced.
A POSITA would be motivated to apply HAN’s multi-resolution segmentation framework to WANG’s system. By doing so, WANG's system could first coarsely locate the combined organic structures (e.g., lungs, heart, stomach) at a low resolution, and then apply WANG's high-resolution pixel-wise probability mapping only to those localized regions. This combination would drastically reduce the processing time required to generate WANG's multi-organ segmentations, making the system much faster and more efficient than conventional segmentation methods.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more.
Under the Alice/Mayo two-step framework, the claims are evaluated to determine whether they are directed to a statutory category (Step 1), whether they are directed to a judicial exception (Step 2A, Prong 1), whether the exception is integrated into a practical application (Step 2A, Prong 2), and whether the claim recites significantly more than the exception itself (Step 2B).
Step 1: Statutory Category
Claims 1-10 are drafted as a system (machine), claims 11-16 are drafted as a computer-implemented method (process), and claims 17-20 are drafted as a computer program product stored on a non-transitory computer-readable medium (article of manufacture). Thus, the claims fall within statutory categories.
Step 2A, Prong 1: Directed to a Judicial Exception
The claims are directed to the abstract idea of classifying and organizing data, mathematical concepts, and mental processes.
Mental Processes & Organization of Data:
Independent claims 1, 11, and 17 recite receiving images with regions of interest (RoIs), determining whether a second RoI relates to a first RoI, and classifying/grouping the RoIs into a "common shared label group" (or assigning segmentation labels to a shared label group). The act of observing an image, recognizing a region of interest, determining if it relates to another region, and grouping them by a shared label is a process of organizing information that can practically be performed in the human mind or by a human using pen and paper.
Mathematical Concepts:
The claims further recite applying the images to a "segmentation model" comprising an "output channel," and outputting a segmented image. A segmentation model (explicitly defined as a "U-net model" in claim 15) is a machine learning algorithm, which is inherently a mathematical construct.
Dependent claims 2-10, 12-16, and 18-20 further limit the abstract idea by adding additional mathematical concepts, data organization steps, and logical rules. For example:
Claims 2, 3, 5, 12, and 13 add logical branching rules for classifying data into secondary shared label groups based on whether labels are disparate.
Claim 4 introduces calculating "common weightings" (a mathematical calculation).
Claims 6, 14, and 20 recite mathematical/logical rules for configuring array sizes (e.g., setting the number of output channels to "equal the greater of" two numbers, or "updating... the segmentation model to include a third channel").
Claims 7, 16, and 19 recite using mathematical or observational algorithms (e.g., "similarity of shape", "pixel level", "patch level") to determine data groupings.
Claims 8-10 recite "training" the model incrementally and determining "convergence," which are mathematical algorithms used to adjust the weights of a neural network.
Thus, all claims are directed to an abstract idea.
Step 2A, Prong 2: Integration into a Practical Application
The claims do not integrate the abstract idea into a practical application. Under MPEP 2106.04(d), an abstract idea is not integrated into a practical application if it merely recites the abstract idea along with highly generalized computer components or instructions to apply the exception on a computer.
While the specification may suggest that sharing labels reduces the number of required channels (improving the efficiency of the model), an improvement to a mathematical algorithm or an algorithmic efficiency does not constitute an improvement to the functioning of a computer as a computer. The claims do not recite any specific hardware improvement, physical transformation, or control of a physical machine. Re-configuring the "output channels" of a model is merely redefining the dimensions of a data vector/tensor in software, which is a mathematical and logical data manipulation, not a physical hardware improvement.
Furthermore, limiting the application of the model to "medical images" (Claim 18) or a specific environment (first and second tasks) constitutes a mere field-of-use limitation, which is insufficient to integrate the abstract idea into a practical application. Outputting a "segmented image" is merely a display or output of the results of the mathematical process, which is insignificant extra-solution activity.
Step 2B: Significantly More
The claims do not recite additional elements that amount to significantly more than the abstract idea itself. Looking at the claims as a whole, the physical components recited include "at least one processor," "at least one memory," a "device," and a "non-transitory computer-readable medium." These are generic, off-the-shelf computer components invoked at a high level of generality merely to execute the abstract classification and mathematical steps. Gathering data ("receiving a set of images") and outputting data ("outputting... at least one segmented image") are well-understood, routine, and conventional activities in the computer arts.
The combination of grouping image labels (a mental/organizational process) and running them through a neural network (a mathematical concept) using generic processors and memory does not add an inventive concept that transforms the nature of the claim into a patent-eligible application. Accordingly, claims 1-20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
BAKER: US20250118050A1, GROTH: US20220319160A1, HAROUNI: US20200051238A1, SCHULTER: US20240378874A1, TAHER: US20240290076A1 and TSAI: US20220148189A1: teach image segmentation systems.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEN WU HUANG whose telephone number is (571)272-7852. The examiner can normally be reached Mon-Fri 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wesley Kim can be reached at (571) 272-7867. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEN W HUANG/ Primary Examiner, Art Unit 2648
Prosecution Insights