DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Zhu et al. (CN 110942462 A, hereinafter “Zhu”)
Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”)
Rong et al. (US 20210383616 A1, hereinafter “Rong”)
Response to Arguments
Applicant Arguments/Remarks filed on 11/25/2025 have been fully considered and they are not persuasive.
Applicant’s Arguments/Remarks: Applicant argues that Grecchi does not point to any features relevant to the non-image information. Accordingly, Applicant respectfully submits that Grecchi does not disclose or suggest "obtaining non-image information associated with at least one of the medical images or the patient, the non-image information referring to information other than image data or image information that is used for generating the first image; and determining an ROI of the medical image based on the medical image, the non-image information, and an image segmentation model" recited in amended claim 1. Clearly, the cited references, Zhu and Grecchi, either alone or in any combination, do not pass this muster.
Examiner’s Response: examiner disagrees with applicant assertion above that the combination of the cited prior arts does not teach or suggest applicant invention as claimed. Zhu teaches using additional information such as discrete set of features comprises imaging center feature, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more. etc., which are non-image information, and perform segmentation on an image accordingly (Zhu [0026]-[0030]). based on the word model (bag of word) the elements of each discrete feature represented by one-invert single hot vector (vector) represented by the word vector (vector);
discrete feature of this embodiment further preferably comprises a collecting center, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more. the feature vector obtained by the fusion in the step (3) and then input to the semantic segmentation front fully connected layer pre-neural network in fully connected layer after the fusion feature vector is converted into a one-dimensional feature vector and the to-be-processed medical image pixel or voxel with the same number; (5) the two-dimensional or three-dimensional matrix of one-dimensional characteristic vector obtained in the step (4) in weight and size of the medical image to be processed the same representation; (6) the obtained two-dimensional or three-dimensional matrix representation in the step (5) with the to-be-processed medical image fusion, the fusion result as semantic segmentation input of deep learning network, thereby performing semantic dividing the network training (Claim 1). We clearly see that how the non-image information can be converted into vectors and vector converted in to image. Furthermore, Grecchi teaches a location of a bounding box 10 (see also FIG. 2) is determined within the 3D medical image. The latter is depicted as white rectangles in the third column, also indicating the size and position of the bounding box 10 in the three orthogonal planes. The dimensions of the bounding box 10 are predetermined based on a-priori information relating to the object (i.e. organ or body part) to be segmented and/or the image dataset (Grecchi [0033]). In other words, the combination teaches the segmentation and the determination of the region of interest are performed based on non-image information such as the type of organ, the body part, gender, age, etc. which makes sense since those can be of different shapes and sizes by providing the organ or the body part to be segmented segmentation error can be reduced by eliminating organs or body parts that are too big or too small to corresponding to the organ or body parts (ROI) expected.
Applicant is reminded that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Claims 17 and 34 contains similar limitations as claim 1 and are therefore not allowable for the same reason given above with regard to claim 1.
Claims 2-15, 67-69 depend at least in part on claim 1 and are therefore not allowable for the same reason given above.
Applicant’s arguments/remarks: claim 2 recites features relating to the first model configured to transform the non-image information into a second image in an image format, which are also not disclosed or taught by the cited prior arts.
Examiner’s Response: examiner disagrees with applicant assertion above that the feature of claim 2 is not taught or suggest by the cited prior arts. For example, Zhu teaches in another example embodiment, according to the order defined gender (male, female) as a discrete feature of the gender set element, then the gender can be expressed independently heat vector length is 2. wherein male and female are [1, 0], [0, 1];
the feature vector obtained by the fusion in the step (3) and then input to the semantic segmentation front fully connected layer pre-neural network in fully connected layer after the fusion feature vector is converted into a one-dimensional feature vector and the to-be-processed medical image pixel or voxel with the same number; (5) the two-dimensional or three-dimensional matrix of one-dimensional characteristic vector obtained in the step (4) in weight and size of the medical image to be processed the same representation; (6) the obtained two-dimensional or three-dimensional matrix representation in the step (5) with the to-be-processed medical image fusion, the fusion result as semantic segmentation input of deep learning network, thereby performing semantic dividing the network training (claim 1).
Applicant’s Arguments/Remarks: claim 10 recites features relating to training the first preliminary model using the sample non-image information of the plurality of training samples and the plurality of target second sample images corresponding to the sample non-image information, which are also not disclosed or taught by the cited prior arts.
Examiner’s Response: examiner disagrees with applicant assertion above the combination of the cited prior arts does not teach or suggest applicant invention as claimed. Zhu teaches training a plurality of models using plurality of corresponding samples which include the samples generated based on the non-image data such as organ type, size, etc. since every machine learning model needs to be trained so that they can perform reasonable as expected to provide the benefit desired.
Therefore, examiner maintains that the claims as presented and argued above are not allowable over the cited prior arts.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-11, 17-18, 20 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (CN 110942462 A, hereinafter “Zhu”) in view of Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”).
Regarding claims 1, 17 and 34:
Zhu teaches a system for image segmentation (Zhu [0001]-[0003], where Zhu teaches an invention belonging to medical imaging and artificial intelligence technology field, relating to a medical image organ deep learning segmentation method, device and storage medium for fusion discrete features), comprising:
at least one storage device including a set of instructions; and at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, (Zhu [0023]-[0024], where Zhu teaches a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, an instruction of said one or more program comprising a medical image organ deep learning segmentation method performs the blending discrete features),
the at least one processor is configured to direct the system to perform operations including:
obtaining a first image of a subject (Zhu [0013], in the step (1), the discrete set of features comprises imaging center feature, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more);
obtaining non-image information associated with at least one of the first image or the subject (where Zhu teaches in the network input image learning network, discrete characteristic information and the image allows all kinds of non-image but related to the image entering the network to learn together; 2, the image auxiliary information into the image segmentation training the neural network to make the network according to different auxiliary information self-adaptively learning and optimizing the segmentation parameter for medical image segmentation. discrete feature of this embodiment further preferably comprises a collecting centre, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more ); and
Zhu fail to explicitly teach determining a region of interest (ROI) of the first image based on the first image, the non-image information, and an image segmentation model. And wherein the non-image information referring to information other than image data or image information that is used for generating the first image.
However, Zhu teaches each discrete feature set word vector input to a vector representation of the single heat to semantic segmentation embedded layer (Embedding layer) front-neural network in the embedded layer the single hot vector for each discrete feature set are respectively converted into a real feature vector of the same fixed length;
(3) the semantic segmentation front pre-neural network in the real characteristic vector obtained in the step (2) for fusion, to obtain the characteristic vector after each discrete feature fusion;
(4) the feature vector obtained by the fusion in the step (3) and then input to the semantic segmentation front fully connected layer (full connection layer) - neural network in fully connected layer after the fusion feature vector is converted into a one-dimensional feature vector and the to-be-processed medical image pixel or voxel with the same number.
the each discrete feature set word vector to a vector representation of the single heat as semantic segmentation front input pre-processing neural network are input in the embedded layer, each nested layer respectively heat the single vector on each discrete feature set is converted into a real feature vector of the same fixed length;
embedded by word (Word embedding) heat the single vector of different discrete feature into a real vector of the same fixed length. thus, it can realize different discrete feature of the characteristic length of the uniform and can avoid heat vector representing the problem sparse height. word embedding operation of deep learning network architecture can be embedding such as TensorFlow-Embedding layer Log function or Keras simple to realize (Zhu [0023]-[0025], [0037]-[0039]).
one of ordinary skill in the art can see that the single heat representing a semantic segmentation can be considered a selected region of interest (ROI) corresponding to applicant claimed limitation. Furthermore, Grecchi teaches a determining unit, the determining unit is configured to determine the position of the boundary frame in the 3 D medical image based on the 2 D segmentation data; the boundary frame has a predetermined size; and a 3 D dividing unit, the 3 D dividing unit is configured to the object in the part corresponding to the boundary frame of the 3 D medical image for the 3 D division. In another exemplary embodiment, there is provided a computer readable medium, the computer readable medium comprises instructions, the instructions when executed by a computer to cause the computer to perform the method. An automatic segmentation method according to the subject matter disclosed herein can reduce the processing time and/or resource, and can establish the effective location of the boundary frame of the region of interest. (Grecchi [0018]-[0020]).
On the basis of the fused (or combined) evaluation of the 2D segmentation data in the axial, coronal and sagittal orientations, being represented as respective white shape contours of the object in the third column of FIG. 1, a location of a bounding box 10 (see also FIG. 2) is determined within the 3D medical image. The latter is depicted as white rectangles in the third column, also indicating the size and position of the bounding box 10 in the three orthogonal planes. The dimensions of the bounding box 10 are predetermined based on a-priori information relating to the object (i.e. organ or body part) to be segmented and/or the image dataset (Grecchi [0033]).
Therefore, taking the teachings of Zhu and Grecchi as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to consider information related to the the object (i.e. organ or body part) to be segmented and/or the image, to use non image information such as the type of organ to be segmented, the size of the organ to be segmented, etc, in order to increase the quality of the segmentation process, for example by knowing the expected size of an organ too small or too big one can be rule out as being not the organ to be segmented, thereby improving the system.
Regarding claims 2 and 18:
Zhu in view of Grecchi teaches wherein the image segmentation model includes a first model configured to transform the non-image information into a second image in an image format (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020],[0033] where the combination teaches based on the word model (bag of word) the elements of each discrete feature represented by one-invert single hot vector (vector) represented by the word vector; (2) each discrete feature set word vector input to a vector representation of the single heat to semantic segmentation embedded layer (Embedding layer) front-neural network in the embedded layer the single hot vector for each discrete feature set are respectively converted into a real feature vector of the same fixed length; (3) the semantic segmentation front pre-neural network in the real characteristic vector obtained in the step (2) for fusion, to obtain the characteristic vector after each discrete feature fusion; (4) the feature vector obtained by the fusion in the step (3) and then input to the semantic segmentation front fully connected layer (full connection layer) - neural network in fully connected layer after the fusion feature vector is converted into a one-dimensional feature vector and the to-be-processed medical image pixel or voxel with the same number; (5) the one-dimensional characteristic vector reconstruction (reshape) obtained in the step (4) to the to-be-processed medical image with the same size two-dimensional or three-dimensional matrix representation; (6) the obtained two-dimensional or three-dimensional matrix representation in the step (5) with the to-be-processed medical image fusion, the fusion result input to the semantic segmentation deep learning network, thereby performing semantic dividing the network training).
Regarding claim 3:
Zhu in view of Grecchi teaches wherein the determining the ROI of the first image includes: determining a vector based on the non-image information; and determining the second image by inputting the vector into the first model (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020], [0033]).
Regarding claims 4 and 20:
Zhu in view of Grecchi teaches wherein the image segmentation model further includes a second model configured to segment the first image based at least on the second image (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020], the obtained two-dimensional or three-dimensional matrix representation in the step (5) with the to-be-processed medical image fusion, the fusion result as semantic segmentation input of deep learning network, thereby performing semantic dividing the network training).
Regarding claim 5:
Zhu in view of Grecchi teaches wherein the second model includes a multichannel neural network (Zhu [0003]-[0011], [0026]-[0030]).
Regarding claim 6:
Zhu in view of Grecchi teaches wherein the non-image information includes at least one of: information relating to a user associated with the first image or the subject, biological information of the subject, or image acquisition information of the first image (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020], [0033]).
Regarding claim 7:
Zhu in view of Grecchi teaches wherein the image segmentation model is obtained by a training process including:
obtaining a plurality of training samples each of which includes a first sample image of a sample subject, sample non-image information associated with the first sample image and the sample subject, and a target ROI of the first sample image; and
generating the image segmentation model by training a preliminary image segmentation model using the plurality of training samples. Zhu teaches the current network to realize the medical image organ based on deep learning segmentation, there is no energy, in the discrete information is effectively integrated into the deep learning network model directly as effective characteristics of auxiliary network training. most data enhanced for data of different center, different scanning machine for multi-source to enhance the robustness of the model learning, but this method to collecting as much of the data for data cost, operability of the data collection, network training difficulty and time are the great challenge, in fact, various sample can not to get all possible by exhaustive fundamentally, in other words, it would be obvious for a plurality of samples to be used in order to properly determine and the model parameters (Zhu [0002]). And Grecchi teaches In another exemplary embodiment, there is provided a system for automatically segmenting a 3 D medical image, the 3 D medical image comprises an object to be segmented, the system comprising: a 2 D dividing unit, the 2 D dividing unit uses a machine learning model and is configured to at least two orthogonal orientation in the first orthogonal orientation, the second orthogonal orientation and the third orthogonal orientation, the object of the slice form of the 3 D medical image for 2 D division to derive the 2 D division data; a determining unit, the determining unit is configured to determine the position of the boundary frame in the 3 D medical image based on the 2 D segmentation data; the boundary frame has a predetermined size; and a 3 D dividing unit, the 3 D dividing unit is configured to the object in the part corresponding to the boundary frame of the 3 D medical image for the 3 D division.
In another exemplary embodiment, there is provided a computer readable medium, the computer readable medium comprises instructions, the instructions when executed by a computer to cause the computer to perform the method.
An automatic segmentation method according to the subject matter disclosed herein can reduce the processing time and/or resource, and can establish the effective location of the boundary frame of the region of interest (Grecchi [0033], [0036]-[0040]).
in other words, it would be obvious for a plurality of samples to be used in order to properly determine and the model parameters.
Regarding claim 8:
Zhu in view of Grecchi teaches wherein the preliminary image segmentation model includes a first preliminary model configured to transform the sample non- image information of a sample subject into a second sample image (Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 9:
Zhu in view of Grecchi teaches wherein the preliminary image segmentation model further includes a second preliminary model configured to segment the first sample image of a subject(Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 10:
Zhu in view of Grecchi teaches wherein the generating the image segmentation model includes:
determining the first model by training the first preliminary model using the sample non-image information of the plurality of training samples and a plurality of target second sample images corresponding to the sample non-image information; and
determining, based on the first model, the second model by training the second preliminary model using the first sample images and the target ROIs of the first sample images of the plurality of training samples (Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 11:
Zhu in view of Grecchi teaches wherein the generating the image segmentation model includes:
determining the first model and the second model simultaneously based on the first preliminary model, the second preliminary model, and the plurality of training samples (Grecchi [0017]-[0020], [0033], [0036]-[0040], [0059]-[0062],).
Regarding claim 67:
Zhu in view of Grecchi teaches wherein the non-image information is input into the first model, and the first model outputs the second image in the image format (Grecchi [0017]-[0020], [0033], [0036]-[0040]), [0059]-[0062], fig. 4, using multiple training models is obvious and well-known as shown by the cited prior arts.
Regarding claim 68:
Zhu in view of Grecchi teaches wherein the first image and the second image are input into the second model, and the second model outputs the first image in which the ROI is identified (Grecchi [0017]-[0020], [0033], [0036]-[0040], [0059]-[0062], fig. 4).
Regarding claim 69:
Zhu in view of Grecchi teaches wherein the generating the image segmentation model by training a preliminary image segmentation model using the plurality of training samples includes:
for each of the plurality of training samples, generating an estimated second sample image by applying a trained first model generated before the training of a second model, the trained first model being generated by training a first preliminary model using sample non-image information
of the plurality of training samples and a plurality of target second sample images corresponding to the sample non-image information of the plurality of training samples;
generating an estimated ROI by inputting a first sample image and the corresponding estimated second sample image into an updated second model determined in a previous second iteration;
determining, based on the estimated ROI and the target ROI corresponding to the first sample image, an assessment result(Grecchi [0017]-[0020], [0033], [0036]-[0040]), [0059]-[0062], fig. 4).
Claims 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (CN 110942462 A, hereinafter “Zhu”) in view of Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”) and in view of Rong et al. (US 20210383616 A1, hereinafter “Rong”).
Regarding claim 12:
The combination of the cited references fails to teach wherein the generating the image segmentation model further includes:
assessing a loss function that relates to the first model and the second model.
However, Rong teaches can include generating refined image data. The refined image data can be generated by the refinement model in response to the processing of the image segmentation masks and the augmented image data. At (312), the method 300 can include comparing the refined image data and the training image data. The method may further include evaluating a loss function that compares the refined image data and the training image data. The loss function can be a perceptual loss or a GAN loss. Furthermore, the method may further include modifying the parameters of the refinement model in response to the comparison of the refined image data and the training image data (Rong [0147]-[0148).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to use loss function to optimize all the models in the system of Zhu and in view of Grecchi, since, it has been shown and is well known in the art that loss function is a well define function that is often use to optimize the parameters of network models with reasonable expectation of success.
Regarding claim 13:
Zhu in view of Grecchi and in view of Rong teaches wherein the generating the image segmentation model further includes assessing a first loss function that relates to the first model (Rong [0147]-[0148; se rejection to claim 12 above).
Regarding claim 14:
Zhu in view of Grecchi and in view of Rong teaches wherein the generating the image segmentation model further includes assessing a second loss function that relates to the second model (Rong [0147]-[0148], see rejection to claim 12 above).
Regarding claim 15:
Zhu in view of Grecchi and in view of Rong teaches wherein the image segmentation model is a machine learning model (Zhu [0002]; Grecchi claim 1; Rong claim 1).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 February 9, 2026
Prosecution Insights