DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Zhu et al. (CN 110942462 A, hereinafter “Zhu”)
Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”)
Rong et al. (US 20210383616 A1, hereinafter “Rong”)
in view of Hughes et al. (EP 3770840 A1, hereinafter “Hughes”)
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/12/2026 has been entered.
Response to Arguments
Applicant Arguments/Remarks filed on 05/12/2026 have been fully considered and they are moot in view of the new ground of rejection set forth below
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5-11, 17, 34 and 67-74 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (CN 110942462 A, hereinafter “Zhu”) in view of Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”) and in view of Hughes et al. (EP 3770840 A1, hereinafter “Hughes”)
Regarding claims 1, 17 and 34:
Zhu teaches a system for image segmentation (Zhu [0001]-[0003], where Zhu teaches an invention belonging to medical imaging and artificial intelligence technology field, relating to a medical image organ deep learning segmentation method, device and storage medium for fusion discrete features), comprising:
at least one storage device including a set of instructions; and at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, (Zhu [0023]-[0024], where Zhu teaches a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, an instruction of said one or more program comprising a medical image organ deep learning segmentation method performs the blending discrete features),
the at least one processor is configured to direct the system to perform operations including:
obtaining a first image of a subject (Zhu [0013], in the step (1), the discrete set of features comprises imaging center feature, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more);
obtaining non-image information associated with at least one of the first image or the subject (where Zhu teaches in the network input image learning network, discrete characteristic information and the image allows all kinds of non-image but related to the image entering the network to learn together; 2, the image auxiliary information into the image segmentation training the neural network to make the network according to different auxiliary information self-adaptively learning and optimizing the segmentation parameter for medical image segmentation. discrete feature of this embodiment further preferably comprises a collecting center, scan machine, scan sequence characteristics, gender, age, geographical feature, a human feature or the like in the one kind of or more ); and
Zhu fail to explicitly teach determining a region of interest (ROI) of the first image based on the first image, the non-image information, and an image segmentation model. And wherein the non-image information referring to information other than image data or image information that is used for generating the first image;
the image segmentation model includes a first model configured to transform the non-image information into a second image in an image format, the second image referring to an image that represents the non-image information associated with at least one of the first image or the subject, the image segmentation model further includes a second model configured to segment the first image based at least on the second image, the non-image information is input into the first model, and the first model outputs the second image in the image format, and the first image and the second image are input into the second model, and the second model outputs the first image in which the ROI is identified.
However, Zhu teaches each discrete feature set word vector input to a vector representation of the single heat to semantic segmentation embedded layer (Embedding layer) front-neural network in the embedded layer the single hot vector for each discrete feature set are respectively converted into a real feature vector of the same fixed length;
(3) the semantic segmentation front pre-neural network in the real characteristic vector obtained in the step (2) for fusion, to obtain the characteristic vector after each discrete feature fusion;
(4) the feature vector obtained by the fusion in the step (3) and then input to the semantic segmentation front fully connected layer (full connection layer) - neural network in fully connected layer after the fusion feature vector is converted into a one-dimensional feature vector and the to-be-processed medical image pixel or voxel with the same number.
the each discrete feature set word vector to a vector representation of the single heat as semantic segmentation front input pre-processing neural network are input in the embedded layer, each nested layer respectively heat the single vector on each discrete feature set is converted into a real feature vector of the same fixed length;
embedded by word (Word embedding) heat the single vector of different discrete feature into a real vector of the same fixed length. thus, it can realize different discrete feature of the characteristic length of the uniform and can avoid heat vector representing the problem sparse height. word embedding operation of deep learning network architecture can be embedding such as TensorFlow-Embedding layer Log function or Keras simple to realize (Zhu [0023]-[0025], [0037]-[0039]).
one of ordinary skill in the art can see that the single heat representing a semantic segmentation can be considered a selected region of interest (ROI) corresponding to applicant claimed limitation. Furthermore, Grecchi teaches a determining unit, the determining unit is configured to determine the position of the boundary frame in the 3 D medical image based on the 2 D segmentation data; the boundary frame has a predetermined size; and a 3 D dividing unit, the 3 D dividing unit is configured to the object in the part corresponding to the boundary frame of the 3 D medical image for the 3 D division. In another exemplary embodiment, there is provided a computer readable medium, the computer readable medium comprises instructions, the instructions when executed by a computer to cause the computer to perform the method. An automatic segmentation method according to the subject matter disclosed herein can reduce the processing time and/or resource, and can establish the effective location of the boundary frame of the region of interest. (Grecchi [0018]-[0020]).
On the basis of the fused (or combined) evaluation of the 2D segmentation data in the axial, coronal and sagittal orientations, being represented as respective white shape contours of the object in the third column of FIG. 1, a location of a bounding box 10 (see also FIG. 2) is determined within the 3D medical image. The latter is depicted as white rectangles in the third column, also indicating the size and position of the bounding box 10 in the three orthogonal planes. The dimensions of the bounding box 10 are predetermined based on a-priori information relating to the object (i.e. organ or body part) to be segmented and/or the image dataset (Grecchi [0033]).
Hughes further teaches in the model proposed, the first step is to convert textual meta-data to an image format suitable for processing with a convolutional neural network. This is shown in Figure 4. The first step in is to convert textual metadata to an image format such that it can be processed with a convolutional neural network.
As shown in Figure 5, the approach separately considers product image data and product metadata in two separate paths 501, 502. The first path incorporates a convolutional neural network. In the embodiment of Figure 5 this neural network is a deep convolutional neural network. This neural network is implemented to describe the semantic content of an image of a product. This semantic content is used to generate the image side of the input pair 5022.
The second path incorporates a second neural network. This second path processes the product metadata to convert textual content into semantic representations. The second neural network is a shallow convolutional neural network which converts textual content into semantic representations 5021. Shallow neural networks have typically a single hidden layer where as deep neural networks have a plurality of hidden layers.
The outputs of both the first path 501 and the second path 502 are concatenated to provide the input into a fully connected multi-layer perceptron network 504. It will be appreciated that as all of the layers are connected, the entire model is trained as a single entity and all the layers are updated during the training phase. The output embedding, output from the fully connected neural network represents the product (Hughes [0034], [0037]-[0040], figs. 4 and 5.
Therefore, taking the teachings of Zhu and Grecchi and Hughes as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to consider information (metadata) related to the object (i.e. organ or body part) to be segmented and/or the image, to use non image information (metadata) such as the type of organ to be segmented, the size of the organ to be segmented, etc., and generate corresponding image using a neural network, in order to increase the quality of the segmentation process, for example by knowing the expected size of an organ too small or too big one can be rule out as being not the organ to be segmented, thereby improving the system.
Regarding claim 3:
Zhu in view of Grecchi and in view of Hughes teaches wherein the determining the ROI of the first image includes: determining a vector based on the non-image information; and determining the second image by inputting the vector into the first model (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020], [0033]).
Regarding claim 5:
Zhu in view of Grecchi and in view of Hughes teaches wherein the second model includes a multichannel neural network (Zhu [0003]-[0011], [0026]-[0030]).
Regarding claim 6:
Zhu in view of Grecchi and in view of Hughes teaches wherein the non-image information includes at least one of: information relating to a user associated with the first image or the subject, biological information of the subject, or image acquisition information of the first image (Zhu [0003]-[0011], [0026]-[0030]; Grecchi Abstract [0017]-[0020], [0033]).
Regarding claim 7:
Zhu in view of Grecchi and in view of Hughes teaches wherein the image segmentation model is obtained by a training process including:
obtaining a plurality of training samples each of which includes a first sample image of a sample subject, sample non-image information associated with the first sample image and the sample subject, and a target ROI of the first sample image; and
generating the image segmentation model by training a preliminary image segmentation model using the plurality of training samples. Zhu teaches the current network to realize the medical image organ based on deep learning segmentation, there is no energy, in the discrete information is effectively integrated into the deep learning network model directly as effective characteristics of auxiliary network training. most data enhanced for data of different center, different scanning machine for multi-source to enhance the robustness of the model learning, but this method to collecting as much of the data for data cost, operability of the data collection, network training difficulty and time are the great challenge, in fact, various sample can not to get all possible by exhaustive fundamentally, in other words, it would be obvious for a plurality of samples to be used in order to properly determine and the model parameters (Zhu [0002]). And Grecchi teaches In another exemplary embodiment, there is provided a system for automatically segmenting a 3 D medical image, the 3 D medical image comprises an object to be segmented, the system comprising: a 2 D dividing unit, the 2 D dividing unit uses a machine learning model and is configured to at least two orthogonal orientation in the first orthogonal orientation, the second orthogonal orientation and the third orthogonal orientation, the object of the slice form of the 3 D medical image for 2 D division to derive the 2 D division data; a determining unit, the determining unit is configured to determine the position of the boundary frame in the 3 D medical image based on the 2 D segmentation data; the boundary frame has a predetermined size; and a 3 D dividing unit, the 3 D dividing unit is configured to the object in the part corresponding to the boundary frame of the 3 D medical image for the 3 D division.
In another exemplary embodiment, there is provided a computer readable medium, the computer readable medium comprises instructions, the instructions when executed by a computer to cause the computer to perform the method.
An automatic segmentation method according to the subject matter disclosed herein can reduce the processing time and/or resource, and can establish the effective location of the boundary frame of the region of interest (Grecchi [0033], [0036]-[0040]).
in other words, it would be obvious for a plurality of samples to be used in order to properly determine the model parameters.
Regarding claim 8:
Zhu in view of Grecchi and in view of Hughes teaches wherein the preliminary image segmentation model includes a first preliminary model configured to transform the sample non- image information of a sample subject into a second sample image (Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 9:
Zhu in view of Grecchi and in view of Hughes teaches wherein the preliminary image segmentation model further includes a second preliminary model configured to segment the first sample image of a subject(Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 10:
Zhu in view of Grecchi and in view of Hughes teaches wherein the generating the image segmentation model includes:
determining the first model by training the first preliminary model using the sample non-image information of the plurality of training samples and a plurality of target second sample images corresponding to the sample non-image information; and
determining, based on the first model, the second model by training the second preliminary model using the first sample images and the target ROIs of the first sample images of the plurality of training samples (Grecchi [0017]-[0020], [0033], [0036]-[0040]).
Regarding claim 11:
Zhu in view of Grecchi and in view of Hughes teaches wherein the generating the image segmentation model includes:
determining the first model and the second model simultaneously based on the first preliminary model, the second preliminary model, and the plurality of training samples (Grecchi [0017]-[0020], [0033], [0036]-[0040], [0059]-[0062],).
Regarding claim 67:
Zhu in view of Grecchi and in view of Hughes teaches wherein the non-image information is input into the first model, and the first model outputs the second image in the image format (Grecchi [0017]-[0020], [0033], [0036]-[0040]), [0059]-[0062], fig. 4, using multiple training models is obvious and well-known as shown by the cited prior arts.
Regarding claim 68:
Zhu in view of Grecchi and in view of Hughes teaches wherein the first image and the second image are input into the second model, and the second model outputs the first image in which the ROI is identified (Grecchi [0017]-[0020], [0033], [0036]-[0040], [0059]-[0062], fig. 4).
Regarding claim 69:
Zhu in view of Grecchi and in view of Hughes teaches wherein the generating the image segmentation model by training a preliminary image segmentation model using the plurality of training samples includes:
for each of the plurality of training samples, generating an estimated second sample image by applying a trained first model generated before the training of a second model, the trained first model being generated by training a first preliminary model using sample non-image information
of the plurality of training samples and a plurality of target second sample images corresponding to the sample non-image information of the plurality of training samples;
generating an estimated ROI by inputting a first sample image and the corresponding estimated second sample image into an updated second model determined in a previous second iteration;
determining, based on the estimated ROI and the target ROI corresponding to the first sample image, an assessment result(Grecchi [0017]-[0020], [0033], [0036]-[0040]), [0059]-[0062], fig. 4).
Regarding claims 70 and 74:
Zhu in view of Grecchi and in view of Hughes teaches wherein the first model is determined by training a first preliminary model using sample non-image information of a plurality of training samples and a plurality of target second sample images corresponding to the sample non-image information (Hughes [0031]-[0034], figs. 4 and 5).
Regarding claim 71:
Zhu in view of Grecchi and in view of Hughes teaches wherein the second preliminary model is downstream to the first preliminary model (Grecchi [0017]-[0020], [0033], [0036]-[0040]; Hughes [0031]-[0034], figs. 4 and 5).
Regarding claim 72:
Zhu in view of Grecchi and in view of Hughes teaches wherein the first model and the second model are determined successively based on the first preliminary model, the second preliminary model, and the plurality of training samples to generate the image segmentation model (Grecchi [0017]-[0020], [0033], [0036]-[0040]; Hughes [0031]-[0034], figs. 4 and 5).
Claims 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (CN 110942462 A, hereinafter “Zhu”) in view of Grecchi et al. (US 20210125707 A1, hereinafter “Grecchi”), in view of Hughes et al. (EP 3770840 A1, hereinafter “Hughes”) and in view of Rong et al. (US 20210383616 A1, hereinafter “Rong”).
Regarding claim 12:
The combination of the cited references fails to teach wherein the generating the image segmentation model further includes:
assessing a loss function that relates to the first model and the second model.
However, Rong teaches can include generating refined image data. The refined image data can be generated by the refinement model in response to the processing of the image segmentation masks and the augmented image data. At (312), the method 300 can include comparing the refined image data and the training image data. The method may further include evaluating a loss function that compares the refined image data and the training image data. The loss function can be a perceptual loss or a GAN loss. Furthermore, the method may further include modifying the parameters of the refinement model in response to the comparison of the refined image data and the training image data (Rong [0147]-[0148).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to use loss function to optimize all the models in the system of Zhu, in view of Grecchi and in view of Hughes, since, it has been shown and is well known in the art that loss function is a well define function that is often use to optimize the parameters of network models with reasonable expectation of success.
Regarding claim 13:
Zhu in view of Grecchi, in view of Hughes and in view of Rong teaches wherein the generating the image segmentation model further includes assessing a first loss function that relates to the first model (Rong [0147]-[0148; se rejection to claim 12 above).
Regarding claim 14:
Zhu in view of Grecchi, in view of Hughes and in view of Rong teaches wherein the generating the image segmentation model further includes assessing a second loss function that relates to the second model (Rong [0147]-[0148], see rejection to claim 12 above).
Regarding claim 15:
Zhu in view of Grecchi, in view of Hughes and in view of Rong teaches wherein the image segmentation model is a machine learning model (Zhu [0002]; Grecchi claim 1; Rong claim 1).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 July 23, 2026
Prosecution Insights