DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The Amendment filed 22 June 2026 (hereinafter “the Amendment”) has been entered and considered. Claims 1, 5-6, 15 and 19-20 have been amended. Claims 2-4 have been canceled. Claims 1 and 5-20, all the claims pending in the application, are rejected. All new grounds of rejection set forth in the present action were necessitated by Applicant’s claim amendments; accordingly, this action is made final.
Response to Amendment
Independent claims 1, 19, and 20 have been similarly amended to incorporate the features of now-canceled dependent claims 2-4 along with some additional features. In view of the substantive amendments to the independent claims, the previously-applied prior art rejections are withdrawn. Applicant’s arguments are rendered moot in view of the new grounds of rejection set forth below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 5-20 are rejected under 35 U.S.C. 103 as being unpatentable over “All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation” by Xu et al. (hereinafter “Xu”) in view of “Optimization Methods for Large-Scale Machine Learning” by Bottou et al. (hereinafter “Bottou”).
According to MPEP § 2153.01(a), “If…the application names fewer joint inventors than a publication (e.g., the application names as joint inventors A and B, and the publication names as authors A, B and C), it would not be readily apparent from the publication that it is an inventor-originated disclosure and the publication would be treated as prior art under AIA 35 U.S.C. 102(a)(1)”.
The Examiner notes that Xu appears to have four authors in common with the inventors of listed in the subject application (XU, Zhe; LU, Donghuan; MA, Kai; and ZHENG, Yefeng). However, five additional people (Yixin Wang, Lequan Yu, Jiangpeng Yan, Jie Luo, and Raymond Kai-yu Tong) are listed as co-authors of the Xu publication. Therefore, it is not readily apparent from the Xu publication that it is an inventor-originated disclosure. Based upon the earlier publication date of the Xu reference (9/28/21) relative to the effective filing date of the subject application (11/10/21), according to MPEP § 2153.01(a), Xu constitutes prior art under 35 U.S.C. 102(a)(1).
Further, the publication date of the reference falls within the one year grace period of the effective filing date of the subject application. Accordingly, this rejection under 35 U.S.C. 102(a)(1) might be overcome by: (1) a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor of this application and is thus not prior art in accordance with 35 U.S.C.102(b)(1)(A); or (2) a showing under 37 CFR 1.130(b) of a prior public disclosure under 35 U.S.C. 102(b)(1)(B). See MPEP § 717.01.
As to independent claim 1, Xu discloses an image segmentation model training method, performed by an electronic device (Abstract discloses that Xu is directed to “a novel cyclic prototype consistency learning (CPCL) framework” which is used, for example, for “brain tumor segmentation from MRI and kidney segmentation from CT images”; Section IV(A)(4) discloses that the framework is “implemented in Python with PyTorch, using NVIDIA GeForce RTX 3090 GPU with 24GB memory”), the image segmentation model training method comprising: acquiring one or more first images, one or more second images, and one or more labeled images, each labeled image among the one or more labeled images being an image segmentation result obtained by labeling a corresponding first image among the one or more first images; for a first image (Xl) among the one or more first images, a second image (Xu) among the one or more second images, and a labeled image (Yl) among the one or more labeled images corresponding to the first image (Section III(A) discloses a training set comprising labeled dataset which includes a labeled image Xl and a “ground-truth segmentation label” Yl of the labeled image Xl, wherein the training set further comprises an unlabeled set which includes unlabeled image Xu; see also Section IV(A) which discloses training datasets comprising multiple sets of the above data; see also Fig. 2, reproduced and annotated below, which shows the operation for each set of Xl, Xu, and Yl); acquiring a first predicted image (Pl) according to a first network model (Enc-Dec), the first predicted image being an image segmentation result obtained by predicting the first image (Fig. 2 shows a “student model” comprising encoder Enc and decoder Dec which predicts a segmentation result Pl of the labeled image Xl); acquiring a second predicted image (Pu) according to a second network model (EMA Enc – EMA Dec), the second predicted image being an image segmentation result obtained by predicting the second image (Fig. 2 shows a “self-ensembling teacher model” comprising encoder EMA Enc and decoder EMA Dec which predicts a segmentation result Pu of the unlabeled image Xu; see also Section III); determining a reference image (Pl2u) of the second image based on the second image (Xu) and the labeled image (Yl), the reference image being an image segmentation result obtained by calculating the second image (Fig. 2 shows probability map Pl2u is obtained based on the “unlabeled features” of the unlabeled image Xu (such features being calculated, as claimed, by Enc EMA extracting the unlabeled features from the unlabeled image Xu) and an expanded “labeled prototype” that is generated based on the “labeled features” Fl of the labeled image Xl combined with “ground-truth segmentation label” Yl; see also Section III); updating a model parameter of the first network model based on the first predicted image (Pl), the labeled image (Yl), the second predicted image (Pu), and the reference image (Pl2u) to obtain an image segmentation model (Section III and Fig. 2 discloses that the CPCL framework (including the “student model” corresponding the claimed first network) is trained to update “student weights” according to loss function L in equation 11 which is based on segmentation result Pl and ground-truth label Yl (see equation 12) and segmentation result Pu and probability map Pl2u (see equation 6); and updating a model parameter of a decoder of the second network model by applying an exponential moving average to a model parameter of a decoder of the first network model (Fig. 2 and Section III(A) discloses that “Enc and Dec represent the student model, while EMA Enc and EMA Dec denote the self-ensembling teacher model updated as the exponential moving average (EMA) of student weights” and that “we update the teacher model’s weights ˜θt at the training step t by means of the exponential moving average (EMA) approach”; see also Equation (2)), wherein the determining the reference image of the second image comprises: acquiring a feature map (Fl) of the first image, the feature map for characterizing semantic information of the first image (Section III and Fig. 2 discloses “feature map” Fl which is “extracted by the encoder for the labeled image” Xl which necessarily carries semantic information of the image, wherein equations 3 and 4 disclose that the foreground and background prototypes are calculated based on the feature map Fl), determining, in the feature map, a first voxel feature of a first voxel with a spatial position located in the first region and a second voxel feature of a second voxel with a spatial position located in another region except the first region in the first image based on the labeled image (Section III and Fig. 2 discloses determining a foreground prototype pl(fg) and a background prototype pl(bg) (See “labeled prototype” in Fig. 2) from based on the “labeled features” extracted from the labeled image Xl combined with the ground-truth segmentation label Yl; notably, the foreground and background targets are mutually exclusive portions in view of the “binary segmentation” masks described in Section III); calculating an average value of the first voxel feature as the foreground prototype, the foreground prototype being a reference feature of the first region in the first image; and calculating an average value of the second voxel feature as the background prototype, the background prototype being a reference feature of the another region except the first region in the first image (Section III and Fig. 2 discloses that the foreground and background prototypes are calculated via “masked average pooling” according to “the spatial location for each voxel” (x, y, z) and the features in the feature map Fl at that spatial location based on the ground-truth segmentation label Yl; see equations 3 and 4); and determining the reference image based on the foreground prototype, the background prototype, and the second image (Section III and Fig. 2 discloses that probability map Pl2u is obtained based on the “unlabeled features” of the unlabeled image Xu and the “labeled prototype” including foreground pl(fg) and background pl(bg) prototypes; see equation 5).
PNG
media_image1.png
638
1056
media_image1.png
Greyscale
PNG
media_image2.png
98
474
media_image2.png
Greyscale
PNG
media_image3.png
102
500
media_image3.png
Greyscale
Xu’s equations 3-4, reproduce above, necessarily teach both a scenario in which a single labeled (first) image is used in which k=1, and another scenario in which multiple labeled (first) images are used in which k>1. Thus, if a user opted for single-sample training (i.e., stochastic gradient descent), then the averaging across images would effectively not take place since the constant 1/K would become 1, and the summation over the k images in the respective formulas would only be performed for the single image, thus resulting in the values for that particular image. However, the lone example disclosed by Xu includes a batch size set to 4, including 2 labeled images and 2 unlabeled images in each mini-batch (Section IV(A)(4)). That is, Xu does not expressly disclose the claimed condition: based on a quantity of the one or more first images being one, despite the fact that equations 3-4 of Xu clearly contemplate that scenario.
However, Bottou discusses the “trade-offs” between using the “stochastic gradient descent method (SG)…corresponding to one sample” and “batch” optimization methods (Section 3.2). For example, Bottou discloses that “SG employs information more efficiently than a batch method” (Section 3.3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to replace Xu’s mini-batch approach of using multiple images with a stochastic gradient descent method with one sample (such that k=1), as taught by Bottou, to arrive at the claimed invention discussed above. Such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, Xu’s mini-batch approach and Bottou’s SG approach perform the same general and predictable function, the predictable function being to train the deep learning framework. Since each individual element and its function are shown in the prior art, albeit shown in separate references, the difference between the claimed subject matter and the prior art rests not on any individual element or function but in the very combination itself - that is in the substitution of Xu’s mini-batch approach by replacing it with Bottou’s SG approach using a single sample. Thus, the simple substitution of one known element for another producing a predictable result renders the claim obvious. It is predictable that the proposed modification would have “employ[ed] information more efficiently” (Section 3.3 of Bottou).
As to claim 5, Xu as modified above further teaches that, based on the quantity of the one or more first images being one, the determining the reference image based on the foreground prototype, the background prototype, and the second image comprises: acquiring a feature map of the second image, the feature map for characterizing semantic information of the second image; and determining the reference image based on the foreground prototype, the background prototype, and the feature map (Section III and Fig. 2 of Xu discloses acquiring “unlabeled feature map” Fu from teacher encoder EMA Enc which extracts the unlabeled features from the unlabeled image Xu, the unlabeled feature necessarily carrying semantic information, wherein the probability map Pl2u is calculated based on the foreground prototype pl(fg), background prototype pl(bg), and unlabeled feature map Fu; see equations 3-5; Section 3.3 of Bottou discloses the advantages of using single-sample training over the mini-batch approach of Xu; the reasons for combining the references are the same as those discussed above in conjunction with claim 1).
As to claim 6, Xu as modified above further teaches that the determining the reference image based on the foreground prototype, the background prototype, and the feature map of the second image comprises: determining, for a voxel in the second image, a voxel feature of the voxel based on the feature map of the second image; calculating a foreground similarity between the voxel feature of the voxel and the foreground prototype and a background similarity between the voxel feature of the voxel and the background prototype; determining a foreground probability that the voxel belongs to a foreground region based on the foreground similarity; determining a background probability that the voxel belongs to a background region based on the background similarity; determining a segmentation result of the voxel based on the foreground probability and the background probability; and determining the reference image based on the segmentation result of the voxel (Section III, equation 5, and Fig. 2 of Xu discloses calculating the “cosine distance” between a feature in the unlabeled feature map Fu at the spatial location (x, y, z) of a voxel in the unlabeled image Xu and the each of the foreground and background prototypes pl “to produce the probability map Pl2u” over the foreground and background classes).
As to claim 7, Xu as modified above further teaches that updating the model parameter comprises: determining a loss value of the first network model based on the first predicted image, the labeled image, the second predicted image, and the reference image; and updating the model parameter based on the loss value to obtain the image segmentation model (Section III and Fig. 2 discloses that the CPCL framework (including the “student model” corresponding the claimed first network) is trained to update “student weights” according to loss function L in equation 11 which is based on segmentation result Pl and ground-truth label Yl (see equation 12) and segmentation result Pu and probability map Pl2u (see equation 6)).
As to claim 8, Xu as modified above further teaches that determining the loss value of the first network model comprises: determining a first loss value based on the first predicted image and the labeled image; determining a second loss value based on the second predicted image and the reference image; and determining the loss value of the first network model based on the first loss value and the second loss value (Section III and Fig. 2 discloses that the total loss function L in equation 11 is calculated based on a “supervised loss” Ls, and “forward and backward prototype consistency loss[es]” Lfpc and Lbpc, wherein the supervised loss Ls is based on segmentation result Pl and ground-truth label Yl (see equation 12) and the forward prototype consistency loss Lfpc is based on segmentation result Pu and probability map Pl2u (see equation 6)).
As to claim 9, Xu as modified above further teaches that determining the loss value of the first network model comprises: determining the loss value of the first network model based on the model parameter of the first network model, a second model parameter of the second network model, the first predicted image, the labeled image, the second predicted image, and the reference image (Section III and Fig. 2 discloses that the CPCL framework is trained according to loss function L in equation 11, wherein the loss function is determined according to “the weights of the student model and the teacher model” (θ and θ˜, respectively); see equation 1; Section III discloses that the loss function of equation 11 is further based on segmentation result Pl and ground-truth label Yl (see equation 12) and segmentation result Pu and probability map Pl2u (see equation 6)).
As to claim 10, Xu as modified above further teaches that the second model parameter is determined based on the model parameter of the first network model (Fig. 2 discloses that the weights of the “self-ensembling teacher model [are] updated as the exponential moving average (EMA) of student weights”).
As to claim 11, Xu as modified above further teaches determining a first weight (α) of a third model parameter of a third network model (θ~t-l) and a second weight (1-α) of the model parameter of the first network model (θt); and performing weighted summation on the third model parameter and the model parameter of the first network model based on the first weight and the second weight to obtain the second model parameter (θ~t) of the second network model (See equation 2 of Section III).
As to claim 12, Xu as modified above further teaches that after acquiring the second predicted image, the image segmentation model training method further comprises: determining a first reference image of the first image based on the first image and the second predicted image, the first reference image being an image segmentation result obtained by calculating the first image (Section III and Fig. 2 shows probability map Pu2l which is a segmentation result obtained based on the “labeled features” Fl of the labeled image Xl (such features being calculated, as claimed, by Enc extracting the labeled features from the labeled image Xl) and an expanded “unlabeled prototype” that is generated based on the “unlabeled features” Fu of the unlabeled image Xu combined with the unlabeled segmentation result Pu); and wherein the determining the loss value of the first network model comprises: determining the loss value of the first network model based on the first predicted image, the labeled image, the first reference image, the second predicted image, and the reference image of the second image (Section III and Fig. 2 discloses that the CPCL framework is trained to update “student weights” according to total loss function L in equation 11 which is based on segmentation result Pl and ground-truth label Yl (see equation 12) and segmentation result Pu and probability map Pl2u (see equation 6) and the probability map Pu2l (see equation 10)).
As to claim 13, Xu as modified above further teaches that the determining the first reference image comprises: determining a foreground prototype of the second image and a background prototype of the second image based on the second predicted image, the foreground prototype being a reference feature of a second region in the second image, and the background prototype being a reference feature of another region except the second region in the second image; and determining the first reference image based on the foreground prototype, the background prototype, and the first image (Section III and Fig. 2 discloses calculating “the foreground and background holistic prototypes for unlabeled data” Xu based on segmentation result Pu (See equations 7 and 8), wherein the foreground and background regions are necessarily mutually exclusive, wherein probability map Pu2l is calculated based on the foreground and background holistic prototypes and features Fl of the labeled image Xl (see equation 9)).
As to claim 14, Xu as modified above further teaches that the determining the foreground prototype and the background prototype comprises: acquiring a second feature map of the second image; and determining the foreground prototype and the background prototype based on the second feature map and the second predicted image (Section III and Fig. 2 discloses acquiring feature map Fu of the unlabeled image Xu and determining the foreground and background holistic prototypes based thereon (see equation 8)).
As to claim 15, Xu as modified above further teaches that the determining the foreground prototype and the background prototype based on the second feature map and the second predicted image comprises: determining, in the second feature map, a voxel feature of a third voxel with a spatial position located in the second region and a voxel feature of a fourth voxel with a spatial position located in the another region based on the second predicted image; and calculating an average value of the voxel feature of the third voxel as the foreground prototype; and calculating an average value of the voxel feature of the fourth voxel as the background prototype (Section III and Fig. 2 discloses that the foreground and background holistic prototypes are respectively calculated via “masked average pooling” according to the spatial location for each voxel (x, y, z) and the feature in the feature map Fu at that spatial location based on an unlabeled binary prediction mask calculated according to the predicted segmentation result Pu of the unlabeled image Xu; see equations 7 and 8).
As to claim 16, Xu as modified above further teaches that determining the first reference image based on the foreground prototype, the background prototype, and the first image comprises: acquiring a feature map of the first image; and determining the first reference image based on the foreground prototype, the background prototype, and the feature map of the first image (Section III and Fig. 2 discloses acquiring feature map Fl of the labeled image Xl and determining the probability map Pu2l based on the foreground and background holistic prototypes and the feature map Fl (see equation 9)).
As to claim 17, Xu as modified above further teaches that the determining the first reference image of the first image based on the foreground prototype, the background prototype, and the feature map of the first image comprises: determining, for a voxel in the first image, a voxel feature of the voxel based on the feature map of the first image; calculating a foreground similarity between the voxel feature of the voxel and the foreground prototype and a background similarity between the voxel feature of the voxel and the background prototype; determining a foreground probability that the voxel belongs to a foreground region based on the foreground similarity; determining a background probability that the voxel belongs to a background region based on the background similarity; determining a segmentation result of the voxel based on the foreground probability and the background probability; and determining the first reference image based on the segmentation result of the voxel in the first image (Section III, equation 9, and Fig. 2 discloses calculating the “cosine distances” between a feature in the labeled feature map Fl at the spatial location (x, y, z) of a voxel in the labeled image Xl and the each of the foreground and background holistic prototypes pu “to produce the U2L probability map Pu2l” over the foreground and background classes).
As to claim 18, Xu as modified above further teaches that the determining the loss value of the first network model comprises: determining a first loss value based on the first predicted image and the labeled image (equation 12 shows a supervised loss Ls calculated based on the predicted segmentation result Pl and the ground-truth segmentation label Yl); determining a second loss value based on the second predicted image and the reference image (equation 6 discloses forward prototype consistency loss Lfpc calculated based on predicated segmentation result Pu and probability map Pl2u); determining a third loss value based on the labeled image and the first reference image (equation 10 discloses backward prototype consistency loss Lbpc calculated based on the ground-truth segmentation label Yl and probability map Pu2l); and determining the loss value of the first network model based on the first loss value, the second loss value, and the third loss value (equation 11 discloses that the total loss is calculated based on each of the above-discussed losses).
Independent claim 19 recites an image segmentation model training apparatus, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code (Section IV(A)(4) of Xu discloses that the framework is “implemented in Python with PyTorch, using NVIDIA GeForce RTX 3090 GPU with 24GB memory”, wherein the Python code is necessarily stored in memory), the program code comprising code configured to cause at least one of the at least one processor to perform the steps recited in the method of independent claim 1. Accordingly, claim 19 is rejected for reasons analogous to those discussed above in conjunction with claim 1 mutatis mutandis.
Independent claim 20 recites a non-transitory computer-readable storage medium storing computer code which, when executed by at least one processor, causes the at least one processor to (Section IV(A)(4) of Xu discloses that the framework is “implemented in Python with PyTorch, using NVIDIA GeForce RTX 3090 GPU with 24GB memory”, wherein the Python code is necessarily stored on some medium) at least perform the steps recited in the method of independent claim 1. Accordingly, claim 20 is rejected for reasons analogous to those discussed above in conjunction with claim 1 mutatis mutandis.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN M CONNER whose telephone number is (571)272-1486. The examiner can normally be reached 10 AM - 6 PM Monday through Friday, and some Saturday afternoons.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN M CONNER/Primary Examiner, Art Unit 2663
