DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The Amendment filed 8 April 2026 (hereinafter “the Amendment”) has been entered and considered. Claims 21, 24-27, 29, 31-32, and 40 have been amended. Claims 21, 28 and 35-39 have been canceled. Claims 41-47 have been added. Claims 21-22, 24-27, 29-34, and 40-47 are currently pending. Claims 21-22, 24-27, and 33 are rejected. Claims 29-32 and 34 are objected to. Claims 40-47 are allowed. All new grounds of rejection set forth in the present action were necessitated by Applicant’s claim amendments; accordingly, this action is made final.
Response to Amendment
Election/Restriction
The provisional election of Group I, claims 21-34 and 40, made during a telephone conversation with Applicant’s representative, Yangzhou Du, on 6 January 2026 has neither been affirmed nor traversed by Applicant. Accordingly, the Restriction Requirement set forth in the previous action is made final.
Double Patenting
The terminal disclaimer filed on 8 April 2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. Patent No. 11,769,249 has been reviewed and is accepted. The terminal disclaimer has been recorded. In view of the terminal disclaimer, the double patenting rejections are withdrawn.
Claim Objections
In view of the cancellation of claim 23 and the amendment to claim 27, the claim objections are withdrawn.
Claim Rejections - 35 USC § 112
In view of the amendment to claim 32, the rejection under 35 USC 112 is withdrawn.
Prior Art Rejections
In view of the amendments to independent claims 21 and 40, the previously-applied prior art rejections are withdrawn.
Independent claim 40 and its dependent claims 41-47 are allowed.
New grounds of rejection of amended independent claim 21 are set forth below. Since the rejection of claim 21 relies on prior art applied in the previous action, Applicant’s arguments are addressed here.
On pages 15-19, Applicant argues that Zhang does not teach or suggest normalizing image values, as required by independent claim 21. In support of this assertion, Applicant contends that the object of normalization in Zhang is the displacement quantities of the neighboring pixels, not the image values themselves (pages 18-19 of the Amendment). The Examiner respectfully submits that Applicant misinterprets the rejection and maintains that the claim limitation in question is indeed taught by Zhang.
As acknowledged by the Applicant on page 18 of the Amendment, Zhang’s “weighting factors for the four neighboring pixels are set based on these normalized displacements”. Importantly, the normalized geometric displacement quantities themselves are not relied upon for teaching the claimed normalized image values, as Applicant appears to imply; rather, it is Zhang’s weights applied to the neighboring pixel values, as derived based on the normalized geometric displacement quantities, that correspond to the claimed normalized image values of the neighboring points.
Thus, Zhang does indeed teach obtaining normalized image values of the one or more neighboring points by normalizing image values of the one or more neighboring points (Zhang discloses weighting the neighboring pixel values), contrary to Applicant’s assertions. In particular, Zhang’s weights are applied to the image values of the neighboring points (as Applicant acknowledges), and they serve a “normalizing” function by virtue of being derived directly from the geometric displacement normalization process discussed by the Applicant.
Nothing in claim language precludes this interpretation. However, if Applicant believes that the normalization of the present invention is different from Zhang’s normalization-derived weighting, the Examiner recommends amending the claim to highlight any such difference.
On pages 19-20 of the Amendment, Applicant argues that the purpose of Zhang’s normalization of displacement is to determine special positions, rather than handling sample points located on boundaries between different tissues.
Initially, as noted above, the Examiner does not rely on Zhang’s displacement normalization for teaching the claimed normalizing of image values of the neighboring points
Furthermore, while the Examiner appreciates the discussion of features related to the disclosed invention, these features are not recited in the rejected claims. In particular, the claim is silent about handling sample points located on boundaries between different tissues. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Moreover, as discussed above, Zhang teaches obtaining normalized image values of the one or more neighboring points by normalizing image values of the one or more neighboring points (Zhang discloses weighting the neighboring pixel values), contrary to Applicant’s assertions.
On pages 24-26 of the Amendment, Applicant contends that Ishii discloses determining the likelihood that adjacent pixels belong to the same tissue category which is allegedly different from claim 21 in which a neighboring point is arbitrarily selected from the neighboring point set and a determination is made as to whether the label of that arbitrarily selected neighboring point is the same as a tissue label of the tissue set.
Initially, the claim is silent about “arbitrary” selection of pixels. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claim language encompasses arbitrary and systematic or intentional selection.
Moreover, in order to perform the very processing that the Applicant acknowledges to be taught by Ishii, the reference necessarily teaches the claimed steps. For example, in order to “determin[e] the likelihood that adjacent pixels belong to the same category”, an adjacent pixel must be selected (arbitrarily, even). In order for Ishii to determine whether the adjacent pixels belong to the same tissue category, a comparison must be made between the tissue labels of the adjacent pixels. If there is a match, as contemplated by Ishii, then the tissue label of the neighboring point is necessarily a tissue label of the tissue set – namely, the tissue label of the target pixel. Thus, contrary to Applicant’s assertions, Ishii does indeed teach the newly added features of claim 21.
For all the foregoing reasons, the applied art renders amended independent claim 21 obvious, as further detailed below.
Claim Objections
Claims 45-47 are objected to because of the following informalities:
Claim 45 recites “The system of claim 40, The system of claim 40…”. One of these phrases should be deleted for clarity.
Claims 46-47 inherit this deficiency by virtue of their dependency on claim 45.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 34 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 34 is dependent on now-canceled claim 23 which renders the scope of the claim unclear. When considering claim 34 on its merits, the claim will be interpreted as being dependent on claim 21.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 21-22 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over “Virtual Endoscopy for Preoperative Planning and Training of Endonasal Transsphenoidal Pituitary Surgery” by Neubauer et al. (cited in the IDS filed 24 October 2023; hereinafter “Neubauer”) in view of U.S. Patent Application Publication No. 2016/0364840 to Zhang (hereinafter “Zhang”) and further in view U.S. Patent Application Publication No. 2014/0334705 to Ishii et al. (hereinafter “Ishii”).
As to independent claim 21, Neubauer discloses a method implemented on at least one machine each of which has at least one processor and at least one storage device (Abstract discloses that Neubauer is directed to “STEPS, a virtual endoscopy system” which is “CPU-based”; p. 36 discloses that the system include “memory”), the method comprising: obtaining an image relating to volume data of a plurality of tissues organized in a tissue set (pp. 28, 32, 84 discloses using one of a variety of imaging modalities to obtain “volume data” which is used for volume rendering the “different tissue types” in the volume data, wherein the tissue types are distinguished by “color coding” (e.g., red for soft tissue and white for bone)); selecting a sample point based on the volume data; obtaining one or more neighboring points of the sample point organized in a neighboring point set; obtaining an interpolation result of the sample point based on an interpolation of the image values of the one or more neighboring points; and determining a color of the sampling point based on the interpolation result (pp. 56-37, 96-102 disclose that the final color of a voxel in the rendered volume may be determined through interpolation between colors of its neighbors, for example, between the colors of 8 neighboring voxels which form a cube that encloses the voxel and are necessarily organized in a set).
Neubauer does not expressly disclose obtaining normalized image values of the one or more neighboring points by normalizing image values of the one or more neighboring points or that the interpolation result is based on interpolation of the normalized image values.
Zhang, like Neubauer, is directed to interpolation in images (Abstract). In particular, Zhang discloses performing “normalization on the displacement” of a sample pixel dot in multiple directions with respect to the “four pixel dots adjacent to” and surrounding the pixel dot, setting weighting factors for the four adjacent pixel dots according to the normalized displacements, and interpolating the sample pixel dot according to the weighting factors for the four adjacent pixel dots ([0086-0104] and Fig. 5a).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Neubauer to perform the interpolation based on normalized values for the neighboring points, as taught by Zhang, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have created a smooth color representation of the rendered volume since each sample point would reflect a properly weighted color value of its neighboring points.
Neubauer as modified by Zhang does not expressly disclose selecting a neighboring point in the neighboring point set; determining whether a label of the neighboring point belongs to a tissue label of the tissue set based on the tissue set and the neighboring point set or that performing the normalizing/interpolating is in response to determining that the label of the neighboring point belongs to the tissue label of the tissue set and based on the tissue label.
Ishii, like Neubauer, is directed to image interpolation using medical images (Abstract and [0159]). Ishii discloses determining “if neighboring pixels are in the same tissue class” prior to calculating a “spatial interpolation” in the image ([0157-0159]). This presupposes the selection of an adjacent pixel and a comparison between the tissue labels of the adjacent pixel and the target pixel. If there is a match, as contemplated by Ishii, then the tissue label of the neighboring point is necessarily a tissue label of the tissue set – namely, the tissue label of the target pixel.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Neubauer and Zhang to determine whether the neighboring image elements are in the same tissue class (target tissue) prior to performing the normalized interpolation based on that selected tissue class, as taught by Ishii, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have “control[ed] a level of smoothness the pixel value x should acquire if neighboring pixels are in the same tissue class” ([0157] of Ishii).
As to claim 22, Neubauer as modified above further teaches obtaining a volume rendering result of the plurality of tissues based on the color of the sample point (pp. 35-37 of Neubauer discloses volume rendering the final colors of the voxels).
As to claim 33, Neubauer as modified above further teaches that the interpolation includes at least one of a linear interpolation, a nonlinear interpolation, an interpolation based on a regularization function, or a diffusion interpolation based on a partial differential equation (p. 96 of Neubauer discloses acquiring the color of the sample point through “linear interpolation”).
Claims 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Neubauer in view of Zhang and Ishii and further in view of U.S. Patent Application Publication No. 2017/0003366 to Jafari-Lhouzani et al. (hereinafter “Jafari-Lhouzani”).
As to claim 24, Ishii discloses a binary determination (0 or 1) – rather than a probability – of whether the neighboring pixels belong to the same tissue class as the pixel of interest ([0152]). Thus, Neubauer as modified above does not expressly disclose that the determining whether a label of the neighboring point belongs to a tissue label of the tissue set based on the tissue set and the neighboring point set includes: obtaining a probability of label of the neighboring point belongs to the tissue label of the tissue set; and determining whether the label of the neighboring point belongs to the tissue label of the tissue set based on the probability.
Jafari-Lhouzani, like Neubauer, is directed to image interpolation using medical images (Abstract). Similar to Zhang, Jafari-Lhouzani discloses that weighting factors for points surrounding a sample point to be interpolated are set based on a normalization ([0051-0052]). Jafari-Lhouzani notes the problem that “inaccurate interpolation” can occur when “neighboring voxels, from which an unknown voxel is estimated, may not have the same tissue type as the unknown voxel” ([0028]). To address this problem, Jafari-Lhouzani discloses that “the probability of neighbors having similar tissue type” is calculated, “weights are generated” using “the probability that voxels v and k have similar tissue types”, and the weights are “used in the interpolation” ([0050-0054]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Neubauer, Zhang and Ishii to determine a probability of whether an unknown voxel belongs to a same tissue type as a neighboring voxel, as taught by Jafari-Lhouzani, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have addressed the problem that “inaccurate interpolation” can occur when “neighboring voxels, from which an unknown voxel is estimated, may not have the same tissue type as the unknown voxel” ([0028] of Jafari-Lhouzani). It is also predictable that Jafari-Lhouzani’s probability is a more nuanced metric than Ishii’s binary determination.
As to claim 25, the proposed combination of Neubauer, Zhang, Ishii, and Jafari-Lhouzani further teaches that the probability of the label of the neighboring point belongs to the tissue label of the tissue set is determined based on a filter corresponding to a tissue with the tissue label, the filter being determined based on an attribute of the tissue with the tissue label ([0051-0056] of Jafari-Lhouzani discloses that the “probability of neighbors having similar tissue type” is expressed in equation 5 which involves a feature vector F of “tissue propert[ies]” of neighboring voxels determined using “filters”; the reasons for combining the references are the same as those discussed above in conjunction with claim 24).
Claims 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Neubauer, in view of Zhang, Ishii and Jafari-Lhouzani and further in view of “ADNet++: A Few-Shot Learning Framework for Multi-Class Medical Image Volume Segmentation with Uncertainty-guided Feature Refinement” by Hansen et al. (hereinafter “Hansen”).
As to claim 26, Neubauer as modified above does not expressly disclose that the probability of the label of the neighboring point belongs to the tissue label of the tissue set is determined based on a trained machine learning model.
Hansen, like Neubauer, is directed to “multi-class segmentation” of a “medical image volume” (Abstract and Title). In particular, Hansen discloses a trained deep learning framework ADNet++ which inputs the medical image volume and outputs a softmax probability of tissue type class for each voxel therein, the tissue classes including “left kidney”, “right kidney”, “spleen” and “liver” (Sections 3-4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Neubauer, Zhang, Ishii, and Jafari-Lhouzani to use a trained machine learning model to determine a probability of each voxel in the volume belonging to a tissue in the tissue set, as taught by Hansen, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have provided “more trustworthy and more accurate predictions” for each voxel, as deep learning frameworks such as Hansen’s outperform classic image-based techniques of classification (Section 1 of Hansen).
As to claim 27, Neubauer as modified above does not expressly disclose that the trained machine learning model is trained according to a training process including: obtaining a training set of data, the training set of data including inputs each of which has a known output, each of the inputs including sample volume data and a reference probability of a sample point belongs to a tissue in the plurality of tissues in the sample volume data; and performing, based on the training set of data, an iteration process including multiple iterations until a termination condition is satisfied.
Hansen, like Neubauer, is directed to “multi-class segmentation” of a “medical image volume” (Abstract and Title). In particular, Hansen discloses a trained deep learning framework ADNet++ which inputs the medical image volume and outputs a softmax probability of tissue type class for each voxel therein, the tissue classes including “left kidney”, “right kidney”, “spleen” and “liver” (Sections 3-4). Hansen discloses that the training process is performed using “a training dataset with base classes” including “ground-truth segmentations” which label each voxel with the expected output tissue class, wherein the training is performed “over 25k iterations” which is a stopping condition (Section 3 and 4.1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Neubauer, Zhang, Ishii, and Jafari-Lhouzani to use a trained machine learning model to determine a probability of each voxel in the volume belonging to a tissue in the tissue set, wherein the machine learning model is trained using a training set of volume image data and corresponding ground-truth labels over 25K iterations of weight optimization, as taught by Hansen, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have provided “more trustworthy and more accurate predictions” for each voxel, as deep learning frameworks such as Hansen’s outperform classic image-based techniques of classification (Section 1 of Hansen).
Allowable Subject Matter
Claims 29-32 and 34 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 40-47 are allowed.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN M CONNER whose telephone number is (571)272-1486. The examiner can normally be reached 10 AM - 6 PM Monday through Friday, and some Saturday afternoons.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN M CONNER/Primary Examiner, Art Unit 2663
Prosecution Insights