DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 8/6/2026.
Claims 1-5, 7-13, 16-18 are pending. Claims 1, 2, 7, 17-18 have been amended. Claims 6, 14-15 cancelled.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a receiving module, a scaling module, a determining module, a rendering module in claim 17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-5, 8-12, 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Engel (US 20210090325 A1) in view of Grbic et al. (US 20160267673 A1, hereinafter Grbic), further in view of Chaitanya et al. ("Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder," ACM Transactions on Graphics, 2017, hereinafter "Chaitanya"), and Petkov (US 10311631 B2).
Regarding Claim 1, Engel teaches a computer-implemented method for volume rendering and/or surface rendering of a medical imaging data set based on optical properties per sampling points the method comprising (Engel, Paragraph [0013], "the present invention refers to a computer-implemented method for rendering medical volumetric images from volumetric data, received from an imaging modality or from a reconstruction or storage unit (like e.g. PACS), using a cinematic rendering approach. The method uses or executes a Monte Carlo path tracing algorithm (MCPT algorithm). The MCPT algorithm uses at least one and usually several microfacet-based bidirectional reflectance distribution functions (also abbreviated as BRDF) for computing a probability how light is reflected at an implicit surface which is used for shading the implicit surface"):
[[ receiving an uncertainty indicator per voxel and/or per surface element in relation to a segmentation mask of and/or an anatomical structure comprised in the medical imaging data set; ]] and scaling a randomization of one or more of the sampling points (Engel, Paragraph [0015], "modifying the computation of a local gradient in the BRDF by perturbing the respective received volumetric data, required for the gradient computation, by applying a noise function for simulating a roughness of the implicit surface"; [0077], "A roughness parameter [0.,1.] is used for modifying or scaling the amount of noise added to each gradient computation sample, thus allowing any surfaces roughness from zero (leading to a perfect mirror surface) to one (1) (for a diffuse surface).") [[ based on the received uncertainty indicator ]];
determining at least one optical property per sampling point as randomized (Engel, Paragraph [0007], "a global transfer function assigns color and opacity values <read on optical property> to each voxel <read on per sampling point> of the volume"; [0013], "The MCPT algorithm uses at least one and usually several microfacet-based bidirectional reflectance distribution functions (also abbreviated as BRDF) for computing a probability how light is reflected at an implicit surface which is used for shading the implicit surface"; [0104], "the renderer comprises a modifier for modifying (perturbing) the local gradients by small random numbers for simulating a roughness of an implicit surface"; [0125], "// random number generator” section shows a random number generator applied to each volume sample position, confirming the sampling point is randomized; [0079], "By modifying the computation of the local gradient in the BRDF, the BRDF will be modified accordingly. The so modified BRDF will be used for computation of the shading by the shader"); rendering a volume based on the voxels and/or a surface based on the surface elements, wherein the rendering is based on the determined at least one optical property per sampling point (Engel, Paragraph [0016], "shading the implicit surfaces by using the modified BRDF, as mentioned in the preceding 'modifying' step for rendering the received volumetric data."; [0122], "in step S2 the computation of the local gradients in the BRDF are modified. This modification is based on perturbing the respective received volumetric data, which are required for the gradient computation, by applying a noise function for simulating a roughness of the implicit surface. Subsequently in step S3 a shading for the implicit surfaces is computed used for rendering the received volumetric data.")
But Engel does not explicitly disclose receiving an uncertainty indicator per voxel and/or per surface element in relation to a segmentation mask of and/or an anatomical structure comprised in the medical imaging data set … that the scaling of the randomization is based on the received uncertainty indicator … in a presence of a user interaction, performing the rendering without accumulating multiple stochastic passes over the medical imaging data set, wherein the rendering in the presence of the user interaction comprises: performing an interactive reconstruction of a Monte Carlo image sequence using a recurrent denoising autoencoder combining spatial and temporal denoising using Monte Carlo samples from current and previous frames, and in an absence of the user interaction, accumulating multiple stochastic passes over the medical imaging data set to render the volume and/or the surface.
However, Grbic teaches receiving an uncertainty indicator (Grbic, Paragraph [0024], "Uncertainty 106 is estimated on surface model 104. In one embodiment, uncertainty is estimated for each vertex of surface model 104 to calculate a range of acceptable boundary points using any machine learning based algorithm, such as, e.g., a probabilistic boosting tree classifier trained with image features"); per voxel and/or per surface element (Grbic, Paragraph [0025] "A level of uncertainty for each voxel may be visually represented on surface model 104"; [0037], "uncertainty determination module 212 is configured to estimate uncertainty at each vertex of the surface model to provide an uncertainty model") in relation to a segmentation mask of and/or an anatomical structure comprised in the medical imaging data set Grbic, Paragraph [0003], "generating a surface model of a target anatomical object from medical imaging data of a patient... Uncertainty is estimated at each of a plurality of vertices of the surface model"; Paragraph [0023], "Surface model 104 of the target anatomical object is generated using imaging data 102. Surface model 104 <read on segmentation mask> may be a 3D surface model of the target anatomical object. Surface model 104 may be generated using any segmentation algorithm"; Paragraph [0038], "Each vertex corresponds to a point on the boundary of the segmented surface")
scaling of the randomization … based on the received uncertainty indicator (Grbic, Paragraph [0048], "In another embodiment, stochastic methods may be applied where point configurations can be drawn based on the uncertainty surface model <read on based on the received uncertainty indicator>").
Grbic and Engel are analogous since both are directed to processing and rendering medical imaging data of anatomical structures, and both address how properties derived from the underlying data are used to drive the visual appearance of the rendered output. Engel provided a way of rendering medical volumetric images using Monte Carlo path tracing with perturbation of the local gradient by a noise function applied at sampling points along implicit surfaces, modifying the BRDF used for shading. Grbic provided a way of computing a per-vertex/per-voxel uncertainty indicator derived from the segmentation of anatomical structures in medical imaging data, and visualizing that uncertainty on the surface model. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the per-voxel/per-surface-element segmentation uncertainty indicator taught by Grbic into the modified invention of Engel such that the perturbation applied to sampling points during rendering is scaled based on the uncertainty indicator derived from the segmentation mask, rather than by a fixed noise function. The motivation is to leverage the segmentation uncertainty information already computed as part of the segmentation pipeline to directly control the degree of perturbation in the rendering step, allowing regions of higher segmentation uncertainty to receive a larger randomization of sampling positions and thus a smoother rendered appearance at uncertain boundaries.
The combination of Engel and Grbic, Chaitanya and Petkov does not explicitly disclose wherein the rendering in the presence of the user interaction comprises: performing an interactive reconstruction of a Monte Carlo image sequence using a recurrent denoising autoencoder combining spatial and temporal denoising using Monte Carlo samples from current and previous frames.
However, Chaitanya teaches wherein the rendering in the presence of the user interaction comprises (Chaitanya, Page 98:1, Abstract, "Our primary focus is on reconstruction of global illumination with extremely low sampling budgets at interactive rates <read on the rendering in the presence of the user interaction>"; Page 98:2, Section 1 INTRODUCTION, "These advances allow us to interactively <read on in the presence of the user interaction> generate plausible image sequences with global illumination at extremely low sampling budgets, consistently outperforming the quality of the state-of-the-art in this regime"):performing an interactive reconstruction of a Monte Carlo image sequence using a recurrent denoising autoencoder (Chaitanya, Page 98:1, Title, "Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder"; Page 98:1, Abstract, "We describe a machine learning technique for reconstructing image sequences rendered using Monte Carlo methods. Our primary focus is on reconstruction of global illumination with extremely low sampling budgets at interactive rates"; Page 98:4, Section 4.1, "our network architecture includes distinct encoder and decoder stages that operate on decreasing and increasing spatial resolutions, respectively. This makes it similar to the Flownet [Fischer et al. 2015] and U-Net [Ronneberger et al. 2015] architec- tures that give good results in optical ow estimation and image segmentation, respectively, and also emphasizes the connection to denoising autoencoders [Vincent et al. 2008]. Autoencoders are networks that learn to reconstruct their inputs from some internal representation, and denoising autoencoders also learn to remove noise from the inputs. We use the term denoising autoencoder be-cause we reconstruct from noisy inputs. "); combining spatial and temporal denoising (Chaitanya, Page 98:1, Abstract, "Our primary contribution is the addition of recurrent connections to the network in order to drastically improve temporal stability for sequences of sparsely sampled input images"; Page 98:4, Section 4.2, "In order to retain temporal features at multiple scales, we intro-duce fully convolutional recurrent blocks after every encoding stage"; Page 98:5, Section 4.2, " This provides a sufficient temporal receptive field and, together with the multi-scale cascade of such recurrent blocks, allows to efficiently track and retain image features temporally at multiple scales "); using Monte Carlo samples from current and previous frames (Chaitanya, Page 98:1, “We describe a machine learning technique for reconstructing image sequences rendered using Monte Carlo methods”; Page 98:5, Section 4.2, "Each recurrent block consists of three convolution layers with a 3× 3-pixel spatial support. One layer processes the input features from the previous layer of the encoder. It then concatenates the results with the features from the previous hidden state <read on previous frames>, and passes it through two remaining convolution layers. The result becomes both the new hidden state and the output of the recurrent block"; Page 98:4, Section 4.2, "in order to denoise a continuous stream of images, we need to achieve temporally stable results. Second, because our inputs are very sparse, the recurrent connections also gather more information about the illumination over time").
Chaitanya and the modified invention of Engel and Grbic are analogous since both are directed to reconstruction of images rendered using Monte Carlo methods. The modified invention of Engel and Grbic provided a way of rendering medical volumetric images using a Monte Carlo path tracing algorithm at interactive rates with per-voxel/per-surface-element uncertainty-driven perturbation of the sampling positions. Chaitanya provided a machine-learning technique for interactively reconstructing image sequences produced by Monte Carlo rendering at extremely low sampling budgets, using a recurrent denoising autoencoder whose recurrent connections carry a hidden state from one frame to the next and whose hierarchical convolutional structure captures spatial neighborhoods at multiple scales. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the recurrent denoising autoencoder for interactive Monte Carlo image sequence reconstruction taught by Chaitanya into the modified invention of Engel and Grbic such that, during the interactive rendering phase of the medical Monte Carlo path tracer, the rendering is performed by an interactive reconstruction of the Monte Carlo image sequence using a recurrent denoising autoencoder combining spatial and temporal denoising using Monte Carlo samples from current and previous frames. The motivation is to interactively generate plausible image sequences with extremely low sampling budgets and to obtain temporally stable, high-quality reconstruction during interactive rendering, as expressly taught by Chaitanya at Page 98:1 Abstract and Page 98:5 Section 4.2.
The combination of Engel, Grbic and Chaitanya does not explicitly disclose in a presence of a user interaction, performing the rendering without accumulating multiple stochastic passes over the medical imaging data set; and in an absence of the user interaction, accumulating multiple stochastic passes over the medical imaging data set to render the volume and/or the surface.
However, Petkov teaches in a presence of a user interaction, performing the rendering without accumulating multiple stochastic passes over the medical imaging data set (Petkov, Paragraph [0073], "In one embodiment, the image changes over time, such as due to user interaction, on-going acquisition of medical scan data, and/or continued addition of information from further light path simulation. For example, the user positions a clipping plane or planes so that different parts of the patient volume are masked. As a result, the rendering shows the internal region from the scan data not clipped or that intersects the clipping plane"; Paragraph [0073], "One rendered image is replaced with another rendered image based on the change in the rendering parameter, such as the view direction, segmentation, or clipping plane position <read on performing the rendering without accumulating multiple stochastic passes during user interaction; the image is replaced rather than accumulated when the user changes a rendering parameter>"); in an absence of the user interaction, accumulating multiple stochastic passes over the medical imaging data set to render the volume and/or the surface (Petkov, Paragraph [0073], "continued addition of information from further light path simulation <read on accumulating multiple stochastic passes in an absence of the user interaction>"; Paragraph [0051], "The physically-based rendering result builds up over time as the rendering may rely on probabilistic scattering and tracing millions of light paths"; Paragraph [0014], "Evaluation of the rendering integral in physically-based volume rendering may require thousands of stochastic samples per pixel <read on multiple stochastic passes over the medical imaging data set> to produce an acceptably noise-free image").
Petkov and the modified invention of Engel, Grbic and Chaitanya are analogous since both are directed to physically-based Monte Carlo path tracing for visualization of medical volumetric imaging data. The modified invention of Engel, Grbic and Chaitanya provided a way of rendering medical volumetric images using Monte Carlo path tracing with uncertainty-driven perturbation of sampling positions and, during interactive rendering, using a recurrent denoising autoencoder to reconstruct image sequences from sparsely sampled input frames. Petkov provided a way of updating the rendered medical image during user interaction by replacing one rendered image with another rendered image in response to a change in the rendering parameter, and improving the rendered image quality through continued addition of information from further light path simulation when no user interaction is occurring, recognizing that thousands of stochastic samples per pixel are needed to produce an acceptably noise-free image. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the interactive-mode / idle-mode rendering behavior taught by Petkov into the modified invention of Engel, Grbic and Chaitanya such that in a presence of a user interaction, the rendering is performed without accumulating multiple stochastic passes over the medical imaging data set (with the recurrent denoising autoencoder of Chaitanya providing the interactive-mode reconstruction), and in an absence of the user interaction, multiple stochastic passes are accumulated over the medical imaging data set to render the volume and/or the surface. The motivation is to provide a responsive interactive medical rendering experience during user manipulation while maximizing rendered image quality through multi-pass stochastic accumulation during idle periods, as taught by Petkov at Paragraph [0073] in conjunction with the noise-free-image objective of Paragraph [0014].
Regarding Claim 3, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches the received uncertainty indicator is determined by a segmentation algorithm, which provides the segmentation mask (Grbic, Paragraph [0023],, "Surface model 104 of the target anatomical object is generated using imaging data 102. Surface model 104 model <read on segmentation mask> may be generated using any segmentation algorithm”; [0024], "Uncertainty 106 is estimated on surface model 104"; it is noted since the uncertainty indicator is determined from that same surface model, i.e., the uncertainty indicator is determined by the segmentation algorithm that provided the surface model/segmentation mask).
As explained in the rejection of Claim 1, the obviousness for combining the uncertainty indicator teaching of Grbic into Engel is provided above.
Regarding Claim 4, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches scaling of the randomization comprises scaling a distance to a sample center as a function of the received uncertainty indicator (Engel, Paragraph [0123], “each volume sample required for the gradient computation is modified by a small noise value. A roughness parameter [0.,1.] is used for modifying or scaling the amount of noise added to each gradient computation sample") — each volume sample position <read on sample center> is offset by a noise value <read on distance to a sample center> whose magnitude is scaled by the roughness parameter <read on uncertainty indicator>, directly teaching scaling a distance to a sample center as a function of the received uncertainty indicator”; this is further confirmed by the GLSL code at [0125]: “float rand(float seed){return fract(sin(seed)*43758.5453123);} // get perturbed volume sample, perturbation is scaled using the roughness factor float getVolumeSampleG(vec3 gradPos, float roughness) { float v = texture(volumeTexture, gradPos) .x; v += ((rand(length(gradPos *100.))-.5)*roughness/15.); return v;}; where gradPos is the sample center and roughness scales the noise offset magnitude).
Regarding Claim 5, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches combining the at least one optical property of the more than one sampling points(Engel, Paragraph [0094], "comparing if accumulated <read on combining> opacity values <read on optical property> along the path are exceeding a pre-defined threshold value"; it is noted since it’s along the path, it will across more than one sampling points) wherein the rendering is based on the combined at least one optical property of the more than one sampling points (Engel, Paragraph [0007], "The color and intensity <read on optical property> of a pixel in the image plane is determined by a global radiance map... and whether the traced paths (originating from that pixel) are absorbed in the volume, whether they pass through the volume, or whether they undergo volumetric scattering or surface scattering in the volume").
\
Regarding Claim 7, the combination of Engel, Grbic and Petkov teaches the invention in Claim 1.
The combination further teaches the denoising comprises using random number generator sequences for sampling the medical imaging data (Petkov, Detailed Description, "In Monte-Carlo path tracing, a random number generator determines up to what opacity a ray is traced through the volume.") .
As explained in the rejection of Claim 1, the obviousness for combining the denoising teaching of Petkov into Engel is provided above.
Regarding Claim 8, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches the medical imaging data set is acquired by a medical scanner comprising at least one of the listed modalities (Engel, Paragraph [0013], "the present invention refers to a computer-implemented method for rendering medical volumetric images from volumetric data, received from an imaging modality or from a reconstruction or storage unit (like e.g. PACS)") — teaches medical imaging data received from an imaging modality <read on medical scanner>”);
wherein the medical scanner comprises: an X-ray device; an ultrasound device; a positron emission tomography device; a computed tomography device; a single-photon emission computed tomography device; or a magnetic resonance tomography device (Engel, Paragraph [0127], "Default values may e.g. be: 10.0 for CT data <read on computed tomography device>, 1.0 for MR data <read on magnetic resonance tomography device>”).
Regarding Claim 9, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches the at least one optical property comprises at least one of: an opacity; a reflectance; a color; and/or at least one value indicative of a chromatic scattering (Engel, Paragraph [0007], "a global transfer function assigns color and opacity values to each voxel of the volume"; [0013], "The MCPT algorithm uses at least one and usually several microfacet-based bidirectional reflectance distribution functions (also abbreviated as BRDF) for computing a probability how light is reflected at an implicit surface"; [0130], , "Classify sample (transfer function lookup yielding color and o[pacity])\rAccumulate opacity\r...\rSurface shading computation with BRDF\rChange scatter ray direction in hemisphere <read on chromatic scattering >"; it is noted the pseudocode explicitly shows color, opacity, and BRDF-based shading (reflectance) determined at each sampled point, and by Change scatter ray direction in hemisphere which is chromatic scattering and following the BRDF computation, which governs the directional distribution of light scattering across color channels ).
Regarding Claim 10, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches wherein the rendering comprises applying a reconstruction filter (Engel, Paragraph [0096], "In another preferred embodiment, the local gradient is computed by modifying central difference samples of a central difference algorithm or by modifying higher order difference samples of a higher order difference algorithm. Generally, higher-order differences provide gradients with higher quality and avoid artefacts or image failures"; it is noted the central difference and higher-order difference algorithms used for gradient computation constitute reconstruction filters applied to the volumetric data, reconstructing the continuous gradient field from discrete voxel samples. The use of higher-order differences specifically to "avoid artefacts or image failures" confirms their role as reconstruction filters improving sample quality).
Regarding Claim 11, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches the rendering comprises ray casting, Monte Carlo path tracing, subsurface scattering, and/or rendering a color mesh (Engel, Paragraph [0013], "The Method uses or executes a Monte Carlo path tracing algorithm (MCPT algorithm)" [0126], "After a scatter event has been triggered (and detected), a probability for a surface scatter event is computed based on a surface scatter probability density function PDF" [0130], pseudo code shows “Generate camera ray Intersect camera ray with bounding box of volume data Set sampling position to ray entry point of bounding box”; Pseudo code also shows "If surface scatter event is triggered …Surface shading computation with BRDF…Change scatter ray direction in hemisphere…Else…Change ray direction using volumetric phase function"; it is noted since the surface scatter event with BRDF computation and the volumetric phase function directing the ray into the scattering hemisphere teach subsurface scattering behavior).
Regarding Claim 12, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 11.
The combination further teaches wherein the received [[ uncertainty ]] indicator is mapped to at least one property of the subsurface scattering (Engel, Paragraph [0083], "The roughness parameter <read on property of subsurface scattering> may be determined by using a local grey value, segmentation or object classification information (a segmentation or classification step may be provided as a preceding step in order to provide a segmented image or classified objects within the image data) and/or other image processing parameters, including gradient magnitude and/or curvature"; [0089], "for each of the segmented or classified objects different BRDFs may be applied, which may be scaled by different roughness parameters"; [0127], "wherein sd is a surface modification factor or parameter, which may be user controlled. Default values may e.g. be: 10.0 for CT data, 1.0 for MR data. The factor or parameter defines how the surfaces are rendered by shading.").
Engel does not explicitly disclose but Grbic teaches receiving an uncertainty indicator (Grbic, [0024], "Uncertainty 106 is estimated on surface model 104. In one embodiment, uncertainty is estimated for each vertex of surface model 104 to calculate a range of acceptable boundary points using any machine learning based algorithm, such as, e.g., a probabilistic boosting tree classifier trained with image features"; [0025] "A level of uncertainty for each voxel may be visually represented on surface model 104").
As explained in the rejection of Claim 1, the obviousness for combining the uncertainty indicator teaching of Grbic into Engel is provided above.
Claim 12: taught by D1 (in combination with D2 as applied to Claim 1).
Regarding Claim 16, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches wherein the rendering comprises rendering an inhomogeneous representation of the volume and/or of the surface (Engel, Paragraph [0088], "it is possible to render the objects of a first category (of one type, e.g. implants) with a first shading and objects of a second category (e.g. bone or other body structures) in a second (different) shading"; it is noted rendering different objects with a first shading and a second (different) shading is inhomogeneous representation of the volume); comprising at least partially the segmentation mask and at least partially the anatomical structure (Engel, Paragraph [0085], "segmenting or classifying different objects or materials or tissue types <read on segmentation mask> in the received volumetric data"; [0088], "objects of a second category (e.g. bone or other body structures <read on anatomical structure>)")
Regarding Claim 17, it recites limitations similar in scope to the limitations of claim 1, but in a system. As shown in the rejection, the combination of Engel and Grbic, Chaitanya and Petkov disclose the limitations of claims 1. Additionally, Engel discloses an system that maps to Paragraph [0040] (Engel, Paragraph [0040], electrical device disclosed herein may include any number of microcontrollers, a graphics processor unit (GPU), integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variants thereof), and software which co-act with one another to perform operation (s) disclosed herein). Thus, Claim 11 is met by Engel according to the mapping presented in the rejection of claims 1, given the method corresponds to the system.
Regarding Claim 18, it recites limitations similar in scope to the limitations of claim 1 and the combination of Engel and Grbic, Chaitanya and Petkov teaches all the limitations as of Claim 1. And Engel discloses these features can be implemented on a computer-readable storage medium (Engel, Paragraph [0022], “the invention relates to a computer-readable medium on which program elements are stored that can be read and executed by a computer, in order to perform steps of the method for rendering medical volumetric images according to the method of an embodiment”).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Engel (US 20210090325 A1) in view of Grbic et al. (US 20160267673 A1, hereinafter Grbic), further in view of Chaitanya et al. ("Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder," ACM Transactions on Graphics, 2017, hereinafter "Chaitanya"), and Petkov (US 10311631 B2) as applied to Claim 1 above and further in view of Tan et al. (US 20230386022 A1, hereinafter Tan)
Regarding Claim 2, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination further teaches the received [[ uncertainty ]] indicator is based on a noise measurement within a local neighborhood within the segmentation mask (Chaitanya, page 98:4, “Our image reconstruction algorithm is a data-driven method that learns a mapping from noisy input image sequences to noise-free output image sequences based on a large number of training pairs”; “architectures that give good results in optical -ow estimation and image segmentation, respectively … The receptive eld of all the deeper layers is also several times larger in the input image, allowing us to consider larger pixel neighborhoods and therefore better deal with very sparse inputs).
The combination does not explicitly disclose but Tan teaches the received uncertainty indicator is based on a noise measurement (Tan, Paragraph [0047], "The quality information can also include one or more noise measures that represent levels of noise in the image segmentations"); within a local neighborhood within the segmentation mask and/or across segmentation masks (Tan, Paragraph [0043], "The CT segmentation data 204 provides segmentation data in the form of a segmentation mask over the entirety of the brain as depicted in the CT image data 203 … The MR segmentation data 202 provides segmentation data in the form of segmentation boundary marks or contours around various defined anatomical objects"; it is noted two distinct segmentation masks (MR and CT) of the same anatomical region teach within the segmentation mask and across segmentation masks), and/or within the anatomical structure and/or across anatomical structures [[ and/or across anatomical structures ]] (Tan, Paragraph [0043], "segmentation boundary marks or contours around various defined anatomical objects (e.g., tissues, vessels, regions of interest (ROIs))"; [0064], "each of the different images may include multiple different organ/object segmentations").
Tan and Engel are analogous since both of them are directed to rendering and processing of medical volumetric image data comprising segmented anatomical structures. Engel provided a way of rendering medical volumetric images using an uncertainty indicator derived from a segmentation algorithm. Tan provided a way of computing noise measures within and across segmentation masks and anatomical structures as part of the uncertainty evaluation of multimodal image segmentations. Therefore, it would have been obvious to one of ordinary skill in the art to incorporate the noise-measurement-based uncertainty indicator within and across segmentation masks and anatomical structures taught by Tan into modified invention of Engel such that the received uncertainty indicator is based on a noise measurement within a local neighbourhood within the segmentation mask and/or across segmentation masks, and/or within the anatomical structure and/or across anatomical structures. The motivation is to improve the quality and reliability of the uncertainty indicator used to modulate the rendering, by grounding it in measurable noise characteristics of the segmentation data.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Engel (US 20210090325 A1) in view of Grbic et al. (US 20160267673 A1, hereinafter Grbic), further in view of Chaitanya et al. ("Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder," ACM Transactions on Graphics, 2017, hereinafter "Chaitanya"), and Petkov (US 10311631 B2) as applied to Claim 1 above and further in view of Kim (US 9589387 B2).
Regarding Claim 13, the combination of Engel and Grbic, Chaitanya and Petkov teaches the invention in Claim 1.
The combination does not explicitly disclose but Kim teaches wherein a subsurface attenuation of one or more wavelength bands (Kim, Column 13, Line 12-20, "the final image producer 272f may decide attenuation constants that are applied to color values of each pixel of the projection image, according to a distance between a first surface point and a second surface point corresponding to the pixel of the projection image, and applies the attenuation constants to the color values of the pixel of the projection image, thereby producing a final image"); comprises the at least one property and is modulated according to the received uncertainty indicator (Kim, Column 13, Line 25-30, “the attenuation constants that are applied to the R, G, and B values of each pixel may be decided based on attenuation constant graphs for R, G, and B channels <read on one or more wavelength bands>"; Column 1, Line 55-60, "apply a subsurface scattering effect according to virtual lighting information, to the projection image with respect to a user's viewpoint").
Kim and Engel are analogous since all are directed to photorealistic rendering of medical volumetric/surface data using physically-based light interaction models including surface scattering. Engel provided a way of rendering medical volumetric images with BRDF-based surface scattering modulated by a roughness/uncertainty parameter. Kim provided a way of applying subsurface scattering effects with wavelength-band attenuation constants modulated by rendering parameters.
Therefore, it would have been obvious to one of ordinary skill in the art to incorporate wavelength-band attenuation constants modulated according to the uncertainty indicator taught by Kim into the modified invention of Engel such that subsurface attenuation of one or more wavelength bands is modulated according to the received uncertainty indicator. The motivation is to produce a physically accurate, photorealistic rendering of anatomical structures where different tissue types exhibit different wavelength-dependent light attenuation properties, improving diagnostic differentiation.
Response to Arguments
Applicant’s arguments with respect to claim 1, 17, 18, filed on 8/6/2026, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. it has now been taught by the combination of Engel and Grbic, Chaitanya and Petkov.
In regard to Claims 2-5, 7-13, 16, they directly/indirectly depends on independent Claim 1. Applicant does not argue anything other than the independent claim 1. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US8817021B1 System for writing, interpreting, and translating three-dimensional (3D) scenes
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights