DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
2. The information disclosure statements (IDS) submitted on the following dates are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner: 01/28/2025.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
4. Claims 1-7 and 11-19 are rejected under 35 U.S.C. 103 as being unpatentable over Bruckner et al., (“Bruckner”) “Efficient Volume Visualization of Large Medical Datasets”
Regarding claim 1, Bruckner discloses a computer-implemented method for volume rendering a set of medical images of a patient (Bruckner- page 31, section CHAPTER 3, VOLUME RENDERING OF LARGE DATASETS), the method comprising:
obtaining the set of medical images defining a voxel grid comprising voxels, the voxels being each associated to a value (Bruckner- Figs. 2.4 and 2.5 show Volume rendering using 2D textures; page 17, section 3.1.1 System Requirements, at least discloses The system will be primarily used to visualize medical datasets of anatomic nature acquired by CT OF MRI; page 19, 1st paragraph, at least discloses The system will support voxels of up to 12 bits, as this is the common format for medical datasets; page 28, section 3.2.2 Classification, at least discloses Since 12 bit voxels are standard for medical datasets [set of medical images defining a voxel grid comprising voxels] [Wingdings font/0xE0] refers to 3D pixels in volumetric datasets where each voxel holds 12 bits of data);
segmenting the voxel grid defined by the set of medical images by associating to each voxel a respective label among a set of labels each corresponding to a respective region of interest of the patient (Bruckner- Fig. 3.4 show segmented dataset. Four objects have been segmented: skullcap, blood vessels, brain, and background as region of interest of the patient; page 28, section 3.2.2 Classification, at least discloses we support segmentation by assigning an object index [a respective label] to each voxel. For every object, an independent transfer function can be defined. Since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information, as the data has to be aligned to byte boundaries for better performance); and
modifying values of the voxel grid by adding to each value to be modified an offset value which depends on the respective label associated to the voxel (Bruckner- Fig. 3.4 showing a dataset of a patient's head, volume-rendered in the described manner, which shows the different parts of skull, brain tissue and vessels visualized with different transfer functions of colour and opacity; page 19, 1st paragraph, at least discloses The system will support voxels of up to 12 bits, as this is the common format for medical datasets. Due to performance issues, the voxels have to be aligned to byte boundaries, which leaves 4 bits. These 4 bits will be used to store segmentation information, which allows up to 16 different objects; page 28, section 3.2.2 Classification, 2nd paragraph, at least discloses For every object, an independent transfer function can be defined. Since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information […] This allows up to 16 objects to be defined. A separate transfer function can be assigned to each of these objects and individual objects can be enabled or disabled (see Figure 3.4).
Bruckner does not directly teach adding to each value to be modified an offset value. However, Bruckner teaches since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information […] This allows up to 16 objects to be defined (pages 28-29). It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the use of the four most significant bits of the voxel intensity value for modifying the voxel-intensity of one of the 16 segment labels amounts to the addition of a label-dependent offset value because doing so would provide an approach for interactive high-quality rendering of large medical data. In addition, it suggests that the offset value equals n multiplied by 213, where n=0 to 15 designates a particular label number.
Regarding claim 2, Bruckner discloses the computer-implemented method of claim 1, and further discloses wherein the offset value added to each value is equal to a result of a multiplication of the respective label associated to the voxel by a predetermined factor (Bruckner- page 28, section 3.2.2 Classification, 2nd paragraph, at least discloses For every object, an independent transfer function can be defined. Since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information […] This allows up to 16 objects to be defined. A separate transfer function can be assigned to each of these objects and individual objects can be enabled or disabled (see Figure 3.4) [Wingdings font/0xE0] It suggests implicitly that the offset value equals n multiplied by 213, where n"'0 to 15 designates a particular label number.
Bruckner does not directly teach the offset value added to each value is equal to a result of a multiplication of the respective label associated to the voxel by a predetermined factor. However, Bruckner teaches since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information […] This allows up to 16 objects to be defined (pages 28-29). It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the use of the four most significant bits of the voxel intensity value for modifying the voxel-intensity of one of the 16 segment labels amounts to the addition of a label-dependent offset value because doing so would provide an approach for interactive high-quality rendering of large medical data.
Regarding claim 3, Bruckner discloses the computer-implemented method of claim 1, and further discloses wherein the modifying includes computing an offset voxel grid Vm based on a following formula:
Vᵐ = V + tS
wherein V is the voxel grid defined by the set of medical images, t is a predetermined factor and S is the segmented voxel grid, and wherein the method further comprises transmitting the computed offset voxel grid to a viewer (Bruckner- Fig. 3.4 show segmented dataset. Four objects have been segmented: skullcap, blood vessels, brain, and background as region of interest of the patient; page 28, section 3.2.2 Classification, at least discloses we support segmentation by assigning an object index to each voxel [segmented voxel grid]. For every object, an independent transfer function can be defined. Since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information, as the data has to be aligned to byte boundaries for better performance […] This allows up to 16 objects to be defined. A separate transfer function can be assigned to each of these objects and individual objects can be enabled or disabled (see Figure 3.4) [Wingdings font/0xE0] It suggests implicitly that the offset value equals n multiplied by 213, where n=0 to 15 designates a particular label number).
Bruckner does not directly disclose the computed offset voxel grid. However, Bruckner discloses since 12 bit voxels are standard for medical datasets, the remaining 4 bits can be used for the segmentation information […] This allows up to 16 objects to be defined (pages 28-29). It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the use of the four most significant bits of the voxel intensity value for modifying the voxel-intensity of one of the 16 segment labels amounts to the addition of a label-dependent offset value because doing so would provide an approach for interactive high-quality rendering of large medical data.
Regarding claim 4, Bruckner discloses the computer-implemented method of claim 1, and further discloses wherein each segment is associated to a respective transfer function, the respective transfer function outputting, for each voxel of the segment to which the function is associated , a respective value for a set of at least two appearance parameters including a color parameter and an opacity parameter (Bruckner- page 28, section 3.2.2 Classification, 1st - 2nd paragraphs, at least discloses Classification is the process of assigning a color and opacity [two appearance parameters] to a reconstructed function value: Transfer functions, usually implemented as lookup tables, are used for this mapping. During rendering, the reconstructed function value serves as an index for the lookup tables, which contain color and opacity values. Levoy first suggested the use of one-dimensional piecewise linear transfer functions […] Multi-dimensional transfer functions are a more general approach which has proven to provide more control over the appearance of the rendering […] We therefore support one dimensional transfer functions including optional opacity modulation based on the gradient magnitude. Additionally, we support segmentation by assigning an object index to each voxel. For every object, an independent transfer function can be defined), the set of at least two appearance parameters including at least one additional appearance parameter such as a metalness parameter and/or a rugosity parameter (Bruckner- Fig. 3.6 shows Visual comparison of specular highlights [metalness parameter] obtained with Phong's and Schlick's approach. (a) Phong. (b) Schlick. The specular exponent is 16 in both cases; page 31, 2nd paragraph, at least discloses The diffuse term (Equation 3.9) is based on Lambert's cosine law which states that the reflection of & perfect rough surface [rugosity parameter] is proportional to the cosine of the angle a between the light vector L and the surface normal N).
Regarding claim 5, Bruckner discloses the computer-implemented method of claim 4, and further discloses wherein one or more of the transfer functions are predetermined (Bruckner- page 28, section 3.2.2 Classification, 2nd paragraph, at least discloses We therefore support one dimensional transfer functions including optional opacity modulation based on the gradient magnitude. Additionally, we support segmentation by assigning an object index to each voxel. For every object, an independent transfer function can be defined).
Regarding claim 6, Bruckner discloses the computer-implemented method of claim 4, and further discloses wherein one or more of the transfer functions are user-defined (Bruckner- page 28, section 3.2.2 Classification, 2nd paragraph, at least discloses We therefore support one dimensional transfer functions including optional opacity modulation based on the gradient magnitude. Additionally, we support segmentation by assigning an object index to each voxel. For every object, an independent transfer function can be defined).
Regarding claim 7, Bruckner discloses the computer-implemented method of claim 4, and discloses the method further comprising:
combining the respective transfer functions in a single piecewise transfer function (Bruckner- page 28, section 3.2.2 Classification, 1st paragraph, at least discloses Transfer functions, usually implemented as lookup tables, are used for this mapping. During rendering, the reconstructed function value serves as an index for the lookup tables, which contain color and opacity values. Levoy first suggested the use of one-dimensional piecewise linear transfer functions […] Multi-dimensional transfer functions are a more general approach which has proven to provide more control over the appearance of the rendering); and
transmitting the single piecewise transfer function to a viewer (Bruckner- page 28, section 3.2.2 Classification, 1st paragraph, at least discloses Transfer functions, usually implemented as lookup tables, are used for this mapping. During rendering, the reconstructed function value serves as an index for the lookup tables, which contain color and opacity values. Levoy first suggested the use of one-dimensional piecewise linear transfer functions […] This allows up to 16 objects to be defined. A separate transfer function can be assigned to each of these objects and individual objects can be enabled or disabled (see Figure 3.4); page 55, section 3.5.2 Entry Point Buffer, 2nd paragraph, at least discloses We store the minimum and maximum value of the samples contained in a block and use a summed area table of the opacity transfer function to determine the visibility of the block [19]).
Regarding claim 11, Bruckner discloses the computer-implemented method of claim 1, and further discloses wherein the set of medical images has been produced by a CT scanner or an MRI scanner (Bruckner- page 79, section 5.5 Visualization Results, at least discloses visualization results for clinical datasets in Figures 5.8. The images show anatomic features and/or pathologies. Fig 5.8 shows CT scan of colon. Bones and colon are displayed in the top image. The bottom image shows the colon without bones).
Regarding claim 12, Bruckner discloses a non-transitory computer readable storage medium having recorded thereon a computer program having instructions for performing a method for volume rendering a set of medical images of a patient (Bruckner- page 4, CHAPTER 1. INTRODUCTION, at least discloses A high-quality volume visualization system has been developed which is capable of interactively handling large datasets on commodity hardware […] In Chapter 3, we propose several techniques to enable the handling of large datasets, which are combined in a single framework to form a high-performance volume rendering algorithm. We discuss an alternative storage scheme that can significantly improve the cache behavior of a volume rendering algorithm. Furthermore, we present parallelization strategies which are well suited for commodity hardware, and introduce memory efficient acceleration data structures), the method comprising the method of claim 1.
Regarding claims 13-15, all claim limitations are set forth as claims 2-4 in non-transitory computer readable storage medium having recorded thereon a computer program having instructions and rejected as per discussion for claims 2-4.
Regarding claim 16, Bruckner discloses a computer system comprising:
a processor coupled to a memory, the memory having recorded thereon a computer program having instructions for volume rendering a set of medical images of a patient (Bruckner- Figure 3.13: Comparison of conventional CPU and Hyper-Threading CPU. (a) conventional CPU with single architectural state. (b) Hyper-Threading CPU with duplicated architectural state, one for each logical processor; page 4, CHAPTER 1. INTRODUCTION, at least discloses A high-quality volume visualization system has been developed which is capable of interactively handling large datasets on commodity hardware […] In Chapter 3, we propose several techniques to enable the handling of large datasets, which are combined in a single framework to form a high-performance volume rendering algorithm. We discuss an alternative storage scheme that can significantly improve the cache behavior of a volume rendering algorithm. Furthermore, we present parallelization strategies which are well suited for commodity hardware, and introduce memory efficient acceleration data structures; page 17, section 3.1.1 System Requirements, at least discloses Medical Datasets The system will be primarily used to visualize medical datasets of anatomic nature acquired by CT or MRI […] The system has to be capable of handling large datasets D 512*) on commodity hardware. In general, a medical visualization system consists of several modules; page 46, section 3.4.1 Symmetric Multiprocessing, at least discloses Architectures using multiple processors within the same computer are referred to as Symmetric Multiprocessing systems. Multiprocessor architectures improve overall performance by allowing threads to execute in parallel; page 92, section 6.3.2 Simultaneous Multithreading, at least discloses Executing two threads simultaneously on one processor has the advantage of more independent instructions being, available, thus increasing CPU utilizations. This can be achieved by duplicating state registers, which only leads to little increases in manufacturing costs. Intel's SMT implementation is called Hyper-Threading and was first available on the Pentium 4 CPU. Currently, two logical CPUs per physical CPU are supported) that when executed by the processor causes the processor to be configured to perform the method of claim 1.
The system of claims 17-19 are similar in scope to the functions performed by the method of claims 2-4 and therefore claims 17-19 are rejected under the same rationale.
5. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Bruckner in view of Kaemmerer et al., (“Kaemmerer”) [US-2016/0250476-A1]
Regarding claim 9, Bruckner discloses the computer-implemented method of claim 2, and does not directly teach wherein the predetermined factor is higher than an amplitude of the values of the voxel grid (Kaemmerer- ¶0099, at least discloses an initial amplitude value (102) to be less than the predetermined maximum. In some examples, the predetermined maximum amplitude value is 10.0 volts […] clinician may select the predetermined maximum value to be the amplitude value (or other stimulation parameter value or combination of values) at which the stimulation intensity is at a maximum desired intensity for patient 12 or a group of patients; ¶0147, at least discloses processor 80 may decrease the volume of the VTA by the scaling factor for patients that have tissue having a higher impedance than the tissue impedance indicated by the general electrical field mode).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the teachings of Kaemmerer, and apply the amplitude value into the Bruckner’s teachings in order the predetermined factor is higher than an amplitude of the values of the voxel grid.
Doing so would adjust electrical stimulation therapy delivered to a patient by a medical device based on the values of the voxels.
6. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Bruckner in view of Takemura et al. (machine translation of JP-2016016250-A with citation below, hereinafter “Takemura”)
Regarding claim 10, Bruckner discloses the computer-implemented method of claim 1, and further discloses wherein the regions of interest of the patient are selected among:
one or more organs such as brain, heart, lungs, liver, pancreas, kidneys, skin and/or gut (Bruckner- Fig. 3.4 showing a dataset of a patient's head, volume-rendered in the described manner, which shows the different parts of skull, brain tissue);
one or more vessels such as blood vessels (Bruckner- Fig. 3.4 showing blood vessels)
.
Bruckner does not directly discloses one or more tissues such as epithelial tissues, connective tissues (e.g., adipose tissues), muscular tissues and/or nerve tissues; and/or one or more pathological forms such as tumors.
However, Takemura discloses
one or more tissues such as epithelial tissues, connective tissues (e.g., adipose tissues), muscular tissues and/or nerve tissues (Takemura- ¶0015, at least discloses The medical image processing system 1 reproduces the target area (hereinafter referred to as the "target area") as a three-dimensional image in order to allow medical professionals such as doctors to understand the location and structure of tissues (e.g., bones and nerves) in the area to be treated (hereinafter referred to as the "target area") as accurately as possible before the medical procedure.); and/or
one or more pathological forms such as tumors (Takemura- ¶0068, at least discloses In the above embodiments, bone and nerves were given as examples of tissues in the target area, but the present invention is not limited thereto. For example, it could be a blood vessel, portal vein, organ, tumor, etc.).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the teachings of Takemura, and apply the nerves tissues and tumor into the Bruckner’s teachings in order one or more tissues such as epithelial tissues, connective tissues (e.g., adipose tissues), muscular tissues and/or nerve tissues; and/or one or more pathological forms such as tumors.
Doing so would provide a medical image processing program that are effective in more accurately reproducing the positional relationship of different tissues in a predetermined target area in an image.
7. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Bruckner in view of Ljung Patric et al., (“Ljung”) “State of the Art in Transfer Functions for Direct Volume Rendering”
Regarding claim 20, Bruckner discloses computer system of claim 16, and discloses the computer system further comprising a viewerconfigured for displaying the volume rendering of the set of medical images (Bruckner- page 70, section 4.1.5 Viewers, at least discloses A viewer manages a set of renderers and manipulators. The viewer treats each renderer as a separate layer. The output images of all renderers are collected and composed into a final image for display. Events are passed on to the corresponding renderer or manipulator. The viewer base class defines interfaces for communication with renderers and manipulators).
Bruckner does not directly disclose, but Ljung discloses
the viewer having a graphical user interface configured for displaying the volume rendering (Ljung- page 683, section 8. User Interfaces, at least discloses Interactive volume rendering is a powerful tool for visual exploration of volumetric data. Interaction and exploration are thus key aspects that converge in the user interface for the TF […] in this section, we discuss the user interface aspects of TFsand provide an overview of what has been accomplished so far).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Bruckner to incorporate the teachings of Ljung, and apply the user interface in interactive volume rendering into the Bruckner’s teachings in order the viewer having a graphical user interface configured for displaying the volume rendering of the set of medical images.
Doing so would provide expertise to determine the most suitable TF for specific data and/or application.
Allowable Subject Matter
8. Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
9. The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 8, the combination of prior arts teaches the method of Claim 1.
However in the context of claim 1, 4, 7 and 8 as a whole, the combination of prior arts does
not teach the single piecewise transfer function is in a form:
PNG
media_image1.png
33
757
media_image1.png
Greyscale
is the single piecewise transfer function, fi is the respective transfer function associated to segment i, vₘᵢₙ_maxe respectively the minimum and maximum values of the voxel grid and t is a predetermined factor.
Therefore, Claim 8 in the context of claim 1, 4, 7 as a whole does comprise allowable subject matter.
Conclusion
10. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. They are as recited in the attached PTO-892 form.
11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LE whose telephone number is (571)272-5330. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Primary Examiner, Art Unit 2614
Prosecution Insights