DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-5 and 7-22 remain pending in the application.
Response to Arguments
The applicant argues that Shekhar does not teach remote patch storage and fails to disclose excluding patch resources already loaded for the requested plane. Those arguments, see Pages 5-7 of Remarks, filed July 13, 2026, with respect to the rejection(s) of claim(s) 1-5 and 7-13 under 35 USC 103 have been fully considered and are persuasive. The rejection of claims 1-5 and 7-13 of March 11, 2026 has been withdrawn.
Regarding claims 14-22, Applicant's arguments filed July 13, 2026, have been fully considered but they are not persuasive.
First, the applicant argues that Shekhar does not teach remote patch storage and fails to disclose excluding patch resources already loaded for the requested plane. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., remote storage of patches and excluding previously loaded patches) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Claim 14 does not recite the patches being remotely stored nor that previously loaded patches are excluded.
Regarding applicant’s argument that Shekhar does not teach non-overlapping patches, Shekhar teaches both overlapping patches (Paragraph 1 in 2nd Col. of Page 2 – “bricks overlap by a single layer of voxels”) and non-overlapping patches (see Fig. 1 of Shekhar). However, the configuration used in the rejection of claim 14 discloses non-overlapping patches. See the rejection of claim 14 below.
In response to applicant's argument that nothing in Pyo suggests a modification, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). One of ordinary skill in the art reading both Pyo and Shekhar would understand that there are ways in which the generation of MPR can be improved. For instance, Shekhar discloses: “the entire volume is rarely required for most visualization tasks. This is especially true for the cine MPR, in which only those voxels either immediately in front of or behind the cutting plane are needed. Volume subdivision provides the “granularity” to reject unnecessary data, thus lowering the data requirement (see Fig. 1) and consequently improving the performance” (Shekhar: Paragraph 2 in 2nd Col. of Page 2). Therefore, if Pyo and Shekhar were combined, one of ordinary skill in the art would have had a motivation to improve Pyo’s standard MPR generation method, such as by using the volume subdivision (patch storage) in Shekhar.
Finally, in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine Pyo and Shekhar to cause a skilled artisan to retrieve only patch resources intersecting the requested MPR plane or target region, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, there is a motivation suggested in Shekhar that would cause one of ordinary skill in the art to retrieve only patches intersecting the requested MPR plane or target region: “the entire volume is rarely required for most visualization tasks. This is especially true for the cine MPR, in which only those voxels either immediately in front of or behind the cutting plane are needed. Volume subdivision provides the “granularity” to reject unnecessary data, thus lowering the data requirement (see Fig. 1) and consequently improving the performance” (Shekhar: Paragraph 2 in 2nd Col. of Page 2). In other words, the MPR system would be more efficient if only the relevant data is loaded for MPR generation, rather than loading all data or unnecessary data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 14 is rejected under 35 U.S.C. 103 as being unpatentable over Pyo et al. (US 2004070584 A1) in view of Shekhar et al. (Cine MPR: Interactive Multiplanar Reformatting of Four-Dimensional Cardiac Data Using Hardware-Accelerated Texture Mapping), hereinafter Pyo and Shekhar respectively.
Regarding claim 14, Pyo teaches a method for generating a patch-loaded Multi-Planar Reconstruction (MPR) (Paragraph 0011 – “a three-dimensional multi-planar image reconstruction method, which is to display a multi-planar image of a region of interest in a reference image”), said method comprising the steps of:
loading stored volumetric data corresponding to a targeted region of interest (Paragraph 0044 – “The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: the identified volume image data is loaded) requested by a user for display in at least one of a plane (Paragraph 0042, 0055 – “the reference image processor 210 processes the volume data stored in the input/storing section 100 to display the three-dimensional reference image from the volume data, and receives a region of interest entered by the user via the input section 400 in the form of straight line, curve, or free-formed curve data… it is checked in step 110 whether or not the user selects the basic MPR. If the basic MPR is chosen, the respective points of the straight line presenting a selected plane are sampled and arrange”; Note: the user indicates a region of interest and selects a plane for an MPR) of the generated MPR (Paragraph 0015, 0044 – “when the shape of the displayed section is in a basic multi-planar image mode, sampling sample points at intervals of unit length from a straight line representing a plane selected by the user…The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: an MPR is generated for the region of interest using the loaded volumetric data).
Pyo does not teach parsing a volumetric image into a plurality of non-overlapping patches with recordation of 3-D coordinates for storage, nor the “stored patch” in the limitation: “loading the stored patch corresponding to a targeted region of interest requested by a user for display in at least one of a plane of the generated load-patched MPR”. However, Shekhar teaches parsing a volumetric image into a plurality of non-overlapping patches (Fig. 1, Paragraph 5 in 1st Col. of Page 2, Paragraph 1 in 2nd Col. of Page 2 – “Volume subdivision divides a 3-D image into smaller 3-D bricks of equal size. For spatiotemporal 4-D images, each frame is subdivided individually and identically”; Note: the volumetric image is parsed/divided into bricks, which are equivalent to patches. Fig. 1 shows the non-overlapping patches/bricks; see screenshot of Fig. 1 below) with recordation of 3-D coordinates for storage (Paragraph 3-5 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… When a user interacts and changes the orientation of the reformatted plane(s), the calculation of intersected bricks, polygons vertices, and texture coordinates is repeated… All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick. The texture memory thus acts like a cache, whose initialization and updating directly affects the performance”; Note: the bricks/patches are stored. It would be obvious to one of ordinary skill in the art that the bricks are stored along with their coordinates because location is an integral part of defining the bricks and retrieving them efficiently later on) and loading the stored patch corresponding to a targeted region of interest requested by a user for display in at least one of a plane of the generated load-patched MPR (Paragraph 3, 5 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence”; Note: the needed/identified bricks are loaded from storage and are rendered for display of the MPR).
PNG
media_image1.png
383
630
media_image1.png
Greyscale
Screenshot of Fig. 1 (taken from Shekhar)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Shekhar to parse the volumetric image into patches because “the entire volume is rarely required for most visualization tasks. This is especially true for the cine MPR, in which only those voxels either immediately in front of or behind the cutting plane are needed. Volume subdivision provides the “granularity” to reject unnecessary data, thus lowering the data requirement (see Fig. 1) and consequently improving the performance” (Shekhar: Paragraph 2 in 2nd Col. of Page 2). It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Shekhar to store the patches with 3D coordinates for the benefit of being able to efficiently identify and retrieve patches for future use. Additionally, in general, a person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the stored volumetric data of Pyo could have been substituted for the stored patches/bricks of Shekhar because both the stored volumetric data and stored patches serve the purpose of being retrieved to generate an MPR. Furthermore, a person of ordinary skill in the art would have been able to carry out the substitution. Finally, the substitution achieves the predictable result of being loaded from storage for an MPR. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the stored volumetric data of Pyo for the stored patches of Shekhar according to known methods to yield the predictable result of being loaded from storage for an MPR.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar, Padmanabhan (US 20130036193 A1), and Lai et al. (Improving Web Browsing on Wireless PDAs Using Thin-Client Computing), hereinafter Padmanabhan and Lai respectively.
Regarding claim 15, Pyo in view of Shekhar teaches the method of claim 14. Pyo does not teach wherein the loading is performed by sending HTTP requests to load patches stored as individual resources identified by URI into thin-client browser. However, Padmanabhan teaches wherein the loading is performed by sending HTTP requests to load patches stored as individual resources identified by URI (Paragraph 0046-0047 – “the dynamic image sprite is transmitted to the client device to enable the web browser to efficiently render the web page…generating the dynamic web sprite may include the web service generating Hypertext Transfer Protocol (HTTP) requests to at least one image hosting server to retrieve a plurality of images corresponding to the plurality of image URIs”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Padmanabhan for the benefit of enabling “the web browser to efficiently render the web page by reducing the number of HTTP requests issued by the web browser to retrieve images” (Padmanabhan: Paragraph 0046). Furthermore, Pyo modified by Padmanabhan does not teach a thin-client browser. However, Lai teaches a thin-client browser (Fig. 2 – See screenshot 4 below). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Lai because “thin clients provide better web browsing performance than fat clients across a wide variety of web content, including general consumer content, medical imaging content, and text-based clinical information content widely used in a major academic medical center” (Lai: Page 153, Paragraph 2 of Conclusions and Future Work).
PNG
media_image2.png
212
756
media_image2.png
Greyscale
Screenshot 4 (taken from Fig. 2 of Lai)
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar, Padmanabhan, and Robertson (Oracle Application Server Concepts), hereinafter Robertson.
Regarding claim 16, Pyo in view of Shekhar teaches the method of claim 14. Pyo does not teach receiving, by a middle tier component that includes a Web server and a wireless application server, inbound HTTP requests and executing appropriate logic in response to the requests. However, Padmanabhan teaches receiving, by a middle tier component that includes a Web server and an application server, inbound HTTP requests and executing appropriate logic in response to the requests (Fig. 1 122, 120; Paragraph 0013-0014, 0047 – “an application program interface (API) server 118 and a web server 120 are coupled to, and provide programmatic and web interfaces respectively to, one or more application servers 122…the web server 120 may send and receive data to and from a toolbar or webpage on a browser application (e.g., web client 110) operating on a client machine (e.g., client machine 106). The API server 118 may send and receive data to and from an application (e.g., client application 112 or third party application 116) running on another client machine (e.g., client machine 108 or third party server 114)…generating the dynamic web sprite may include the web service generating Hypertext Transfer Protocol (HTTP) requests to at least one image hosting server to retrieve a plurality of images corresponding to the plurality of image URIs”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Padmanabhan for the benefit of handling HTTP requests more efficiently and thus enhancing the user experience (Padmanabhan: Paragraph 0003). Furthermore, Pyo modified by Padmanabhan does not teach a wireless application server. However, Robertson teaches a wireless application server (Page 1 of Chapter 4 – “A component of Oracle Application Server, Oracle Application Server Wireless enables enterprises and service providers to efficiently build, manage, and maintain wireless and voice applications”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Robertson to have wireless application servers for the benefit of making applications more accessible to other devices (Robertson: Page 1 of Chapter 4).
Regarding claim 17, Pyo in view of Shekhar, Padmanabhan, and Robertson teaches the method of claim 16. Pyo does not teach wherein the execution includes session management, content management, and integration to a back-end system. However, Robertson teaches wherein the execution includes session management (Page 21 of Chapter 2 – “The session manager configures and manages the session as a singleton within the application”; Note: the Oracle application server has a session manager to manage sessions), content management (Page 35 of Chapter 2 – “Oracle Application Server includes the Oracle Content Management Software Development Kit”), and integration to a back-end system (Page 2 of Chapter 6 – “Oracle provides a suite of integration adapters that implement bi-directional connectivity between applications and various back-end systems”; Note: Oracle’s application server provides integration features to connect the front and back end). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Robertson for the benefit of providing support for the application (Robertson: Page 20 of Chapter 2) and enabling fast and flexible integration (Robertson: Page 2 of Chapter 6).
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar and Azad et al. (Combining Appearance-based and Model-based Methods for Real-Time Object Recognition and 6D Localization), hereinafter Azad.
Regarding claim 18, Pyo in view of Shekhar teaches the method of claim 14. Pyo does not teach wherein positioning of targeted regions is based on localized objects in each patch produced by an AI segmentation model. However, Azad teaches wherein positioning of targeted regions (Fig. 5, Page 5342 – “Before a segmented region can be used as input for appearance-based calculations it has to be transformed into a normalized representation…the region has to be normalized in size. This is done by resizing the region to a squared window of 64 × 64 pixels”) is based on localized objects in each patch produced by an AI segmentation model (Fig. 4, Page 5343-5344 – “Segmented objects whose edges are visible to a sufficient extent are recognized and localized with a rate of 100%”; Note: Color segmentation model assists in localizing the objects). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Azad for the benefit of “efficient acquisition of object representations and real-time recognition and localization” (Azad: Page 5339).
Regarding claim 19, Pyo in view of Shekhar and Azad teaches the method of claim 18. Pyo does not teach using spatial information of objects in the target region of interest in any of the stored patches to generate the MPR. However, Azad teaches using spatial information of objects in the target region of interest in any of the stored patches to generate the MPR (Fig. 10 – A 3D visualization of the localized objects is shown). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Azad for the benefit of further assisting the user in learning more about the objects within the region of interest.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar, Azad, Chen, and MedDream (https://www.youtube.com/watch?v=yGcYFT8hMlA) hereinafter MedDream.
Regarding claim 20, Pyo in view of Shekhar and Azad teaches the method of claim 19. Pyo does not teach wherein the spatial information includes a panoramic reformat of a CBCT, wherein each object is localized on the reformat and selecting the object opens an MPR in the region of interest. However, Chen teaches wherein the spatial information includes a panoramic reformat of a CBCT (Col. 3 lines 34-38, Fig. 5 and 6A), wherein each object is localized on the reformat (Col. 3 lines 37-38, Fig. 6A – The object of interest is localized by a bounding box). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Chen for the benefit of providing a fuller, unfolded view of teeth with reduced distortion (Chen: Col. 5 lines 24-47). Additionally, localizing the teeth with bound boxes “help to define starting points for algorithmic techniques that detect gaps indicating edges between teeth” (Chen: Col. 8 lines 46-58). Furthermore, Pyo modified by Chen does not teach selecting the object opens an MPR in the region of interest. However, MedDream teaches selecting the object opens an MPR in the region of interest (Screenshots 5 and 6 below – Teaches selecting MPR button to open the MPR). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of MedDream to use a selection feature to open the MPR. Doing so would make the interface more user-friendly by increasing accessibility to an MPR view.
PNG
media_image3.png
1199
1919
media_image3.png
Greyscale
Screenshot 5 (taken from https://www.youtube.com/watch?v=yGcYFT8hMlA)
PNG
media_image4.png
1199
1920
media_image4.png
Greyscale
Screenshot 6 (taken from https://www.youtube.com/watch?v=yGcYFT8hMlA)
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar, Azad, and MedDream.
Regarding claim 21, Pyo in view of Shekhar and Azad teaches the method of claim 19. Pyo does not teach wherein the spatial information is a 3D model of the object localized and selecting the object opens an MPR in the region of interest. However, Azad teaches wherein the spatial information is a 3D model of the object localized (Fig. 10 – A 3D visualization of the localized objects is shown). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Azad for the benefit of further assisting the user in learning more about the objects within the region of interest. Furthermore, Pyo modified by Azad does not teach selecting the object opens an MPR in the region of interest. However, MedDream teaches selecting the object opens an MPR in the region of interest (Screenshots 5 and 6 above – Teaches selecting MPR button to open the MPR). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of MedDream to use a selection feature to open the MPR. Doing so would make the interface more user-friendly by increasing accessibility to an MPR view.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Pyo in view of Shekhar, Azad, and Chen.
Regarding claim 22, Pyo in view of Shekhar and Azad teaches the method of claim 19. Pyo does not teach wherein the object is at least one of a tooth, a landmark, or maxillofacial organ. However, Chen teaches wherein the object is at least one of a tooth, a landmark, or maxillofacial organ (Fig. 2 – The objects are teeth). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pyo to incorporate the teachings of Chen for the benefit of providing “a better solution for teeth position identification in a three dimensional dental image volume” (Chen: Col. 2 lines 43-45).
Allowable Subject Matter
Claims 1-5 and 7-13 are allowed.
Claim 1 would be allowable for disclosing:
storing the patches remotely with recordation of three-dimensional (3-D) spatial coordinates, defining bounds of the patch;
computing volumetric coordinates for pixels on the requested view, based on the recorded 3-D spatial coordinates of the stored patches;
loading only the identified patches from the remote storage, excluding patches previously loaded for the user-selected plane.
Regarding claim 1,
Pyo teaches a method for generating a Multi-planar Reconstruction (MPR) of a targeted region of a volumetric image (Paragraph 0011 – “a three-dimensional multi-planar image reconstruction method, which is to display a multi-planar image of a region of interest in a reference image”), comprising: receiving the volumetric image (Paragraph 0040, 0042 – “The input/storing section 100 externally receives volume data containing density values of a three-dimensional structure having a predefined characteristic… the reference image processor 210 processes the volume data stored in the input/storing section 100 to display the three-dimensional reference image from the volume data”; Note: there is a 3D image of the volume data); storing the volumetric data (Paragraph 0040 – “The input/storing section 100 externally receives volume data containing density values of a three-dimensional structure having a predefined characteristic, and stores the received volume data for three-dimensional multi-planar image reconstruction”; Note: the volumetric data is stored); receiving a user request for a particular MPR view, including a user-selected plane and a targeted region of interest (Paragraph 0042, 0055 – “the reference image processor 210 processes the volume data stored in the input/storing section 100 to display the three-dimensional reference image from the volume data, and receives a region of interest entered by the user via the input section 400 in the form of straight line, curve, or free-formed curve data… it is checked in step 110 whether or not the user selects the basic MPR. If the basic MPR is chosen, the respective points of the straight line presenting a selected plane are sampled and arrange”; Note: the user indicates a region of interest and selects a plane for an MPR); computing volumetric coordinates for pixels on the requested view (Paragraph 0055-0056, 0061, 0064 – “it is checked in step 110 whether or not the user selects the basic MPR. If the basic MPR is chosen, the respective points of the straight line presenting a selected plane are sampled and arranged, in step 112. The sample points that are the basis in the generation of the corresponding MPR image, preferably the basic MPR image, are then stored, in step 114…The sample points are contained in a straight line (or curve) drawn (or selected) on the three-dimensional reference image by the user, and they become the points that constitute the one side (the left side or the lower base according to the direction of view) of the final MPR image. In the case of the basic MPR, the storage of the sample points is achieved by sampling the sample points at intervals of unit length from the straight line presenting the plane selected by the user…To generate the MPR image directly from the three-dimensional volume data, the two-dimensional sample points obtained in the above procedures are converted to three-dimensional sample points, in step 150. More specifically, the conversion of the two-dimensional sample points to three-dimensional ones involves multiplying the coordinate of each point by the inverse matrix of viewing matrix A…The value corresponding to the unit voxel is then obtained using the direction vectors starting from the respective sample points. Applying this procedure to all the sample points obtains the MPR image”; Note: points are sampled for the user selected region of interest and converted to volumetric (3D) coordinates); identifying stored volumetric data having recorded bounds intersecting the user-selected plane in the targeted region of interest (Paragraph 0043, 0044 – “The converter 220 extracts three-dimensional coordinates corresponding to the individual points constituting a line, a curve, or a free-formed curve on the reference image fed into the reference image processor 210 from the two-dimensional position data of the points…The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest”; Note: volume data corresponding to (intersecting) the user-selected plane and region of interest are identified); loading only the identified volumetric data from the storage (Paragraph 0044 – “The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: the identified volume image data is loaded); and generating the MPR, wherein at least one of the planes displays the user- requested targeted region of interest, using the loaded volumetric data (Paragraph 0015, 0044 – “when the shape of the displayed section is in a basic multi-planar image mode, sampling sample points at intervals of unit length from a straight line representing a plane selected by the user…The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: an MPR is generated for the region of interest using the loaded volumetric data).
Shekhar teaches parsing the volumetric image into a plurality of non-overlapping patches (Fig. 1, Paragraph 5 in 1st Col. of Page 2, Paragraph 1 in 2nd Col. of Page 2 – “Volume subdivision divides a 3-D image into smaller 3-D bricks of equal size. For spatiotemporal 4-D images, each frame is subdivided individually and identically”; Note: the volumetric image is parsed/divided into bricks, which are equivalent to patches. Fig. 1 shows the non-overlapping patches/bricks); storing the patches with recordation of three-dimensional (3-D) spatial coordinates, defining bounds of the patch (Paragraph 3-5 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… When a user inter acts and changes the orientation of the reformatted plane(s), the calculation of intersected bricks, polygons vertices, and texture coordinates is repeated… All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick. The texture memory thus acts like a cache, whose initialization and updating directly affects the performance”; Note: the bricks/patches are stored. It would be obvious to one of ordinary skill in the art that the bricks are stored along with their coordinates because location is an integral part of defining the bricks and retrieving them efficiently later on. The polygon vertices and brick edge coordinates define the bounds); identifying stored patches having recorded bounds intersecting the user- selected plane in the targeted region of interest (Fig. 1, Paragraph 3-5 in 2nd Col. of Page 2 – “the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… The bricks needed for each reformatted plane are determined individually. Likewise, the calculation of polygon vertices and texture coordinates is repeated for each view…All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick”; Note: the bricks/patches that intersect with the cutting plane, which corresponds to the user-selected plane taught by Pyo, are identified. The intersection is shown in Fig. 1); loading only the identified patches (Paragraph 5 in 2nd Col. of Page 2 – “All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick”; Note: the needed/identified bricks are loaded from storage); and generating the MPR, wherein at least one of the planes displays the user- requested targeted region of interest, using the loaded patches (Paragraph 2 in 2nd Col. of Page 2, Paragraph 5 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons. Fig. 2 shows a reformatted plane in which each polygon is colored differently to illustrate the underlying mosaic and the seamless tiling of the polygons. For cine MPR, the rendering of the reformatted plane is repeated for each frame of the sequence…All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick”; Note: an MPR is generated using the loaded bricks/patches).
Feulner et al. (US 20100205142 A1), hereinafter Feulner, teaches storing the patches remotely with recordation of coordinates (Paragraph 0019, 0043 – “the processing platform may be further operable to allocate image data portions of the image data set. An image data portion is associated to a patch of the slice of the body, wherein a number of patches covers the slice of the body…The arrangement according to FIG. 1 allows determining automatically which portion of the human body 2 is shown by CT volume images generated by the CT apparatus and stored in the storage device 5. A CT volume image comprises usually a plurality of image data sets each set corresponding to the scan of a slice of the body…The processing platform 2 analyzes the image data ID, in particular image data sets corresponding to the slices of the body, and attaches labels to the data sets including a body coordinate value. The body coordinate value designates a position along the body axis and therefore relates to certain body regions if the scale of the coordinates is chosen appropriately”).
However, none of the prior art directly and fully teaches the claim limitations of storing the patches remotely with recordation of three-dimensional (3-D) spatial coordinates, defining bounds of the patch; computing volumetric coordinates for pixels on the requested view, based on the recorded 3-D spatial coordinates of the stored patches; nor loading only the identified patches from the remote storage, excluding patches previously loaded for the user-selected plane. Therefore, the combination of the features is considered allowable.
Claims 2-5 and 7-8 are allowable due to their dependence on claim 1.
Claim 9 would be allowable for disclosing:
parse a volumetric image into a plurality of non-overlapping patches with recordation of three-dimensional coordinates for storage;
compute volumetric coordinates for pixel on the requested view, based on the recorded 3-D spatial coordinates of the stored patches;
load only the identified patches from remote storage, excluding patches previously loaded for the user-selected plane.
Regarding claim 9,
Pyo teaches a system for generating a patch-loaded Multi-Planar Reconstruction (MPR) (Paragraph 0010 – “a three-dimensional multi-planar image reconstruction system”), comprising: a processor (Paragraph 0009, Paragraph 0041 – “a recording medium readable by a computer storing the three-dimensional multi-planar image reconstruction method…The multi-planar image reconstructor 200, which comprises a reference image processor 210, a converter 220, and a reconstructor 230”; Note: it is implied that there is a processor in the computer order to perform the method); a memory storing instructions that, when executed by the processor (Paragraph 0009 – “a recording medium readable by a computer storing the three-dimensional multi-planar image reconstruction method”; Note: the recording medium is equivalent to memory), cause the processor to: receive a user request for a particular MPR view, including a user-selected plane and a targeted region of interest (Paragraph 0042, 0055 – “the reference image processor 210 processes the volume data stored in the input/storing section 100 to display the three-dimensional reference image from the volume data, and receives a region of interest entered by the user via the input section 400 in the form of straight line, curve, or free-formed curve data… it is checked in step 110 whether or not the user selects the basic MPR. If the basic MPR is chosen, the respective points of the straight line presenting a selected plane are sampled and arrange”; Note: the user indicates a region of interest and selects a plane for an MPR); compute volumetric coordinates for pixels on the requested view (Paragraph 0055-0056, 0061, 0064 – “it is checked in step 110 whether or not the user selects the basic MPR. If the basic MPR is chosen, the respective points of the straight line presenting a selected plane are sampled and arranged, in step 112. The sample points that are the basis in the generation of the corresponding MPR image, preferably the basic MPR image, are then stored, in step 114…The sample points are contained in a straight line (or curve) drawn (or selected) on the three-dimensional reference image by the user, and they become the points that constitute the one side (the left side or the lower base according to the direction of view) of the final MPR image. In the case of the basic MPR, the storage of the sample points is achieved by sampling the sample points at intervals of unit length from the straight line presenting the plane selected by the user…To generate the MPR image directly from the three-dimensional volume data, the two-dimensional sample points obtained in the above procedures are converted to three-dimensional sample points, in step 150. More specifically, the conversion of the two-dimensional sample points to three-dimensional ones involves multiplying the coordinate of each point by the inverse matrix of viewing matrix A…The value corresponding to the unit voxel is then obtained using the direction vectors starting from the respective sample points. Applying this procedure to all the sample points obtains the MPR image”; Note: points are sampled for the user selected region of interest and converted to volumetric (3D) coordinates); identify stored volumetric data having recorded bounds intersecting the user-selected plane in the targeted region of interest (Paragraph 0043, 0044 – “The converter 220 extracts three-dimensional coordinates corresponding to the individual points constituting a line, a curve, or a free-formed curve on the reference image fed into the reference image processor 210 from the two-dimensional position data of the points…The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest”; Note: volume data corresponding to (intersecting) the user-selected plane and region of interest are identified); load only the identified volumetric data (Paragraph 0044 – “The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: the identified volume image data is loaded); and generate the MPR for display (Paragraph 0015, 0044 – “when the shape of the displayed section is in a basic multi-planar image mode, sampling sample points at intervals of unit length from a straight line representing a plane selected by the user…The reconstructor 230 acquires image information from the three-dimensional image using the three-dimensional coordinates corresponding to the individual points received from the converter 220 and the viewing vector of a multi-planar image of interest, and reconstructs the image information into a three-dimensional multi-planar image corresponding to a region of interest designated by the user from the volume data”; Note: an MPR is generated for the region of interest using the loaded volumetric data).
Shekhar teaches parsing the volumetric image into a plurality of non-overlapping patches (Fig. 1, Paragraph 5 in 1st Col. of Page 2, Paragraph 1 in 2nd Col. of Page 2 – “Volume subdivision divides a 3-D image into smaller 3-D bricks of equal size. For spatiotemporal 4-D images, each frame is subdivided individually and identically”; Note: the volumetric image is parsed/divided into bricks, which are equivalent to patches. Fig. 1 shows the non-overlapping patches/bricks; see screenshot of Fig. 1 above) with recordation of three-dimensional (3-D) spatial coordinates for storage (Paragraph 3-5 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… When a user inter acts and changes the orientation of the reformatted plane(s), the calculation of intersected bricks, polygons vertices, and texture coordinates is repeated… All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick. The texture memory thus acts like a cache, whose initialization and updating directly affects the performance”; Note: the bricks/patches are stored. It would be obvious to one of ordinary skill in the art that the bricks are stored along with their coordinates because location is an integral part of defining the bricks and retrieving them efficiently later on. The polygon vertices and brick edge coordinates define the bounds); identifying stored patches having recorded bounds intersecting the user- selected plane in the targeted region of interest (Fig. 1, Paragraph 3-5 in 2nd Col. of Page 2 – “the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons… The bricks needed for each reformatted plane are determined individually. Likewise, the calculation of polygon vertices and texture coordinates is repeated for each view…All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick”; Note: the bricks/patches that intersect with the cutting plane, which corresponds to the user-selected plane taught by Pyo, are identified. The intersection is shown in Fig. 1); loading only the identified patches from the storage (Paragraph 5 in 2nd Col. of Page 2 – “All bricks needed for cine MPR must exist in the texture memory before they can be used. Since the brick size is significantly smaller than the size of the texture memory, thousands of bricks can reside in the texture memory simultaneously. When a specific brick is needed, the texture memory is checked first for its existence. If the brick does not exist, it is copied to the texture memory, overwriting an existing brick”; Note: the needed/identified bricks are loaded from storage); and generating the patch-loaded Multi-Planar Reconstruction (MPR) for display (Paragraph 2 in 2nd Col. of Page 2 – “The MPR of a 3-D image is the intersection of a cutting plane, coinciding with the reformatted plane, with the image volume. In our case, the intersection of the cutting plane with the subdivided volume yields a mosaic of polygons. Following the calculation of polygon vertices and texture coordinates, the reformatted plane is rendered by texture mapping each of the polygons. Fig. 2 shows a reformatted plane in which each polygon is colored differently to illustrate the underlying mosaic and the seamless tiling of the polygons. For cine MPR, the rendering of the reformatted plane is repeated for each frame of the sequence”; Note: an MPR is generated using the mosaic of polygons (patch-loading)).
However, none of the prior art directly and fully teaches the claim limitations of causing the processor to: parse a volumetric image into a plurality of non-overlapping patches with recordation of three-dimensional coordinates for storage; compute volumetric coordinates for pixel on the requested view, based on the recorded 3-D spatial coordinates of the stored patches; and load only the identified patches from remote storage, excluding patches previously loaded for the user-selected plane. Therefore, the combination of the features is considered allowable.
Claims 10-13 are allowable due to their dependence on claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Tizhoosh et al. (US 20200176102 A1) teaches a method of dividing images into patches, encoding representative patches, and generating index identifiers for storing the images based on the encoded patches.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE HAU MA whose telephone number is (571)272-2187. The examiner can normally be reached M-Th 7-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE HAU MA/Examiner, Art Unit 2617
/KING Y POON/Supervisory Patent Examiner, Art Unit 2617
