DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 10, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Qui (U.S. Publication 2020/0367970) in view of Wilson (U.S. Publication 2014/0267239).
As to claim 1, Qiu discloses a processor-implemented method for generating three-Dimensional (3D) meshes of volumetric in-vivo images of a structure and rendering the 3D mesh for each node connection in the structure for visualization, comprising:
segmenting the volumetric in-vivo images of the structure into fine structures within the structure using at least one segmentation method (p. 11, sections 0153-0156; p. 22, section 0283; a marching cubes method is used to segment volumetric surgical images, which would be in-vivo images of a human body; the images are segmented into ROIS that better delineate structure, reading on segmenting into fine structures), wherein the volumetric in-vivo images of the structure are received from an image-capturing device, wherein the volumetric in-vivo images comprising medical imaging data from a plurality of magnetic resonance imaging (MRI) scans and Computed tomography (CT) scans (p. 18, section 0236; image volumes are received from CT or MRI scans of blood vessels or tumors, which would be in-vivo);
generating 3D meshes by applying a voxel sub-division method on the fine structures, wherein each 3D mesh is a representation of a geometric structure of the segmented fine structures areas (p. 11, sections 0153-0156; p. 22, section 0283; a marching cubes method is used to sub-divide voxels of designated regions and produce a 3D mesh model);
constructing a scene graph representation of the 3D meshes by combining the spatial relationships of the 3D mesh (p. 13, section 0180; p. 17, section 0222-p. 18, section 0232; a scene graph is constructed to represent combined spatial relationships between model representations, which the 3D meshes are part of as noted above, devices, and tools);
Qiu does not disclose, but Wilson discloses rendering the 3D mesh of the structure by rendering the scene graph representation (p. 1, section 0008; p. 2, sections 0022-0025; p. 4, section 0056; a representation of a hierarchical scene description, reading on a scene graph, is rendered) and analyzing the connections of nodes within the scene graph representation (p. 4, sections 0056-0064; as part of rendering, connections/paths between nodes are analyzed to determine parent and leaf nodes and merge nodes if necessary), wherein each node represents fine structures with attributes comprising at least one of geometry, or texture (p. 2, section 0025; p. 2, sections 0032-0033; p. 4, section 0056; nodes can represent objects or meshes with geometry or texture attributes) and providing the 3D meshes of each node connection on a user device for visualization (p. 2, section 0025; p. 2, sections 0032-0033; p. 4, section 0056; p. 4, section 0064-p. 5, section 0067; objects associated with merged node connections, which can be meshes representing a 3D scene, are rendered and displayed). The motivation for this is to represent similar elements with less overhead (p. 2, section 0024). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu to render the 3D mesh of the structure by rendering the scene graph representation and analyze the connections of nodes within the scene graph representation, wherein each node represents fine structures with attributes comprising at least one of geometry, or texture and provide the 3D meshes of each node connection on a user device for visualization in order to represent similar elements with less overhead as taught by Wilson.
As to claim 10, see the rejection to claim 1, Further, Qiu discloses a server, memory, and processor implementing the method (p. 7, sections 0097-0098; p. 15, section 0199; p, 21, sections 0272-0274) as well as that the mesh generation is marching cubes (p. 11, section 0156).
As to claim 19, see the rejection to claim 1. Further, Qiu discloses one or more non-transitory computer-readable storage mediums storing one or sequences of instructions, which when executed by one or more processors, causes a processor-implemented method (p. 7, sections 0097-0098).
Claims 2 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu in view of Wilson and further in view of Kim (U.S. Publication 2023/0306677).
As to claim 2, Qiu discloses ensuring accurate spatial relationships in the scene graph representation (p, 18, section 0236-p. 19, section 0237). Qiu does not disclose, but Kim discloses that accuracy is achieved (p. 1-2, section 0010; accuracy of matching points/spatial relationships is higher with the automatic method of the reference compared to manual methods) using a method that comprises aligning the 3D mesh of the node with the corresponding segmented volumetric in-vivo images (p. 1, section 0003; p. 1, section 0007; p. 2, sections 0028-0030; meshes of nodes/landmarks are aligned with volumetric data segmented into particular landmark regions and representing a maxillofacial and oral cavity, which would be in-vivo). The motivation for this is that it is helpful in establishing a procedure and surgical plan considering the aesthetic shape and color as well as the anatomical information of the inside of the head and neck region. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu and Wilson to in order to help establish a procedure and surgical plan considering the aesthetic shape and color as well as the anatomical information of the inside of the head and neck region as taught by Kim.
As to claim 11, see the rejection to claim 2.
Claims 3, 7, 8, 12, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu in view of Wilson and further in view of Rapaka (U.S. Publication 2016/0306942).
As to claim 3, Qiu does not disclose but Rapaka discloses wherein the method comprises enabling a user to interact with the 3D meshes on the user device, wherein the user interaction comprises actions for rotating, zooming, selecting, or modifying the 3D meshes (p. 3, section 0027-p. 4, section 0028; p. 5, section 0036; a patient’s brain activity is visualized on a gridded mesh representation which can be modified by a user interacting by rotating or zooming). The motivation for this is to allow a user to see different perspectives of the data using segmented anatomical structures representing a cerebral cortex. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu and Wilson to enable a user to interact with the 3D meshes on the user device, wherein the user interaction comprises actions for rotating, zooming, selecting, or modifying the 3D meshes in order to allow a user to see different perspectives of the data as taught by Rapaka.
As to claim 7, Qiu does not disclose but Rapaka discloses wherein at least one segmentation method is selected from Brain Suite, Free Surfer, and ITK-Snap (p. 3, section 0027). Motivation for the combination is given in the rejection to claim 3.
As to claim 8, Qiu discloses incorporating additional data comprising volume images of the structure into the scene graph representation to provide context for the 3D meshes (p. 18, section 0234-p. 19, section 0237; various volume images are used for proper alignment, which provides a context, in constructing a scene graph). Qiu does not disclose, but Rapaka discloses wherein the method comprises incorporating additional data comprising tractography (p. 3, section 0026-p. 4, section 0028; p. 4, section 0032; tractography data is used to provide context by identifying fiber pathways between points in the mesh model and weight node connections). Motivation for the combination is given in the rejection to claim 3.
As to claim 12, see the rejection to claim 3.
As to claim 16, see the rejection to claim 7.
As to claim 17, see the rejection to claim 8.
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu, Wilson and Rapaka and further in view of Frommhold (U.S. Publication 2020/0327740).
As to claim 4, Qui does not disclose, but Frommhold discloses wherein the method comprises updating the scene graph representation and the visualization of the 3D meshes in response to the user interaction to reflect real-time changes to the 3D meshes (p. 2, section 0015-0018; p. 2-3, section 0021; p. 3, section 0028; p. 5, section 0039; p. 5-6, section 0049; a mesh-based scene graph is updated or manipulated by various users and the visualization is updated; changes are updated real-time or close to real-time by compensating for latency). The motivation for this real-time updating is to allow users to use content correlated with a real-world object (p. 1-2, section 0014) . It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu, Wilson and Rapaka to update the scene graph representation and the visualization of the 3D meshes in response to the user interaction to reflect real-time changes to the 3D meshes in order to allow users to use content correlated with a real-world object as taught by Frommhold.
As to claim 13, see the rejection to claim 4.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu in view of Wilson and further in view of Hellge (U.S. Publication 2023/0260215).
As to claim 5, Qui does not disclose, but Hellge discloses wherein the method comprises exporting the scene graph representation in an advanced user visualization (AVU) format for storing or sharing the current state of the 3D meshes (p. 10, section 0138-0145;p. 13, sections 0194-0195; the scene graph with associated mesh states can be exported into glTF format, which would read on an advanced user visualization format). The motivation for this is that this format is vendor and runtime-neutral and can be loaded with minimal processing. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu and Wilson to export the scene graph representation in an advanced user visualization format for storing or sharing the current state of the 3D meshes in order to use a format that is vendor and runtime-neutral and can be loaded with minimal processing as taught by Hellge.
As to claim 14, see the rejection to claim 5.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu in view of Wilson and further in view of Huang (U.S. Publication 2024/0212139).
As to claim 6, Qiu does not disclose, but Huang discloses wherein the method comprises pre-processing the volumetric in-vivo images using at least one preprocessing technique, wherein the at least one preprocessing technique is selected from a normalization method, a contrast adjustment method, or artifact removal method (p. 2-3, section 0037; volumetric in-vivo images such as a collection of MRI/CT images are enhanced by contrast adjustment during pre-processing). The motivation for this is to make alignment more accurate. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu and Wilson to pre-process volumetric in-vivo images using at least one preprocessing technique, wherein the at least one preprocessing technique is selected from a normalization method, a contrast adjustment method, or artifact removal method in order to make alignment more accurate as taught by Huang.
As to claim 15, see the rejection to claim 6.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Qiu in view of Wilson and further in view of Ladavac (U.S. Publication 2023/0394767)
As to claim 9, Qiu does not disclose, but Ladavac discloses, wherein the method comprises generating the 3D mesh from the segmented fine structures by,
dividing the segmented fine structures into a plurality of equal-sized cubic units or voxels (p. 3, section 0033; p. 4, section 0043-p. 5, section 0044; p. 5, section 0053-p. 6, section 0054; a structure segmented into surfaces or object portions, or previously segmented to voxels, can be voxelized);
identifying whether neighboring voxels belong to the same segmented structure or different segmented structures (p. 10, section 0089; p. 12, section 0113; p. 13-14, section 0128; p. 15, section 0138; adjacent voxels are processed for same surface determination);
detect boundaries between the neighboring voxels belonging to different segmented structures (p. 10, section 0092; boundaries/edges are detected and voxels or voxel portions are determined to be above or below surfaces);
determining intersection points along voxel edges where transitions occur between the segmented fine structures for the plurality of voxels belonging to different segmented structures (p. 10, section 0092; intersection points are determined where region transition occurs between above/below surface portions);
constructing sub-surfaces between the identified intersection points and corresponding voxel positions to define boundary regions (p. 10, section 0097; p. 15, section 0141-p. 17, section 0156; meshes can be subdivided into sub-surfaces based on the intersections that represent boundaries and other factors such as curvature); and
aggregating the generated sub-surfaces to construct the 3D mesh representing the boundary of the segmented structure (p. 16, section 0156-p. 17, section 0167, the mesh with sub-surfaces is finalized based on the collection/aggregation of sub-surfaces created from the boundaries). The motivation for this is to provide more accurate approximations (p. 1, section 0003). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Qiu and Wilson to divide the segmented fine structures into a plurality of equal-sized cubic units or voxels, identify whether neighboring voxels belong to the same segmented structure or different segmented structures, detect boundaries between the neighboring voxels belonging to different segmented structures, determine intersection points along voxel edges where transitions occur between the segmented fine structures for the plurality of voxels belonging to different segmented structures, construct sub-surfaces between the identified intersection points and corresponding voxel positions to define boundary regions, and aggregate the generated sub-surfaces to construct the 3D mesh representing the boundary of the segmented structure in order to provide more accurate approximations as taught by Ladavac.
As to claim 18, see the rejection to claim 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/Primary Examiner, Art Unit 2617
