Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant’s Response
In Applicant’s Response dated 6/24/26, the Applicant amended Claims 1, 9-17 and argued Claims previously rejected in the Office Action dated 3/24/26. Claims 1-24 are pending examination.
In light of the Applicant’s amendments and remarks, the rejections under 35 USC 101 have been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 5, 6, 7, 9, 10, 13-15, 17, 18 and 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Kluckner, in view of Koo et al., “Automatic, global registration in laparoscopic liver surgery” (hereinafter “Koo”).
Claim 1:
Kluckner discloses:
A method comprising:
generating virtual 3D camera poses with respect to a 3D model previously constructed to model a 3D target organ and 3D anatomical structures associated therewith, wherein each of the virtual 3D camera poses corresponds to a perspective to view the 3D model (see paragraphs [0015] and [0054]). Kluckner teaches generating poses from a 3D model of an organ. The poses corresponding to view of the 3D model;
creating virtual 2D images corresponding to the virtual 3D camera poses by projecting the 3D model in accordance with corresponding perspectives, wherein each of the virtual 2D images includes 2D projected target organ and/or 2D structures of some of the 3D anatomical structures visible from a corresponding perspective (see paragraph [0023]). Kluckner teaches creating 2D images corresponding to the 3D poses by projecting the 3D model and getting 2D images and structures; and
Kluckner fails to expressly disclose obtaining 2d geometric features.
Koo discloses:
obtaining 2D feature/camera pose mapping models based on the 2D features extracted from the virtual 2D images and the corresponding virtual 3D camera poses, wherein the 2D features comprise geometric features of the target organ including a 2D ridge line that is a 2D projection of a 3D ridge on the target organ represented in the 3D model (see page 2, related works). Koo teaches obtaining 2d contours of the organ based on the images and ridge line represented in the model.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Kluckner to include obtaining 2d geometric features based on the model for the purpose of having augmented reality systems more intuitive and usable for surgeons and easier to translate to operating rooms for accurate registration, as taught by Koo.
Claim 2:
Kluckner discloses:
wherein the 3D model models at least one of:
the target organ, at least one blood vessel; at least one tumor; and one or more 3D ridges on the target organ (see paragraph [0015]). Kluckner teaches the 3D models a target organ.
Claim 5:
Kluckner discloses:
wherein the step of obtaining 2D feature/camera pose mapping models comprises:
pairing each of the virtual 3D camera poses with 2D features extracted from a corresponding virtual 2D image created by projecting the 3D model in accordance with a perspective determined based on the virtual 3D camera pose; and creating the 2D feature/camera pose mapping models based on the pairs of the 2D features and the virtual 3D camera poses (see paragraph [0028]-[0030] and [0033]). Kluckner teaches pairing the 3D image data with 2D features and creating a look up table and mappings based on the pairing.
Claim 6:
Kluckner discloses:
wherein the 2D feature/camera pose mapping models correspond to a look-up table comprising the pairs of the 2D features and the virtual 3D camera poses so that given input 2D features extracted from a 2D image, at least one 3D camera pose is identified from a pair in the look-up table that has stored 2D features similar to the input 2D features (see paragraph [0033]). Kluckner teaches a look up table to look up the pairs of 2D features and 3D pose images.
Claim 7:
Kluckner discloses:
wherein the step of creating the 2D feature/camera pose mapping tools comprises:
generating training data based on the pairs of the 2D features and the virtual 3D camera poses (see paragraph [0011]). Kluckner teaches generating training data based on the mapping of 3D and 2D features;
performing machine learning, using the training data, to learn the 2D feature/camera pose mapping tools (see paragraph [0013]). Kluckner teaches performing machine learning using training data to map 3D to 2D.
Claim 9, 10, 13-15:
Although Claims 9, 10, 13-15 are machine-readable medium claims, they are interpreted and rejected for the same reasons as the method of Claims 1, 2, 5-7, respectively.
Claim 17, 18, 21-23:
Although Claims 17, 18, 21-23 are system claims, they are interpreted and rejected for the same reasons as the method of Claims 1, 2, 5-7, respectively.
Claims 3, 4, 8, 11, 12, 16, 19, 20 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Kluckner, in view of Koo, in further view of Lurie et al., United States Patent Publication 20170046833 (hereinafter “Lurie”).
Claim 3:
Kluckner and Koo fail to express disclose six degrees of freedom.
Lurie discloses:
each of the virtual 3D camera poses is characterized in terms of six-degrees of freedom (see paragraph [0163] and [0167]). Lurie teaches the poses characterized in terms of six-degrees of freedom;
and the virtual 3D camera poses are generated to cover different viewing angles with respect to the 3D model with an increment in each of the six-degrees of freedom according to a pre-determined resolution (see paragraph [0163] and [0167]). Lurie teaches poses are generated to cover every angle with respect to the model.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Kluckner and Koo to include the camera poses in terms of six-degrees of freedom for the purpose of efficiently covering different angles of the 3D model, as taught by Lurie.
Claim 4:
Kluckner and Koo fail to express disclose generating mask and using ridge lines for model reconstruction.
Lurie discloses:
wherein the 2D features extracted from each of the virtual
2D images include one or more of:
a 2D structure corresponding to a 2D projection of the target organ in the virtual 2D image (see paragraphs [0056]). Lurie teaches the 2D images of the target image correspond to 2D structure;
a mask of the 2D structure corresponding to the target organ (see paragraph [0010] and [0011]). Lurie teaches a mask of the 2D structure for the organ;
a 2D ridge projected from a 3D ridge on the target organ modeled by the 3D model (see paragraph [0112] and [0113]). Lurie teaches the 2D ridge on the image is projected from the 3D ridge of the model.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Kluckner and Koo to include generating the masking and ridges of the 2D images for the purpose of efficiently processing 2D image data relating to 3D models, as taught by Lurie.
Claim 8:
Kluckner and Koo fail to express disclose generating 3D model reconstruction.
Lurie discloses:
receiving, during a medical procedure, a 2D image acquired by a camera inserted into a patient's body near the target object to capture surrounding information [0056]). Lurie teaches receiving 2D image-based data taken with endoscope inside the body near the target organ;
detecting, from the 2D image, a 2D object corresponding to the target organ and/or 2D structures corresponding to some of the 3D anatomical structures (see paragraph [0060]). Lurie teaches detecting a 2D object corresponding to the image based on structure from the 3D model;
extracting 2D features of the detected 2D object and/or 2D structures (see paragraph [0060]). Lurie teaches extracting features from the detected object;
predicting, based on the 2D feature/camera pose mapping models, an estimated 3D camera pose of the camera (see paragraph [0060]). Lurie teaches predicting a pose based on the camera pose; and
projecting the 3D model to visualize the target organ and/or some of the anatomical structures associated therewith in accordance with a perspective determined based on the estimated 3D camera pose (see paragraph [0059] and [0064]). Lurie teaches projects the model to visualize the target organ in a 3d space based on the pose and mapping.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Kluckner and Koo to include generating a 3D model reconstruction from 2D images for the purpose of efficiently reconstructing 3D models from 2D image data, as taught by Lurie.
Claim 11, 12, 16:
Although Claims 11, 12, 16 are machine-readable medium claims, they are interpreted and rejected for the same reasons as the method of Claims 3, 4, 8, respectively.
Claim 19, 20, 24:
Although Claims 19, 20, 24 are system claims, they are interpreted and rejected for the same reasons as the method of Claims 3, 4, 8, respectively.
Response to Arguments
Applicant’s arguments, see REM, filed 6/24/26, with respect to the rejections of claims 1-24 under 35 USC 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new grounds of rejection is made in view of Kluckner and Koo.
Applicant argues the Applicant has further amended claim 1 to recite, "wherein the 2D features comprise geometric features of the target organ including a 2D ridge line that is a 2D projection of a 3D ridge on the target organ represented in the 3D model." As described above, such features are not disclosed or adequately suggested by Kluckner.
The Examiner agrees.
The Examiner introduced new art, Koo, to teach obtaining the geometric features of the target organ based on the 3D model. See the above rejection of Claim 1. Thus, the combination of Kluckner and Koo discloses the limitations of the claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIONNA M BURKE whose telephone number is (571)270-7259. The examiner can normally be reached M-F 8a-4p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571)272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIONNA M BURKE/Examiner, Art Unit 2178 9/14/26
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights