DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendment was received on 6/8/26 and has been entered and made of record. Currently, claims 1-7, 9-18, 20-31, and 33-35 are pending.
Response to Arguments
Applicant’s arguments, see pages 9-11 of the remarks, filed 6/8/26, with respect to the rejection(s) of claim(s) 1 under 35 USC 103(a) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of newly found prior art.
Applicant's arguments, see pages 11-12 of the remarks, filed 6/8/26, with respect to the rejection of claim 22 under 35 USC 103(a) have been fully considered but they are not persuasive. The applicant asserts O’Connor et al. (WO 2021/055522) and Shelton, IV et al. (US 2021/0196109) do not disclose determining at least one attribute associated with resection of at least a portion of the anatomy of interest using the first image data but without using the pixels of the first image data that are associated with the intersection between the obstruction and the anatomy of interest. The Examiner respectfully disagrees as Shelton discloses the above-mentioned feature. Particularly, Shelton discloses an imaging system 142 that can detect a tumor 2332, an artery 2334, and various abnormalities 2338 based on image data from an image sensor 135. The image sensor 135 captures a surgical site and visualizes the tissue and/or structures, which can be either visible or obscured (paras 176-177). A tumor, artery, various abnormalities, tissue, structure, etc. are all attributes associated with resection. The Examiner suggests further defining the attribute referred to in the claims as the term is quite broad. Shelton does not state the pixels associated with the intersection between the obstruction and the anatomy of interest are used to determine the attribute. As such, this satisfies the negative limitation set forth in the claim. Negative limitations do not require the prior art to specifically recite not utilizing data of information, only that such data or information is not utilized. Therefore, the combination of O’Connor and Shelton disclose that which is set forth in claim 22, and similar claim 27.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 2, 4-7, 9-13, 15-18, 20-21, 28, 30-31, and 33-34 are rejected under 35 U.S.C. 103(a) as being unpatentable over O’Connor et al. (WO 2021/055522), cited in the IDS dated 7/23/24, in view of Shelton IV et al. (2021/0196109) and further in view of Goel et al. (US 2020/0210721).
Regarding claims 1 and 12, O’Connor discloses a system for determining an attribute associated with anatomy of interest of a patient, the system comprising one or more processors, memory, and one or more programs stored in the memory for execution by the one or more processors and a method for determining an attribute associated with anatomy of interest of a patient comprising:
receiving first image data capturing the anatomy of interest of the patient and at least one obstruction obscuring at least a portion of the anatomy of interest of the patient (see paras 19 and 30, a first image of an anatomy of interest is captured by an image capture device 103, obstruction of the anatomy of interest is detected);
generating, using a second machine learning model, second image data in which at least a portion of the obstruction is replaced, wherein the second machine learning model is trained to generate the second image data by filling in a region of the first image data that corresponds to the at least a portion of the obstruction with a representation of the at least a portion of the anatomy of interest that was generated by the second machine learning model based on the first image data (see paras 19, 29-30, 43, and 46-47, combinational logic/algorithms, such as a machine learning model, neural network, or AI, are used to detect an obstruction that is obstructing the anatomy of interest and removing the obstruction or making the obstruction translucent such that the anatomy of interest that was hidden becomes visible).
O’Connor does not disclose expressly identifying the at least one obstruction in the first image data using a first machine learning model and determining at least one attribute associated with resection of at least a portion of the anatomy of interest based on the second image data.
Goel discloses identifying the at least one obstruction in the first image data using a first machine learning model (see Figs. 3 and 4 and paras 10, 37-38, 60-62, a first machine learning model is used to classify an object, such as an obstruction, and send information to a second machine learning model).
Shelton discloses determining at least one attribute associated with resection of at least a portion of the anatomy of interest based on the second image data (see paras 118, 123, 141, 177, and 331, a critical structure is determined, a tumor 2332 can be identified for removal, or resection, surgical visualization system 100 utilizes images of the anatomy of interest that does not contain obstructions to display image data to a clinician when an obstruction is detected).
Regarding claims 28 and 34, O’Connor discloses a system for compensating for an obstruction in imaging of anatomy of a patient, the system comprising one or more processors, memory, and one or more programs stored in the memory for execution by the one or more processors and a method for compensating for an obstruction in imaging of anatomy of a patient comprising:
receiving image data capturing anatomy of interest of the patient and at least one obstruction obscuring a portion of the anatomy of interest (see paras 19 and 30, a first image of an anatomy of interest is captured by an image capture device 103, obstruction of the anatomy of interest is detected);
detecting the at least one obstruction in the image data using a machine learning model (see para 43, combinational logic/algorithms, such as a machine learning model, neural network, or AI, are used to detect an obstruction that is obstructing the anatomy of interests);
generating a data set from the image data in which at least a portion of the at least one obstruction is altered based on the anatomy of interest using a second machine learning model, wherein the second machine learning model is trained to generate the data set by filling in a region of the image data that corresponds to the at least a portion of the at least one obstruction with a representation of the portion of the anatomy of interest that was generated by the second machine learning model based on the image data (see paras 19, 29-30, 43, and 46-47, combinational logic/algorithms, such as a machine learning model, neural network, or AI, are used to detect an obstruction that is obstructing the anatomy of interest and removing the obstruction or making the obstruction translucent such that the anatomy of interest that was hidden becomes visible); and
generating a visual guidance associated with the at least a portion of the anatomy of interest based on the determined at least one attribute; and displaying the visual guidance (see paras 58 and 64, visual guidance is displayed to the user about the anatomy of interest and the removal of the obstruction from the anatomy of interest).
O’Connor does not disclose expressly detecting the at least one obstruction in the image data using a machine learning model and determining at least one attribute associated with resection of at least a portion of the anatomy of interest based on the data set and generating a visual guidance associated with the resection.
Goel discloses detecting the at least one obstruction in the image data using a machine learning model (see Figs. 3 and 4 and paras 10, 37-38, 60-62, a first machine learning model is used to classify an object, such as an obstruction, and send information to a second machine learning model).
Shelton discloses determining at least one attribute associated with resection of at least a portion of the anatomy of interest based on the data set (see paras 118, 123, 141, 177, and 331, a critical structure is determined, a tumor 2332 can be identified for removal, or resection, surgical visualization system 100 utilizes images of the anatomy of interest that does not contain obstructions to display image data to a clinician when an obstruction is detected); and
generating a visual guidance associated with the resection of the at least a portion of the anatomy of interest based on the determined at least one attribute; and displaying the visual guidance (see paras 118-119, 123, 126-127, 133, 158, and 161, surgical visualization system 100 displays visual guidance to a clinician to aid in the resection of a tumor).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the use of multiple machine learning models, as described by Goel, and the guidance for resection based on an identified attribute, as described by Shelton, with the system of O’Connor.
The suggestion/motivation for doing so would have been to ensure proper visual depiction of an anatomy of interest allowing a practitioner to accurately perform a surgical procedure thereby reducing risk to the patient.
Therefore, it would have been obvious to combine Shelton and Goel with O’Connor to obtain the invention as specified in claims 1, 12, 28, and 34.
Regarding claims 2 and 13 O’Connor further discloses generating a visual guidance associated with the anatomy of interest based on the determined at least one attribute and adding the visual guidance to the second image data (see paras 58 and 64, visual guidance is displayed to the user about the anatomy of interest and the removal of the obstruction from the anatomy of interest).
Regarding claims 4 and 15, O’Connor further discloses wherein the second image data is displayed intraoperatively for guiding a surgical procedure (see paras 58 and 64, visual guidance is displayed to a surgeon in real time during a procedure about the anatomy of interest and the removal of the obstruction from the anatomy of interest).
Regarding claims 5, 16, and 30 O’Connor further discloses wherein determining the at least one attribute comprises identifying at least a portion of a perimeter of the anatomy of interest based at least in part on the representation of the at least a portion of the anatomy of interest obscured by the obstruction (see paras 26 and 29-30, an obstruction is identified in the anatomy of interest, pixels associated with the intersection between the obstruction and the anatomy of interest are identified).
Regarding claims 6, 17, and 31 O’Connor further discloses wherein the obstruction obscures the at least a portion of the perimeter in the first image data (see paras 26 and 29-30, an obstruction is identified in the anatomy of interest, pixels associated with the intersection between the obstruction and the anatomy of interest are identified).
Regarding claims 7 and 18, O’Connor further discloses wherein the first image data is an X-ray image (see paras 30 and 52, the first image is an X-ray image).
Regarding claims 9, 20, and 33 O’Connor further discloses displaying the second image data with a representation of the at least one obstruction overlaid on the representation of the at least a portion of the anatomy of interest (see paras 19 and 58, an obstruction that is obstructing the anatomy of interest is replaced with an image overlay).
Regarding claims 10 and 21, O’Connor further discloses wherein the at least one obstruction is at least one surgical instrument (see paras 55, a surgical instrument that is obstructing the anatomy of interest is detected).
Regarding claim 11, O’Connor further discloses wherein the at least one machine learning model comprises a diffusion-based machine learning model (see paras 43, the system can use a diffusion-based machine learning model).
Regarding claim 35, Shelton further discloses wherein the at least one attribute is determined based on the at least a portion of the anatomy of interest that was obscured by the at least one obstruction (see paras 117-119, 124, 126-128, 160, and 163, a first image of an anatomy of interest is captured by a camera of an imaging device 120, obstruction the anatomy of interest is detected).
Claims 3, 14, and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over O’Connor, Shelton, and Goel as applied to claims 1 and 12 above, and further in view of Quaid III (US 2004/0034282).
O’Connor, Shelton, and Goel do not disclose expressly wherein the visual guidance provides guidance for bone removal.
Quaid III discloses wherein the visual guidance provides guidance for bone removal (see paras 10, 63, 93, 96, 110, and 120, bone resections/removal is performed based on visual guidance).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the guidance for bone removal utilizing X-ray images, as described by Quaid III, with the system of O’Connor, Shelton and Goel.
The suggestion/motivation for doing so would have been to ensure proper visual depiction of an anatomy of interest allowing a practitioner to accurately perform a surgical procedure thereby reducing risk to the patient.
Therefore, it would have been obvious to combine Quaid III with O’Connor, Goel, and Shelton to obtain the invention as specified in claims 3, 14, and 29.
Claims 22-23 and 25-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over O’Connor et al. (WO 2021/055522), cited in the IDS dated 7/23/24, in view of Shelton IV et al. (2021/0196109).
Regarding claims 22 and 27, O’Connor discloses a system for determining an attribute associated with anatomy of interest of a patient, the system comprising one or more processors, memory, and one or more programs stored in the memory for execution by the one or more processors and a method for determining an attribute associated with anatomy of interest of a patient comprising:
receiving first image data capturing the anatomy of interest of the patient and at least one obstruction obscuring at least a portion of the anatomy of interest of the patient (see paras 19 and 30, a first image of an anatomy of interest is captured by an image capture device 103, obstruction of the anatomy of interest is detected); and
determining a location of the obstruction relative to the anatomy of interest within the first image data using at least one machine learning model, wherein determining the location of the obstruction relative to the anatomy of interest comprises identifying pixels of the first image data that are associated with an intersection between the obstruction and the anatomy of interest (see paras 19, 26, 29-30, and 43, combinational logic/algorithms, such as a machine learning model, neural network, or AI, are used to detect an obstruction that is obstructing the anatomy of interest and removing the obstruction or making the obstruction translucent such that the anatomy of interest that was hidden becomes visible, pixels associated with the intersection between the obstruction and the anatomy of interest are identified).
O’Connor does not disclose expressly determining at least one attribute associated with resection of at least a portion of the anatomy of interest using the first image data but without using the pixels of the first image data that are associated with the intersection between the obstruction and the anatomy of interest.
Shelton discloses determining at least one attribute associated with resection of at least a portion of the anatomy of interest using the first image data but without using the pixels of the first image data that are associated with the intersection between the obstruction and the anatomy of interest (see paras 118, 123, 141, 177, and 331, a critical structure is determined, a tumor 2332 can be identified for removal, or resection, surgical visualization system 100 utilizes images of the anatomy of interest that does not contain obstructions to display image data to a clinician when an obstruction is detected).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the guidance for resection based on an identified attribute, as described by Shelton, with the system of O’Connor.
The suggestion/motivation for doing so would have been to ensure proper visual depiction of an anatomy of interest allowing a practitioner to accurately perform a surgical procedure thereby reducing risk to the patient.
Therefore, it would have been obvious to combine Shelton with O’Connor to obtain the invention as specified in claims 22 and 27.
Regarding claim 23, O’Connor further discloses generating a visual guidance associated with the anatomy of interest based on the determined at least one attribute and adding the visual guidance to the second image data (see paras 58 and 64, visual guidance is displayed to the user about the anatomy of interest and the removal of the obstruction from the anatomy of interest).
Regarding claim 25, O’Connor further discloses wherein determining the at least one attribute comprises identifying at least a portion of a perimeter of the anatomy of interest based at least in part on the representation of the at least a portion of the anatomy of interest obscured by the obstruction (see paras 26 and 29-30, an obstruction is identified in the anatomy of interest, pixels associated with the intersection between the obstruction and the anatomy of interest are identified).
Regarding claim 26, O’Connor further discloses wherein the obstruction obscures the at least a portion of the perimeter in the first image data (see paras 26 and 29-30, an obstruction is identified in the anatomy of interest, pixels associated with the intersection between the obstruction and the anatomy of interest are identified).
Claim 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over O’Connor and Shelton as applied to claim 22 above, and further in view of Quaid III (US 2004/0034282).
O’Connor and Shelton do not disclose expressly wherein the visual guidance provides guidance for bone removal.
Quaid III discloses wherein the visual guidance provides guidance for bone removal (see paras 10, 63, 93, 96, 110, and 120, bone resections/removal is performed based on visual guidance).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the guidance for bone removal utilizing X-ray images, as described by Quaid III, with the system of O’Connor and Shelton.
The suggestion/motivation for doing so would have been to ensure proper visual depiction of an anatomy of interest allowing a practitioner to accurately perform a surgical procedure thereby reducing risk to the patient.
Therefore, it would have been obvious to combine Quaid III with O’Connor and Shelton to obtain the invention as specified in claim 24.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK R MILIA whose telephone number is (571) 272-7408. The examiner can normally be reached Monday-Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached at 571-270-3438. The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK R MILIA/ Primary Examiner, Art Unit 2681
Prosecution Insights