DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 12, and 20, the last line of claims 1, 12, and 20 recites “tracking positions of the at one object”. It is unclear what “the at one object” is referring to. Claims 1, 12, and 20 previously recite “selected material of at least one object”. Therefore the limitation will be interpreted as “tracking positions of the at least one object”. Claims 2-11 and 13-19 are also rejected due to their dependency.
Regarding claim 10, lines 1-2 of claim 10 recites “determines the a respective position of each object of the one or more objects”. It is unclear if this is referring to the respective position of each object of the at least one object of claim 1 or if this is defining a new respective position of each object. For examination purposes this limitation will be interpreted as “determines the respective position of each object of the at least one object”. Claim 11 is also rejected due to its dependency.
Regarding claim 10, line 4 of claim 10 recites “placement of arrays for tracking object position optically”. It is unclear if this is referring to the one or more objects, the at least one object, or each object of the at least one object of claim 1. For examination purposes this limitation will be interpreted as “placement of arrays for tracking the at least one object position optically”. Claim 11 is also rejected due to its dependency.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 8, 10, 12-15, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yardibi (US 20230020346).
Regarding claims 1, 12, and 20, Yardibi discloses a computer-implemented method (Abstract – “A method of operating a surgical visualization system”, [0041] – “The depicted control system (20) includes a control circuit (21) in signal communication with a memory (22). The memory (22) stores instructions executable by the control circuit (21) to determine and/or recognize critical structures”) comprising: [claim 1]
a computer system comprising: a memory; and a processing circuit in communication with the memory, wherein the computer system is configured to perform a method (Abstract – “A method of operating a surgical visualization system”, [0041] – “The depicted control system (20) includes a control circuit (21) in signal communication with a memory (22). The memory (22) stores instructions executable by the control circuit (21) to determine and/or recognize critical structures”) comprising: [claim 12]
a computer program product comprising: a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method (Abstract – “A method of operating a surgical visualization system”, [0041] – “The depicted control system (20) includes a control circuit (21) in signal communication with a memory (22). The memory (22) stores instructions executable by the control circuit (21) to determine and/or recognize critical structures”) comprising: [claim 20]
imaging, using at least one spectral imaging camera, an area comprising one or more objects ([0034] – “The imaging device (17) of the present example includes an emitter (18), which is configured to emit spectral light in a plurality of wavelengths to obtain a spectral image of hidden structures”), wherein the imaging comprises obtaining intensity signals for a selective one or more wavelengths or wavelength ranges that correlate to selected material of at least one object of the one or more objects (Figs. 4-6, paragraph [0056] describes Figs. 4-6 by reciting they are graphical representations of the spectral signatures of different tissues as a function of wavelength.);
using the obtained intensity signals to determine a respective position of each object of the at least one object in space ([0032] – “these features of the surgical visualization system (10) may determine a position of a critical structure”, [0045] – “integrates spectral signature tissue identification and structured light tissue positioning to identify critical structures”); and
tracking positions of the at one object in space over time ([0107] – “the method (310) includes providing real-time feedback to the user (e.g., the surgeon) on the display (130) to aid in identification of critical structures (11 a-11 b, 512, 514, 516) and/or background structures (520). The real-time feedback may include electronically displaying the mask (522 a-b) on at least one critical structure (11 a-11 b, 512, 514, 516)”, therefore the critical structure is identified and tracked in real-time).
Regarding claims 2 and 13, Yardibi further discloses wherein the tracking comprises repeating the imaging and the using one or more times at different points in time ([0033] – “camera…configured to provide real-time views”, [0045] – “integrates spectral signature tissue identification and structured light tissue positioning to identify critical structures”, [0107] – “the method (310) includes providing real-time feedback to the user (e.g., the surgeon) on the display (130) to aid in identification of critical structures (11 a-11 b, 512, 514, 516) and/or background structures (520). The real-time feedback may include electronically displaying the mask (522 a-b) on at least one critical structure (11 a-11 b, 512, 514, 516)”).
Regarding claims 3 and 14, Yardibi further discloses wherein the at least one spectral imaging camera comprises at least one selected from the group consisting of: (i) one or more hyperspectral imaging cameras for hyperspectral imaging of the area and (ii) one or more multispectral imaging cameras for multispectral imaging of the area ([0033] – “an imaging device (17) includes a spectral camera (e.g., a hyperspectral camera, multispectral camera”).
Regarding claims 4 and 15, Yardibi further discloses wherein the area comprises a surgical scene and wherein the at least one object comprises patient anatomy, the patient anatomy comprising bone or other selected anatomy ( [0107] – “the method (310) includes providing real-time feedback to the user (e.g., the surgeon) on the display (130) to aid in identification of critical structures (11 a-11 b, 512, 514, 516) and/or background structures (520). The real-time feedback may include electronically displaying the mask (522 a-b) on at least one critical structure (11 a-11 b, 512, 514, 516)”, [0030] – “Critical structures (11 a, 11 b) may be any anatomical structures of interest”).
Regarding claim 8, Yardibi further discloses wherein the using comprises using at least one algorithm to correlate the patient anatomy to a preoperative dataset or modified version of the preoperative dataset and return a location/pose of the patient anatomy ([0053] “Preoperative data (75) from a CT or MRI scan may be employed to register or align certain three-dimensional deformable tissue in various instances”, [0102] – “the pre-operative information is information that is not specific to any particular patient that is subsequently used by the surgical visualization system (110) to aid in identification of at least one anatomical structure in the anatomical field (510)… This pre-operative information may be used to train (e.g., through machine learning) the surgical visualization system (110) to better identify anatomical structures in the anatomical field (510)”).
Regarding claim 10, Yardibi further discloses wherein the using determines the a respective position of each object of the one or more objects absent use or reliance on (i) tracking of fiducials or other markers on the one or more objects or in the area comprising the one or more objects, (ii) placement of arrays for tracking object position optically, and (iii) beacons and RADAR-based tracking ([0032] – “the surgical visualization system (10) incorporates tissue identification and geometric surface mapping in combination with a distance sensor system (14). In combination, these features of the surgical visualization system (10) may determine a position of a critical structure”, [0045] – “integrates spectral signature tissue identification and structured light tissue positioning to identify critical structures”, Yardibi does not use or rely on tracking fiducial or markers, arrays for tracking object position optically, or beacons and RADAR-based tracking).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 5, 6, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yardibi (US 20230020346) as applied to claims 4 and 15 above, and further in view of Ben-Yishai (US 20220354691).
Regarding claims 5 and 16, Yardibi discloses all the elements of the claimed invention as cited in claims 1, 4, 12 and 15.
Yardibi discloses in paragraph [0053] “Preoperative data (75) from a CT or MRI scan may be employed to register or align certain three-dimensional deformable tissue in various instances” and [0102] “the pre-operative information is information that is not specific to any particular patient that is subsequently used by the surgical visualization system (110) to aid in identification of at least one anatomical structure in the anatomical field (510)”.
Conversely Yardibi does not teach further comprising correlating the determined respective position of the patient anatomy to a prior-obtained model of the patient anatomy or modified version of the prior-obtained model.
However Ben-Yishai discloses further comprising correlating the determined respective position of the patient anatomy to a prior-obtained model of the patient anatomy or modified version of the prior-obtained model ([0050] – “The intraoperative image can be obtained via…multi-spectral imaging”, [0073] – “image registration between a first image that is a preoperative image and a second image that is an intraoperative image…the first image can be a rendered image of a model of the outer surface of the cortex from CT/MRI that is registered to the intraoperative image (e.g. based on sulci, gyri, blood vessels that can appear in the 3D model)”).
The disclosure of Ben-Yishai is an analogous art considering it is in the field of multi-spectral imaging.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system of Yardibi to incorporate the registration of a preoperative model to an intraoperative multi-spectral image of Ben-Yishai to achieve the same results. One would have motivation to combine because it would provide a more detailed view of the location of the anatomical structure in relation to other structures that aren’t visible in the multi-spectral image.
Regarding claims 6 and 17, Yardibi and Ben-Yishai disclose all the elements of the claimed invention as cited in claims 1, 4, 5, 12, 15, and 16.
Conversely Yardibi does not teach wherein the prior-obtained model comprises a preoperative two-dimensional or three-dimensional model of the patient anatomy.
However Ben-Yishai discloses wherein the prior-obtained model comprises a preoperative two-dimensional or three-dimensional model of the patient anatomy ([0073] – “image registration between a first image that is a preoperative image and a second image that is an intraoperative image…the first image can be a rendered image of a model of the outer surface of the cortex from CT/MRI that is registered to the intraoperative image (e.g. based on sulci, gyri, blood vessels that can appear in the 3D model)”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system of Yardibi to incorporate the preoperative three-dimensional model of Ben-Yishai to achieve the same results. One would have motivation to combine because it would provide a more detailed view of the location of the anatomical structure in relation to other structures that aren’t visible in the multi-spectral image.
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yardibi (US 20230020346) and Ben-Yishai (US 20220354691) as applied to claims 6 and 17 above, and further in view of Hendriks (WO 2018002347).
Regarding claims 7 and 18, Yardibi and Ben-Yishai disclose all the elements of the claimed invention as cited in claims 1, 4, 5, 6, 12, 15, 16, and 17.
Conversely Yardibi does not teach tracking alterations to the patient anatomy during a surgical procedure and updating the prior-obtained model according to the tracked alterations to provide the modified version of the prior-obtained model; and correlating the altered patient anatomy as observed from the imaging to the modified version of the prior-obtained model.
However Hendriks discloses tracking alterations to the patient anatomy during a surgical procedure and updating the prior-obtained model according to the tracked alterations to provide the modified version of the prior-obtained model; and correlating the altered patient anatomy as observed from the imaging to the modified version of the prior-obtained model (pg. 16 lines 8-9 – “the tomographic images taken by the imaging machine 36 (which may be pre -operative images)”, pg. 4 lines 31-32 – “The endoscopic imaging data may be obtained from a spectral, hyperspectral, multispectral or thermographic imaging device”, pg. 7 lines 2-5 – “registering the tomographic imaging data and the endoscopic imaging data based on registration of the structures identified in the endoscopic imaging data and the subsurface structures of the model produced from segmentation of the tomographic imaging data”, pg. 5 lines 17-21 – “The at least one processor may be adapted to register the tomographic imaging data and the endoscopic imaging data using an elastic registration process. In this way, deformation in the region of interest…can be determined from the endoscopic imaging data and used, based on the registration process, to update the tomographic imaging data”).
The disclosure of Hendriks is an analogous art considering it is in the field of spectral imaging.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system of Yardibi to incorporate the elastic registration of Hendriks to achieve the same results. One would have motivation to combine because “the surgeon is guided to take into account the deformed position of the tumor 152 and blood vessels 154 when carrying out a procedure” (Hendricks – pg. 16 lines 1-2).
Claim 9 are rejected under 35 U.S.C. 103 as being unpatentable over Yardibi (US 20230020346) as applied to claim 8 above, and further in view of Hendriks (WO 2018002347).
Regarding claim 9, Yardibi discloses all the elements of the claimed invention as cited in claims 1, 4, and 8.
Conversely Yardibi does not teach tracking alterations to the patient anatomy during a surgical procedure and updating the preoperative dataset according to the tracked alterations to provide the modified version of the preoperative dataset; and correlating the altered patient anatomy as observed from the imaging to the modified version of the preoperative dataset.
However Hendriks discloses further comprising: tracking alterations to the patient anatomy during a surgical procedure and updating the preoperative dataset according to the tracked alterations to provide the modified version of the preoperative dataset; and correlating the altered patient anatomy as observed from the imaging to the modified version of the preoperative dataset (pg. 16 lines 8-9 – “the tomographic images taken by the imaging machine 36 (which may be pre -operative images)”, pg. 4 lines 31-32 – “The endoscopic imaging data may be obtained from a spectral, hyperspectral, multispectral or thermographic imaging device”, pg. 7 lines 2-5 – “registering the tomographic imaging data and the endoscopic imaging data based on registration of the structures identified in the endoscopic imaging data and the subsurface structures of the model produced from segmentation of the tomographic imaging data”, pg. 5 lines 17-21 – “The at least one processor may be adapted to register the tomographic imaging data and the endoscopic imaging data using an elastic registration process. In this way, deformation in the region of interest…can be determined from the endoscopic imaging data and used, based on the registration process, to update the tomographic imaging data”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system of Yardibi to incorporate the elastic registration of Hendriks to achieve the same results. One would have motivation to combine because “the surgeon is guided to take into account the deformed position of the tumor 152 and blood vessels 154 when carrying out a procedure” (Hendricks – pg. 16 lines 1-2).
Claims 11 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yardibi (US 20230020346) as applied to claims 10 and 12 above, and further in view of Wood (US 20190388160).
Regarding claims 11 and 19, Yardibi discloses all the elements of the claimed invention as cited in claims 1, 10, and 12.
Conversely Yardibi does not teach wherein the using comprises applying an artificial intelligence (Al) model to identify the at least one object, the Al model configured to identify selected materials based on training the Al model using machine learning and at least one dataset providing reflection or absorption of various wavelengths for varying specific materials.
However Wood discloses wherein the using comprises applying an artificial intelligence (Al) model to identify the at least one object, the Al model configured to identify selected materials based on training the Al model using machine learning and at least one dataset providing reflection or absorption of various wavelengths for varying specific materials ([0072] – “the sample 1198 (FIG. 9) is illuminated with light 1197 (FIG. 9) that comprises specific wavelength bands”, [0077] – “the hyperspectral dataset 1280 is then processed to extract the tissue specific information…if the spectral peaks or features of chemical(s) of interest are known, the spectra is processed, through either peak or feature detection algorithms, to detect the peaks or features to give an indication of the chemical presence and some indication of the concentration or quality…In one embodiment, the spectra of specific tissues or tissue states of interest can be acquired and stored in a database…Spectra then acquired during the surgery can be compared to the spectra stored in the database for similarity and if sufficiently similar to give an indication of what tissue or tissue type the spectra was acquired from”, [0078] – “predict the state of a new sample based on the acquired spectrum. Some of the more commonly used employed techniques include principal component regression (PCR), partial least squares (PLS), and neural networks (NN)”).
The disclosure of Wood is an analogous art considering it is in the field of spectral imaging.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system of Yardibi to incorporate the machine learning for tissue identification based on the spectra of Wood to achieve the same results. One would have motivation to combine because “the results of the analysis can be obtained in near-real time for appropriate use by a surgeon” (Wood – [0078]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENEE C LANGHALS whose telephone number is (571)272-6258. The examiner can normally be reached Mon.-Thurs. alternate Fridays 8:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached at 571-272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.C.L./Examiner, Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797