Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 5, 2026 has been entered.
Claim Interpretation
Claims 25-35 are interpreted as claiming subject matter that is eligible under 35 U.S.C. 101 (Alice., 573 U.S. at 216, 110 USPQ2d at 1980; Mayo, 566 U.S. at 79080, 86-87, 101 USPQ2d at 1968-69, 1971), for claiming a process that cannot practically performed in the human mind. That is, independent Claim 25 claims a method for synthesizing surgical robotic tool pose information and a deformable 2D reconstruction of tissue into a common coordinate frame after receiving pose information of a surgical robotic tool from a tool tracker and receiving a deformable 3D tissue reconstruction from a tissue tracker, both of which require electronic sensors. As such, the position the tool requires the use of a physical sensor, just as a GPS receiver is required for determining the position thereof. See MPEP 2106.04(a)(2)(III)(A) and SiRF Tech., 601 F.3d at 1331-33, 94 USPQ2d at 1616-17. As such, independent Claim 25, as well as its dependent Claims 26-35, meet the conditions of Prong One of Step 2A of an Alice/Mayo test, and are statutory under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(g)(1) during the course of an interference conducted under section 135 or section 291, another inventor involved therein establishes, to the extent permitted in section 104, that before such person’s invention thereof the invention was made by such other inventor and not abandoned, suppressed, or concealed, or (2) before such person’s invention thereof, the invention was made in this country by another inventor who had not abandoned, suppressed, or concealed it. In determining priority of invention under this subsection, there shall be considered not only the respective dates of conception and reduction to practice of the invention, but also the reasonable diligence of one who was first to conceive and last to reduce to practice, from a time prior to conception by the other.
Claims 25, 28-30, and 32-34 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Barbagli et al. (US PGPUB 2017/0281139 – “Barbagli”).
Regarding Claim 25, Barbagli discloses:
A method for synthesizing surgical robotic tool pose information and a deformable 3D reconstruction of tissue into a common coordinate frame, comprising:
receiving images from an endoscopic camera (Barbagli FIG. 2, visualization system 231 and medical tool 226; Barbagli paragraph [0040], “medical tool 226 may be an image capture probe that includes a distal portion with a stereoscopic or monoscopic camera at or near the distal end 218 of the flexible catheter body 216 for capturing images (including video images) that are processed by a visualization system 231 for display”);
segmenting the images into a first dataset that includes image data of the surgical robotic tool (Barbagli FIG. 4, medical instrument 307) and a second dataset that includes image data of tissue (Barbagli FIG. 4, virtual image 301 including virtual image of target structure 302 together with an image of medical instrument 307);
passing the first and second datasets to a tool tracker (Barbagli FIG. 2, tracking system 230; Barbagli paragraph [0044], “tracking system 230 may be sent to a navigation system 232 where it is combined with information from the visualization system 231 and/or the preoperatively obtained models to provide the surgeon or other operator with real-time position information on the display system 110 for use in the control of the instrument 200”; Barbagli paragraph [0006], “a medical instrument including a catheter and a tool deployable from the catheter”; Examiner interprets Barbagli’s tracking system as tracking a surgical tool deployed from a medical instrument, such as an endoscope.) and a tissue tracker (Barbagli FIG. 1, display system 110 and sensor system 108; Barbagli paragraph [0025], “a display system 110 for displaying an image or representation of the surgical site and medical instrument system(s) 104 generated by sub-systems of the sensor system 108”), respectively;
receiving pose information of the surgical robotic tool from the tool tracker and receiving the deformable 3D tissue reconstruction from the tissue tracker, wherein the tissue tracker performs tissue tracking and fusion on the second dataset to generate the deformable 3D tissue reconstruction (Examiner interprets Barbagli’s tissue tracker as being an integral component of Barbagli’s system for generating the 3D tissue reconstruction shown in Barbagli FIG. 4, just as the tissue tracing and fusion module 30 shown in FIG. 1 of the present patent application is combined with surgical tool tracking module 25 to enable synthesize tracking module 35 to generate the surgical scene reconstruction 40.); combining the pose information and the deformable 3D tissue reconstruction into a common coordinate frame to provide information for generating a virtual surgical environment captured by the endoscopic camera (Barbagli FIG. 4, showing real-time image of medical instrument 307 superimposed on virtual image 301 of target structure 302; Barbagli paragraph [0027], “intra-operative image data may be presented as two-dimensional, three-dimensional, or four-dimensional (including e.g., time based or velocity based information) images or as images from models created from the pre-operative or intra-operative image data sets”; Barbagli paragraph [0049], “The composite virtual image 301 thus indicates the location of the distal tip of the medical instrument within the lung.”).
Regarding Claim 28, Barbagli discloses the features of Claim 25, as described above.
Barbagli further discloses wherein segmenting the images further includes rendering of the surgical robotic tool to remove pixel information associated with the surgical robotic tool so that a remainder of the images includes the second dataset and excludes the pixel information associated with the tool (Barbagli paragraph [0028], “In some embodiments, the display 110 may display a virtual navigational image in which the actual location of the medical instrument 104 is registered (i.e., dynamically referenced) with the preoperative or concurrent images/model to present the clinician or surgeon S with a virtual image of the internal surgical site…Alternatively, the instrument 104 may not be visible in the virtual image.”).
Regarding Claim 29, Barbagli discloses the features of Claim 25, as described above.
Barbagli further discloses wherein the common coordinate frame is an endoscopic camera frame (Barbagli paragraph [0024], “visualization system (e.g., visualization system 231 of FIG. 2) may include a viewing scope assembly that records a concurrent or real-time image of the surgical site and provides the image to the clinician…in alternative embodiments, a separate endoscope, attached to a separate manipulator assembly may be used with the medical instrument to image the surgical site”).
Regarding Claim 30, Barbagli discloses the features of Claim 25, as described above.
Barbagli further discloses wherein the tissue tracker performs tissue tracking and fusion (Barbagli paragraph [0044], “information from the tracking system 230 may be sent to a navigation system 232 where it is combined with information from the visualization system 231 and/or the preoperatively obtained models to provide the surgeon or other operator with real-time position information on the display system 110 for use in the control of the instrument 200.”).
Regarding Claim 32, Barbagli discloses the features of Claim 25, as described above.
Barbagli further discloses wherein the tissue tracker updates the deformable 3D tissue reconstruction using subsequently received image data of the tissue (Barbagli FIG. 4, showing image of instrument 307; Barbagli paragraph [0049], “The instrument image may change to depict the advancement or withdrawal of the instrument through the bronchial passageways.” Examiner interprets the changes to the image of the instrument 396 as being updates to previous instrument images as caused by the advancement or withdrawal of the instrument through the bronchial passageways 304 shown in Barbagli FIG. 4.)
Regarding Claim 33, Barbagli discloses the features of Claim 25, as described above.
Barbagli further discloses wherein the tissue tracker tracks deformation of tissue represented in the second dataset (Barbagli paragraph [0032], “The composite representation and any virtual images generated by the composite representation may represent the static posture of a deformable anatomic region during one or more phases of motion (e.g., during an inspiration/expiration cycle of a lung).” Examiner interprets Barbagli’s display system 110 and sensor system 108 detecting changes in the shape (deformation) of a tissue being tracked, such as a moving lung that deforms/changes shape during inspiration/expiration cycles.).
Regarding Claim 34, Barbagli discloses the features of Claim 32, as described above.
Barbagli further discloses wherein updating the deformable 3D tissue reconstruction comprises adding, deleting, or fusing portions of the deformable 3D tissue reconstruction based on subsequently received image data (Barbagli FIG. 4, showing image of instrument 307; Barbagli paragraph [0049], “The instrument image may change to depict the advancement or withdrawal of the instrument through the bronchial passageways.” Examiner interprets the changes to the image of the instrument 396 as being additions, deletions, or fusions to previous images the deformable 3D tissue reconstruction to updates the images based on the advancement or withdrawal of the instrument through the bronchial passageways 304 shown in Barbagli FIG. 4.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Barbagli et al. (US PGPUB 2017/0281139 – “Barbagli”) in view of Popovic et al. (US PGPUB 2017/0007350 – “Popovic”) and Kitamura et al . (US PGPUB 2013/0023730 – “Kitamura”).
Regarding Claim 26, Barbagli discloses the features of Claim 25, as described above.
Barbagli does not explicitly disclose includes specified information between the tool tracker and the tissue tracker for improving the pose information and the deformable 3D tissue reconstruction, wherein the specified information includes surgical robotic tool manipulation data from the pose information and collision information from the deformable 3D tissue reconstruction.
Popovic teaches specified information that includes surgical robotic tool manipulation data from the pose information (Popovic FIG. 1, robot system 108; Popovic paragraph [0034], “image guidance module 106 provides image-based control of the device 102, e.g., an endoscope, preferably by controlling the robot system 108”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Popovic’s robotic control with the method disclosed by Barbagli. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a robotic system that uses both real images and virtual images for tool navigation, in order to observe otherwise hidden features of the region of interest (see Popovic paragraph [0046]).
Barbagli in view of Popovic do not explicitly teach collision information from the deformable 3D tissue reconstruction. Kitamura teaches collision information from the deformable 3D tissue reconstruction (Kitamura FIG. 12, warming message WM on structure of interest P1; see also Kitamura paragraph [0103]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Kitamura’s collision warning with the method taught by Barbagli in view of Popovic. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a robotic method that avoids damaging a region of interest within a patient (see Kitamura paragraph [0103]).
Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over Barbagli et al. (US PGPUB 2017/0281139 – “Barbagli”) in view of Popovic et al. (US PGPUB 2017/0007350 – “Popovic”), Kitamura et al . (US PGPUB 2013/0023730 – “Kitamura”), and Shats (US PGPUB 2015/0351612 – “Shats”).
Regarding Claim 27, Barbagli in view of Popovic and Kitamura teach the features of Claim 26, as described above.
Popovic further teaches wherein the surgical robotic tool manipulation data includes tensioning, cautery and dissecting data (Popovic FIG. 7, device 107 with cutting tips 410; Popovic paragraph [0030], “medical device 102 may include…an electrode”).
Popovic does not explicitly teach the electrode is for cautery.
Shats teaches a cauterizing structure (Shats FIG. 3, cauterizing structure 66).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Shat’s cauterizing structure with the method taught by Barbagli in view of Popovic and Kitamura. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that cauterizes tissue/vessels in order to stop bleeding during a surgical procedure that cuts into the tissue/polyp (see Shats paragraph [0027]).
Claims 31 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Barbagli et al. (US PGPUB 2017/0281139 – “Barbagli”) in view of Yan et al. (US PGPUB 2019/0026943 – “Yan”).
Regarding Claim 31, Barbagli discloses the features of Claim 25, as described above.
Barbagli does not explicitly disclose wherein the deformable 3D tissue reconstruction is a 3D surfel model.
Yan teaches wherein the deformable 3D tissue reconstruction is a 3D surfel model (Yan paragraph [0008], “a first data structure having a plurality of surfels representing a 3D environment, each surfel having a 3D position…an intensity…and a surface normal, the 3D position of each surfel being in a first coordinate space”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Yan’s use of surfels (surface elements) with the method disclosed by Barbagli. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that generates a virtual image using surfels that take into account intensities, rather than only positions, when generating a virtual image (see Yan paragraph [0027]).
Regarding Claim 35, Barbagli discloses the features of Claim 25, as described above.
Barbagli does not explicitly disclose wherein the deformable 3D tissue reconstruction comprises a 3D surfel model.
Yan teaches wherein the deformable 3D tissue reconstruction comprises a 3D surfel model (Yan paragraph [0008], “a first data structure having a plurality of surfels representing a 3D environment, each surfel having a 3D position…an intensity…and a surface normal, the 3D position of each surfel being in a first coordinate space”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Yan’s use of surfels (surface elements) with the method disclosed by Barbagli. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that generates a virtual image using surfels that take into account intensities, rather than only positions, when generating a virtual image (see Yan paragraph [0027]).
Response to Arguments
Applicant's arguments, see pages 7-10, filed June 5, 2026, with respect to the rejection of Claims 25-31 under 35 U.S.C. 103, have been fully considered but they are not persuasive.
On pages 7-8, Applicant asserts that the operation of tissue tracking and generating a deformable 3D tissue reconstruction is not taught by the display system 110 and sensor system 108 shown in Barbagli FIG. 1,, as described in Barbagli paragraph [0025], “a display system 110 for displaying an image or representation of the surgical site and medical instrument system(s) 104 generated by sub-systems of the sensor system 108”. Specifically, page 8 of Applicant’s response states that amended Claim 25 claims a singular “tissue tracker that performs tissue tracking” and will “generate the deformable three-dimensional tissue reconstruction”. Examiner notes that module 30 shown in Applicant’s FIG. 1 exchanges data with the synthesize tracking module 35, which generates the three-dimensional tissue reconstruction. As stated in Applicant’s paragraph [0062], “The output from the synthesize tracking module 35 is the information necessary for generating a virtual surgical environment, which is generated using the endoscopic image data as input, passing the appropriate image(s) data to the appropriate module, and finally combining the outputs of the surgical tool tracking module 25 and tissue tracking and fusion module 30 into a common coordinate frame.”. Applicant appears to be relying on a sentence in Applicant’s paragraph [0057] that states “The tissue tracking and fusion module 30 shown in FIG. 1 takes in endoscopic image data and outputs a deformable 3D reconstruction of the actual tissue in the surgical site.” However, this is contrary to FIG. 1 and the rest of Applicant’s disclosure. That is, paragraphs [0044], [0062], and [0065] of Applicant’s disclosure are clear in describing the role of tissue tracking and fusion module 30 is that of supplying outputs to synthesize tracking module 35, which generates the surgical scene reconstruction 40. The interpretation of the claimed tissue tracker being suggested by Applicant, if maintained, would result in rejections under 35 U.S.C. 112a/b and/or an objection to the figures.
On pages 8-9, Applicant asserts that Barbagli fails to disclose tissue tracking and fusion. As described above, Examiner interprets displaying time-based images of tissue as tissue tracking, since the present patent application provides no definition that conflicts with this definition. Likewise, Examiner interprets changes to the image of the instrument 396 shown in Barbagli FIG. 4 as being additions, deletions, or fusions to previous images the deformable 3D tissue reconstruction, which result in updates the images based on the advancement or withdrawal of the instrument through the bronchial passageways 304 shown in Barbagli FIG. 4.
On page 9, Applicant further raises a concern regarding a “deformable 3D tissue reconstruction”. Since the present patent application provides no alternative/unique definition of what “deformable” describes, Examiner interprets “deformable 3D tissue” as simply being tissue having shape that can be distorted/changed. Examiner also notes in the rejection of Claim 33 that Examiner interprets Barbagli’s display system 110 and sensor system 108 detecting changes in the shape (deformation) of a tissue being tracked, such as a moving lung that deforms/changes shape during inspiration/expiration cycles.
As such, the rejections of Claims 25-31, as well as new Claims 32-35, under 35 U.S.C. 102/103 are maintained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIM BOICE whose telephone number is (571)272-6565. The examiner can normally be reached Monday-Friday 9:00am - 5:00pm Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571)272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIM BOICE
Examiner
Art Unit 3795
/JAMES EDWARD BOICE/Examiner, Art Unit 3795
Prosecution Insights