Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 14 is objected to because of the following informalities:
Grammar: “a location sensor embed at a distal tip”
Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 2-7, 11, 14-19 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Ye US20190365209.
Ye discloses for claim 2, “A method for a robotic endoscopic apparatus, the method comprising:
(a) collecting a first set of sensor data using a location sensor embedded at a distal tip of the robotic endoscopic apparatus (0155-0156 describes location sensor data used for tracking the 3D location of an instrument, including in the x, y, and z directions);
(b) generating, based at least in part on the first set of sensor data, a first transformation between an orientation of the robotic endoscopic apparatus and an orientation of the location sensor (0065, 'The command module 515 receives data passed from the endoscopic tool. The type of received data depends on the corresponding type of instrument attached. For example, example received data includes sensor data (e.g., image data, EM data), robot data (e.g., endoscopic and 1DM physical motion data), control data, and/or video data.'; 0066, 'In order to track a sensor through the patient's anatomy, the EM tracking system 505 may require a process known as "registration," where the system finds the geometric transformation that aligns a single object between different coordinate systems. For instance, a specific anatomical site on a patient has two different representations in the 3D model coordinates and in the EM sensor coordinates. To be able to establish consistency and common language between these two different coordinate systems, the EM tracking system 505 needs to find the transformation that links these two representations, i.e., registration. For example, the position of the EM tracker relative to the position of the EM field generator may be mapped to a 3D coordinate system to isolate a location in a corresponding 3D model."; 0072, 'In a sensor-based approach to localization, a sensor, such as an EM tracker, may be coupled to the distal working end of the endoscopic tool to provide a real-time indication of the progression of the endoscopic tool.'; a location sensor is provided at the distal tip of the endoscope 118, and provides a first set of sensor data for tracking purposes; the first transformation is establishing the coordinates that orient the EM sensor at the distal end of the endoscope 118 within the coordinates of the surgical robotic system 100), wherein the orientation of the robotic endoscopic apparatus or the orientation of the location sensor comprises at least a z-axis (0155-0156 describes location sensor data used for tracking the 3D location of an instrument, including in the x, y, and z directions);
(c) generating a second transformation between a coordinate frame of the robotic endoscopic apparatus and a coordinate frame of a model representing an anatomical luminal network (Fig. 6A & 6B; 0020 'FIGS. 6A-6B show an example anatomical lumen and an example 3D model of the anatomical lumen') based at least in part on the first transformation and a second set of sensor data (Fig. 1A-1F & 6A-6B; 0048 'The console base 201 may include a central processing unit, a memory unit, a data bus, and associated data communication ports that are responsible for interpreting and processing signals such as camera imagery and tracking sensor data, e.g., from the endoscope 118 shown in FIG. 1.'; 0065 'The command module 515 receives data passed from the endoscopic tool. The type of received data depends on the corresponding type of instrument attached. For example, example received data includes sensor data (e.g., image data, EM data), robot data (e.g., endoscopic and 1DM physical motion data), control data, and/or video data.'; 0066 'In order to track a sensor through the patient's anatomy, the EM tracking system 505 may require a process known as "registration," where the system finds the geometric transformation that aligns a single object between different coordinate systems. For instance, a specific anatomical site on a patient has two different representations in the 3D model coordinates and in the EM sensor coordinates. To be able to establish consistency and common language between these two different coordinate systems, the EM tracking system 505 needs to find the transformation that links these two representations, i.e., registration. For example, the position of the EM tracker relative to the position of the EM field generator may be mapped to a 3D coordinate system to isolate a location in a corresponding 3D model."; 0067 'FIGS. 6A-6B show an example anatomical lumen 600 and an example 3D model 620 of the anatomical lumen, according to one embodiment); and
(d) updating, based at least in part on a third set of sensor data, the second transformation, wherein the third set of sensor data is collected using the location sensor (0075 During the EM tracking between generation of the graph 810 and generation of graph 820, additional data points 803 have been recorded by the EM tracking system but the registration has not yet been updated based on the newly collected EM data. As a result, the data points 803 in FIG. 8B are clustered along a visible path 814, but that path differs in location and orientation from the planned navigation path 802 the endoscope tip is being directed by the operator to travel along. Eventually, once sufficient data (e.g., EM data) is accumulated, compared with using only the 3D model or only the EM data, a relatively more accurate estimate can be derived from the transform needed to register the EM coordinates to those of the 3D model.'; 0076 'In some embodiments, as further data are collected, the registration transform may be updated to increase accuracy. In some cases, the data used to determine the registration transformation may be a subset of data chosen by a moving window, so that the registration may change over time, which gives the ability to account for changes in the relative coordinates of the EM and 3D models-for example, due to movement of the patient)”.
Ye discloses for claim 3, “The method of claim 2, wherein the location sensor is an electromagnetic (EM) sensor (0066 describes EM sensors and data for tracking)”.
Ye discloses for claim 4, “The method of claim 3, wherein the first transformation registers the orientation of the EM sensor to the orientation of the distal tip of the robotic endoscopic apparatus (0065-0066 as described above and 0072)”.
Ye discloses for claim 5, “The method of claim 4, wherein the first transformation registers a magnetic field generator's z-axis to the z-axis of the distal tip of the robotic endoscopic apparatus (0072-0073 describes registration between coordinate systems where 0066 describes the registration as an alignment, i.e. the axes with their corresponding axes)”.
Ye discloses for claim 6, “The method of claim 5, wherein the magnetic field generator is placed relative to the robotic endoscopic apparatus without pre-known location or orientation (the reference does not describe a pre-known location or orientation).
Ye discloses for claim 7, “The method of claim 2, wherein the first transformation is continuously updated based on newly connected sensor data from the location sensor until an accuracy threshold is met (0075-0076 describes the determination of sufficient data for transformation “may be made by threshold criteria” and as further data are collected the registration transformation may be updated to increase accuracy)”.
Ye discloses for claim 11, “The method of claim 2, wherein the coordinate frame of the model representing an anatomical luminal network is generated using a pre-operative imaging system (0053 describes preoperative CT scans)”.
Ye discloses for claim 14, “A system for a robotic endoscopic apparatus, the system comprising: a location sensor (0072 describes a location sensor is provided at the distal tip of the endoscope) embed at a distal tip to the robotic endoscopic apparatus; and one or more processors (0008, 0078 describes the processors to execute the functions of the medical robotic system) in communication with the location sensor and the robotic endoscopic apparatus and configured to execute a set of instructions to cause the system to:
(a) collect a first set of sensor data using the location sensor (0155-0156 describes location sensor data used for tracking the 3D location of an instrument, including in the x, y, and z directions);
(b) generate, based at least in part on the first set of sensor data, a first transformation between an orientation of the robotic endoscopic apparatus and an orientation of the location sensor (0065, 'The command module 515 receives data passed from the endoscopic tool. The type of received data depends on the corresponding type of instrument attached. For example, example received data includes sensor data (e.g., image data, EM data), robot data (e.g., endoscopic and 1DM physical motion data), control data, and/or video data.'; 0066, 'In order to track a sensor through the patient's anatomy, the EM tracking system 505 may require a process known as "registration," where the system finds the geometric transformation that aligns a single object between different coordinate systems. For instance, a specific anatomical site on a patient has two different representations in the 3D model coordinates and in the EM sensor coordinates. To be able to establish consistency and common language between these two different coordinate systems, the EM tracking system 505 needs to find the transformation that links these two representations, i.e., registration. For example, the position of the EM tracker relative to the position of the EM field generator may be mapped to a 3D coordinate system to isolate a location in a corresponding 3D model."; 0072, 'In a sensor-based approach to localization, a sensor, such as an EM tracker, may be coupled to the distal working end of the endoscopic tool to provide a real-time indication of the progression of the endoscopic tool.'; a location sensor is provided at the distal tip of the endoscope 118, and provides a first set of sensor data for tracking purposes; the first transformation is establishing the coordinates that orient the EM sensor at the distal end of the endoscope 118 within the coordinates of the surgical robotic system 100), wherein the orientation of the robotic endoscopic apparatus or the orientation of the location sensor comprises at least a z-axis (0155-0156 describes location sensor data used for tracking the 3D location of an instrument, including in the x, y, and z directions);
(c) generate a second transformation between a coordinate frame of the robotic endoscopic apparatus and a coordinate frame of a model representing an anatomical luminal network (Fig. 6A & 6B; 0020 'FIGS. 6A-6B show an example anatomical lumen and an example 3D model of the anatomical lumen') based at least in part on the first transformation and a second set of sensor data (Fig. 1A-1F & 6A-6B; 0048 'The console base 201 may include a central processing unit, a memory unit, a data bus, and associated data communication ports that are responsible for interpreting and processing signals such as camera imagery and tracking sensor data, e.g., from the endoscope 118 shown in FIG. 1.'; 0065 'The command module 515 receives data passed from the endoscopic tool. The type of received data depends on the corresponding type of instrument attached. For example, example received data includes sensor data (e.g., image data, EM data), robot data (e.g., endoscopic and 1DM physical motion data), control data, and/or video data.'; 0066 'In order to track a sensor through the patient's anatomy, the EM tracking system 505 may require a process known as "registration," where the system finds the geometric transformation that aligns a single object between different coordinate systems. For instance, a specific anatomical site on a patient has two different representations in the 3D model coordinates and in the EM sensor coordinates. To be able to establish consistency and common language between these two different coordinate systems, the EM tracking system 505 needs to find the transformation that links these two representations, i.e., registration. For example, the position of the EM tracker relative to the position of the EM field generator may be mapped to a 3D coordinate system to isolate a location in a corresponding 3D model."; 0067 'FIGS. 6A-6B show an example anatomical lumen 600 and an example 3D model 620 of the anatomical lumen, according to one embodiment); and
(d) update, based at least in part on a third set of sensor data, the second transformation, wherein the third set of sensor data is collected using the location sensor (0075 During the EM tracking between generation of the graph 810 and generation of graph 820, additional data points 803 have been recorded by the EM tracking system but the registration has not yet been updated based on the newly collected EM data. As a result, the data points 803 in FIG. 8B are clustered along a visible path 814, but that path differs in location and orientation from the planned navigation path 802 the endoscope tip is being directed by the operator to travel along. Eventually, once sufficient data (e.g., EM data) is accumulated, compared with using only the 3D model or only the EM data, a relatively more accurate estimate can be derived from the transform needed to register the EM coordinates to those of the 3D model.'; 0076 'In some embodiments, as further data are collected, the registration transform may be updated to increase accuracy. In some cases, the data used to determine the registration transformation may be a subset of data chosen by a moving window, so that the registration may change over time, which gives the ability to account for changes in the relative coordinates of the EM and 3D models-for example, due to movement of the patient)”.
Ye discloses for claim 15, “The system of claim 14, wherein the location sensor is an electromagnetic (EM) sensor (0066 describes EM sensors and data for tracking)”.
Ye discloses for claim 16, “The system of claim 15, wherein the first transformation registers the orientation of the EM sensor to the orientation of the distal tip of the robotic endoscopic apparatus (0065-0066 as described above and 0072)”.
Ye discloses for claim 17, “The system of claim 16, wherein the first transformation registers a magnetic field generator's z-axis to the z-axis of the distal tip of the robotic endoscopic apparatus (0072-0073 describes registration between coordinate systems where 0066 describes the registration as an alignment, i.e. the axes with their corresponding axes)”.
Ye discloses for claim 18, “The system of claim 17, wherein the magnetic field generator is placed relative to the robotic endoscopic apparatus without pre-known location or orientation (the reference does not describe a pre-known location or orientation)”.
Ye discloses for claim 19, “The system of claim 14, wherein the first transformation is continuously updated based on newly connected sensor data from the location sensor until an accuracy threshold is met (0075-0076 describes the determination of sufficient data for transformation “may be made by threshold criteria” and as further data are collected the registration transformation may be updated to increase accuracy)”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8, 12, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye as applied to claim 2 above, and further in view of Bleunven US20220296302.
Ye does not disclose for claim 8, “The method of claim 2, wherein the second set of sensor data comprises a point cloud data collected by the location sensor”. Bleunven teaches in the same field of endeavor, location data collection in the form of a point cloud (0103). Since Ye fails to disclose the nature of the sensor data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable sensor data configuration known in the art, including the one taught by Bleunven, to achieve the predictable result of capturing and storing location data.
Ye does not disclose for claim 12, “The method of claim 2, wherein the third set of sensor data is sampled from a point cloud data collected by the location sensor”. Bleunven teaches in the same field of endeavor, location data collection in the form of a point cloud (0103). Since Ye fails to disclose the nature of the sensor data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable sensor data configuration known in the art, including the one taught by Bleunven, to achieve the predictable result of capturing and storing location data.
Ye does not disclose for claim 20, “The system of claim 14, wherein the second set of sensor data comprises a point cloud data collected by the location sensor”. Bleunven teaches in the same field of endeavor, location data collection in the form of a point cloud (0103). Since Ye fails to disclose the nature of the sensor data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable sensor data configuration known in the art, including the one taught by Bleunven, to achieve the predictable result of capturing and storing location data.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye and Bleunven as applied to claim 12 above, and further in view of Mountney US20140241600.
Ye does not disclose for claim 13, “The method of claim 12, wherein the third set of sensor data is used to calculate a registration error for the second transformation and the registration error is used to trigger updating the second transformation”. Mountney teaches in the same field of endeavor, an iterative registration process until a registration error value is below a threshold (0012). Further, the registration provides refinement of the initial registration, capable of correcting for error in the initial registration, where inputs to the registration are the two views to be registered (0032). 0034 describes executing the algorithm until the registration error is below a predetermined registration threshold is met. It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Mountney into the invention of Ye in order to configure the method e.g. as claimed because it helps to reduce noise inaccuracies in the registration (0032).
Claim(s) 9, 10, 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye and Bleunven as applied to claim 8 above, and further in view of Gutmann US20210312625.
Ye does not disclose for claim 9, “The method of claim 8, wherein the second transformation is generated utilizing an iterative closest points algorithm”. Gutmann teaches in the same field of endeavor, registration utilizing an iterative closest point algorithm and a coherent point drift algorithm (0134 “In some examples, one or more relevant algorithms may be used by the software to calculate the transformation matrix, such as an iterative point cloud or iterative closest point (ICP) algorithm and/or a coherent point drift (CPD) algorithm”). Since Ye fails to disclose the nature of the transformation algorithm, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable transformation algorithm known in the art, including the one taught by Gutmann, to achieve the predictable result of configuring a transformation algorithm.
Modified Ye discloses for claim 10, “The method of claim 9, wherein the iterative closest points algorithm is modified by a coherent point drift algorithm to reduce noise (0134 describes using both algorithms)”.
Ye does not disclose for claim 21, “The system of claim 14, wherein the second transformation is generated utilizing an iterative closest points algorithm”. Gutmann teaches in the same field of endeavor, registration utilizing an iterative closest point algorithm and a coherent point drift algorithm (0134 “In some examples, one or more relevant algorithms may be used by the software to calculate the transformation matrix, such as an iterative point cloud or iterative closest point (ICP) algorithm and/or a coherent point drift (CPD) algorithm”). Since Ye fails to disclose the nature of the transformation algorithm, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable transformation algorithm known in the art, including the one taught by Gutmann, to achieve the predictable result of configuring a transformation algorithm.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANH TUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795
6/6/26
Prosecution Insights