DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 16, 2026 has been entered.
This Office action is responsive to an amendment filed February 16, 2025. Claims 1-2, 4-12 & 14-22 are pending. Claims 3 & 13 have been canceled. Claims 1, 11 & 21-22 have been amended.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 4, 11, 14 & 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Witt (US 2015/0215614) in view of Craig et al. (US 2020/0100647) (“Craig” hereinafter).
In regards to claim 1, Witt discloses a system comprising:
a) a first image capture device configured to capture a first image of an interior of a cavity of a patient (see at least par 0042-0044);
PNG
media_image1.png
561
469
media_image1.png
Greyscale
b) a second image capture device configured to capture a second image of the interior of the cavity of the patient (see at least par 0042-0044);
c) a two-dimensional display (14, 40) (see at least fig. 1 and par 0037);
d) a processor (13, 41) (see at least par 0041-0043, 0046 & 0049-0052);
e) a laparoscope (see par 0036) housing the first image capture device (see at least figs. 3A-B and par 0036 & 0042-0044); and
g) a non-transitory computer readable medium storing instructions operable to, when executed by the processor (13, 41), cause the processor (13, 41) to perform a set of acts comprising:
i) display, on the two-dimensional display (14, 40), a two-dimensional image of the interior of the cavity of the patient (see at least figs. 4-6 & 9 and par 0071-0072);
ii) determine a three-dimensional distance between a plurality of points (i.e., any one of points 90, 91, 92, 93) on the two-dimensional image (see at least fig. 9); and
PNG
media_image2.png
339
410
media_image2.png
Greyscale
iii) display, on the two-dimensional display (14, 40), the three-dimensional distance (see at least figs. 4-6 & 9 and par 0071-0072).
Witt discloses a system, as described above, that fails to explicitly teach a system comprising: f) an inertial measurement unit ("IMU") coupled to the laparoscope and b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) generate a plurality of representations of the interior of the cavity of the patient, wherein each of the plurality of representations corresponds to a time from a plurality of times; and ii) for each time from the plurality of times, determine a pose corresponding to that time, based on: A) movement information captured from the IMU at that time; and B) the representation from the plurality of representations corresponding to that time, wherein the representation from the plurality of representations corresponding to that time overlaps with another representation from the plurality of representations.
However, Craig teaches that it is known to provide a system comprising: f) an inertial measurement unit ("IMU") (25, 27) (see par 0017) coupled to the laparoscope (see par 0003) and b) the non-transitory computer readable medium (see par 0039) further stores instructions operable to, when executed by the processor 16, cause the processor 16 to: i) generate a plurality of representations (e.g., image frames) of the interior of the cavity of the patient, wherein each of the plurality of representations (e.g., image frames) corresponds to a time from a plurality of times; and ii) for each time from the plurality of times, determine a pose corresponding to that time (e.g., a pose associated with an image frame) (see at least par 0017), based on: A) movement information captured from the IMU at that time (see at least par 0017); and B) the representation (e.g., image frame) from the plurality of representations (e.g., image frames) corresponding to that time, wherein the representation from the plurality of representations (e.g., image frames) corresponding to that time overlaps with another representation from the plurality of representations (see at least par 0017 & 0029-0030).
Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt comprising: f) an inertial measurement unit ("IMU") coupled to the laparoscope and b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) generate a plurality of representations of the interior of the cavity of the patient, wherein each of the plurality of representations corresponds to a time from a plurality of times; and ii) for each time from the plurality of times, determine a pose corresponding to that time, based on: A) movement information captured from the IMU at that time; and B) the representation from the plurality of representations corresponding to that time, wherein the representation from the plurality of representations corresponding to that time overlaps with another representation from the plurality of representations as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as removing navigation errors by associating each image frame or a sampling of image frames with a discrete distal IMU pose data point to create a discrete image pose datum (see at least par 0017 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 4, Witt discloses the system of claim 1, wherein the instructions stored on the non-transitory computer readable medium are operable to, when executed by the processor (13, 41), cause the processor (13, 41) to:
a) create a depth (i.e., color or chrominance saturation) map based on the first image and the second image; and
b) determine, using the depth (i.e., color or chrominance saturation) map, the three-dimensional distance between the plurality of points (i.e., any one of points 90, 91, 92, 93) on the two-dimensional image as a distance between a first point (i.e., any one of points 90, 91, 92, 93) and a second point (i.e., any other of points 90, 91, 92, 93) from the plurality of points(i.e., any one of points 90, 91, 92, 93) along a surface of the interior of the cavity of the patient on a plane comprising a straight line connecting the first point and the second point (see at least par 0047, 0061-0062, 0069-0076 & 0079-0084).
In regards to claim 11, Witt discloses a method comprising:
a) capturing a first image of an interior of a cavity of a patient and a second image of the interior of the cavity of the patient (see at least par 0042-0044);
PNG
media_image1.png
561
469
media_image1.png
Greyscale
d) displaying, on a two-dimensional display (14, 40), a two-dimensional image of the interior of the cavity of the patient (see at least figs. 4-6 & 9 and par 0071-0072);
c) determining a three-dimensional distance between a plurality of points (i.e., any one of points 90, 91, 92, 93) on the two-dimensional image (see at least fig. 9); and
PNG
media_image2.png
339
410
media_image2.png
Greyscale
d) displaying, on the two-dimensional display (14, 40), the three-dimensional distance (see at least figs. 4-6 & 9 and par 0071-0072).
Witt discloses the method, as described above, that fails to explicitly teach a method wherein the method further comprises: a) generating a plurality of representations of the interior of the cavity of the patient, wherein each of the plurality of representations corresponds to a time from a plurality of times; and b) for each time from the plurality of times, determining a pose corresponding to that time, based on: i) movement information captured from an inertial measurement unit ("IMU") coupled to a laparoscope housing first and second image capture devices used to capture the first and second images of the interior of the cavity of the patient at that time; and ii) the representation from the plurality of representations corresponding to that time, wherein the representation from the plurality of representations corresponding to that time overlaps with another representation from the plurality of representations.
However, Craig teaches that it is known to provide a method wherein the method further comprises: a) generating a plurality of representations (e.g., image frames) of the interior of the cavity of the patient, wherein each (e.g., image frame) of the plurality of representations (e.g., image frames) corresponds to a time from a plurality of times; and b) for each time from the plurality of times, determining a pose corresponding to that time, based on: i) movement information captured from an inertial measurement unit ("IMU") (25, 27) coupled to a laparoscope housing first and second image capture devices (e.g. stereo cameras, see par 0027 & 0030) used to capture the first and second images of the interior of the cavity of the patient at that time; and ii) the representation (e.g., image frame) from the plurality of representations (e.g., image frames) corresponding to that time, wherein the representation (e.g., image frame) from the plurality of representations (e.g., image frames) corresponding to that time overlaps with another representation (e.g., image frame) from the plurality of representations (e.g., image frames) (see at least par 0017 & 0029-0030).
Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt method wherein the method further comprises: a) generating a plurality of representations of the interior of the cavity of the patient, wherein each of the plurality of representations corresponds to a time from a plurality of times; and b) for each time from the plurality of times, determining a pose corresponding to that time, based on: i) movement information captured from an inertial measurement unit ("IMU") coupled to a laparoscope housing first and second image capture devices used to capture the first and second images of the interior of the cavity of the patient at that time; and ii) the representation from the plurality of representations corresponding to that time, wherein the representation from the plurality of representations corresponding to that time overlaps with another representation from the plurality of representations. as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as removing navigation errors by associating each image frame or a sampling of image frames with a discrete distal IMU pose data point to create a discrete image pose datum (see at least par 0017 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 14, Witt discloses the method of claim 11, wherein the method further comprises:
a) creating a depth (i.e., color or chrominance saturation) map based on the first image and the second image; and
b) determining, using the depth (i.e., color or chrominance saturation) map, the three-dimensional distance between the plurality of points (i.e., any one of points 90, 91, 92, 93) as a distance between a first point and a second point from the plurality of points (i.e., any one of points 90, 91, 92, 93) along a surface of the interior of the cavity of the patient on a plane comprising a straight line connecting the first point (i.e., any one of points 90, 91, 92, 93) and the second point (i.e., any other of points 90, 91, 92, 93) (see at least par 0047, 0061-0062, 0069-0076 & 0079-0084).
In regards to claim 21, while Witt discloses a system comprising b) determining the three-dimensional distance between the plurality of points on the two dimensional image comprises using a most recently determined pose to determine the three-dimensional distance based on topography of a scene depicted in the two-dimensional image of the interior of the cavity of the patient (see at least fig. 9), Witt discloses the system of claim 1 that fails to disclose a system, wherein: a) for each of one or more times from the plurality of times: i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time; ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time; and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time; and C) averaging the representation pose change and the IMU pose change. However, Craig teaches that it is known to provide a system wherein: a) for each of one or more times from the plurality of times: i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time (see par 0017); ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time (see par 0017); and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation (e.g., image frame) depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time (see at least par 0017, 0027 & 0029-0030). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt wherein: a) for each of one or more times from the plurality of times: i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time; ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time; and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as removing navigation errors by associating each image frame or a sampling of image frames with a discrete distal IMU pose data point to create a discrete image pose datum (see at least par 0017 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
Moreover, Witt as modified by Craig discloses a system, as described above, that fails to explicitly teach a system comprising averaging the representation pose change and the IMU pose change. However, the Office takes Official notice that it is known to take at least three position measurements to provide trend analysis thereof, see at least par 0110 of Robaina et al., US 2021/0177370), it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig comprising averaging the representation pose change and the IMU pose change as claimed in order to provide trend analysis of the position measurements.
In regards to claim 22, while Witt discloses a method comprising b) determining the three-dimensional distance between the plurality of points on the two dimensional image comprises using a most recently determined pose to determine the three-dimensional distance based on topography of a scene depicted in the two-dimensional image of the interior of the cavity of the patient (see at least fig. 9), Witt discloses the method of claim 11 that fails to disclose a method, wherein: a) for each of one or more times from the plurality of times: i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time; ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time; and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time; and C) averaging the representation pose change and the IMU pose change. However, Craig teaches that it is known to provide a method , wherein: a) for each of one or more times from the plurality of times (see at least par 0017): i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time (see at least par 0017); ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time (see at least par 0017); and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time (see at least par 0017, 0027 & 0029-0030). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt wherein: a) for each of one or more times from the plurality of times: i) the representation corresponding to that time is a frame depicting the interior of the cavity of the patient at that time; ii) the pose corresponding to that time is a position and orientation of the laparoscope at that time; and iii) determining the pose corresponding to that time comprises: A) determining a representation pose change, wherein the representation pose change is a change in a position and orientation of the laparoscope at that time relative to a previous time from the plurality of times, based on the representation depicting the interior of the cavity of the patient at that time and a representation depicting the interior of the cavity of the patient at the previous time; B) determining an IMU pose change, wherein the IMU pose change is a change in a position and orientation of the laparoscope at that time relative to the previous time from the plurality of times, based on the movement information captured by the IMU at that time as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as removing navigation errors by associating each image frame or a sampling of image frames with a discrete distal IMU pose data point to create a discrete image pose datum (see at least par 0017 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
Moreover, Witt as modified by Craig discloses a system, as described above, that fails to explicitly teach a method comprising averaging the representation pose change and the IMU pose change. However, the Office takes Official notice that it is known to take at least three position measurements to provide trend analysis thereof, see at least par 0110 of Robaina et al., US 2021/0177370), it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt as modified by Craig comprising averaging the representation pose change and the IMU pose change as claimed in order to provide trend analysis of the position measurements.
Claim(s) 2 & 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Witt (US 2015/0215614) in view of Craig (US 2020/0100647) further in view of Ogawa (US 6,937,268).
In regards to claim 2, while Witt discloses the system of claim 1, wherein:
a) the two-dimensional display (14, 40) is a touch display (see at least par 0069); b) the plurality of points on the two-dimensional image comprises a first point (i.e., any one of points 90, 91, 92, 93) and a second point (i.e., any other of points 90, 91, 92, 93); and c) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor (13, 41), cause the processor (13, 41) to: i) receive the plurality of points (i.e., any one of points 90, 91, 92, 93) on the two-dimensional image as user input provided by touching the touch display (see at least fig. 9 and par 0070-0071); ii) determine a three-dimensional location of each of the plurality of points (i.e., any one of points 90, 91, 92, 93) using triangulation based on the first image and the second image (see at least fig. 9 and par 0071); iii) determine the three-dimensional distance 95 as the length of a straight line connecting, and having endpoints (90, 91) at, the first point 90 and the second point 91 in the three-dimensional space (see at least fig. 9); iv) identify a cutting (i.e., incision) plane comprising the straight line connecting, and having endpoints at, the first point 90 and the second point 91 (see at least par 0072); and v) display, on the two-dimensional display (14, 40) simultaneously with the two- dimensional image of the interior of the cavity of the patient, a depiction of the straight line connecting, and having endpoints at, the first point and the second point in three-dimensional space on an image (see at least fig. 9), Witt as modified by Craig discloses a system, as described above, that fails to explicitly teach a system with the first point and the second point in three-dimensional space on an image selected from a group consisting of: A) a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane; and B) a three-dimensional reconstruction of the interior of the cavity of the patient which highlights a surface of the interior of the cavity of the patient intersecting the cutting plane. However, Ogawa teaches that it is known to provide a system with the first point and the second point in three-dimensional space on an image of a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane (see at least abstract, figs. 1-3, 5-7 & 15-16 and col. 4, lines 32-37, col. 6, lines 2-65, col. 7, lines 26-52). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig with the first point and the second point in three-dimensional space on an image of a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane as taught by Ogawa since such a modification would amount to applying a known technique (i.e., as taught by Ogawa) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as making it easy to designate a cutting plane whose measurement is desired (see col. 1, lines 61-65 of Ogawa)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 12, while Witt discloses the method of claim 11, wherein: a) the two-dimensional display (14, 40) is a touch display (see at least par 0069); b) the plurality of points on the two-dimensional image comprises a first point (i.e., any one of points 90, 91, 92, 93) and a second point (i.e., any other one of points 90, 91, 92, 93); and c) the method further comprises: i) receiving the plurality of points (i.e., any one of points 90, 91, 92, 93) on the two-dimensional image as user input provided by touching the touch display (see at least fig. 9 and par 0070-0071); ii) determining a three-dimensional location of each of the plurality of points using triangulation based on the first image and the second image (see at least fig. 9 and par 0071); iii) determining the three-dimensional distance as the length of a straight line connecting, and having endpoints at, the first point (i.e., any one of points 90, 91, 92, 93) and the second point (i.e., any other one of points 90, 91, 92, 93) in the three-dimensional space (see at least fig. 9); iv) identifying a cutting (i.e., incision) plane comprising the straight line connecting, and having endpoints at, the first point and the second point (see at least par 0072); and v) displaying, on the two-dimensional display (14, 40) simultaneously with the two- dimensional image of the interior of the cavity of the patient, a depiction of the straight line connecting, and having endpoints at, the first point (i.e., any one of points 90, 91, 92, 93) and the second point (i.e., any other one of points 90, 91, 92, 93) in three-dimensional space on an image (see at least fig. 9), Witt as modified by Craig discloses a method, as described above, that fails to explicitly teach a method with the first point and the second point in three-dimensional space on an image selected from a group consisting of: A) a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane; and B) a three-dimensional reconstruction of the interior of the cavity of the patient which highlights a surface of the interior of the cavity of the patient intersecting the cutting plane. However, Ogawa teaches that it is known to provide a method with the first point and the second point in three-dimensional space on an image of a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane (see at least abstract, figs. 1-3, 5-7 & 15-16 and col. 4, lines 32-37, col. 6, lines 2-65, col. 7, lines 26-52). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt as modified by Craig with the first point and the second point in three-dimensional space on an image of a cross-sectional view of a portion of the interior of the cavity of the patient taken on the cutting plane as taught by Ogawa since such a modification would amount to applying a known technique (i.e., as taught by Ogawa) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as making it easy to designate a cutting plane whose measurement is desired (see col. 1, lines 61-65 of Ogawa)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
Claim(s) 5 & 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Witt (US 2015/0215614) in view of Craig (US 2020/0100647) further in view of Boctor et al. (US 2015/0031990) (“Boctor” hereinafter).
In regards to claim 5, Witt as modified by Craig discloses the system of claim 1, as described above, that fails to explicitly a system wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) highlight the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlight the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient. However, Boctor teaches that it is known to provide a system wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) highlight (i.e., blue) the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlight (i.e., green) the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient (see at least fig. 8 and par 0069). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) highlight the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlight the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient as taught by Boctor since such a modification would amount to applying a known technique (i.e., as taught by Boctor) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as facilitating identification of the different tissue types within safe margins--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 15, Witt as modified by Craig discloses the method of claim 11, as described above, that fails to explicitly teach a method wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the method further comprises: i) highlighting the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlighting the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient. However, Boctor teaches that it is known to provide a method wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the method further comprises: i) highlighting the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlighting the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient (see at least fig. 8 and par 0069). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig wherein: a) the plurality of points on the two-dimensional image comprise: i) a point on a border of an anatomical object in the interior of the cavity of the patient; and ii) a point on an outer edge of a resection margin surrounding the anatomical object in the interior of the cavity of the patient; and b) the method further comprises: i) highlighting the border of the anatomical object in the two-dimensional image of the interior of the cavity of the patient; and ii) highlighting the outer edge of the resection margin surrounding the anatomical object in the two-dimensional image of the interior of the cavity of the patient as taught by Boctor since such a modification would amount to applying a known technique (i.e., as taught by Boctor) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as facilitating identification of the different tissue types within safe margins--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
Claim(s) 6-10 & 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Witt (US 2015/0215614) in view of Craig (US 2020/0100647) further in view of Zhao et al. (US 2013/0046137) (“Zhao” hereinafter).
In regards to claim 6, Witt as modified by Craig discloses a system that fails to explicitly teach a system comprising b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: iii) generate a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations. However, Zhao teaches that it is known to provide a system comprising b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: iii) generate a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations (see at least par 0017-0018, 0060, 0077, 0080 & 0090). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig comprising b) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: iii) generate a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as providing information on surgical instruments that are not in the field of view of the laparoscope as well as information on the anatomy of the patient that is not in the field of view of laparoscope so as to provide peripheral vision for the surgeon (see at least par 0080 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 7, Witt as modified by Zhao discloses the system of claim 6, that fails to explicitly teach a system wherein for each time from the plurality of times, determining the pose corresponding to that time comprises: a) determining a set of potential poses by, for each potential pose from the set of potential poses, determining that potential pose based on: i) the representation corresponding to that time, and ii) a different representation from the plurality of representations corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set of potential poses; and c) determining the pose corresponding to that time based on: i) the representation pose corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU at that time. However, Craig teaches that it is known to provide a system wherein for each time from the plurality of times, determining the pose 162 corresponding to that time comprises: a) determining a set 168 of potential poses by, for each potential pose from the set 168 of potential poses, determining that potential pose based on: i) the representation 162 corresponding to that time, and ii) a different representation from the plurality of representations 164 corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set 168 of potential poses; and c) determining the pose 162 corresponding to that time based on: i) the representation pose 162 corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU 80 at that time (see at least par 0076-0078). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Zhao wherein for each time from the plurality of times, determining the pose corresponding to that time comprises: a) determining a set of potential poses by, for each potential pose from the set of potential poses, determining that potential pose based on: i) the representation corresponding to that time, and ii) a different representation from the plurality of representations corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set of potential poses; and c) determining the pose corresponding to that time based on: i) the representation pose corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU at that time as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as providing giving the viewer a perception of a three-dimensional rendering showing the movement of the location the catheter moving in three-dimensions, i.e., left, right, up, down, forward, backward, etc. superposed onto a live 3D image of the interior of a patient’s body that changes over time as different images are captured over by the catheter system (see at least par 0072-0073 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 8, Witt discloses the system of claim 6, wherein:
a) each representation from the plurality of representations is a three-dimensional representation of the interior of the cavity of the patient; and
b) the non-transitory computer readable medium stores instructions operable to, when executed by the processor (13, 41), cause the processor (13, 41), to generate each representation from the plurality of representations based on a pair of images captured by the first and second image capture devices at the corresponding time for that representation (see at least par 0007-0010).
In regards to claim 9, Witt as modified by Craig discloses the system of claim 1, as described above, that fails to explicitly teach a system wherein: a) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) analyze the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determine, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool, and the second point associated with the second surgical tool. However, Zhao teaches that it is known to provide a system wherein: a) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) analyze the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determine, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool 1610L, and the second point associated with the second surgical tool 1610R (see at least abstract, figs. 5B & 16 and par 0031-0032, 0054-0055, 0061, 0164, 0193, 0195 & 0207). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig wherein: a) the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to: i) analyze the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determine, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool, and the second point associated with the second surgical tool as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as using the surgical tools to take measurements or determine a surface profile (see par 0207 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 10, Witt as modified by Craig discloses the system of claim 9 that fails to explicitly teach system wherein the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to display, on the two-dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool and the second point associated with the second surgical tool. However, Zhao teaches that it is known to provide a system wherein the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to display, on the two-dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool 1610L and the second point associated with the second surgical tool 1610R (see at least abstract, figs. 5B & 16 and par 0031-0032, 0054-0055, 0061, 0164, 0193, 0195 & 0207). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Craig wherein the non-transitory computer readable medium further stores instructions operable to, when executed by the processor, cause the processor to display, on the two-dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool and the second point associated with the second surgical tool as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as using the surgical tools to take measurements or determine a surface profile (see par 0207 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 16, Witt as modified by Craig discloses a method, as described above, that fails to explicitly teach a method comprising c) generating a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations. However, Zhao teaches that it is known to provide a method comprising c) generating a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations (see at least par 0017-0018, 0060, 0077, 0080 & 0090). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt as modified by Craig comprising c) generating a panoramic view of the interior of the cavity of the patient based on combining the plurality of representations of the interior of the cavity of the patient using the poses corresponding to the times corresponding to those representations as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as providing information on surgical instruments that are not in the field of view of the laparoscope as well as information on the anatomy of the patient that is not in the field of view of laparoscope so as to provide peripheral vision for the surgeon (see at least par 0080 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 17, Witt as modified by Zhao discloses the method of claim 16, that fails to explicitly teach a method wherein for each time from the plurality of times, determining the pose corresponding to that time comprises: a) determining a set of potential poses by, for each potential pose from the set of potential poses, determining that potential pose based on: i) the representation corresponding to that time, and ii) a different representation from the plurality of representations corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set of potential poses; and c) determining the pose corresponding to that time based on: i) the representation pose corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU at that time. However, Craig teaches that it is known to provide a method wherein for each time from the plurality of times, determining the pose corresponding to that time comprises: a) determining a set of potential poses by, for each potential pose from the set 168 of potential poses, determining that potential pose based on: i) the representation corresponding to that time, and ii) a different representation from the plurality of representations corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set of potential poses; and c) determining the pose corresponding to that time based on: i) the representation pose corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU (25, 27) at that time (see at least par 0017, 0027 & 0029-0030). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the system of Witt as modified by Zhao wherein for each time from the plurality of times, determining the pose corresponding to that time comprises: a) determining a set of potential poses by, for each potential pose from the set of potential poses, determining that potential pose based on: i) the representation corresponding to that time, and ii) a different representation from the plurality of representations corresponding to a previous time; b) determining a representation pose corresponding to that time based on the set of potential poses; and c) determining the pose corresponding to that time based on: i) the representation pose corresponding to that time; and ii) an IMU pose based on the movement information captured from the IMU at that time as taught by Craig since such a modification would amount to applying a known technique (i.e., the IMU/camera navigation/tracking technique of Craig) to a known device (i.e., as taught Witt) ready for improvement to achieve a predictable result such as removing navigation errors by associating each image frame or a sampling of image frames with a discrete distal IMU pose data point to create a discrete image pose datum (see at least par 0017 of Craig)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 18, Witt discloses the method of claim 16, wherein the method comprises generating each representation from the plurality of representations based on a pair of images captured by the first and second image capture devices at the corresponding time for that representation (see at least par 0007-0010).
In regards to claim 19, Witt as modified by Craig discloses the method of claim 11, that fails to explicitly a method wherein: a) the method further comprises: i) analyzing the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determining, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool, and the second point associated with the second surgical tool. However, Zhao teaches that it is known to provide a method wherein: a) the method further comprises: i) analyzing the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determining, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool 1610L, and the second point associated with the second surgical tool 1610R (see at least abstract, figs. 5B & 16 and par 0031-0032, 0054-0055, 0061, 0164, 0193, 0195 & 0207). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt as modified by Craig wherein: a) the method further comprises: i) analyzing the first image and the second image to identify a first surgical tool and a second surgical tool; and ii) determining, based on the analyzing, a first point associated with the first surgical tool and a second point associated with the second surgical tool; and b) the plurality of points on the two-dimensional image comprises the first point associated with the first surgical tool, and the second point associated with the second surgical tool as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as using the surgical tools to take measurements or determine a surface profile (see par 0207 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
In regards to claim 20, Witt discloses the method of claim 19, that fails to explicitly teach a method wherein the method further comprises displaying, on the two- dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool and the second point associated with the second surgical tool. However, Zhao teaches that it is known to provide a method wherein the method further comprises displaying, on the two-dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool 1610L and the second point associated with the second surgical tool 1610R (see at least abstract, figs. 5B & 16 and par 0031-0032, 0054-0055, 0061, 0164, 0193, 0195 & 0207). Therefore, it would have been obvious to one of ordinary skill in the art at the time Applicant’s invention was filed to provide the method of Witt wherein the method further comprises displaying, on the two- dimensional display simultaneously with the two-dimensional image of the interior of the cavity of the patient, a depiction of a straight line connecting, and having endpoints at, the first point associated with the first surgical tool and the second point associated with the second surgical tool as taught by Zhao since such a modification would amount to applying a known technique (i.e., as taught by Zhao) to a known device (i.e., as taught by Witt) ready for improvement to achieve a predictable result such as using the surgical tools to take measurements or determine a surface profile (see par 0207 of Zhao)--See KSR, 550 U.S. at___, 82 USPQ2d at 1396 (See MPEP § 214 3 for a discussion of the rationale(s) listed above. See also MPEP § 2144 - §2144.09 for additional guidance regarding support for obviousness determinations).
Response to Arguments
Applicant’s arguments, filed February 16, 2026, with respect to the 101 rejections have been fully considered and are persuasive as the claim(s) integrates the abstract idea into a practical application. The 101 rejections have been withdrawn.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENE T TOWA whose telephone number is (313)446-6655. The examiner can normally be reached Mon-Fri, 9:00 AM-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason M. Sims can be reached on 571-272-7540. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RENE T TOWA/ Primary Examiner, Art Unit 3791
Prosecution Insights