Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/27/2022, 02/06/2023 and 02/26/2026 has been considered by the examiner.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/26/2026 has been entered. Claims 31, 34, 36, and 38-63 remain pending in the application.
Response to Amendment
Claims 60-63 are added, and claims 31, 34, 36, and 38-63 remain pending in the application in response to the applicant’s amendments to the rejections previously set forth in the Final Office Action mailed 12/01/2025.
Response to Arguments
Applicant’s arguments filed 02/26/2026 with respect to claim(s) 31 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Specification
The disclosure filed 08/01/2024 is objected to because it contains an embedded hyperlink and/or other form of browser-executable code. Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code (see pg. 1, 2, 5, 18, 20, 22, 23, 25, 26, 30, 32 of applicant’s specification). See MPEP § 608.01.
Claim Objections
Claims 31, 39, 44, 47, 49, 51, 53 are objected to because of the following informalities:
Although the courts have found that the use of the term “and/or” would not be indefinite, (Employers Mut. Liability Ins. Co. v. Tollefsen, 219 Wis. 434 (1935)), the board did note that the preferred way of writing the claim is through use of “at least one of A and B" in the future. Therefore, the Examiner object to the terms "and/or" in claim 53 such that it is written in accordance with the courts preferred way.
For claims 31 and 51, “projector/s” should be “projector” for clarity.
For claim 31, the examiner assumes “to generate therefrom depth maps” should be “to generate from said active portions depth maps” for clarity.
For claim 39, “the 3d image” should be “the 3D image”.
For claim 39, “the hardware processor assigns” should be “the hardware processor is further configured to assign” for clarity.
For claim 44, the examiner assumes “the tracker” should be “the at least one tracker” for clarity.
For claim 47, “a 3d camera” should be “a 3D camera”.
For claim 49, “wherein said at least one projector comprises but a single projector, said at least one image sensor comprises but a single image sensor” should be “wherein said at least one projector comprises a single projector, said at least one image sensor comprises a single image sensor” for clarity.
For claim 53, “Tubular Laminotomy” should be “tubular laminotomy”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 36, 42-43, 49 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 31 recites the limitation "the image area". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the image are” should be “the at least one 3D images” for clarity.
Claim 31 recites the limitation "the plural depth maps". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the plural depth maps” should be “the depth maps” for clarity.
Claims 31 and 62 recites the limitation " said continuous 3D depth map". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “said continuous 3D depth map” should be “said continuous map” for clarity.
Claims 34, 36, 38-46, 50, and 60-63 are dependent of claim 31, and therefore, rejected under these 112(b) rejections as well.
For claim 36, the limitation “wherein the imaging system includes at least one mechanical subsystem configured to secure the imaging system at a fixed location and orientation vs. markers that track the tube” is indefinite. It is unclear what the markers are fixed to. For the purpose of advancing prosecution, the examiner assumes the markers are fixed to the tube of the imaging system.
Claim 46 is dependent of claim 36, and therefore rejected under this 112(b) rejection as well.
Claim 38 recites the limitation " said data". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes the hardware processor is configured to generate a 3D image.
Claims 48-49 are dependent of claim 38, and therefore rejected under this 112(b) rejection as well.
Claims 42-43 recites the limitation "the tube retractor". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the tube retractor” should be “the tube” for clarity.
Claims 47 and 51 recites the limitation "the image depth maps". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the image depth maps” should be “at least one image” for clarity.
Claims 47 and 51 recites the limitation "the depth maps". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the depth maps” should be “at least one image” for clarity.
Claims 52-56 are dependent of claim 51, and therefore rejected under these 112(b) rejections as well.
For claim 49, the limitation “wherein pattern correlation and measurement of sub-pattern displacement are used for said triangulation or for depth estimation” is indefinite. It is unclear what is “sub-pattern displacement” and “pattern correlation” (i.e., what is the pattern that is being correlated, what the pattern is correlated to, what of the pattern is being displaced, and what is “sub” about a pattern). For the purpose of advancing prosecution, the examiner assumes the pattern refers to the projected light pattern of claim 31.
Claim 50 recites the limitation " the larger pre-mapped topology". There is insufficient antecedent basis for this limitation in the claim. For the purpose of advancing prosecution, the examiner assumes “the larger pre-mapped topology” should be “the larger topology” (see claim 41) for clarity.
For claim 55, “MIS TLIF (Transforaminal Interbody Fusion)” is indefinite. It is unclear what is MIS. For the purpose of advancing prosecution, the examiner reads the limitation as “transforaminal interbody fusion”.
Claim 56 is dependent of claim 55, and therefore rejected under this 112(b) rejection as well.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 31, 34, 36, 38-46, 48-49, 57, 60-62 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dehghani et al. (US 20220377217 A1, published November 24, 2022 with a priority date of August 21, 2019), hereinafter referred to as Dehghani.
Regarding claim 31, Dehghani teaches an imaging system operative in conjunction with a tube having two open ends and an inner diameter (Fig. 1-3), the system comprising:
active portions small enough to fit into the tube (Fig. 3, housing 2210 as tube; see para. 0250 – “The scope assembly may comprise an imaging unit 2220 [active portions] comprising one or more sensors (e.g., a camera, a depth sensor, or both) that is operatively coupled to (e.g., connected to) an elongated scope 2300 [tube]. The ecosystem may comprise an illumination source 2400 in optical communication with the elongated scope 2300.”); and
an electronic subsystem including a hardware processor operative to receive at least one image from said active portions and to generate therefrom depth maps comprising at least one 3D images of a scene visible via one of the tube's open ends (Fig. 3, imaging unit 2220 (hardware processor) generating depth maps via tube’s (housing 2210) open end; see para. 0249 – “The imaging processor 2500 [hardware processor] may be configured to analyze or combine data, image(s), or video(s) generated by the scope assembly 2200 and the optical adapter 2600.”; see para. 0168 – “The scope assembly of the present disclosure can allow visualization of structures or features of a target site (e.g., an internal tissue site) of a subject with depth perception (or 3D imaging) [3D images].”),
wherein said depth maps are combined to form a continuous map of the image area, yielding integration of the plural depth maps which increases accuracy of 3D depth estimation relative to using only a projected pattern for 3D depth estimation (see para. 0168 – “The scope assembly of the present disclosure can allow visualization of structures or features of a target site (e.g., an internal tissue site) of a subject with depth perception (or 3D imaging).”; see para. 0263 – “The imaging processor 2500 may use the plurality of IMU signals to transform the plurality of images in camera space to base space, thus allowing stitching of the depth map for 3D reconstruction [depth maps are combined to form a continuous map].”), and
wherein said active portions comprise:
image sensors or cameras oriented to have a partially or totally overlapping field of view (see para. 0257 – “A distal end of the imaging unit of the scope assembly is illustrated in FIG. 4A. The distal end of the imaging unit may comprise a camera and separately a depth sensor (e.g., a stereo sensor). The camera may comprise a first lens, and the depth sensor may comprise a plurality of lenses (e.g., a left lens and a right lens) that are different from the first lens of the camera.”), and
at least one structured light projector/s projecting a known pattern onto the partially or totally overlapping field of view of the image sensors (see para. 0248 – “The scope assembly 2200 may be operatively coupled to (or may enclose at least a portion of) an elongated scope 2300. The ecosystem may comprise an illumination source 2400 [light projector] in optical communication with the elongated scope 2300. The illumination source 2400 may be configured to provide one or more light beams (e.g., a combined light beam) through the elongated scope 2300, through the scope assembly 2200, and toward the target site(s) 2100.”; see para. 0163 – “The additional depth computation improvement technologies may comprise, for example, a stereoscopic depth sensor, a structured light imaging sensor, and/or a time of flight sensor. In some cases, a pattern projector could be incorporated in the sensor head. The pattern projector can be used in active stereo sensor applications to allow depth computation of targets including, e.g., featureless targets.”),
the system also comprising a tracker configured to be secured to the tube to monitor an absolute location of the tube in conjunction with a 3D camera which tracks markers on the tracker enabling transfer of 3D coordinates from a coordinate system of said image sensors to a coordinate system of said 3D camera (see para. 0258 – “As shown in FIG. 4C, the imaging unit may further comprise one or more motion sensors (e.g., an IMU) [tracker] configure to sense position, orientation, and/or sudden accelerations of the imaging unit [of tube] during the medical imaging.”; see para. 0261 – “The scope assembly 2200 may further generate one or more IMU signals from one or more motion sensors. The spatial signals may be used to provide 3D imaging comprising depth perception. The spatial signals may be combined with the IMU signals (registered) for real-time 3D reconstruction of the target site within the subject's body.”),
wherein said continuous 3D depth map has sub-mm accuracy (see para. 0314 – “The size of the filtering kernel may be adjustable based on a measured depth of one or more features in the target region in order to maintain a desired imaging resolution at said measured depth…This may permit filtering more heavily for closer depths while maintaining sub-mm resolutions [sub-mm accuracy].”).
Furthermore, regarding claim 34, Dehghani further teaches wherein at least one dimension of said active portions is smaller than the tube's inner diameter (see para. 0257 – “A distal end of the imaging unit of the scope assembly is illustrated in FIG. 4A. The distal end of the imaging unit may comprise a camera and separately a depth sensor (e.g., a stereo sensor).”).
Furthermore, regarding claim 36, Dehghani further teaches wherein the imaging system includes at least one mechanical subsystem configured to secure the imaging system at a fixed location and orientation vs. markers that track the tube (markers are fixed to the tube of the imaging system; see para. 0262 – “Referring to FIG. 6A, the scope assembly may comprise a first motion sensor 2700 (e.g., IMU) [markers] at a first hinge [mechanical subsystem] of the imaging unit arm 2225 [tube].”).
Furthermore, regarding claim 38, Dehghani further teaches wherein at least one image sensor is deployed at an offset from at least one structured light projector and wherein the offset is known to the hardware processor and is used for triangulation which generates said 3D image from said data (see para. 0148 – “The cameras or sensors 16 can be properly calibrated with each other to enable, e.g., stereoscopic depth computation, since their positions relative to one another are known and fixed [offset is known] to the same rigid plane relative to the sensor head 14.”; see para. 0225 – “The depth sensor may utilize one or more imaging techniques including, but are not limited to, stereo triangulation (e.g., stereoscopic imaging)…”).
Furthermore, regarding claim 39, Dehghani further teaches wherein the hardware processor assigns absolute coordinates of the 3D camera which tracks markers on the tracker to the 3D image of the surgical field (see para. 0262 – “Referring to FIG. 6A, the scope assembly may comprise a first motion sensor 2700 (e.g., IMU) [markers] at a first hinge [mechanical subsystem] of the imaging unit arm 2225 [tube].”).
Furthermore, regarding claim 40, Dehghani further teaches wherein the hardware processor is also configured to monitor a tool which wherein said tool is tracked by the 3D camera as said tool moves, hence absolute coordinates of said tool, by a coordinate system of the 3D camera which tracks markers on the tracker are known, and is moving (see para. 0197 – “In addition to the elongated scope, the housing unit of the scope assembly may be configured to enclose (or releasably couple to) at least one additional device. In some cases, the at least one additional device may comprise one or more endoscopic surgical tools.”).
Furthermore, regarding claim 41, Dehghani further teaches wherein the hardware processor is configured to recognize a location of the scene within a larger topology (see para. 0284 – “In some cases, the one or more depth maps may provide a surgeon with spatial information about the surgical scene to optimally maneuver a scope, robotic camera, robotic arm, or surgical tool relative to one or more features within the surgical scene.”).
Furthermore, regarding claim 42, Dehghani further teaches wherein said tool is deployed inside the tube retractor (see para. 0197 – “In addition to the elongated scope, the housing unit of the scope assembly may be configured to enclose (or releasably couple to) at least one additional device. In some cases, the at least one additional device may comprise one or more endoscopic surgical tools.”).
Furthermore, regarding claim 43, Dehghani further teaches wherein said tool is deployed outside the tube retractor (see para. 0197 – “In addition to the elongated scope, the housing unit of the scope assembly may be configured to enclose (or releasably couple to) at least one additional device. In some cases, the at least one additional device may comprise one or more endoscopic surgical tools.”).
Furthermore, regarding claim 44, Dehghani further teaches at least one tracker attached to the tube, and wherein the hardware processor is configured to use data from the tracker to be presented to a human user, thereby enabling the human user to monitor a current position of the tube (see para. 0258 – “As shown in FIG. 4C, the imaging unit may further comprise one or more motion sensors (e.g., an IMU) [tracker] configure to sense position, orientation, and/or sudden accelerations of the imaging unit [of tube] during the medical imaging.”; see para. 0261 – “The spatial signals may be combined with the IMU signals (registered) for real-time 3D reconstruction of the target site within the subject's body. Image or video stitching may be used for the real-time 3D reconstruction. The 3D imaging, the 3D reconstruction, or any 2D images/videos from a third party camera 2310 may be transmitted as an output 2610, e.g., to be displayed on a monitor or stored in a database in communication with the control unit of the scope assembly”).
Furthermore, regarding claim 45, Dehghani further teaches wherein the hardware processor is configured to superimpose the 3D image of a miniature scene onto an earlier captured image of a larger scene which is larger than, and includes, the miniature scene visible via one of the tube's open ends, thereby to generate a superimposed image, and to display the superimposed image to a human user (see para. 0332 – “The processor may be configured to use the translational and rotational motion of the scope to generate a minimap. The minimap may comprise a visual representation of which portion of a global map is currently being displayed/imaged/visualized using the scope assembly.”).
Furthermore, regarding claim 46, Dehghani further teaches wherein at least one dimension of the mechanical subsystem is larger in size than the tube's inner diameter (see para. 0262 – “Referring to FIG. 6A, the scope assembly may comprise a first motion sensor 2700 (e.g., IMU) [markers] at a first hinge [mechanical subsystem] of the imaging unit arm 2225 [tube].” The hinge (mechanical subsystem) is outside of the tube, so hinge is larger than tube’s inner diameter).
Furthermore, regarding claim 48, Dehghani further teaches wherein said at least one image sensors comprises two image sensors, and wherein said triangulation comprises stereo triangulation (see para. 0148 – “The cameras or sensors 16 can be properly calibrated with each other to enable, e.g., stereoscopic depth computation, since their positions relative to one another are known and fixed [offset is known] to the same rigid plane relative to the sensor head 14.”; see para. 0225 – “The depth sensor may utilize one or more imaging techniques including, but are not limited to, stereo triangulation (e.g., stereoscopic imaging)…”).
Furthermore, regarding claim 49, Dehghani further teaches wherein said at least one projector comprises but a single projector, said at least one image sensor comprises but a single image sensor, and wherein pattern correlation and measurement of sub-pattern displacement are used for said triangulation or for depth estimation (see para. 0163 – “In some cases, a pattern projector could be incorporated in the sensor head. The pattern projector can be used in active stereo sensor applications to allow depth computation of targets including, e.g., featureless targets.”).
Furthermore, regarding claim 57, Dehghani further teaches also comprising at least one tool tracker and wherein the hardware processor is configured to use data from the tracker to be presented to a human user, thereby enabling the human user to monitor a current position of the tool (see para. 0258 – “As shown in FIG. 4C, the imaging unit may further comprise one or more motion sensors (e.g., an IMU) [tracker] configure to sense position, orientation, and/or sudden accelerations of the imaging unit [tool] during the medical imaging.”).
Furthermore, regarding claim 60, Dehghani further teaches wherein said depth map directly provides depth values (see para. 0236 – “The depth map (or a disparity map) may be a 2D representation of the 3D visual scene (e.g., the target site within the subject's body), wherein a value of each element (e.g., pixel) of the 2D representation may be indicative of the distance between the depth sensor and the respective position of the target site within the subject's body.”).
Furthermore, regarding claim 61, Dehghani further teaches wherein said depth map comprises a map that contains depth information (see para. 0236 – “The depth map (or a disparity map) may be a 2D representation of the 3D visual scene (e.g., the target site within the subject's body), wherein a value of each element (e.g., pixel) of the 2D representation may be indicative of the distance between the depth sensor and the respective position of the target site within the subject's body.”).
Furthermore, regarding claim 62, Dehghani further teaches wherein said continuous 3D depth map has an accuracy of <0.3 mm (see para. 0314 – “The size of the filtering kernel may be adjustable based on a measured depth of one or more features in the target region in order to maintain a desired imaging resolution at said measured depth…This may permit filtering more heavily for closer depths while maintaining sub-mm resolutions.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 47, 50-51, 58-59, and 63 are rejected under 35 U.S.C. 103 as being unpatentable over Dehghani in view of Paulsen et al. (US 11406471 B1, August 9, 2022 with a priority date of October 6, 2018), hereinafter referred to as Paulsen.
Regarding claim 47, Dehghani teaches a computer program product, comprising a non-transitory tangible computer readable medium having computer readable program code embodied therein, said computer readable program code adapted to be executed to implement an imaging method operative in conjunction with a tube having two open ends, the method comprising:
receiving at least one image from active portions, of a 3d camera, which are small enough to fit into the tube and using a hardware processor to generate therefrom at least one from the image depth maps comprising 3D images of a scene (see para. 0250 – “The scope assembly may comprise an imaging unit 2220 [active portions] comprising one or more sensors (e.g., a camera, a depth sensor, or both) that is operatively coupled to (e.g., connected to) an elongated scope 2300 [tube]. The ecosystem may comprise an illumination source 2400 in optical communication with the elongated scope 2300.”; see para. 0168 – “The scope assembly of the present disclosure can allow visualization of structures or features of a target site (e.g., an internal tissue site) of a subject with depth perception (or 3D imaging).”),
wherein the depth maps are combined with each other to form a high accuracy depth map allowing sub-mm accuracy (see para. 0261 – “The spatial signals may be used to provide 3D imaging comprising depth perception. The spatial signals may be combined with the IMU signals (registered) for real-time 3D reconstruction of the target site within the subject's body. Image or video stitching may be used for the real-time 3D reconstruction.”; see para. 0314 – “The size of the filtering kernel may be adjustable based on a measured depth of one or more features in the target region in order to maintain a desired imaging resolution at said measured depth…This may permit filtering more heavily for closer depths while maintaining sub-mm resolutions.”).
Dehghani teaches registering images (see para. 0262 – “One or more motion sensors of the scope assembly may allow real-time registration and stitching of a plurality of images (e.g., 2D or 3D images) to reconstruct a 3D view of the target site. The plurality of images in camera space may be registered in base space (e.g., space with respect to the subject's body) using the one or more motion sensors.”), and imaging bone (see para. 0170 – “The scope assembly of the present disclosure may be useable for a number of medical applications, e.g., general surgery, neurosurgical procedures, orthopedic procedures, and spinal procedures.”; see para. 0271 – “Examples of the target site within the subject's body can include, but are not limited to,…bone…”), but does not explicitly teach registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT.
Whereas, Paulsen, in an analogous field of endeavor, teaches registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT (see col. 2, lines 48-50 – “The reconstructed intraoperative stereovision surfaces (iSV) are registered with preoperative CT (pCT; supine position) in a nonrigid fashion…”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified generating a 3D depth map, as disclosed in Dehghani, by also registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT, as disclosed in Paulsen. One of ordinary skill in the art would have been motivated to make this modification in order to generate updated CT images (uCT) and correct for vertebral posture and alignment changes, as taught in Paulsen (see col. 2, lines 50-52).
Furthermore, regarding claim 50, Paulsen further teaches wherein at least one pre-operative image, having a resolution, represents the larger pre-mapped topology, and wherein the pre-operative image comprises a CT image (see col. 2, lines 48-50 – “The reconstructed intraoperative stereovision surfaces (iSV) are registered with preoperative CT (pCT; supine position) in a nonrigid fashion…” where an image inherently has a resolution).
Regarding claim 51, Dehghani teaches an imaging method operative in conjunction with a tube having two open ends, the method comprising:
providing a first 3d camera with active portions small enough to fit into the tube and using an electronic subsystem including a hardware processor operative to receive at least one image from said active portions and to generate from the image depth maps (see para. 0250 – “The scope assembly may comprise an imaging unit 2220 [active portions] comprising one or more sensors (e.g., a camera, a depth sensor, or both) that is operatively coupled to (e.g., connected to) an elongated scope 2300 [tube]. The ecosystem may comprise an illumination source 2400 in optical communication with the elongated scope 2300.”; see para. 0168 – “The scope assembly of the present disclosure can allow visualization of structures or features of a target site (e.g., an internal tissue site) of a subject with depth perception (or 3D imaging).”),
wherein the depth maps are combined with each other to form a high accuracy depth map allowing sub-mm accuracy (see para. 0261 – “The spatial signals may be used to provide 3D imaging comprising depth perception. The spatial signals may be combined with the IMU signals (registered) for real-time 3D reconstruction of the target site within the subject's body. Image or video stitching may be used for the real-time 3D reconstruction.”; see para. 0314 – “The size of the filtering kernel may be adjustable based on a measured depth of one or more features in the target region in order to maintain a desired imaging resolution at said measured depth…This may permit filtering more heavily for closer depths while maintaining sub-mm resolutions.”),
wherein said active portions comprise:
image sensors oriented to have a partially or totally overlapping field of view (see para. 0257 – “A distal end of the imaging unit of the scope assembly is illustrated in FIG. 4A. The distal end of the imaging unit may comprise a camera and separately a depth sensor (e.g., a stereo sensor). The camera may comprise a first lens, and the depth sensor may comprise a plurality of lenses (e.g., a left lens and a right lens) that are different from the first lens of the camera.”), and
at least one structured light projector/s projecting a known pattern onto the partially or totally overlapping field of view of the image sensors (see para. 0248 – “The scope assembly 2200 may be operatively coupled to (or may enclose at least a portion of) an elongated scope 2300. The ecosystem may comprise an illumination source 2400 [light projector] in optical communication with the elongated scope 2300. The illumination source 2400 may be configured to provide one or more light beams (e.g., a combined light beam) through the elongated scope 2300, through the scope assembly 2200, and toward the target site(s) 2100. “),
wherein a tracker is secured to the tube, to monitor an absolute location of the tube in conjunction with a second 3D camera which tracks markers on the tracker enabling transfer of 3D coordinates from a coordinate system of said image sensors to a coordinate system of said 3D camera (see para. 0258 – “As shown in FIG. 4C, the imaging unit may further comprise one or more motion sensors (e.g., an IMU) [tracker] configure to sense position, orientation, and/or sudden accelerations of the imaging unit [of tube] during the medical imaging.”).
Dehghani teaches registering images (see para. 0262 – “One or more motion sensors of the scope assembly may allow real-time registration and stitching of a plurality of images (e.g., 2D or 3D images) to reconstruct a 3D view of the target site. The plurality of images in camera space may be registered in base space (e.g., space with respect to the subject's body) using the one or more motion sensors.”), and imaging bone (see para. 0170 – “The scope assembly of the present disclosure may be useable for a number of medical applications, e.g., general surgery, neurosurgical procedures, orthopedic procedures, and spinal procedures.”; see para. 0271 – “Examples of the target site within the subject's body can include, but are not limited to,…bone…”), but does not explicitly teach registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT.
Whereas, Paulsen, in an analogous field of endeavor, teaches registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT (see col. 2, lines 47-50 – “…acquire intraoperative profiles of the exposed spine in prone position. The reconstructed intraoperative stereovision surfaces (iSV) are registered with preoperative CT (pCT; supine position) in a nonrigid fashion…”),
wherein use of said first 3D camera inserted into plural incisions per vertebra provides said sub-mm accuracy in registration of each vertebra (see col. 13, lines 49-56 – “The system and method can also realize a low-cost hand-held or mounted iSV [intraoperative stereovision] image acquisition system that is more accurate than the current iSV platform, which is mounted to an operating microscope and used to generate the preliminary results presented here (tracking accuracy ≤0.5 mm, calibration accuracy ≤0.5 mm, reconstructed surface accuracy ≤1 mm)…”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified generating a 3D depth map, as disclosed in Dehghani, by also registering exposed bone areas from a 3D depth map to bone surface extracted from pre-op CT, as disclosed in Paulsen. One of ordinary skill in the art would have been motivated to make this modification in order to generate updated CT images (uCT) and correct for vertebral posture and alignment changes, as taught in Paulsen (see col. 2, lines 50-52).
Furthermore, regarding claim 58, Paulsen further teaches wherein the surgeon marks at least one exposed area on the pre-operational CT and the bone structure is registered to the bone structure derived from said pre-operational CT, to define a registration stage whose output relates each vertebra in the pre-operational CT to the vertebra's location and orientation as measured by said cameras during surgery (see col. 2, lines 47-50 – “…acquire intraoperative profiles of the exposed spine in prone position. The reconstructed intraoperative stereovision surfaces (iSV) are registered with preoperative CT (pCT; supine position) in a nonrigid fashion…”).
Furthermore, regarding claim 59, Paulsen further teaches wherein said at least one exposed area comprises multiple areas within the same vertebra and wherein the surgeon makes multiple incisions with multiple retractors accordingly and wherein said registration stage is performed over said multiple areas of the same vertebra, thereby to yield accurate registration of the vertebra position to the pre-operational CT (col. 9, lines 13-20 – “The system and method provides a portable iSV [intraoperative stereovision surfaces] scanner and associated iSV registration algorithms to adjust for changes in spinal alignment. The system and method can overcome factors in live surgery that interfere with patient registration performance, including bleeding and fluid pooling in the surgical field, movement due to respiration and tissue retraction, and changes in vertebral alignment during the surgical procedure.”).
Furthermore, regarding claim 63, Paulsen further teaches wherein registration of bone features, from pre-surgery imagery to imagery generated by said 3D camera which tracks markers, is accurate to a level which yields an error of only 0.5 mm (see col. 13, lines 49-56 – “The system and method can also realize a low-cost hand-held or mounted iSV [intraoperative stereovision] image acquisition system that is more accurate than the current iSV platform, which is mounted to an operating microscope and used to generate the preliminary results presented here (tracking accuracy ≤0.5 mm, calibration accuracy ≤0.5 mm, reconstructed surface accuracy ≤1 mm)…”; see col. 6, lines 56-59 – “The results of an exemplary spatial calibration are shown in FIG. 4. The reconstructed points matched well with tracked points in tracker space, and the overall error was 0.63±0.29 mm using a total of 731 points.”).
The motivation for claims 50, 58-59, 69 was shown previously in claims 47 and 51.
Claims 52-56 are rejected under 35 U.S.C. 103 as being unpatentable over Dehghani in view of Paulsen, as applied to claim 51 above, and in further view of Gildenberg (US 20080243142 A1, published October 2, 2008), hereinafter referred to as Gildenberg.
Regarding claim 52, Dehghani in view of Paulsen teaches all of the elements disclosed in claim 51 above.
Dehghani in view of Paulsen teaches a tube, but does not explicitly teach fiducial markers on a tube.
Whereas, Gildenberg, in an analogous field of endeavor, teaches wherein the tube bears fiducial markers and wherein the tube's location in space is known to said hardware processor due to said markers (see para. 0044 – “The endoscope 6 includes an endoscopic camera 7, and an instrument or resection device 8 on the end for use by the surgeon in excision of the target tissue 5. The endoscope 6 includes at least three fiducial markers…”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a tube, as disclosed in Dehghani in view of Paulsen, by having fiducial markers on the tube, as disclosed in Gildenberg. One of ordinary skill in the art would have been motivated to make this modification in order to register the position and trajectory of the endoscope for incorporation into image compilation (image overlay), as taught in Gildenberg (see para. 0044).
Furthermore, regarding claim 53, Paulsen further teaches wherein at least when an inferior edge of at least one lamina and/or ipsilateral base of spinous process are identified, the 3d camera is secured to a top end of the tube, and the inferior edge of the lamina, as viewed through the bottom end of the tube, is measured, thereby to yield a measured surface; and wherein at least one vertebra's 3D location is presented to a human user, thereby to facilitate performance of Tubular Laminotomy (see col. 12, lines 61-67 – “First, spinous and inferior articular processes can be identified by detecting local maxima in image intensity (i.e., topological height). Adjacent vertebrae can then be partitioned by tracing contours around the spinous/articular processes. 2D segmentation labels can be mapped to 3D through the one-to-one pixel-point correspondence between the depth images and 3D spine surface to fully segment the vertebrae without requiring accurate segmentation of the facet joints between two adjacent vertebra.”), and
Gildenberg further teaches wherein the 3d camera is secured to a top end of the tube and viewed through the bottom end of the tube (see para. 0025 – “Another exemplary use would involve fluoroscopic or x-ray images of a patient's spine for registration and incorporation in the defined stereotactic space allowing for the spine to be displayed in a 3D reconstructed image.”; see para. 0044 – “The endoscope 6 includes an endoscopic camera 7, and an instrument or resection device 8 on the end for use by the surgeon in excision of the target tissue 5. The endoscope 6 includes at least three fiducial markers to register the position and trajectory of the endoscope 6 for incorporation into image compilation (image overlay) 102.”).
Furthermore, regarding claim 54, Paulsen further teaches wherein said vertebra's 3D location is derived by matching the measured surface to a portion of a 3D image of at least a portion of the lamina and from the tube's known location in space (see col. 12, lines 61-67 – “First, spinous and inferior articular processes can be identified by detecting local maxima in image intensity (i.e., topological height). Adjacent vertebrae can then be partitioned by tracing contours around the spinous/articular processes. 2D segmentation labels can be mapped to 3D through the one-to-one pixel-point correspondence between the depth images and 3D spine surface to fully segment the vertebrae without requiring accurate segmentation of the facet joints between two adjacent vertebra.”).
Furthermore, regarding claim 55, Gildenberg further teaches wherein the camera is secured to a top end of the tube and measures an inferior articulating facet, as viewed through the bottom end of the tube, and wherein at least one vertebra's 3D location is presented to a human user, thereby to facilitate performance of MIS TLIF (Transforaminal Interbody Fusion) (see col. 12, lines 61-67 – “First, spinous and inferior articular processes can be identified by detecting local maxima in image intensity (i.e., topological height). Adjacent vertebrae can then be partitioned by tracing contours around the spinous/articular processes. 2D segmentation labels can be mapped to 3D through the one-to-one pixel-point correspondence between the depth images and 3D spine surface to fully segment the vertebrae without requiring accurate segmentation of the facet joints between two adjacent vertebra.” Transforaminal Interbody Fusion is a well-known spinal surgery).
Furthermore, regarding claim 56, Paulsen further teaches said vertebra's 3D location being derived from a 3D image of the facet and from the tube's known location in space (see col. 12, lines 61-67 – “First, spinous and inferior articular processes can be identified by detecting local maxima in image intensity (i.e., topological height). Adjacent vertebrae can then be partitioned by tracing contours around the spinous/articular processes. 2D segmentation labels can be mapped to 3D through the one-to-one pixel-point correspondence between the depth images and 3D spine surface to fully segment the vertebrae without requiring accurate segmentation of the facet joints between two adjacent vertebra.”).
The motivation for claims 53-56 was shown previously in claims 51 and 52.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
von Jako (US 20080177203 A1, published July 24, 2008) discloses a display may be configured to show the real time position and orientation of a model of the at least one surgical instrument or at least one implant attached to the tip or end of the at least one surgical instrument on a registered image of the patient's anatomy.
Kronman (US 20170119474 A1, published May 4, 2017) discloses determining a position of the endoscope within the patient in the endoscope's coordinate system, capturing in an image fiducial markers attached to the endoscope by an external optical tracker, transforming the captured fiducial markers from the endoscope's coordinate system to the optical tracker's coordinate system.
Hughes (US 20220175550 A1, published June 9, 2022 with a priority date of April 11, 2018) discloses the stereotactic frame is rigidly coupled to a robotic arm and the end effector is an articulating drill extension and the method allows for robotically assisted minimally invasive spinal surgery.
Michaeli et al. (US 20110301421 A1, published December 8, 2011) discloses a surgical retractor, after insertion and opening for the purpose of performing spinal minimal invasive neurosurgery.
Klusza et al. (US 20140225985 A1, published August 14, 2014) discloses projecting a dot pattern from a light source onto a plurality of points on a scene, measuring distances to the points, and digitally reconstructing an image or images of the scene, such as a 3D view of the scene. A plurality of images may also be stitched together to re-position an orientation of the view of the scene.
Elbaz et al. (US 20190269485 A1, published September 5, 2019) discloses generating a three-dimensional model of a subject's intraoral region (e.g., teeth) including both surface features and internal features.
Hladio et al. (US 20180064496 A1, published March 8, 2018) discloses detection of spatial attributes, including depth information, of the anatomy for purposes of registration or to create a 3D surface profile of the anatomy.
Katz et al. (US 20120056982 A1, published March 8, 2012) discloses depth values are obtained from each sensor by matching to the structured light pattern, and the depth values are merged to obtain a final depth map.
Yancey et al. (US 20200352686 A1, published November 12, 2020 with a priority date of May 7, 2019) discloses the scanning device can include non-contact active 3D scanners (e.g., time-of-flight, triangulation, conoscopic holography, or any other kind of non-contact active 3D scanner), hand held laser 3D scanners, structured light 3D scanners, modulated light 3D scanners, and non-contact passive 3D scanners (e.g., stereoscopic, photometric, silhouette, or any other kind of non-contact passive 3D scanner).
Hillman (US 20190167081 A1, published June 6, 2019) discloses the SCAPE microscope can be implemented as an endoscopic or laparoscopic inspection instrument.
Nebosis et al. (US 20100097616 A1, published April 22, 2010) discloses the synchronisation of the macroscopic movement of the reference mirror and the focus tracking on the one hand in combination with a two-dimensional detector on the other hand guarantees particularly simple and rapid recording of a plurality of sharp, two-dimensional image sections at different depths of the specimen and so the recording of a complete, three-dimensional set of image data with high image quality.
Wang et al. (US 20160309140 A1, published October 20, 2016) discloses a laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation.
H. Le et al, “Demonstration of a laparoscopic structured-illumination three dimensional imaging system for guiding reconstructive bowel anastomosis”, Journal of Biomedical Optics, vol. 23, no. 5, pp. 056009-1 – 10, May 2018 discloses 3-D reconstruction using SI (structured illumination) method is based on parallax and triangulation between the camera and the structured light from the projector, in relation to the sample surface profile.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nyrobi Celestine whose telephone number is 571-272-0129. The examiner can normally be reached on Monday - Thursday, 7:00AM - 5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached on 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.C./Examiner, Art Unit 3798
Prosecution Insights