DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is in response to Application No. 18/630,226 filed 04/09/2024. Claims 1-21 are pending.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 08/25/2025 have been entered and considered. Initialed copies of the PTO-1449 by the examiner are attached.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim(s) 15, 17-18 and 20 recite limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claims 15, 18 and 20; recite the limitation, “controller configured to …,”.
Claim 17; recites the limitation, “light source configured to …,”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claim(s) 15, 17-18 and 20:
“Controller” (Fig. 8, #70. Paragraph [0031]- “the system 10 includes the controllers 70 denoted as an image processing and tracking controller 70. As previously discussed, the controller 70 may incorporate one or more processors 142, including one or more graphic processors (GPUs) that may be implemented for a feature extraction module 142a or one or more computational processing units (CPUs) that may provide for pose-calculation and alignment module 142b” thus, have sufficient structure or material wherein is any kind of processor including GPUs or CPUs).
“light source” (Fig. 8, #36. Paragraphs [0015] and [0016]- “At least one light source 36 of the surgical imaging system 10 may output the excitation emission 24 in the NIR spectral range” and “In various implementations, the light sources 36 may comprise one or more emitters that may illuminate the surgical field 14 or the field of view 52 of the imagers 50 with light in the range of wavelengths necessary to excite the luminescent material of the fiducial markers 22 as the excitation emission 24” thus, have sufficient structure or material wherein is any kind of emitter capable of emitting light in wavelength to excite luminescent material).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 15 is rejected under 35 U.S.C. 102(a)(2) as being anticipated by Umbdenstock et al. (US 20240104747 A1, hereinafter referred to as “Umbdenstock”).
Regarding claim 15, Umbdenstock teaches a surgical imaging system comprising (“tracking system for assigning marker identities to markers of a tracker” Umbdenstock, abstract; “The present disclosure generally relates to the tracking of objects, for example in the field of computer-assisted surgery” Umbdenstock, [0002]):
at least one camera configured to capture first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view (“The tracking system 10 further comprises a camera system 21. The camera system 21 has an IR camera module configured to capture first image data in at least the IR spectrum and an optical camera module configured to capture second image data in at least the visible spectrum. The IR camera module and the optical camera module are configured or configurable to assume substantially the same viewing angle” Umbdenstock, [0050]); and
at least one controller configured to receive the first image data and the second image data, the at least one controller configured to (“The device 11 for assigning marker identities is configured to receive the first and second image data from the camera system 21 (e.g., via a wired or wireless connection). The device 11 may be a computer or a part of a computer or be at least partially provided by a remote desktop or cloud computing resources” Umbdenstock, [0051]):
identify at least one feature of an object (“The tracker 12 further comprises a reference 14 detectable in a visible light spectrum (e.g., a light spectrum visible to the human eye, for example a light spectrum between wavelengths of 400 nm and 700 nm)” Umbdenstock, [0055]; wherein the feature is that of reference 14 of Fig. 2A and the object is that of tracker 12 of Fig. 2A) associated with a medical procedure in the first image data (“The present disclosure generally relates to the tracking of objects, for example in the field of computer-assisted surgery” Umbdenstock, [0002]);
identify a plurality of fiducial markers affixed to the object in the second image data (“The markers 13 of the tracker are detectable in an infrared spectrum (e.g., between wavelengths of 700 nm and 1 μmm, such as between 800 nm to 900 nm). At least one or more of the markers 13 may be passive markers configured to reflect light in the infrared spectrum. To this end, at least one or more of the markers 13 may each comprise a reflecting material, foil, or dye” Umbdenstock, [0054]), wherein the fiducial markers are present in the field of view with the at least one feature of the object (“In some cases, the first and second image data 34, 36 were captured by the camera system 21 under at least essentially the same viewing angle” Umbdenstock, [0067], Fig. 2A);
calculate a spatial relationship between the at least one object and the plurality of fiducial markers (“FIG. 2B shows an example of a pre-determined relationship 16 between the markers 13 and the reference 14, that may be used with the tracker 12 shown in FIG. 2A. Such pre-determined relationship may be indicative of the “full” geometry of the tracker 12 with its markers 13. As will be appreciated, for each dedicated tracker 12, the markers 13 (in FIG. 2B: indicated as small circles with center points indicative of the marker positions) are arranged in a pre-determined relationship relative to the reference 14 (in FIG. 2B: the inner and outer contour of the frame-shaped substrate 15), and the pre-determined relationship may be indicative of marker identities. The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B. The pre-determined relationship 16, and in particular the model, may be provided in the form of positions of the markers 13 and the reference 14 in a coordinate system or in the form of positions of the markers 13 and the reference 14 relative to each other” Umbdenstock, [0056]; wherein the feature is that of reference 14 of Fig. 2B, the object is that of tracker 12 of Fig. 2B and the fiducial markers are that of markers 13 of Figs. 2A-2B); and
determine at least one of a feature position and a feature orientation of the at least one feature in response to fiducial positions of the fiducial markers based on the spatial relationship (“the pre-determined relationship 16 may define a relative position of an individual position of a marker identity 18 relative to at least one individual feature of the reference 14, e.g., in form of a vector or matrix. Step 110 may then comprise identifying the at least one individual feature in the second image data 34 and determining the position of the individual marker identity 18 based on the at least one individual feature and the relative position.” Umbdenstock, [0072]; “the camera system 21 comprises stereo imaging capabilities in the infrared light spectrum, but also in other cases, additional geometric information can be evaluated in step 110. Such additional geometric information can relate to pre-defined geometric details of the substrate 15 on which the markers 13 are arranged... Based on an orientation of the substrate 15 determined in the visible light spectrum, it can thus be determined which of the markers 13 should be visible by the “left” and the “right” imaging sensor perspective of the camera system 21” Umbdenstock, [0089]; “FIG. 2D shows a third example of the tracker 12. In this example, the reference 14 comprises four reference printings 28A, 28B, 28C, 28D. The reference printings 28A, 28B, 28C, 28D may be printed on the substrate 15 or other surfaces of the tracker 12. The reference printings 28A, 28B, 28C, 28D depicted in FIG. 2D are optically distinguishable in order to reduce ambiguity in regards to orientation of the tracker 12” Umbdenstock, [0060]; additionally see, Umbdenstock [0003]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 10-11 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Umbdenstock et al. (US 20240104747 A1) in view of Azizian et al. (US 20190282307 A1, hereinafter referred to as “Azizian”).
Regarding claim 1, Umbdenstock teaches a method for tracking one or more features of an object in a surgical field, the method comprising (“A method... for assigning marker identities to markers of a tracker” Umbdenstock, abstract; “The present disclosure generally relates to the tracking of objects, for example in the field of computer-assisted surgery” Umbdenstock, [0002]):
capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view (“The tracking system 10 further comprises a camera system 21. The camera system 21 has an IR camera module configured to capture first image data in at least the IR spectrum and an optical camera module configured to capture second image data in at least the visible spectrum. The IR camera module and the optical camera module are configured or configurable to assume substantially the same viewing angle” Umbdenstock, [0050]);
identifying the at least one feature of the object (“The tracker 12 further comprises a reference 14 detectable in a visible light spectrum (e.g., a light spectrum visible to the human eye, for example a light spectrum between wavelengths of 400 nm and 700 nm)” Umbdenstock, [0055]; wherein the feature is that of reference 14 of Fig. 2A and the object is that of tracker 12 of Fig. 2A) associated with a medical procedure in the first image data in the field of view (“The present disclosure generally relates to the tracking of objects, for example in the field of computer-assisted surgery” Umbdenstock, [0002]; “In some cases, the first and second image data 34, 36 were captured by the camera system 21 under at least essentially the same viewing angle” Umbdenstock, [0067]);
identifying the plurality of fiducial markers in the second image data (“The markers 13 of the tracker are detectable in an infrared spectrum (e.g., between wavelengths of 700 nm and 1 μmm, such as between 800 nm to 900 nm). At least one or more of the markers 13 may be passive markers configured to reflect light in the infrared spectrum. To this end, at least one or more of the markers 13 may each comprise a reflecting material, foil, or dye” Umbdenstock, [0054]) in the field of view (“In some cases, the first and second image data 34, 36 were captured by the camera system 21 under at least essentially the same viewing angle” Umbdenstock, [0067], Fig. 2A);
calculating a spatial relationship between the at least one feature and the plurality of fiducial markers (“FIG. 2B shows an example of a pre-determined relationship 16 between the markers 13 and the reference 14, that may be used with the tracker 12 shown in FIG. 2A. Such pre-determined relationship may be indicative of the “full” geometry of the tracker 12 with its markers 13. As will be appreciated, for each dedicated tracker 12, the markers 13 (in FIG. 2B: indicated as small circles with center points indicative of the marker positions) are arranged in a pre-determined relationship relative to the reference 14 (in FIG. 2B: the inner and outer contour of the frame-shaped substrate 15), and the pre-determined relationship may be indicative of marker identities. The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B. The pre-determined relationship 16, and in particular the model, may be provided in the form of positions of the markers 13 and the reference 14 in a coordinate system or in the form of positions of the markers 13 and the reference 14 relative to each other” Umbdenstock, [0056]; wherein the feature is that of reference 14 of Fig. 2B, the object is that of tracker 12 of Fig. 2B and the fiducial markers are that of markers 13 of Figs. 2A-2B); and
determining at least one of a feature position and a feature orientation of the object in response to fiducial positions of the fiducial markers based on the spatial relationship (“the pre-determined relationship 16 may define a relative position of an individual position of a marker identity 18 relative to at least one individual feature of the reference 14, e.g., in form of a vector or matrix. Step 110 may then comprise identifying the at least one individual feature in the second image data 34 and determining the position of the individual marker identity 18 based on the at least one individual feature and the relative position.” Umbdenstock, [0072]; “the camera system 21 comprises stereo imaging capabilities in the infrared light spectrum, but also in other cases, additional geometric information can be evaluated in step 110. Such additional geometric information can relate to pre-defined geometric details of the substrate 15 on which the markers 13 are arranged... Based on an orientation of the substrate 15 determined in the visible light spectrum, it can thus be determined which of the markers 13 should be visible by the “left” and the “right” imaging sensor perspective of the camera system 21” Umbdenstock, [0089]; “FIG. 2D shows a third example of the tracker 12. In this example, the reference 14 comprises four reference printings 28A, 28B, 28C, 28D. The reference printings 28A, 28B, 28C, 28D may be printed on the substrate 15 or other surfaces of the tracker 12. The reference printings 28A, 28B, 28C, 28D depicted in FIG. 2D are optically distinguishable in order to reduce ambiguity in regards to orientation of the tracker 12” Umbdenstock, [0060]; additionally see, Umbdenstock [0003]).
Umbdenstock fails to explicitly teach affixing a plurality of fiducial markers in fixed fiducial positions on the object.
However, Azizian teaches affixing a plurality of fiducial markers in fixed fiducial positions on the object (“deploying fluorescent material on at least one of an organ under surgery” Azizian, [0084]; “the surgeon to mark the anastomosis site by applying fluorescent markers via, for instance miniature clips, spray, paint, tapes, and the like, which can be detected and tracked using the dual-spectrum imaging technology” Azizian, [0085]; “dyes, or markers, can be applied to the targets that are internal as well as targets that are external. The fluorescent dye can be attached to the target by clips, staples, and glue or can be applied by painting or spraying. The dye can also be injected to the tissue to mark specific points or can be injected through blood. The dye can be selected in order to bind with specific types of cells to mark specific structures such, for instance, tumors” Azizian, [0094]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Azizian of having affixing a plurality of fiducial markers in fixed fiducial positions on the object.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein affixing a plurality of fiducial markers in fixed fiducial positions on the object.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 2, Umbdenstock in view of Azizian teach the method according claim 1, Umbdenstock fails to explicitly teach wherein the plurality of fiducial markers is formed by applying a liquid compound to the object, wherein the liquid compound cures forming the plurality of fiducial markers rigidly affixed to the object.
However, Azizian teaches wherein the plurality of fiducial markers is formed by applying a liquid compound to the object, wherein the liquid compound cures forming the plurality of fiducial markers rigidly affixed to the object (“In addition to visibility, as shown in FIG. 12, it is important for a surgical marker to contain a compound to bind to target tissue. When placed in the surgical field, it may be desirable for a marker to remain at its initial placement... the marker includes a cyanoacrylate to achieve sufficient adhesion for the marker to remain fixed to tissue for the entirety of a surgical procedure. Cyanoacrylate has the advantage of being an FDA approved compound for some medical procedures, such as wound closure on superficial tissue. Additionally, cyanoacrylate will polymerize when in contact with water, allowing the marker to harden in the applied shape as it fuses to tissue” Azizian, [0137]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Azizian of having wherein the plurality of fiducial markers is formed by applying a liquid compound to the object, wherein the liquid compound cures forming the plurality of fiducial markers rigidly affixed to the object.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the plurality of fiducial markers is formed by applying a liquid compound to the object, wherein the liquid compound cures forming the plurality of fiducial markers rigidly affixed to the object.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 3, Umbdenstock in view of Azizian teach the method according claim 1, Umbdenstock further teaches wherein the fiducial markers comprise a luminescent material that reflects a nonvisible wavelength in the second wavelength range (“The markers 13 of the tracker are detectable in an infrared spectrum (e.g., between wavelengths of 700 nm and 1 μmm, such as between 800 nm to 900 nm). At least one or more of the markers 13 may be passive markers configured to reflect light in the infrared spectrum. To this end, at least one or more of the markers 13 may each comprise a reflecting material, foil, or dye” Umbdenstock, [0054]).
Regarding claim 4, Umbdenstock in view of Azizian teach the method according claim 3, Umbdenstock fails to explicitly teach wherein the luminescent material absorbs and fluoresces light in the near infrared spectrum.
However, Azizian teaches wherein the luminescent material absorbs and fluoresces light in the near infrared spectrum (“In step S1001, fluorescent dye markers are deployed to a surgical field. The dye markers can be deployed, for example, by spraying, painting, attachment, tissue injection, intravenous injection, and the like. In step S1002, the surgical field can be illuminated with fluorescent and visible light sources... The light captured by the camera may include light emitted within both the visible range and IR range” Azizian, [0120]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Azizian of having wherein the luminescent material absorbs and fluoresces light in the near infrared spectrum.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the luminescent material absorbs and fluoresces light in the near infrared spectrum.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 5, Umbdenstock in view of Azizian teach the method according claim 3, Umbdenstock fails to explicitly teach wherein the luminescent material is a fluorescent dye comprising at least one of indocyanine green, brilliant blue green, infracyanine green, bromophenol blue, and iFluor® 790.
However, Azizian teaches wherein the luminescent material is a fluorescent dye comprising at least one of indocyanine green, brilliant blue green, infracyanine green, bromophenol blue, and iFluor® 790 (“a marker can be made from, at least, near infrared (NIR) fluorophore such as indocyanine green (ICG)” Azizian, [0136]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Azizian of having wherein the luminescent material is a fluorescent dye comprising at least one of indocyanine green, brilliant blue green, infracyanine green, bromophenol blue, and iFluor® 790.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the luminescent material is a fluorescent dye comprising at least one of indocyanine green, brilliant blue green, infracyanine green, bromophenol blue, and iFluor® 790.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 6, Umbdenstock in view of Azizian teach the method according claim 1, Umbdenstock fails to explicitly teach further comprising: illuminating the field of view with light in the second wavelength range.
However, Azizian teaches illuminating the field of view with light in the second wavelength range (“light source 102 is a narrow-band source of light (e.g. in the near infrared range) that is chosen according to the excitation wavelength of the fluorescent material” Azizian, [0105]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Azizian of having further comprising: illuminating the field of view with light in the second wavelength range.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker further comprising: illuminating the field of view with light in the second wavelength range.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 7, Umbdenstock in view of Azizian teach the method according claim 1, Umbdenstock further teaches accessing surgical data comprising at least one of an image of the object, a model of the object, a scan of the object, or a surgical plan associated with the object (“The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B. The pre-determined relationship 16, and in particular the model, may be provided in the form of positions of the markers 13 and the reference 14 in a coordinate system or in the form of positions of the markers 13 and the reference 14 relative to each other” Umbdenstock, [0056]; [0071]).
Regarding claim 8, Umbdenstock in view of Azizian teach the method according claim 7, Umbdenstock further teaches displaying the surgical data in alignment with the feature position and feature orientation of the object based in response to the fiducial position and fiducial orientation of the at least one fiducial marker based on the spatial relationship (“Step 110 may, for example, comprise determining the positions associated with the marker identities 18 in the second image data 36 based on the reference 14 determined in the second image data 36 and the pre-determined relationship 16. To this end, features of the reference 14 of the pre-determined relationship 16 may be aligned with features of the reference determined in the second image data 36. For example, the pre-determined relationship may be a virtual model that is applied or modified until the reference 14 of the virtual model aligns with the reference 14 of the second image data 36. The pre-determined relationship is also indicative of positions of the marker identities 18. During the alignment, the positions of the marker identities move with the virtual model. After the alignment process fulfils a certain abortion criterion, the positions associated with the marker identities 18 of pre-determined relationship are located at (or at least close to) the positions of the markers 13 (associated with the identities 18) in the second image data 36” Umbdenstock, [0071]; [0072]).
Regarding claim 10, Umbdenstock in view of Azizian teach the method according claim 7, Umbdenstock further teaches wherein the surgical data is a three-dimensional representation of the object (“The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B” Umbdenstock, [0056]).
Regarding claim 11, Umbdenstock in view of Azizian teach the method according claim 1, Umbdenstock further teaches wherein the object comprises at least one of: a surgical tool or instrument; and a patient and the at least one feature comprises an anatomic feature visible in the first image data (“The tracker 12 is attached to a patient 32. In the example of FIG. 1, the tracker 12 has a rectangular, frame-like configuration and is sticked (e.g., adhesively attached or glued) to a back of the patient 32. In other cases, the tracker 12 has a different configuration and is attached (e.g., clamped) to a patient bone” Umbdenstock, [0049]).
Regarding claim 16, Umbdenstock in view of Azizian teach the system according claim 15, Umbdenstock fail to explicitly teach wherein a fiducial marker is applied as a liquid compound forming a cement, the cement comprising a luminescent material that emits a nonvisible wavelength in the second wavelength range.
However, Azizian explicitly teaches wherein a fiducial marker is applied as a liquid compound forming a cement (“In addition to visibility, as shown in FIG. 12, it is important for a surgical marker to contain a compound to bind to target tissue. When placed in the surgical field, it may be desirable for a marker to remain at its initial placement... the marker includes a cyanoacrylate to achieve sufficient adhesion for the marker to remain fixed to tissue for the entirety of a surgical procedure. Cyanoacrylate has the advantage of being an FDA approved compound for some medical procedures, such as wound closure on superficial tissue. Additionally, cyanoacrylate will polymerize when in contact with water, allowing the marker to harden in the applied shape as it fuses to tissue” Azizian, [0137]), the cement comprising a luminescent material that emits a nonvisible wavelength in the second wavelength range (“The markers 13 of the tracker are detectable in an infrared spectrum (e.g., between wavelengths of 700 nm and 1 μmm, such as between 800 nm to 900 nm). At least one or more of the markers 13 may be passive markers configured to reflect light in the infrared spectrum. To this end, at least one or more of the markers 13 may each comprise a reflecting material, foil, or dye” Umbdenstock, [0054]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a surgical imaging system comprising: at least one camera configured to capture first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; and at least one controller configured to receive the first image data and the second image data, the at least one controller configured to: identify at least one feature of an object associated with a medical procedure in the first image data; identify a plurality of fiducial markers affixed to the object in the second image data, wherein the fiducial markers are present in the field of view with the at least one feature of the object, with the teachings of Azizian of having wherein a fiducial marker is applied as a liquid compound forming a cement, the cement comprising a luminescent material that emits a nonvisible wavelength in the second wavelength range.
Wherein having Umbdenstock’s method for assigning marker identities to markers of a tracker wherein a fiducial marker is applied as a liquid compound forming a cement, the cement comprising a luminescent material that emits a nonvisible wavelength in the second wavelength range.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 17, Umbdenstock in view of Azizian teach the system according claim 15, Umbdenstock fail to explicitly teach further comprising: at least one light source configured to emit light in the second wavelength range.
However, Azizian explicitly teaches at least one light source configured to emit light in the second wavelength range (“light source 102 is a narrow-band source of light (e.g. in the near infrared range) that is chosen according to the excitation wavelength of the fluorescent material” Azizian, [0105]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a surgical imaging system comprising: at least one camera configured to capture first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; and at least one controller configured to receive the first image data and the second image data, the at least one controller configured to: identify at least one feature of an object associated with a medical procedure in the first image data; identify a plurality of fiducial markers affixed to the object in the second image data, wherein the fiducial markers are present in the field of view with the at least one feature of the object, with the teachings of Azizian of having further comprising: at least one light source configured to emit light in the second wavelength range.
Wherein having Umbdenstock’s method for assigning marker identities to markers of a tracker further comprising: at least one light source configured to emit light in the second wavelength range.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Regarding claim 18, Umbdenstock in view of Azizian teach the system according claim 15, Umbdenstock further teaches wherein the controller is further configured to: access surgical data comprising at least one of an image of the object, a model of the object, a scan of the object, or a surgical plan associated with the object (“The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B. The pre-determined relationship 16, and in particular the model, may be provided in the form of positions of the markers 13 and the reference 14 in a coordinate system or in the form of positions of the markers 13 and the reference 14 relative to each other” Umbdenstock, [0056]; [0071]).
Regarding claim 19, Umbdenstock in view of Azizian teach the system according claim 18, Umbdenstock further teaches wherein the surgical data is three-dimensional scan data of the object (“The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B” Umbdenstock, [0056]).
Regarding claim 20, Umbdenstock in view of Azizian teach the system according claim 18, Umbdenstock further teaches wherein the controller is further configured to: register the surgical data in alignment with the object in response to the fiducial positions based on the spatial relationship (“Step 110 may, for example, comprise determining the positions associated with the marker identities 18 in the second image data 36 based on the reference 14 determined in the second image data 36 and the pre-determined relationship 16. To this end, features of the reference 14 of the pre-determined relationship 16 may be aligned with features of the reference determined in the second image data 36. For example, the pre-determined relationship may be a virtual model that is applied or modified until the reference 14 of the virtual model aligns with the reference 14 of the second image data 36. The pre-determined relationship is also indicative of positions of the marker identities 18. During the alignment, the positions of the marker identities move with the virtual model. After the alignment process fulfils a certain abortion criterion, the positions associated with the marker identities 18 of pre-determined relationship are located at (or at least close to) the positions of the markers 13 (associated with the identities 18) in the second image data 36” Umbdenstock, [0071]; [0072]).
Regarding claim 21, Umbdenstock teaches a method for tracking one or more features of an object in a surgical field, the method comprising (“A method... for assigning marker identities to markers of a tracker” Umbdenstock, abstract; “The present disclosure generally relates to the tracking of objects, for example in the field of computer-assisted surgery” Umbdenstock, [0002]):
capturing first image data in a visible spectrum in a field of view of a camera (“The tracking system 10 further comprises a camera system 21. The camera system 21 has an IR camera module configured to capture first image data in at least the IR spectrum and an optical camera module configured to capture second image data in at least the visible spectrum. The IR camera module and the optical camera module are configured or configurable to assume substantially the same viewing angle” Umbdenstock, [0050]);
capturing second image data in the infrared or near infrared spectrum in the field of view (See above, Umbdenstock, [0050]);
identifying the at least one feature of the object in the first image data (“The tracker 12 further comprises a reference 14 detectable in a visible light spectrum (e.g., a light spectrum visible to the human eye, for example a light spectrum between wavelengths of 400 nm and 700 nm)” Umbdenstock, [0055]; wherein the feature is that of reference 14 of Fig. 2A and the object is that of tracker 12 of Fig. 2A);
identifying the plurality of fiducial markers in the second image data (“The markers 13 of the tracker are detectable in an infrared spectrum (e.g., between wavelengths of 700 nm and 1 μmm, such as between 800 nm to 900 nm). At least one or more of the markers 13 may be passive markers configured to reflect light in the infrared spectrum. To this end, at least one or more of the markers 13 may each comprise a reflecting material, foil, or dye” Umbdenstock, [0054]);
calculating a spatial relationship between the at least one feature and the plurality of fiducial markers (“FIG. 2B shows an example of a pre-determined relationship 16 between the markers 13 and the reference 14, that may be used with the tracker 12 shown in FIG. 2A. Such pre-determined relationship may be indicative of the “full” geometry of the tracker 12 with its markers 13. As will be appreciated, for each dedicated tracker 12, the markers 13 (in FIG. 2B: indicated as small circles with center points indicative of the marker positions) are arranged in a pre-determined relationship relative to the reference 14 (in FIG. 2B: the inner and outer contour of the frame-shaped substrate 15), and the pre-determined relationship may be indicative of marker identities. The pre-determined relationship 16 may be provided in the form a three-dimensional or two-dimensional virtual model of the tracker 12, as indicated in FIG. 2B. The pre-determined relationship 16, and in particular the model, may be provided in the form of positions of the markers 13 and the reference 14 in a coordinate system or in the form of positions of the markers 13 and the reference 14 relative to each other” Umbdenstock, [0056]; wherein the feature is that of reference 14 of Fig. 2B, the object is that of tracker 12 of Fig. 2B and the fiducial markers are that of markers 13 of Figs. 2A-2B); and
tracking at least one of a feature position and a feature orientation of the object in response to the fiducial positions of the fiducial markers based on the spatial relationship (“the pre-determined relationship 16 may define a relative position of an individual position of a marker identity 18 relative to at least one individual feature of the reference 14, e.g., in form of a vector or matrix. Step 110 may then comprise identifying the at least one individual feature in the second image data 34 and determining the position of the individual marker identity 18 based on the at least one individual feature and the relative position.” Umbdenstock, [0072]; “the camera system 21 comprises stereo imaging capabilities in the infrared light spectrum, but also in other cases, additional geometric information can be evaluated in step 110. Such additional geometric information can relate to pre-defined geometric details of the substrate 15 on which the markers 13 are arranged... Based on an orientation of the substrate 15 determined in the visible light spectrum, it can thus be determined which of the markers 13 should be visible by the “left” and the “right” imaging sensor perspective of the camera system 21” Umbdenstock, [0089]; “FIG. 2D shows a third example of the tracker 12. In this example, the reference 14 comprises four reference printings 28A, 28B, 28C, 28D. The reference printings 28A, 28B, 28C, 28D may be printed on the substrate 15 or other surfaces of the tracker 12. The reference printings 28A, 28B, 28C, 28D depicted in FIG. 2D are optically distinguishable in order to reduce ambiguity in regards to orientation of the tracker 12” Umbdenstock, [0060]; additionally see, Umbdenstock [0003]).
Umbdenstock fails to explicitly teach applying a liquid compound comprising a luminescent material that reflects an infrared or near infrared spectrum, wherein the liquid compound cures on the object in the surgical field forming a plurality of fiducial markers on the object in fiducial positions.
However, Azizian teaches applying a liquid compound comprising a luminescent material that reflects an infrared or near infrared spectrum, wherein the liquid compound cures on the object in the surgical field forming a plurality of fiducial markers on the object in fiducial positions (“deploying fluorescent material on at least one of an organ under surgery” Azizian, [0084]; “the surgeon to mark the anastomosis site by applying fluorescent markers via, for instance miniature clips, spray, paint, tapes, and the like, which can be detected and tracked using the dual-spectrum imaging technology” Azizian, [0085]; “dyes, or markers, can be applied to the targets that are internal as well as targets that are external. The fluorescent dye can be attached to the target by clips, staples, and glue or can be applied by painting or spraying. The dye can also be injected to the tissue to mark specific points or can be injected through blood. The dye can be selected in order to bind with specific types of cells to mark specific structures such, for instance, tumors” Azizian, [0094]; “In addition to visibility, as shown in FIG. 12, it is important for a surgical marker to contain a compound to bind to target tissue. When placed in the surgical field, it may be desirable for a marker to remain at its initial placement... the marker includes a cyanoacrylate to achieve sufficient adhesion for the marker to remain fixed to tissue for the entirety of a surgical procedure. Cyanoacrylate has the advantage of being an FDA approved compound for some medical procedures, such as wound closure on superficial tissue. Additionally, cyanoacrylate will polymerize when in contact with water, allowing the marker to harden in the applied shape as it fuses to tissue” Azizian, [0137])
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data in a visible spectrum in a field of view of a camera; capturing second image data in the infrared or near infrared spectrum in the field of view; identifying the at least one feature of the object in the first image data; identifying the plurality of fiducial markers in the second image data, with the teachings of Azizian of having applying a liquid compound comprising a luminescent material that reflects an infrared or near infrared spectrum, wherein the liquid compound cures on the object in the surgical field forming a plurality of fiducial markers on the object in fiducial positions.
Wherein having Umbdenstock’s method for assigning marker identities to markers of a tracker wherein applying a liquid compound comprising a luminescent material that reflects an infrared or near infrared spectrum, wherein the liquid compound cures on the object in the surgical field forming a plurality of fiducial markers on the object in fiducial positions.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Azizian are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Azizian deploys fluorescent markers to detect and track objects to help guide surgeons for manual and robotic surgery procedures. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Azizian et al. (US 20190282307 A1), paragraph [0086].
Claim(s) 9 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Umbdenstock et al. (US 20240104747 A1) in view of Azizian et al. (US 20190282307 A1) and in further view of Chiou (US 12053247 B1, hereinafter referred to as “Chiou”).
Regarding claim 9, Umbdenstock in view of Azizian teaches the method of claim 8, Umbdenstock in view of Azizian fail to explicitly teach wherein the surgical data is shown aligned with the at least one object and superimposed over a scene corresponding to the field of view on a display screen of a head-mounted display.
However, Chiou explicitly teaches wherein the surgical data is shown aligned with the at least one object and superimposed over a scene corresponding to the field of view on a display screen of a head-mounted display (“head mounted display is configured to display the virtual surgical guide superimposed onto a physical joint based at least in part on coordinates of a predetermined position of the virtual surgical guide, and the virtual surgical guide is configured to align the physical surgical guide or a physical saw blade with the virtual surgical guide to guide a bone cut of the joint” Chiou, Col. 16-17 ln 64-3).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Chiou of having wherein the surgical data is shown aligned with the at least one object and superimposed over a scene corresponding to the field of view on a display screen of a head-mounted display.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the surgical data is shown aligned with the at least one object and superimposed over a scene corresponding to the field of view on a display screen of a head-mounted display.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Chiou are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Chiou uses a head mounted display to generates a 3D stereoscopic view of a virtual guide indicating a placement of predetermined coordinates for position and orientation for aligning a physical surgical tool based on markers. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Chiou (US 12053247 B1), Col 6-7 ln 62-15.
Regarding claim 12, Umbdenstock in view of Azizian teaches the method of claim 11, Umbdenstock in view of Azizian fail to explicitly teach wherein the at least one fiducial marker is affixed to a bony portion of the patient presented in the second image data.
However, Chiou explicitly teaches wherein the at least one fiducial marker is affixed to a bony portion of the patient presented in the second image data (“at least one marker, e.g. attached to the patient, for example a bony structure in a spine, knee, hip, shoulder or ankle join” Chiou, Col 23 ln 44-46; wherein the second image data contains “One or more infrared and/or RF markers, active and/or passive markers can be applied to the anatomic structure or near the anatomic structure tracking the coordinates and/or the position and/or orientation of the anatomic structure” Chiou, Col 35 ln 52-56).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Chiou of having wherein the at least one fiducial marker is affixed to a bony portion of the patient presented in the second image data.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the at least one fiducial marker is affixed to a bony portion of the patient presented in the second image data.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Chiou are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Chiou uses a head mounted display to generates a 3D stereoscopic view of a virtual guide indicating a placement of predetermined coordinates for position and orientation for aligning a physical surgical tool based on markers. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Chiou (US 12053247 B1), Col 6-7 ln 62-15.
Regarding claim 13, Umbdenstock in view of Azizian teaches the method of claim 11, Umbdenstock in view of Azizian fail to explicitly teach wherein the bony portion comprises an exposed bone surface accessed via an open procedure.
However, Chiou explicitly teaches wherein the bony portion comprises an exposed bone surface accessed via an open procedure (“non-visualized portions for one or more devices or implants or implant components or surgical instruments or surgical tools, and/or one or more of a predetermined tissue change or alteration, can be superimposed onto and/or aligned with the corresponding anatomic structure, e.g. a target tissue or an exposed joint surface, e.g. an exposed articular surface” Chiou, Col 69 ln 36-43).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Chiou of having wherein the bony portion comprises an exposed bone surface accessed via an open procedure.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the bony portion comprises an exposed bone surface accessed via an open procedure.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Chiou are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Chiou uses a head mounted display to generates a 3D stereoscopic view of a virtual guide indicating a placement of predetermined coordinates for position and orientation for aligning a physical surgical tool based on markers. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Chiou (US 12053247 B1), Col 6-7 ln 62-15.
Regarding claim 14, Umbdenstock in view of Azizian teaches the method of claim 11, Umbdenstock in view of Azizian fail to explicitly teach wherein the bony portion comprises an enclosed bone surface positioned within a patient cavity accessed via a closed surgical procedure.
However, Chiou explicitly teaches wherein the bony portion comprises an enclosed bone surface positioned within a patient cavity accessed via a closed surgical procedure (“Tracking of the one or more image capture system, video capture system, image or video capture system, image and/or video capture system, and/or optical imaging system can, for example, be advantageous when the one or more 3D scanners are integrated into or attached to an instrument, an arthroscope, an endoscope, and/or when they are located internal to any structures, e.g. inside a joint or a cavity” Chiou, Col 28 ln 15-21).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Umbdenstock in view of Azizian of having a method for tracking one or more features of an object in a surgical field, the method comprising: capturing first image data at a first wavelength range in a visible spectrum and second image data at a second wavelength range in an infrared or near infrared spectrum in a field of view; identifying the at least one feature of the object associated with a medical procedure in the first image data in the field of view; identifying the plurality of fiducial markers in the second image data in the field of view, with the teachings of Chiou of having wherein the bony portion comprises an enclosed bone surface positioned within a patient cavity accessed via a closed surgical procedure.
Wherein Umbdenstock’s method for assigning marker identities to markers of a tracker wherein the bony portion comprises an enclosed bone surface positioned within a patient cavity accessed via a closed surgical procedure.
The motivation behind the modification would have been to obtain a method for assigning marker identities to markers of a tracker that tracks objects, since both Umbdenstock and Chiou are methods that track objects using markers and features during a surgery. Wherein Umbdenstock tracking method determines the position and orientation of the tracker and consequently of the surgical object in the surgical procedure, while Chiou uses a head mounted display to generates a 3D stereoscopic view of a virtual guide indicating a placement of predetermined coordinates for position and orientation for aligning a physical surgical tool based on markers. Please see Umbdenstock et al. (US 20240104747 A1), paragraph [0003] and Chiou (US 12053247 B1), Col 6-7 ln 62-15.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dumpe et al. (WO 2025076046 A1) – certain features of objects can be tracked by registering physical properties of the object and associating them with objects that can be tracked, such as fiducial marks fixed to a tool or bone... a three- dimensional surface can be mapped for that bone that is associated with a position and orientation relative to the frame of reference of that fiducial mark. By optically tracking the position and orientation (pose) of the fiducial mark associated with that bone, a model of that surface can be tracked with an environment through extrapolation.
McCombs et al. (US 20100249581 A1) – uses position and/or orientation tracking sensors such as infrared sensors acting stereoscopically or other sensors acting in conjunction with reference structures or reference transmitters to track positions of body parts, surgery-related items; by sensing the position of reference structures or transmitters (fiducials/markers), can display or otherwise output useful data relating to predicted or actual position and orientation of body parts, surgically related items, implants, and virtual constructs for use in navigation, assessment, and otherwise performing surgery or other operations.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMANUEL SILVA-AVINA whose telephone number is (571)270-0729. The examiner can normally be reached Monday - Friday 11 AM - 8 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMMANUEL SILVA-AVINA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights