DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on October 31, 2024 is in compliance with 37 CFR 1.97 and 1.98 and therefore has been considered by the examiner and placed in the file.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The BRI of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
In the following, some of the terms in the claims have been given BRIs in light of the specification. These BRIs are used for purposes of searching for prior art and examining the claims, but cannot be incorporated into the claims. Should Applicant believe that different interpretations are appropriate, Applicant should point to the portions of the specification that clearly support a different interpretation.
The BRI of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Image capture unit in claims 1 and 15.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The image capture unit recited in claim 1 is interpreted based on the present specification as one or more cameras and equivalents.
The image capture unit recited in claim 15 is interpreted based on the present specification as one or more cameras and one or more processors of, or in communication with, the camera(s) for performing the steps recited in claim 15, and equivalents.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-8, 10, 12-13, 15-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publ. Appl. No. 2019/0350658 A1 to Yang et al. (hereinafter referred to as “Yang”) in view of U.S. Pat. No. 11,880,992 B2 to Tang et al. (hereinafter referred to as “Tang”).
Regarding claim 1, Yang discloses a system comprising:
a projector configured to project a pattern of dots within a tracking volume (the BRI for this limitation is that some type of projection device projects a structured light pattern comprising dots onto a surface; in Yang, structured light is used by the surface detection subsystem 10 to measure surface topology, such as the surface topology of the medical instrument and of the patient’s skin, para. [0136] and Fig. 2; the surface detection subsystem 10 can include “structured light imaging systems, which project surface topography detection light onto a region of interest”; para. [0180] and Fig. 5C disclose using structured light to measure surface topology of anatomical features of the patient; Yang does not explicitly disclose that the structured light pattern is a pattern of dots);
a medical instrument having one or more markers (Fig. 2, para. [0137], medical instrument 40 has fiducial markers 45 attached thereto, para. [0137]), the medical instrument being positioned within the tracking volume (para. [0137], Fig. 2, the tracking volume is the volume within the field of view (FOV) of the stereoscopic tracking system employing the two tracking cameras);
an image capture unit configured to capture imagery of the medical instrument and the one or more markers and configured to capture imagery of the pattern of dots within the tracking volume (para. [0137] and Fig. 2, the tracking system 20 comprises the stereoscopic tracking system that employes the two cameras for tracking the medical instrument 40 based on captured images of the markers 45, while the structured light detection system of the surface detection subsystem 10 captures images of the structured light pattern to detect the surface topology of the medical instrument and/or of the patient’s anatomy, para. [0180], Fig. 5C); and
a computing device comprising a memory configured to store instructions and a processor to execute the instructions to perform operations (Fig. 2, processor 110 and memory 115, para. [0145]) comprising:
initiating capture, by the image capture unit, of at least two images of the medical instrument and the one or more markers (para. [0137], the tracking system 20 under the control of the processor 110 initiates capture by the stereoscopic tracking system of images of the medical instrument 40 and the markers 45 affixed to the medical instrument 40; since the stereoscopic tracking system uses two cameras, at least two images are captured; in addition, since tracking of the medical instrument 40 is being performed, this indicates that at least two images are captured);
determining a three-dimensional position of the one or more markers from the captured images of the one or more markers (para. [0137] describes the tracking system 20 determining the “position and orientation” of the medical instrument 40 based on images of the markers 45 acquired by the cameras of the stereoscopic tracking system, which means that the three-dimensional position of the markers 45 is determined);
initiating projection, by the projector, of the pattern of dots within the tracking volume (paras. [0137] and [0180], the structured light detection system of the surface detection subsystem 10, under the control of the processor 110, initiates projection by the structured light detection system of structured light patterns onto the medical instrument and/or onto an anatomical feature of the patient within the tracking volume);
initiating capture, by the image capture unit, of at least two images of a portion of the pattern of dots (para. [0137], the structured light detection system of the surface detection subsystem 10, under the control of the processor 110, initiates capture of the structured light pattern in order to detect the topology of the surface upon which the structured light is projected; since the stereoscopic tracking system uses two cameras, at least two images are acquired; also, as indicated above, since tracking of the medical instrument 40 is being performed, this indicates that at least two images are captured); and
determining three-dimensional positions of dots in the portion of dots from the captured images of the portion of the pattern of dots (since the structure light patterns are being used by the structured light detection system of the surface detection subsystem 10 to determine surface topology, three-dimensional positions of features of the structured light pattern are being determined; as is well known in the art, structured light is used to determine three-dimensional surface topologies).
As indicated above, Yang does not explicitly disclose that the structured light pattern is a pattern of dots. Tang, in the same field of endeavor, discloses using structured light projection and detection of dot patterns to perform 3D reconstruction of surfaces (Col. 11, lines 49-58; Col. 13, lines 12-30).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots as taught by Tang. Yang is silent as to the type of structured light pattern that is used, but it is common in structured light imaging systems to use patterns of lines, grids, strips or dots. Therefore, it would have been obvious to one of ordinary skill in the art to use dot patterns as the structured light patterns in Yang as taught by Tang since there are a finite number of structured light patterns that are used for measuring surface topology and there would be a reasonable expectation of success using dot patterns since dot patterns have been successfully used in the past. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang to project dot light patterns and to detect the reflected dot patterns).
Regarding claim 2, Yang discloses determining the three-dimensional position of the one or more markers and the three-dimensional positions of the dots in a same coordinate system (para. [0157] discloses that during navigation/tracking the tracking system 20 and the surface detection subsystem 10 are in a common coordinate system relative to a reference frame that is attached to the patient and that “[s]ince the reference frame remains in a fixed position relative to the patient, recording position/orientation information from tracked instruments or the surface detection system relative to the reference frame position/orientation…is sufficient to compensate for these apparent motions”; this indicates that the three-dimensional positions of the markers and of the structured light pattern are in the same coordinate system).
As indicated above, Yang does not explicitly disclose that the structured light pattern is a pattern of dots. As indicated above, Tang discloses using structured light projection and detection of dot patterns to perform 3D reconstruction of surfaces (Col. 11, lines 49-58; Col. 13, lines 12-30).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots as taught by Tang. As indicated above, Yang is silent as to the type of structured light pattern that is used, but it is common in structured light imaging systems to use line, grid, strip and dot patterns. Therefore, it would have been obvious to one of ordinary skill in the art to use dot patterns as the structured light patterns in Yang since there are a finite number of structured light patterns that are used for measuring surface topology and there would be a reasonable expectation of success using dot patterns since dot patterns have been successfully used in the past. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang to project dot light patterns and to detect the reflected dot patterns).
Regarding claim 3, Yang discloses tracking patient anatomy using the three-dimensional positions of the structured light pattern (paras. [0117], [0136], [0137], [0157] and [0180], Fig. 5C), but does not disclose that the light pattern is a pattern of dots.
As indicated above, Tang discloses using structured light projection and detection of dot patterns to perform 3D reconstruction of surfaces (Col. 11, lines 49-58; Col. 13, lines 12-30).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots as taught by Tang for the reasons discussed above in the rejection of claims 1-2. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang to project dot light patterns and to detect the reflected dot patterns).
Regarding claim 4, as indicated above, Yang discloses determining the 3D positions of the structured light pattern, but does not explicitly disclose that the structured light pattern is a pattern of dots. As indicated above, Tang discloses using structured light projection and detection of dot patterns to perform 3D reconstruction of surfaces (Col. 11, lines 49-58; Col. 13, lines 12-30). Tang discloses using a portion of the dot pattern comprising a cropped and downscaled portion of the dot pattern for post-processing 3D reconstruction (Col. 33, lines 14-62). Tang teaches that the cropping and downsizing operations reduce the amount of corresponding pixel data that has to be processed to generate depth maps (Col. 2, line 66-Col. 3, line 15).
For the reasons discussed above in the rejections of claims 1-3, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots as taught by Tang.. In addition, a person of ordinary skill in the art would have been motivated to make the modification to reduce the number of pixels making up the dots that have to be processed as taught by Tang. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang to project dot light patterns and to detect the reflected dot patterns).
Regarding claim 6, Yang does not explicitly disclose matching the dots across the captured images of the portion of the pattern of dots. Tang discloses matching the dots across the captured images of the portion of the pattern of dots (Col. 3, lines 55-67 and Col. 31, lines 1-34 discuss performing block matching, which in the context of structured-light dot pattern images, means matching dots of a captured dot-pattern image with corresponding dots of another captured dot-pattern image).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to perform block matching of dots across multiple images as taught by Tang. A person of ordinary skill in the art would have been motivated to make the modification to match the dots of multiple images during tracking to ensure that the medical instrument and/or the anatomical feature surface being measured are properly tracked. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using a block matching algorithm or similar algorithm to match dots across multiple images).
Regarding claim 7, Yang does not explicitly disclose projecting a pattern of dots in time intervals. Tang discloses projecting the pattern of dots at time intervals corresponding to every third video frame (Col. 37, lines 36-52).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots projected in time intervals as taught by Tang. A person of ordinary skill in the art would have been motivated to make the modification to reduce the number of pixels making up the dots that have to be processed as taught by Tang and to reduce the need to use filtering to reduce noise in the RGB images caused by the dot pattern images and vice versa . The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang along with suitable timing control circuitry to project dot light patterns in time intervals and to detect the reflected dot patterns).
Regarding claim 8, Yang does not explicitly disclose that capturing of the at least two images of the portion of the structured light pattern is synchronized with the time intervals. Tang discloses capturing at least two images in at least the 0th and third frames in synchronization with the time intervals (Col. 37, lines 45-48, Fig. 7, frames 302a and 302d).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots projected in time intervals as taught by Tang and capturing the dot-pattern images in synchronization with the time intervals as taught by Tang. A person of ordinary skill in the art would have been motivated to make the modification to reduce the number of pixels making up the dots that have to be processed as taught by Tang and to ensure that the timing of dot-pattern image capture is synchronized with the timing of dot pattern projection. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang along with suitable timing control circuitry to project dot light patterns in time intervals and to detect the reflected dot patterns in synchronization with the projection time intervals).
Regarding claim 10, Yang discloses that the image capture unit comprises multiple cameras (para. [0137], tracking system 20 comprises a stereoscopic tracking system comprising two cameras).
Regarding claim 12, Yang is silent regarding the timing of capturing the images of the markers and of the structured light pattern. Tang discloses capturing RGB images during a first time period (Fig. 7, para. [0137], RGB images are captured in the frames 302b and 302c other than every third frame) and capturing the dot-pattern images in a second time period (Fig. 7, para. [0137], every third frame 302a and 302d), where the first and second time periods are different.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the surface detection subsystem 10 of Yang to measure surface topology by using structured light patterns comprising dots projected in time intervals as taught by Tang and capturing the dot-pattern images in synchronization with the time intervals as taught by Tang while capturing marker images in different time intervals. A person of ordinary skill in the art would have been motivated to make the modification to reduce the number of pixels making up the dots that have to be processed as taught by Tang and to prevent interference between the marker image data and the structured light image data. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using suitable light sources such as lasers or LEDs and/or suitable lens arrangements in the surface detection subsystem 10 of Yang along with suitable timing control circuitry to project and detect dot light patterns in certain time intervals and to capture the marker images during time intervals that are different from the time intervals during which the structured light images are captured).
Regarding claim 13, Yang is silent regarding the location of the projector that projects the structured light. Tang discloses that the projector 106 is mounted to a housing that contains an image capture unit (Fig. 2, Col. 5, line 50 – Col. 6, line 4, camera systems 100a-100n and projector 106 are coupled to the same housings).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system of Yang such that the cameras of the tracking system 20 and the projector of the surface detection subsystem 10 of Yang are mounted to the same housing. A person of ordinary skill in the art would have been motivated to make the modification to make the system compact. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using a common housing for system 20 and subsystem 10).
Regarding claims 15-18, the rejection of claims 1-4 apply mutatis mutandis to claims 15-18, respectively.
Regarding claim 20, the rejection of claim 1 applies mutatis mutandis to claim 20.
Claims 5, 9, 11, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Tang as applied to claims 1-4, 6-8, 10, 12-13, 15-18 and 20 and further in view of U.S. Pat. No. 10,016,243 B2 to Esterberg (hereinafter referred to as “Esterberg”).
Regarding claim 5, the combined teachings of Yang and Tang do not explicitly disclose that determining the three-dimensional positions of the dots comprises determining a centroid. Esterberg, in the same field of endeavor, discloses systems and methods for assisted surgical navigation in which dot patterns are projected onto the surgical field. The captured dot-pattern image is captured and the centroid each dot is determined (Col. 2, line 59-Col. 3, line 6).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system of Yang as modified by Tang further based on the teachings of Esterberg such that the centroids of the dots are determined and used to determine the 3D positions of the dots. A person of ordinary skill in the art would have been motivated to make the modification to determine the 3D positions of the dots with greater precision by ensuring that the centers of the dots are precisely located. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (using a known centroid determination algorithm to calculate the centroids of pixels comprising the dots).
Regarding claim 9, the combined teachings of Yang and Tang do not explicitly teach that the geometry of the dot pattern is geometrically changed between subsequent projections.
Esterberg discloses that the pattern of dots is geometrically changed between subsequent projections (Col. 15, lines 19-33, a denser dot pattern is projected when the surgeon wearing the head-mounted display (HMD) moves closer to the patient than when the surgeon is farther away from the patient, which means that the geometry of the dot pattern being projected changes between subsequent projections).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system of Yang as modified by Tang further based on the teachings of Esterberg such that the pattern of dots being projected changes between subsequent projections as taught by Esterbegr. A person of ordinary skill in the art would have been motivated to make the modification to ensure that data used to represent the anatomy of the patient represented by the dots has higher resolution as the surgeon wearing the HMD moves closer to the patient. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (varying the lens arrangement and/or the light source of the projector of Tang to change the dot pattern based on the position of the surgeon/user relative to the patient).
Regarding claim 11, the combined teachings of Yang and Tang do not explicitly teach that the pattern of dots comprises a pseudorandom pattern. Esterberg discloses that the pattern of dots can comprise a pseudorandom pattern of dots (Col. 12, lines 3-6 and Col. 15, lines 8-11).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system of Yang as modified by Tang further based on the teachings of Esterberg such that the pattern of dots being projected is a pseudorandom pattern as taught by Esterberg. A person of ordinary skill in the art would have been motivated to make the modification to allow correspondence matching between the dots of the pattern generated by the projector and the dots reflected from the surface being measured. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (varying the lens arrangement and/or the light source driver of the projector of Tang to pseudorandomize the dot pattern being projected).
Regarding claim 14, the combined teachings of Yang and Tang do not explicitly teach that the projector is positioned remote from a housing that contains the image capture unit.
Esterberg discloses that the projector can be positioned remote from the housing that contains the image capture unit (Col. 19, lines 19-22, the dot matrix projector 805 can be remote from the HMD housing in which the image capture unit is mounted, such as on a ceiling mount).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system of Yang as modified by Tang further based on the teachings of Esterberg such that the dot pattern projector is mounted remotely from the image capture unit as taught by Esterberg. A person of ordinary skill in the art would have been motivated to make the modification to reduce the size and complexity of the housing that houses the image capture unit. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (remotely mounting the projector).
Regarding claim 19, the rejection of claim 5 applies mutatis mutandis to claim 19.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL J. SANTOS/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667