Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of US patent 12,238,381. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims in the current applicant are broader that in the US patent ‘381.
Patent US 12,239,381
1. A surgical system comprising: a first plurality of fiducial markers configured to be affixed to one or more patient bones; a plurality of cameras configured to capture images of a surgical scene including the first plurality of fiducial markers, the plurality of cameras including both headset cameras and fixed cameras; a plurality of headsets, each comprising an augmented reality display and at least one of the plurality of cameras, configured to be worn by surgical staff; and one or more processors configured to process the images of the surgical scene and to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers, using images captured from at least a subset of the headset cameras and a subset of the fixed cameras, wherein each of the plurality of headsets is configured to overlay contextual information about the at least one anatomical feature via the augmented reality display at a location corresponding to the at least one anatomical feature, based on the three-dimensional patient model, and wherein the one or more processors maintains a headset model of a field of view of plurality of headsets relative to the three-dimensional patient model, wherein the contextual information includes visual feedback of compliance with a surgical plan.
2. The surgical system of claim 1, wherein at least one of the fixed cameras is affixed to a portable cart.
3. The surgical system of claim 1, further comprising a second plurality of fiducial markers affixed to each of the plurality of headsets, wherein the one or more processors are configured to track a pose of each headset by tracking the second plurality of fiducial markers.
4. The surgical system of claim 1, further comprising a video monitor that displays the same contextual information such that surgical staff who are not wearing a headset can view the contextual information.
5. The surgical system of claim 1, further comprising a projector configured to project the contextual information directly onto patient anatomy.
6. The surgical system of claim 1, further comprising a hand tool having fiducial marks that is configured to improve the three-dimensional patient model by registering the extents of the at least one anatomical feature.
7. The surgical system of claim 1, wherein the contextual information includes an identification of a location of a planned resection on the at least one anatomical feature.
8. A method for assisting surgical staff during a procedure comprising steps of: affixing a first plurality of fiducial markers to one or more patient bones; capturing a plurality of images of a surgical scene including the first plurality of fiducial markers, using a plurality of cameras that include both headset cameras and fixed cameras, the headset cameras being part of augmented reality headsets that also comprise an augmented reality display configured to be worn by surgical staff, determining, from the plurality of images captured from at least a subset of the headset cameras and a subset of the fixed cameras, location and orientation information about a patient anatomical feature to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers; and displaying, to a user of an augmented reality headset, contextual information at a predetermined location in the user's field of view corresponding to the patient anatomical feature, wherein the contextual information includes visual feedback of compliance with a surgical plan.
9. The method of claim 8, wherein at least one of the fixed cameras is affixed to a portable cart.
10. The method of claim 8, further comprising a step of tracking a pose of each augmented reality headset by tracking a second plurality of fiducial markers affixed to each of the augmented reality headsets.
11. The method of claim 8, further comprising a step of displaying, on a video monitor, the same contextual information such that surgical staff who are not wearing the augmented reality headsets can view the contextual information.
12. The method of claim 8, wherein the contextual information includes an identification of a location of a planned resection on the patient anatomical feature.
13. The method of claim 8, further comprising a step of manipulating a hand tool having fiducial marks to register the extents of the at least one anatomical feature.
14. The system of claim 1, wherein the contextual information comprises preoperative image data of the at least one anatomical feature.
15. The system of claim 7, wherein the identification of the location of the planned resection on the at least one anatomical feature comprises an indication of an amount of bone to be removed.
16. The method of claim 8, further comprising projecting the contextual information directly onto patient anatomy.
17. The method of claim 8, wherein the contextual information comprises preoperative image data of the at least one anatomical feature.
18. The method of claim 12, wherein the identification of the location of the planned resection on the at least one anatomical feature comprises an indication of an amount of bone to be removed.
1. A surgical system comprising:a first plurality of fiducial markers configured to be affixed to one or more patient bones;a plurality of cameras configured to capture images of a surgical scene including the first plurality of fiducial markers, the plurality of cameras including both headset cameras and fixed cameras;a plurality of headsets, each comprising an augmented reality display and at least one of the plurality of cameras, configured to be worn by surgical staff; andone or more processors configured to process the images of the surgical scene and to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers, using images captured from at least a subset of the headset cameras and a subset of the fixed cameras,wherein each of the plurality of headsets is configured to be used simultaneously to provide additional camera angles and tracking capabilities,wherein each of the plurality of headsets is configured to overlay contextual information about the at least one anatomical feature via the augmented reality display at a location corresponding to the at least one anatomical feature, based on the three-dimensional patient model, andwherein the one or more processors maintains a headset model of a field of view of the plurality of headsets relative to the three-dimensional patient model.
2. The surgical system of claim 1, wherein at least one of the fixed cameras is affixed to a portable cart.
3. The surgical system of claim 1, further comprising a second plurality of fiducial markers affixed to each of the plurality of headsets, wherein the one or more processors are configured to track a pose of each headset by tracking the second plurality of fiducial markers.
4. The surgical system of claim 1, further comprising a video monitor that displays the same contextual information such that surgical staff who are not wearing a headset can view the contextual information.
5. The surgical system of claim 1, further comprising a projector configured to project the contextual information directly onto patient anatomy.
6. The surgical system of claim 1, further comprising a hand tool having fiducial marks that is configured to improve the three-dimensional patient model by registering the extents of the at least one anatomical feature.
7. The surgical system of claim 1, wherein the contextual information includes an identification of a location of a planned resection on the at least one anatomical feature.
8. The surgical system of claim 7, wherein the identification of the location of the planned resection on the at least one anatomical feature comprises an indication of an amount of bone to be removed.
9. The surgical system of claim 1, wherein the contextual information includes visual feedback of compliance with a surgical plan.
10. The surgical system of claim 1, wherein the contextual information comprises preoperative image data of the at least one anatomical feature.
11. A method for assisting surgical staff during a procedure comprising steps of:affixing a first plurality of fiducial markers to one or more patient bones;capturing a plurality of images of a surgical scene including the first plurality of fiducial markers, using a plurality of cameras that include both headset cameras and fixed cameras, the headset cameras being part of augmented reality headsets that also comprise an augmented reality display configured to be worn by surgical staff, the augmented reality headsets configured to be used simultaneously to provide additional camera angles and tracking capabilities;determining, from the plurality of images captured from at least a subset of the headset cameras and a subset of the fixed cameras, location and orientation information about a patient anatomical feature to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers;and displaying, to a user of an augmented reality headset, contextual information at a predetermined location in the user's field of view corresponding to the patient anatomical feature.
12. The method of claim 11, wherein at least one of the fixed cameras is affixed to a portable cart.
13. The method of claim 11, further comprising a step of tracking a pose of each augmented reality headset by tracking a second plurality of fiducial markers affixed to each of the augmented reality headsets.
14. The method of claim 11, further comprising a step of displaying, on a video monitor, the same contextual information such that surgical staff who are not wearing the augmented reality headsets can view the contextual information.
15. The method of claim 11, wherein the contextual information includes an identification of a location of a planned resection on the patient anatomical feature.
16. The method of claim 15, wherein the identification of the location of the planned resection on the at least one anatomical feature comprises an indication of an amount of bone to be removed.
17. The method of claim 11, wherein the contextual information includes visual feedback of compliance with a surgical plan.18. The method of claim 11, further comprising a step of manipulating a hand tool having fiducial marks to register the extents of the at least one anatomical feature.
19. The method of claim 11, further comprising projecting the contextual information directly onto patient anatomy.
20. The method of claim 11, wherein the contextual information comprises preoperative image data of the at least one anatomical feature.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-8, 10-18, 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lang (US 2017/0258526).
With respect to claims 1, 11, Lang discloses a surgical system (figs. 1,14: system 10) comprising: a first plurality of fiducial markers (figs. 1,14: optical markers 22) configured to be affixed to one or more patient bones (par. [0268], first to third sentence; par. [0450], fourth sentence); a plurality of cameras (par. [0905], first sentence) configured to capture images of a surgical scene including the first plurality of fiducial markers (par. [0905], second sentence), the plurality of cameras including both headset cameras and fixed cameras (par. [0905], first sentence; par. [0968]: " can acquire one or more optical measurements or measurement inputs, e.g. of anatomic landmarks, axes from cameras, anatomic axes, biomechanical axes, a mechanical axis of a leg 17, using for example an integrated or attached camera, image capture or video system."); a plurality of headsets (figs. 1,14: OHMDs 11, 12, 13, 14), each comprising an augmented reality display (par. [0102],[0103]) and at least one of the plurality of cameras (par. [0968]: "Any of the OHMD's 11, 12, 13, 14 can acquire one or more optical measurements or measurement inputs, e.g. of anatomic landmarks, axes from cameras, anatomic axes, biomechanical axes, a mechanical axis of a leg 17, using for example an integrated or attached camera, image capture or video system."), configured to be worn by surgical staff; and one or more processors (par. [0103],[(0909]) configured to process the images of the surgical scene and to maintain a three-dimensional patient model of at least one anatomical feature of a patient relative to a reference frame defined by the first plurality of fiducial markers, using images captured from at least a subset of the headset cameras and a subset of the fixed cameras (figs. 1,14: virtual surgical plan 24, registration in common coordinate system 15; par. [0905], [1194]), wherein each of the plurality headsets is configured to overlay contextual information about the at least one anatomical feature via the augmented reality display at a location corresponding to the anatomical feature, based on the three dimensional patient model, and wherein the one or more processors maintains a headset model of the field of view of plurality of headsets relative to the three-dimensional patient model (fig. 14: maintain alignment/superimposition of virtual data).
With respect to claims 2, 12 Lang discloses wherein at least one of the fixed cameras is affixed to a portable cart (see para. 0067).
With respect to claims 3, 13 Lang discloses further comprising a second plurality of fiducial markers affixed to each of the plurality of headsets, wherein the one or more processors are configured to track a pose of each headset by tracking the second plurality of fiducial markers (see para. 0146 see 0421).
With respect to claims 4, 14 Lang discloses further comprising a video monitor that displays the same contextual information such that surgical staff who are not wearing a headset can view the contextual information (see para. 1293).
With respect to claims 5, 15 Lang discloses further comprising a projector configured to project the contextual information directly onto patient anatomy (see para. 0431)
With respect to claims 6, 16 Lang discloses further comprising a hand tool having fiducial marks that is configured to improve the three-dimensional patient model by registering the extents of the at least one anatomical feature (see para. 0427, 0428, 0436,1197).
With respect to claims 8, 18 Lang discloses wherein the contextual information includes visual feedback of compliance with a surgical plan (see Fig. 14).
With respect to claims 10, 20 Lang discloses wherein the contextual information comprises preoperative image data of the at least one anatomical feature (see 0203).
With respect to claims 7, 17 Lang discloses wherein the contextual information includes visual feedback of compliance with a surgical plan (see 0020, 0262).
Examiner’s Comment
With respect to claim 9, 19 the closest prior art fails to disclose the limitations as set forth; however, the Double Patenting rejection as sets forth above is outstanding.
Response to Arguments
Applicant's arguments filed 05/19/2026 have been fully considered but they are not persuasive.
The applicant argues “Lang's disclosure of cameras "separate from the OHMD" is presented as one option in a menu of alternatives-not as a component that operates in conjunction with the headset-integrated cameras. The disjunctive phrasing ("integrated into, attached to or separate from") consistently characterizes these camera locations as interchangeable alternatives, any one (and only one) of which may be selected for a given implementation. Even when stating "[b]y using multiple OHMD's 11, 12, 13, 14 from different view angles with multiple cameras, image capture or video systems, the accuracy of the measurements can optionally be improved," Lang addresses combining cameras among multiple OHMDs (i.e., headset cameras from different headsets) and not combining headset cameras with separate, fixed cameras in a unified tracking pipeline. Even when Lang discloses that "the 3D scanner or image and/or video capture system can be attached to an arm or tripod," the disclosure appears in the context of bone and tissue morphing for surface scanning, not in the context of maintaining a 3D patient model using fused data from both headset- mounted and externally mounted cameras.
At no point does Lang describe a system in which a processor simultaneously receives and fuses image data from headset cameras and fixed cameras to maintain a three-dimensional patient model relative to a fiducial-marker-defined reference frame. The passages cited by the Office Action describe, at most, a system where either headset cameras or cameras separate from the OHMD can be used for optical marker detection and measurement and not a system where both are required to work together for model maintenance.
As a claim is anticipated only if each and every element as set forth in the claim is found, either expressly or inherently described, in a single prior art reference, claims 1 and 11 are not anticipated by Lang. Lang never discloses a single embodiment in which headset camera images and fixed camera images are simultaneously processed to maintain the three-dimensional patient model. For at least these reasons, claims 1 and 11 and the claims that depend thereon are patentable over the cited reference. Applicant respectfully requests withdrawal of this rejection.”
The Examiner respectfully disagrees.
IT should be noted, as pointed by the applicant, Lang discloses camera located in the optical head mounted display (OHMD). Further Lang discloses, also as pointed by the applicant and quoting the applicant response “Even when Lang discloses that "the 3D scanner or image and/or video capture system can be attached to an arm or tripod," the disclosure appears in the context of bone and tissue morphing for surface scanning, not in the context of maintaining a 3D patient model using fused data from both headset- mounted and externally mounted cameras.”
The Examiner points to para. 1193 which discloses] “In some embodiments of the invention, 3D surfaces morphed from 2D pre-operative data, e.g. using one or more pre-operative x-rays, can be combined with 3D surfaces derived intra-operatively, e.g. derived using an intra-operative mechanical and/or opto-electronic and/or laser and/or 3D scanner. For example, the pre-operative morphed surfaces of a femoral head can be matched, aligned, superimposed or merged in this manner with the intra-operative surfaces. Or the pre-operative morphed surfaces of one or both femoral condyles and/or tibial plateaus can be matched, aligned superimposed or merged in this manner with their corresponding intra-operative surfaces. By matching, aligning, superimposing or merging surfaces derived from pre-operative and intra-operative data, axis information obtained on pre-operative data, e.g. standing x-rays can be readily superimposed or merged with intra-operative data. The resultant model can be used to develop, derive and/or modify a virtual surgical plan, for example with subsequent display of one or more cut planes or tissue resections or axes by an OHMD.
In summary, Lang discloses additional 3D scanner or image/or video capture system to acquire images which is separate from the camera image system in the headseat and further used in the model generation.
It should be pointed that the applicants set forth in the arguments “…not in the context of maintaining a 3D patient model using fused data from the headset mounted and externally mounted cameras” (applicant’s response, pg. 7, first paragraph).
The Examiner points that claim 1does not set forth data fusion between headset mounted cameras and externally mounted cameras, but merely data a 3D model is maintained (which also set forth a broad description since the claim neither set forth a step for generating of a 3D model) using images captured from the headset cameras and fixed cameras. There is not fusion limitation, not that the fixed camera is external.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH M SANTOS RODRIGUEZ whose telephone number is (571)270-7782. The examiner can normally be reached Monday-Friday 8:30am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached on 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH M SANTOS RODRIGUEZ/Primary Examiner, Art Unit 3797
Prosecution Insights