DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 6-8, filed 8/11/2026 with respect to 35 USC §102 rejection of claims 22-31 have been fully considered but were not persuasive. Applicant has amended claims and argues previously cited references do not disclose amended claim language. Applicant argues previously cited Cheatham fails to teach a system having, among other things, one or more pieces of controlled equipment that includes one or more of surgical navigation systems, PACS (picture archiving and communication systems), cameras, electrosurgical generators, infusion pumps, vacuum valves, or music players, with a controller configured to adjust a setting of the one or more pieces of controlled equipment based on the user's interaction with a portion of the projected image of the one or more pieces of controlled equipment on the data board. In response, examiner points to paragraph [0031] of Cheatham to disclose a remove sensor such as a camera which can control dynamic display and choose the information to display. Cheatham’s remote camera teaches amended one or more pieces of controlled equipment that includes one or more cameras. Therefor all argued claim limitations are taught by previously cited references.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 22-31, 35-37, 39-40 is/are rejected under 35 U.S.C. 102(a)(2)/(a)(1) as being anticipated by Cheatham III et al (US 20160331460 A1).
Regarding claim 22, Cheatham III discloses a user interface system ([0025] interactive surgical drape device and system), comprising:
a data board ([0027] interactive surgical drape device and system including at least one sensor disposed on or in the surgical drape surgical drape);
a camera configured to capture images of a working environment in which the data board is disposed ([0148] an imaging sensor 760 captures an image 770 (dashed lines) of the face or another feature of the surgery attendant 740, and transmits the information to the processor and controller that in turn direct a remote dynamic display projector 750 to project 760 information 790 (e.g., task list, reminders, images, etc.) onto the surgical drape);
a projector that projects a projected image onto the data board ([0027] alter the content or location area of display of information on or near the drape, for example by way of projection or other dynamic display unit(s));
one or more pieces of controlled equipment that includes one or more of surgical navigation systems, PACS (picture archiving and communication systems), cameras, electrosurgical generators, infusion pumps, vacuum valves, or music players ([0106] the electronic circuitry is configured to communicate with an external computing device. In an embodiment, the external computing device includes at least one of a personal computer, a tablet, a mobile device, a smart TV, a multimedia player, or a game console [0031] the sensing means includes at least one remote sensor, such as a camera or other imaging sensor. For example, a camera capable of imaging the disposition of the interactive surgical drape device itself can provide information to the processor and controller, which can then control the dynamic display to provide optimal viewing of the information displayed thereon, as well as choose the information to display (including in real-time). [0090] one or more imaging sensors (e.g., a camera or laser scanner) remote to the surgical drape can be configured to image the surgical drape and transmit the image to the processor for processing (e.g., for use in mapping the surface of the surgical drape in 2D or 3D));
a controller operatively coupled to the camera, the projector and the one or more pieces of controlled equipment, ([0148] transmits the information to the processor and controller (not shown)),
the controller being configured to:
detect the data board within the captured images by the camera and simultaneously detect one or more of a user, a body part of the user, a surgical instrument, a surgical implant, a stylus, and/or another other instrument held by the user within the captured images to determine how the projected image on the data board is being interacted with or being moved relative to the data board ([0028] the interactive surgical drape device or system includes at least one sensor; transmitter; receiver; transceiver; Bluetooth™; GPS (global positioning system); wireless internet; audio, visual, or haptic output; audio, visual, or haptic input; touchscreen or other input device (e.g., keyboard, mouse, stylus, etc.).).,
determine a position and orientation of the data board within the working environment ([0049] , the interactive surgical drape device or system includes dynamic display, circuitry, and software including active 3D projection mapping, together with 3D tracking, for example using sensors of the sensor assembly disposed within the drape and/or remote sensors such as imaging sensors, which may include interactions with sensors or fiducials on the drape),
control the projector to project the projected image onto the data board based on the position and orientation of the data board ([0049] appropriate portions of the drape as projected by the 3D projection and mapping software), and
adjust a setting ([0158] if so determines new optimum parameters and directs the one or more dynamic display projectors 1207 or the one or more dynamic display devices 1213 to update the presentation of the information, e.g., to update in real time.) of the one or more pieces of controlled equipment based on the user's interaction ([0158] transmit the sensed information to the processor, which determines if there has been a change) with a portion of the projected image of the one or more pieces of controlled equipment on the data board ([0158] one or more sensors 1205, associated with (e.g., in or on) the surgical drape 1217 or remote from the surgical drape 1217, are configured to sense at least one of the disposition or condition of the surgical drape 1217, or a surgery attendant).
Regarding claim 23, Cheatham III discloses wherein the controller is further configured to maintain the projected image on the data board as the data board is moved within the working environment ([0049] real-time tracking of the surgery attendant(s) as informed by the sensors allows for real-time information display on portions of the drape viewable or accessible to the attendant(s), as projected by the 3D projection and mapping software.).
Regarding claim 24, Cheatham III discloses wherein the controller is further configured to calculate a difference in distance-to-camera between the user and the data board ([0039] determine which area of the dynamic display or which of several distinct dynamic display areas is provided with which specific information, based on the availability of planar space, distance from a particular surgery attendant, spatial dimensions, etc., relative to its disposition), and, if the difference in distance-to-camera is below a threshold amount, the controller is configured to direct the system to adjust the setting that corresponds to a location along the data board with respect to which the calculation occurred ([0061] optionally determining if the sensed signal exceeds a threshold value for the at least one property of the disposition of the interactive surgical drape; instructing at least one sensor of the sensor assembly to detect at least one additional property of the disposition of the interactive surgical drape when the threshold value is exceeded, [0158] if so determines new optimum parameters and directs the one or more dynamic display projectors 1207 or the one or more dynamic display devices 1213 to update the presentation of the information, e.g., to update in real time)
Regarding claim 26, Cheatham III discloses wherein the one or more pieces of controlled equipment is procedurally or physically isolated from the working environment ([0110] interactive surgical drape has at least one extension for also draping a second area, such as a surgical tool table).
Regarding claim 27, Cheatham III discloses wherein the one or more pieces of controlled equipment is disposed within the working environment and outside of the working environment (Figure 1, [0110] interactive surgical drape has at least one extension for also draping a second area, such as a surgical tool table ).
Regarding claim 28, Cheatham III discloses wherein the controller comprises a data board identification module configured to receive one or more captured images of the data board and recognize a unique identifier of the data board within the one or more captured images ([0119] , the camera 502 is configured to scan the surgical theater, subject, or surgery attendant, and/or measure at least one feature thereof, such as for movement, distress, assistance with 3D and spatial mapping, identification, etc. and to generate at least one sensed signal indicating detection of one or more feature of the surgical theater, subject, or surgery attendant.).
Regarding claim 29, Cheatham III discloses wherein the controller comprises a data board tracking module that is configured to determine the position or orientation of the data board relative to a reference coordinate system of the working environment ([0063] detecting at least one property of the disposition of the interactive surgical drape includes detecting at least one of a change in movement of the interactive surgical drape, x, y, or z coordinates of at least one dynamic display within the interactive surgical drape, an x, y, or z coordinate of at least a portion of the surgical drape, an x, y, or z coordinate of at least one sensor operably coupled thereto, size of a planar surface of the interactive surgical drape, position or orientation of the interactive surgical drape, three-dimensional shape of the interactive surgical drape, or folding of the interactive surgical drape)
Regarding claim 30, Cheatham III discloses wherein the data board tracking module is further configured to determine changes in position or orientation of the data board as it moves throughout the working environment in substantially real-time ([0063] detecting at least one property of the disposition of the interactive surgical drape includes detecting at least one of a change in movement of the interactive surgical drape, x, y, or z coordinates of at least one dynamic display within the interactive surgical drape, an x, y, or z coordinate of at least a portion of the surgical drape, an x, y, or z coordinate of at least one sensor operably coupled thereto, size of a planar surface of the interactive surgical drape, position or orientation of the interactive surgical drape, three-dimensional shape of the interactive surgical drape, or folding of the interactive surgical drape)
Regarding claim 31, Cheatham III discloses wherein the projector is configured to:
shift the projected image when the data board tracking module determines that the data board has shifted in the X or Y direction, or scale the projected image to match the movement of the data board when the data board tracking module determines that the data board has shifted in the Z direction or is rotated about the Z axis ([0113] receive at least one sensed signal from the sensor assembly 410 and to process the sensed signal in order to provide direction signals to at least one portion of the system 400, for example to the dynamic display component).
Regarding claim 35, Cheatham III discloses a method of tracking ([0025] interactive surgical drape device and system), comprising:
capturing one or more images of a working environment in which a data board is disposed ([0148] an imaging sensor 760 captures an image 770 (dashed lines) of the face or another feature of the surgery attendant 740, and transmits the information to the processor and controller that in turn direct a remote dynamic display projector 750 to project 760 information 790 (e.g., task list, reminders, images, etc.) onto the surgical drape);
determining, using a controller, a position and orientation of the data board within the working environment from said one or more captured images ([0049] , the interactive surgical drape device or system includes dynamic display, circuitry, and software including active 3D projection mapping, together with 3D tracking, for example using sensors of the sensor assembly disposed within the drape and/or remote sensors such as imaging sensors, which may include interactions with sensors or fiducials on the drape), and,
simultaneously determining, using the controller, one or more of a user, a body part of the user, a surgical instrument, a surgical implant, a stylus, and/or another other instrument held by the user within the captured images to determine how the projected image on the data board is being interacted with or being moved relative to the data board ([0028] the interactive surgical drape device or system includes at least one sensor; transmitter; receiver; transceiver; Bluetooth™; GPS (global positioning system); wireless internet; audio, visual, or haptic output; audio, visual, or haptic input; touchscreen or other input device (e.g., keyboard, mouse, stylus, etc.)).;
projecting the projected image onto the data board based on the position and orientation of the data board as the data board is moved within the working environment ([0049] appropriate portions of the drape as projected by the 3D projection and mapping software); and
interacting with ([0158] transmit the sensed information to the processor, which determines if there has been a change), using the controller, one or more pieces of controlled equipment to adjust a setting ([0158] if so determines new optimum parameters and directs the one or more dynamic display projectors 1207 or the one or more dynamic display devices 1213 to update the presentation of the information, e.g., to update in real time.) of the one or more pieces of controlled equipment based on the user's interaction with a portion of the projected image of the one or more pieces of controlled equipment on the data board ([0158] one or more sensors 1205, associated with (e.g., in or on) the surgical drape 1217 or remote from the surgical drape 1217, are configured to sense at least one of the disposition or condition of the surgical drape 1217, or a surgery attendant)
the one or more pieces of controlled equipment that includes one or more of surgical navigation systems, PACS (picture archiving and communication systems), cameras, electrosurgical generators, infusion pumps, vacuum valves, or music players ([0106] the electronic circuitry is configured to communicate with an external computing device. In an embodiment, the external computing device includes at least one of a personal computer, a tablet, a mobile device, a smart TV, a multimedia player, or a game console [0031] the sensing means includes at least one remote sensor, such as a camera or other imaging sensor. For example, a camera capable of imaging the disposition of the interactive surgical drape device itself can provide information to the processor and controller, which can then control the dynamic display to provide optimal viewing of the information displayed thereon, as well as choose the information to display (including in real-time). [0090] one or more imaging sensors (e.g., a camera or laser scanner) remote to the surgical drape can be configured to image the surgical drape and transmit the image to the processor for processing (e.g., for use in mapping the surface of the surgical drape in 2D or 3D));
Regarding claim 36, Cheatham III discloses further comprising adjusting a position of the projected image in response to changes in position or orientation of the data board as it moves throughout the working environment in substantially real-time ([0063] detecting at least one property of the disposition of the interactive surgical drape includes detecting at least one of a change in movement of the interactive surgical drape, x, y, or z coordinates of at least one dynamic display within the interactive surgical drape, an x, y, or z coordinate of at least a portion of the surgical drape, an x, y, or z coordinate of at least one sensor operably coupled thereto, size of a planar surface of the interactive surgical drape, position or orientation of the interactive surgical drape, three-dimensional shape of the interactive surgical drape, or folding of the interactive surgical drape)
Regarding claim 37, Cheatham III discloses further comprising calculating a difference in distance- to-camera between the user's finger and the data board, and, if the difference in distance-to- camera is below a threshold amount ([0039] determine which area of the dynamic display or which of several distinct dynamic display areas is provided with which specific information, based on the availability of planar space, distance from a particular surgery attendant, spatial dimensions, etc., relative to its disposition,
adjusting the setting that corresponds to a location along the data board with respect to which the calculation occurred ([0061] optionally determining if the sensed signal exceeds a threshold value for the at least one property of the disposition of the interactive surgical drape; instructing at least one sensor of the sensor assembly to detect at least one additional property of the disposition of the interactive surgical drape when the threshold value is exceeded, [0158] if so determines new optimum parameters and directs the one or more dynamic display projectors 1207 or the one or more dynamic display devices 1213 to update the presentation of the information, e.g., to update in real time)
Regarding claim 39, Cheatham III discloses wherein the projected image comprises a graphical user interface, and the action comprises generating a control signal to the one or more pieces of controlled equipment procedurally or physically isolated from the working environment ([0149] the imaging devices may be remote to the surgical drape 730, for nonlimiting example, housed with the dynamic display projector)
Regarding claim 40, Cheatham III discloses wherein the projected image comprises a graphical user interface, and the action comprises generating a control signal to the one or more pieces of controlled equipment procedurally or physically isolated from the working environment ([0149] the imaging devices may be remote to the surgical drape 730, for nonlimiting example, housed with the dynamic display projector)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheatham III et al (US 20160331460 A1) as applied to claim 22, 35 above, and further in view of Takano et al (US 20180197344 A1).
Regarding claim 32, Cheatham III is silent to wherein the controller is configured to apply a keystone correction algorithm to the image data such that the projected image maps accurately to a tilted data board without distortion.
Takano discloses wherein the controller is configured to apply a keystone correction algorithm to the image data such that the projected image maps accurately to a tilted data board without distortion ([0082] image processes including a resolution conversion process, various color tone correction processes such as adjustment of luminance and color saturation, a keystone correction process, and the like.).
Cheatham III and Takano are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify interactive surgical drape device and system of Cheatham III to include wherein the controller is configured to apply a keystone correction algorithm to the image data such that the projected image maps accurately to a tilted data board without distortion as described by Takano
The motivation for doing so would have been to allow a user to visually recognize a virtual image and external scenery (Takano, [0009]).
Therefore, it would have been obvious to combine Cheatham III and Takano to obtain the invention as specified in claim 32.
Claim(s) 33-34,38 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheatham III et al (US 20160331460 A1) as applied to claim 22, 35 above, and further in view of Nawana et al (US 20140088990 A1).
Regarding claim 33, Cheatham III is silent to further comprising a pre-planning module configured to execute a procedure that was previously planned, the pre-planning module being configured to set one or more of: a predetermined trajectory of surgical tools, a correction of trajectory of surgical tools, and/or a tabulation of surgical tools used.
Nawana discloses further comprising a pre-planning module configured to execute a procedure that was previously planned, the pre-planning module being configured to set one or more of: a predetermined trajectory of surgical tools, a correction of trajectory of surgical tools, and/or a tabulation of surgical tools used ([0183] simulated surgery can allow the user to simulate the selected surgical procedure from beginning to end. Aspects of the selected surgical procedure that can be simulated can include one or more of an access point on the patient, surgical instruments used in the procedure, a path through and/or retraction of tissue and nervous structures, bone removal, soft tissue removal, tissue and structure movement, correction, movement due to indirect decompression, and placement of all implants).
Cheatham III and Nawana are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify interactive surgical drape device and system of Cheatham III to include further comprising a pre-planning module configured to execute a procedure that was previously planned, the pre-planning module being configured to set one or more of: a predetermined trajectory of surgical tools, a correction of trajectory of surgical tools, and/or a tabulation of surgical tools used as described by Nawana
The motivation for doing so would have been to plan of medical treatment corresponding to each of the patients (Nawana, [0009]).
Therefore, it would have been obvious to combine Cheatham III and Nawana to obtain the invention as specified in claim 33.
Regarding claim 34, Cheatham III is silent to wherein the planned procedure is projected onto the data board.
Nawana discloses wherein the planned procedure is projected onto the data board ([0213] user can thus consult the projected results and decide whether to revise the surgery, such as if the projected results are less than the user would like and/or expect.).
Cheatham III and Nawana are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify interactive surgical drape device and system of Cheatham III to include wherein the planned procedure is projected onto the data board as described by Nawana
The motivation for doing so would have been to plan of medical treatment corresponding to each of the patients (Nawana, [0009]).
Therefore, it would have been obvious to combine Cheatham III and Nawana to obtain the invention as specified in claim 34.
Regarding claim 38, Cheatham III is silent to further comprising executing a pre-planned procedure, the pre-planned procedure including one or more of setting a predetermined trajectory of surgical tools, setting a correction of trajectory of surgical tools, and/or setting tabulation of surgical tools used.
Nawana discloses further comprising executing a pre-planned procedure, the pre-planned procedure including one or more of setting a predetermined trajectory of surgical tools, setting a correction of trajectory of surgical tools, and/or setting tabulation of surgical tools used ([0183] simulated surgery can allow the user to simulate the selected surgical procedure from beginning to end. Aspects of the selected surgical procedure that can be simulated can include one or more of an access point on the patient, surgical instruments used in the procedure, a path through and/or retraction of tissue and nervous structures, bone removal, soft tissue removal, tissue and structure movement, correction, movement due to indirect decompression, and placement of all implants).
Cheatham III and Nawana are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify interactive surgical drape device and system of Cheatham III to include further comprising executing a pre-planned procedure, the pre-planned procedure including one or more of setting a predetermined trajectory of surgical tools, setting a correction of trajectory of surgical tools, and/or setting tabulation of surgical tools used as described by Nawana
The motivation for doing so would have been to plan of medical treatment corresponding to each of the patients (Nawana, [0009]).
Therefore, it would have been obvious to combine Cheatham III and Nawana to obtain the invention as specified in claim 38.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615
Prosecution Insights