DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-13, 15-16, and 18-22 are pending in this application. Claims 14 and 17 are canceled, Claims 19-20 are withdrawn, and Claims 1-13, 15-16, 18, and 21-22 have been examined on the merits.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Aguirre-Valencia (US20160381256A1) in view of Lang (US20210137634A1).
Regarding Claim 1,
Aguirre-Valencia teaches a method of performing and updating a medical procedure on a patient by a practitioner using an augmented reality system (corresponding disclosure in at least [0170], where the method uses augmented reality and updates a surgical procedure “the set of 3D stereoscopic images may be dynamically updated during the surgical procedure if the surgeon deviates from the predefined surgical plan. In particular, the set of 3D stereoscopic images may be displayed on a 3D display (such as a monitor, an augmented-reality headset”),
the method comprising:
providing a procedural plan of the medical procedure (corresponding disclosure in at least [0169], where there is a plan provided of the procedure “the surgeon (or a physician) may visualize patient specific anatomy that has been identified as part of the preoperative plan at its patient specific location through the patient's body and plan out a series of tasks or operations they will perform on a patient during a planned surgical procedure”);
performing the medical procedure on the patient utilizing the procedural plan and the augmented reality system (corresponding disclosure in at least [0170], where the procedure is being performed using augmented reality “In particular, the set of 3D stereoscopic images may be displayed on a 3D display (such as a monitor, an augmented-reality headset…Computer system 600 (FIG. 6) may step through the set of 3D stereoscopic images as the surgical procedure progresses”);
recording a metric during the medical procedure using the augmented reality system to provide a recorded metric (corresponding disclosure in at least [0170], where there is a recorded metric (the location of the tool) “Computer system 600 (FIG. 6) may step through the set of 3D stereoscopic images as the surgical procedure progresses, e.g., based on user input via a user interface or voice commands, or based on information that specifies the surgeon's actions or location in patient's anatomy. For example, the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel)”), the recorded metric including photogrammetry, videogrammetry, ultrasound, and MRI (corresponding disclosure in at least [0047], where there is an input of MRI i.e. “ data engine 110 may receive input data (such as a computed-tomography or CT scan, histology, an ultrasound image, a magnetic resonance imaging or MRI scan, or another type of 2D image slice depicting volumetric information)”) ;
updating the procedural plan based on the recorded metric to provide an updated procedural plan (corresponding disclosure in at least [0170], where the plan is updated based on the recorded metric (analysis of surgical tool location) providing an updated plan “If the surgeon intentionally or unintentionally deviates from the predefined surgical plan (e.g., using on-the-fly or dynamic analysis of the location of a surgical tool relative to the predefined surgical plan), computer system 600 (FIG. 6) may update or revise the displayed 3D stereoscopic images. The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback)”);
and using the updated procedural plan as the procedural plan, thereby establishing a feedback loop for the procedural plan of the medical procedure (corresponding disclosure in at least [0174] and Figure 16, where there is a feedback loop (the steps repeat) which updates the plan “Thus, when generating the stereoscopic images (operation 1614) or preparing the stereoscopic images, information from optionally tracked motion (operation 1610) and/or the optionally tracked interaction may be used to generate or revise the view and projection matrices”).
PNG
media_image1.png
498
316
media_image1.png
Greyscale
Figure 16 of Aguirre-Valencia
Aguirre-Valencia does not teach the medical procedure including treatment of a soft tissue moving due to respiration of the patient, the recorded metric including deformation of the soft tissue, the updated procedural plan allowing visualization of respiratory compensation integration through use of the augmented reality system, and wherein the augmented reality system utilizes a machine learning algorithm to modify a holographic representation and intraoperative guidance of the procedural plan based on a plurality of iterations of: recording the metric during the medical procedure using the augmented reality system; updating the procedural plan based on the recorded metric; and providing the updated procedural plan as the procedural plan, thereby establishing the feedback loop for the procedural plan of the medical procedure.
Lang, in a similar field of endeavor, teaches a similar concept (augmented reality for updating procedures) of the medical procedure including treatment of a soft tissue moving due to respiration of the patient (corresponding disclosure in at least [0409], where there is treatment of a soft tissue moving due to respiration, many of the listed surgeries directly being affected by respiration “The surgical alteration or surgical steps can include, but are not limited to the listed in the following: Exemplary surgical alterations or steps applied to various patient tissues, e.g. bone, skin, fat, organ tissue, e.g. liver, spleen, kidney, intestines, gallbladder, lung, heart, thyroid, brain etc.: Cutting, drilling, pinning, radiofrequency ablation, heat ablation, cryoablation, cauterization, tissue resection, tissue removal, resection of a neoplasm, fracture fixation, trauma repair, trauma reconstruction, soft-tissue repair”);
the recorded metric including deformation of the soft tissue measured by one of photogrammetry, videogrammetry, ultrasound, and MRI (corresponding disclosure in at least [1033] “a deformation field can be determined to model soft tissue deformations between the preoperative and intraoperative images”;
the updated procedural plan allowing visualization of respiratory compensation integration through use of the augmented reality system (corresponding disclosure in at least [1252], where there is visualization of the respiratory movement via augmented reality “Respiratory gating can be performed using any system or method known in the art. In any of the embodiments, see through augmented reality optical head mounted displays can be used”); and
the augmented reality system utilizes a machine learning algorithm to modify a holographic representation and intraoperative guidance of the procedural plan based on a plurality of iterations of (corresponding disclosure in at least [0186], where augmented reality modifies the holographic representation and the guidance plan “ The pre-operative data 16 or live data 18 including intra-operative measurements or combinations thereof can be used to develop, generate or modify a virtual surgical plan 24”):
recording the metric during the medical procedure using the augmented reality system (corresponding disclosure in at least [0269], where the metric is recorded during the procedure “the OHMD can be optionally placed in a fixed position, e.g. mounted on a stand or on a tripod. While the OHMD is placed in the fixed position, live data can be viewed by the surgeon or interventionalist and they can be, optionally recorded with a camera and/or displayed on a monitor. Virtual data can then be superimposed and the matching and registration of virtual data and live data can be performed”);
updating the procedural plan based on the recorded metric; and providing the updated procedural plan as the procedural plan, thereby establishing the feedback loop for the procedural plan of the medical procedure (corresponding disclosure in at least [0392], where the plan is updated based on the metric, or a movement of the patient, which would include respiration, with the plan being provided with each update “the resultant information can, for example, be used to update or adjust or modify a virtual surgical plan or to update or adjust or modify the display of the virtual surgical plan or virtual surgical steps or virtual displays for the movement of the patient”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the recording of deformation of soft tissues, updating procedural plans based on respiration, and updating the plan based on the deformation as taught by Lang. One of the ordinary skill in the art would have been motivated to incorporate this because the tissues and organs move due to respiratory motion, and tracking such motion provides better accuracy for the surgeon during procedures.
Regarding Claim 2, Aguirre-Valencia and Lang teach the limitations of Claim 1, and Aguirre-Valencia further teaches identifying a difference between the procedural plan and the updated procedural plan (corresponding disclosure in at least [0170], where the plan is updated “Furthermore, the set of 3D stereoscopic images may be dynamically updated during the surgical procedure if the surgeon deviates from the predefined surgical plan” and further in [0171], where the plans and updates are all saved, so the differences can be identified “Before, during and/or at the end of the surgical procedure, the surgical plan, the locations of the surgical instruments and the progression of the surgical procedure, and the outcome may be saved in a computer-readable data structure for subsequent analysis and/or use”).
wherein the augmented reality system is configured to track a position and orientation of a medical instrument in real-time during the medical procedure (corresponding disclosure in at least [0170], where the position is tracked in real-time (dynamic tracking) “the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel), using: a local positioning system (such as a 3D local positioning system that tracks the position of one or more surgical tools in the patient, such as relative to one or more anatomical landmarks in the patient)”);
and the method further comprises:
in relation to the medical instrument; and analyzing a postoperative result to compare an outcome of the updated procedural plan with the procedural plan and ascertaining instrument placement and effectiveness of the instrument (corresponding disclosure in at least [0170], where the placement of the medical instrument is ascertained “the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel)” and further where the system will indicate the most effective placement (correcting the deviation/showing the predefined plan if veered off-course) “ If the surgeon intentionally or unintentionally deviates from the predefined surgical plan (e.g., using on-the-fly or dynamic analysis of the location of a surgical tool relative to the predefined surgical plan), computer system 600 (FIG. 6) may update or revise the displayed 3D stereoscopic images. The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback), the current location (relative to the patient's anatomy), and/or how to correct the deviation so that the surgical procedure follows the predefined surgical plan”).
Regarding Claim 3, Aguirre-Valencia and Lang teach the limitations of Claim 1, and Aguirre-Valencia further teaches providing the updated procedural plan as the procedural plan for a subsequent medical procedure (corresponding disclosure in at least [0171], where the updated plan can be saved for subsequent use “Before, during and/or at the end of the surgical procedure, the surgical plan, the locations of the surgical instruments and the progression of the surgical procedure, and the outcome may be saved in a computer-readable data structure for subsequent analysis and/or use”).
Regarding Claim 4, Aguirre-Valencia and Lang teach the limitations of Claim 1, and Aguirre-Valencia further teaches wherein the augmented reality system provides intraoperative guidance during the medical procedure by providing an augmented reality display of an aspect of the procedural plan and tracking of a medical instrument (corresponding disclosure in at least [0170], where there is an augmented reality system using intraoperative guidance showing the plan and tracking of a medical instrument (surgical tool) “the set of 3D stereoscopic images may be displayed on a 3D display (such as a monitor, an augmented-reality headset… the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel)”).
Regarding Claim 5, Aguirre-Valencia and Lang teach the limitations of Claim 4, and Aguirre-Valencia further teaches wherein the intraoperative guidance includes a step of alerting the practitioner to a deviation from the procedural plan (corresponding disclosure in at least [0170], where there is an alert when there is a deviation from the plan “The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback), the current location (relative to the patient's anatomy), and/or how to correct the deviation so that the surgical procedure follows the predefined surgical plan”).
Regarding Claim 6, Aguirre-Valencia and Lang teach the limitations of Claim 5, and Aguirre-Valencia further teaches wherein the step of alerting includes visual or auditory cue (corresponding disclosure in at least [0170], where the alert is visual or auditory ““The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback)”).
Regarding Claim 7, Aguirre-Valencia and Lang teach the limitations of Claim 4, and Aguirre-Valencia further teaches wherein providing the updated procedural plan as the procedural plan includes a step of adjusting a holographic representation by the augmented reality system (corresponding disclosure in at least [0050], where each step is updated via tracking “ one or more optional position sensors 116 (which may be separate from or integrated into display 114) may dynamically track movement of the head of viewer 122 with up to six degrees of freedom, and this head-tracking information (e.g., the positions of the eyes of viewer 122 relative to display 114) may be used by graphics engine 112 to update the view and frustum matrices and, thus, the rendered left-eye and right-eye images” and further in [0049], where it is specified the plans are demonstrated in a holographic representation “These images may be appropriately scaled or sized so that the images match the physical parameters of the viewing geometry (including the position of viewer 122 and size 126 of the display 114). This may facilitate the holographic effect for viewer”); and
the method further includes a step of providing the updated procedural plan as the procedural plan for a subsequent medical procedure (corresponding disclosure in at least [0171], where the updated plan can be saved for subsequent use “Before, during and/or at the end of the surgical procedure, the surgical plan, the locations of the surgical instruments and the progression of the surgical procedure, and the outcome may be saved in a computer-readable data structure for subsequent analysis and/or use”).
Regarding Claim 8, Aguirre-Valencia and Lang teach the limitations of Claim 7, and
Aguirre-Valencia further teaches wherein the holographic representation includes a three- dimensional model of a portion of an anatomy of the patient derived from medical imaging data (corresponding disclosure in at least [0047], where there is an input of a medical imaging data “ data engine 110 may receive input data (such as a computed-tomography or CT scan, histology, an ultrasound image, a magnetic resonance imaging or MRI scan, or another type of 2D image slice depicting volumetric information), including dimensions and spatial resolution.” And further in [0049], where the data is then rendered to be 3D “These model, view and frustum matrices may be used by graphics engine 112 to render images of the 3D objects. For a given eye, the rendered image may provide a 2.5D monoscopic projection view on display 114. By sequentially displaying left-eye and right-eye images that include image parallax (i.e., stereoscopic images), 3D information may be presented on display”).
Regarding Claim 9, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia further teaches wherein the recorded metric obtained during the medical procedure includes data selected from the group consisting of instrument positioning metrics,
instrument interaction metrics, operative action metrics, outcome-related metrics, procedure efficiency metrics, and system interaction metrics (corresponding disclosure in at least [0170], where the recorded metric is the instrument positioning “ Computer system 600 (FIG. 6) may step through the set of 3D stereoscopic images as the surgical procedure progresses, e.g., based on user input via a user interface or voice commands, or based on information that specifies the surgeon's actions or location in patient's anatomy. For example, the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel), using: a local positioning system (such as a 3D local positioning system that tracks the position of one or more surgical tools in the patient, such as relative to one or more anatomical landmarks in the patient), image processing of images acquired in the operating room, etc.”).
Regarding Claim 10, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia further teaches wherein providing the updated procedural plan as the procedural plan includes a step of adjusting a holographic representation by the augmented reality system based on the recorded metric (corresponding disclosure in at least [0170], where the holographic representation by the augmented reality system will be adjusted (the augmented reality headset) “the set of 3D stereoscopic images may be dynamically updated during the surgical procedure if the surgeon deviates from the predefined surgical plan. In particular, the set of 3D stereoscopic images may be displayed on a 3D display (such as a monitor, an augmented-reality headset”).
Regarding Claim 11, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia further teaches wherein the augmented reality system includes a head- mounted display for rendering a holographic representation to the practitioner performing the medical procedure (corresponding disclosure in at least [0170], where the system includes a head-mounted display “the set of 3D stereoscopic images may be displayed on a 3D display (such as a monitor, an augmented-reality headset”)”).
Regarding Claim 12, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia teaches wherein the head-mounted display is configured to overlay the holographic representation within a view by the practitioner of the patient receiving the medical procedure (corresponding disclosure in at least [0179], where the system will continue to update the overlay based on the perspective (view) of the user “ as the user interacts with the 3D stereoscopic images and/or the one or more 2D projections and changes their viewing perspective, the computer system may dynamically update the 3D stereoscopic images and the 2D projections based on the current perspective (operation 1812). In some embodiments, note that the 2D projections are always presented along a perspective direction perpendicular to the user so that motion parallax is registered in the 2D projections”).
Regarding Claim 13, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia further teaches wherein providing the updated procedural plan as the procedural plan, thereby establishing the feedback loop for the procedural plan of the medical procedure, includes a step of analyzing a postoperative result to compare an outcome of the updated procedural plan with the procedural plan within the step of providing the procedural plan (corresponding disclosure in at least [0169], where the procedural plan (the preoperative plan) is compared with an updated plan by allowing surgeons to interact and mark the procedure “ visualize patient specific anatomy that has been identified as part of the preoperative plan at its patient specific location through the patient's body and plan out a series of tasks or operations they will perform on a patient during a planned surgical procedure (i.e., to determine a surgical plan). True 3D may allow the surgeon to view the 3D environment in the patient (including organs and landmarks in the context of the patient's anatomy) as the planned surgical procedure progresses. This may allow the surgeon to visualize the planned surgical procedure and to interact with the 3D stereoscopic images (which may include haptic interaction with a virtual instrument as a surgical tool) so that the surgeon can determine an efficient manner in which to perform the planned surgical procedure. The resulting set of 3D stereoscopic images (including image parallax, motion parallax, prehension and/or stereopsis scaling) may be saved by the surgeon for subsequent viewing before and/or during the surgical procedure”)
and, wherein the step of analyzing the postoperative result further includes steps of ascertaining a placement of a medical instrument and ascertaining an effectiveness of the medical instrument (corresponding disclosure in at least [0170], where the placement of the medical instrument is ascertained “the information may include dynamic tracking of the surgeon and/or one or more of the surgeon's surgical tools (such as a scalpel)” and further where the system will indicate the most effective placement (correcting the deviation/showing the predefined plan if veered off-course) “ If the surgeon intentionally or unintentionally deviates from the predefined surgical plan (e.g., using on-the-fly or dynamic analysis of the location of a surgical tool relative to the predefined surgical plan), computer system 600 (FIG. 6) may update or revise the displayed 3D stereoscopic images. The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback), the current location (relative to the patient's anatomy), and/or how to correct the deviation so that the surgical procedure follows the predefined surgical plan”).
Regarding Claim 15, Aguirre-Valencia and Lang teach the limitations of Claim 1, and
Aguirre-Valencia further teaches wherein the augmented reality system is further configured to integrate and display data from an imaging system and a tracking system (corresponding disclosure in at least [0047], where integrated data includes one from an imaging system (i.e. CT) “ data engine 110 may receive input data (such as a computed-tomography or CT scan, histology, an ultrasound image, a magnetic resonance imaging or MRI scan, or another type of 2D image slice depicting volumetric information), including dimensions and spatial resolution” and further in [0050], where information for a tracking system (head tracking information from optional position sensors) are also used “one or more optional position sensors 116 (which may be separate from or integrated into display 114) may dynamically track movement of the head of viewer 122 with up to six degrees of freedom, and this head-tracking information (e.g., the positions of the eyes of viewer 122 relative to display 114) may be used by graphics engine”).
Claims 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Aguirre-Valencia (US20160381256A1) in view of Liarno (US20230377714A1).
Regarding Claim 16, Aguirre-Valencia and Lang teach the limitations of Claim 1, and Aguirre-Valencia further teaches wherein the augmented reality system is further configured to provide a postoperative review interface to display the recorded metric relative to the procedural plan (corresponding disclosure in at least [0170], where the procedure with the progression is saved alongside the recorded metric relative to the procedural plan (location of the surgical instruments) “If the surgeon intentionally or unintentionally deviates from the predefined surgical plan (e.g., using on-the-fly or dynamic analysis of the location of a surgical tool relative to the predefined surgical plan), computer system 600 (FIG. 6) may update or revise the displayed 3D stereoscopic images. The revised 3D stereoscopic images may reflect or indicate: that a deviation from the predefined surgical plan has occurred (e.g., via visual, auditory and/or sensory feedback), the current location (relative to the patient's anatomy), and/or how to correct the deviation so that the surgical procedure follows the predefined surgical plan” and further in [0171] “Before, during and/or at the end of the surgical procedure, the surgical plan, the locations of the surgical instruments and the progression of the surgical procedure, and the outcome may be saved in a computer-readable data structure for subsequent analysis and/or use”), but does not teach providing adjustment of the procedural plan.
Liarno, in a similar field of endeavor, teaches a similar concept (adaptations for improving surgical plans) of providing adjustment of the procedural plan (corresponding disclosure in at least [0168], where given a postoperative review adjustments can be made “This postoperative plan 9030 may be newly generated based on postoperative data 3000 and/or may be a modification to the postoperative plan 8030 generated using the intraoperative data 2000 (and/or a manually input) and/or the postoperative plan 7030 generated using the preoperative data 1000 (and/or manually input)… manually input an adjustment to the postoperative plan 9030 via an electronic device. The postoperative plan 9030 may be continuously adjusted and/or updated as more postoperative data 3000 is collected”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated providing adjustments of the procedural plan as taught by Liarno. One of the ordinary skill in the art would have been motivated to incorporate this because it provides an improvement to the future plan for assisting further surgical procedures.
Regarding Claim 18, Aguirre-Valencia and Lang teach the limitations of Claim 1, and Aguirre-Valencia further teaches the augmented reality system ([0170]), but does not teach to provide a preemptive action relative to the procedural plan using predictive analytics based on the step of providing the updated procedural plan as the procedural plan, thereby establishing the feedback loop for the procedural plan of the medical procedure.
Liarno, in a similar field of endeavor, teaches to provide a preemptive action relative to the procedural plan using predictive analytics based on the step of providing the updated procedural plan as the procedural plan, thereby establishing the feedback loop for the procedural plan of the medical procedure (corresponding disclosure in at least disclosure in at least [0227], where predictions can be made for further actions (preemptive actions) based on the procedural outputs “The Finite Element Analysis algorithm 4040 may also be configured to assist postoperative algorithms 6000 in making determinations. The Finite Element Analysis algorithm 4040 may be configured to predict preoperative outputs 7000, intraoperative outputs 8000, and/or postoperative outputs 9000, and may further be configured to make determinations based on the predicted outputs 7000, 8000, 9000” with Figure 1 showing the process working in a feedback loop) .
PNG
media_image2.png
428
663
media_image2.png
Greyscale
Figure 1 of Liarno
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated a preemptive action using a predictive analysis as taught by Liarno. One of the ordinary skill in the art would have been motivated to incorporate this because further actions and next steps are determined by taking into account a consistently updated procedural plan, providing the most accurate medical procedure.
Claims 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Aguirre-Valencia (US20160381256A1) in view of Lang (US20210137634A1) as applied in Claim 1 and in view of Allenby (US20170181808A1) and in further view of Ballicki (US20220409292A1).
Regarding Claim 21, Aguirre-Valencia and Lang teach the limitations of Claim 1, and the augmented reality system, but do not teach to provide a heat map to the practitioner, the heat map depicting deformation of the soft tissue relative to treatment of a target in the soft tissue and an effect of a medical device used by the practitioner in performing the medical procedure on the patient.
Allenby, in a similar field of endeavor, teaches a similar concept (surgical systems and updating upon information) of deformation of the soft tissue relative to treatment of a target in the soft tissue and an effect of a medical device used by the practitioner in performing the medical procedure on the patient (corresponding disclosure in at least [0165], where the deformation of soft tissue is depicted or mapped relative to the effect of a medical device, or the instrument “Similarly, if the instrument force is mapped onto the TUI 2702, the deformation amount XUI2 may be larger or smaller than XUI1 according to the change in force to Ft2 from Ft1.”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the deformation of soft tissue relative to treatment of a target as taught by Allenby. One of the ordinary skill in the art would have been motivated to incorporate this because it is critical in accounting for possible mismatch in preoperative images and ensures higher accuracy in navigation.
The combined references noted above do not teach the heat map.
Ballicki, in a similar field of endeavor teaches a similar concept (surgical navigation systems) of a heat map (corresponding disclosure in at least [0055], where a heat map is generated “ limiting deformation on the esophagus to manage perforation risk; elastography, creating a 3D heat map of force loading regions of the esophagus or stomach”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated a heat map as taught by Ballicki. One of the ordinary skill in the art would have been motivated to incorporate this because the heat map helps visualize the information to better identify the trends in the image.
Regarding Claim 22, the combined references noted above teach the limitations of Claim 21, and Ballicki further teaches the medical device used by the practitioner includes a flexible device including one of an ultrasound device, a catheter, and a camera; (corresponding disclosure in at least [0028], where a flexible device, the ultrasound, is used “1 , the ultrasound device 10 includes a probe 12 configured as, for example, a flexible cable or tube that serves as a catheter for insertion into a lumen of the patient”)
and the augmented reality system provides augmented reality tracking (corresponding disclosure in at least [0967] of Lang, where a medical device is tracked within the patient using augmented reality “tracking the one or more optical imaging systems and/or 3D scanners outside and/or inside the patient's body (or any structure of the human body, e.g. an abdominal cavity, a lumen etc.) can be useful for directing the optical imaging system and/or 3D scanner to a desired location”) of the flexible device.
Response to Arguments
Applicant's arguments filed 06/12/2026 regarding the 35 U.S.C. 101 rejections have been fully considered and the rejections are withdrawn in the light of the amendments.
Applicant’s arguments regarding the 35 U.S.C. 102 and 103 rejections have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN KIM whose telephone number is (571)272-1821. The examiner can normally be reached Monday-Friday 6-2 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.E.K./Examiner, Art Unit 3797
/SERKAN AKAR/Primary Examiner, Art Unit 3797
Prosecution Insights