DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending and are currently under consideration for patentability under 37 CFR 1.104.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5, 7-10, and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Miyayashiki (US 2012/0078043).
Regarding claim 1, Miyayashiki discloses a medical system, comprising: an instrument (endoscope apparatus 20, figure 1) configured to be inserted into an anatomy (insertion portion 1, figure 1); a camera (6, figure 1) disposed on a distal end of the instrument and configured to capture images of the anatomy within its field-of-view (FOV) (object image [0039]); and a control system (2, figure 1) configured to: display the images in a graphical user interface (GUI) (3, figure 1 | GUI [0058]); determine an orientation of the FOV relative to the anatomy (gravity information…an inclination angle with respect to plumb/straight down or a vertical/straight up direction [0043] | interpreted the gravity direction to indicate how the patient is laying, which would be relative to the anatomy); and generate a compass for display with the images on the GUI (s6 or s7, figure 2 | see figure 3), the compass indicating the orientation of the FOV depicted by the images relative to the anatomy (see s9, figure 2 | see figure 3).
Regarding claim 2, Miyayashiki further discloses the control system is further configured to: receive user input for toggling the compass on or off (turning on/off the superimposed display…screen showing only the video display [0083]); and selectively display the compass on the GUI based at least in part on the received user input (user operation may switch…[0083]).
Regarding claim 3, Miyayashiki further discloses the control system is further configured to selectively display the compass on the GUI based at least in part on a position, orientation, or movement direction of the instrument (not determined that there is an operation…gravity direction display ends [0056]).
Regarding claim 4, Miyayashiki further discloses the control system is further configured to: determine whether a movement of the instrument is parallel to a first axis (see xyz-axes in figure 3) in a cartesian space (s5, figure 2 | gravity direction is…[0058]); and display the compass on the GUI responsive to determining that the movement of the instrument is parallel to the first axis (either s6 or s7, figure 2 | arrow of the gravity direction GUI [0059]), the compass indicating one or more axes of the cartesian space that are perpendicular to the first axis (see figure 3).
Regarding claim 5, Miyayashiki further discloses the control system is further configured to change an appearance of one or more direction indicators (either s6 or s7, figure 2 | arrow of the gravity direction GUI [0059]) associated with the compass based at least in part on a position, orientation, or movement direction of the instrument (s5, figure 2 | gravity direction is…[0058]).
Regarding claim 7, Miyayashiki further discloses the GUI further comprises an indicator indicating a center of an anatomical channel within which the camera is disposed relative to the orientation of the FOV (best seen with figures 3 and 9 | 6-2, figure 6).
Regarding claim 8, Miyayashiki further discloses the compass includes an orientation indicator indicating at least a first anatomical direction and a second anatomical direction associated with a reference axis of the anatomy (see xyz-axes, figure 3).
Regarding claim 9, Miyayashiki further discloses the orientation indicator is a two- dimensional (2D) orientation indicator having a first axis associated with anterior and posterior anatomical directions and having a second axis associated with medial and lateral anatomical directions (see xyz-axes, figure 3 | the axes may coincide with the anterior, posterior, medial, and lateral anatomical directions dependent on the gravity direction, which would be impacted by how the patient is oriented during the procedure).
Regarding claim 10, Miyayashiki further discloses the orientation indicator is a three- dimensional (3D) orientation indicator (three-dimensional can be displayed [0166]) having a first axis associated with anterior and posterior anatomical directions (see figures 9 | gravity direction and opposite gravity direction can be associated with anterior and posterior directions, depending on how the patient is oriented during the procedure), a second axis associated with medial and lateral anatomical directions (x- or y-axis, figure 3 | see figure 9), and a third axis associated with superior and inferior anatomical directions based on yaw and pitch of the instrument (x- or y-axis, figure 3 | see figure 9).
Regarding claim 18, Miyayashiki discloses a method performed by a medical system, comprising: receiving images of an anatomy (object image [0039]) captured within a field-of-view (FOV) of a camera disposed on a distal end (6, figure 1) of an instrument (endoscope apparatus 20, figure 1) configured to be inserted into the anatomy (insertion portion 1, figure 1); displaying the images in a graphical user interface (GUI) (3, figure 1 | GUI [0058]); determining an orientation of the FOV relative to the anatomy (gravity information…an inclination angle with respect to plumb/straight down or a vertical/straight up direction [0043] | interpreted the gravity direction to indicate how the patient is laying, which would be relative to the anatomy); and generating a compass for display with the images on the GUI (s6 or s7, figure 2 | see figure 3), the compass indicating the orientation of the FOV depicted by the images relative to the anatomy (see s9, figure 2 | see figure 3).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Miyayashiki (US 2012/0078043) as applied to claim 1 above, and further in view of Brown (US 2016/0000414).
Regarding claim 6, Miyayashiki discloses the features of the current invention as shown above in claim 1. Miyayashiki is silent regarding the GUI further comprises a target site indicator indicating a distance or direction of a target relative to the orientation of the FOV.
Brown teaches an electromagnetic navigation (EMN) system (10, figure 1) with a bronchoscope (50, figure 1), a catheter guide assembly (90, 100, figure 1), an electromagnetic tracking system (70, figure 1), a tracking module (72, figure 1), and a workstation (80, figure 1) with an application (81, figure 1). The workstation utilizes CT image data for generating and viewing a 3D model, enables the identification of target tissue on the 3D model, and allows for the selection of a pathway through the patient’s airway to the target tissue ([0042]). During navigation, EM sensor (94, figure 1) in conjunction with the tracking system (70, figure 1), enables tracking of EM sensor as it is advanced through the patient’s airway. The 3D dynamic view (606, figure 4) presents the dynamic 3D model (614, figure 4) and the orientation of the 3D model automatically updates based on movement of the EM sensor within the patient’s airway ([0050]).
It would have been obvious to modify the system of Miyayashiki with the EM sensor (94, figure 1) on the distal end of the instrument, the electromagnetic tracking system (70, figure 1), a tracking module (72, figure 1), and a workstation (80, figure 1) with an application (81, figure 1) as taught by Brown. Doing so would provide a 3D map dynamic view (606, figure 4) that automatically updates based on movement of the EM sensor ([0050]). The modified system would have the GUI further comprises a target site indicator (606, figure 4; Brown) indicating a distance or direction of a target relative to the orientation of the FOV (automatically updates based on movement of the EM sensor [0050]).
Claim(s) 11-12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Miyayashiki (US 2012/0078043) as applied to claims 1 and 18 above, and further in view of Kambe (US 2022/0330790).
Regarding claim 11, Miyayashiki discloses all of the features in the current invention as shown above in claim 1. Miyayashiki is silent regarding the control system is further configured to: determine a degree of articulation by the instrument; determine a direction of articulation by the instrument; and display an articulation indicator on the GUI having a length indicating the degree of articulation and a position relative to the GUI indicating the direction of articulation.
Kambe teaches an endoscope system (1000G, figure 54) with a drive controller (260B, figure 27) of a drive device (200B, figure 54). The drive controller transmits the bending amount of the first joint in the UD and LR direction and the bending amount of the second joint in the UD and LR direction to an input and output control portion ([0572]). A processor generates a three-dimensional image of the endoscope viewed in a determined direction based on each bending amount ([0572]). A meter can be displayed that indicates the bending amount of the first joint in the UD or LR direction (see MT1 and MT2, figure 66). The needle (ND1 and ND2, figure 66) shows the current value of the bending amount, where the end portion of the range in which the needles can be displayed indicates the maximum value of the bending amount in that direction ([0611]).
It would have been obvious to modify the system of Miyayashiki with the endoscope system (1000G, figure 54) and drive device (200B, figure 54) as taught by Kambe. Doing so would display the current value and maximum value of bending amount in the up/down and left/right direction of the instrument (see MT1 and MT2, figure 6 | [0611]). The modified system would determine a degree of articulation by the instrument (bending amount…[0572]); determine a direction of articulation by the instrument (MT1 or MT2, figure 66); and display an articulation indicator on the GUI having a length indicating the degree of articulation (location of ND1 and ND2, figure 66) and a position relative to the GUI indicating the direction of articulation (MT1 or MT2, figure 66; Kambe).
Regarding claim 12, Kambe further teaches the control system is further configured to: determine that the degree of articulation by the instrument is equal to a maximum articulation (see end portion of the range of the needles in MT1 and MT2, figure 66; Kambe); and change at least one property associated with the articulation indicator based on determining that the degree of articulation by the instrument is equal to the maximum articulation (needle ND1 or ND2 would move to the end portion of the range in MT1 or MT2, figure 66).
Regarding claim 19, Miyayashiki discloses all of the features in the current invention as shown above in claim 18. Miyayashiki is silent regarding determining a degree of articulation by the instrument; determining a direction of articulation by the instrument; and displaying an articulation indicator on the GUI having a length indicating the degree of articulation and a position relative to the GUI indicating the direction of articulation.
Kambe teaches an endoscope system (1000G, figure 54) with a drive controller (260B, figure 27) of a drive device (200B, figure 54). The drive controller transmits the bending amount of the first joint in the UD and LR direction and the bending amount of the second joint in the UD and LR direction to an input and output control portion ([0572]). A processor generates a three-dimensional image of the endoscope viewed in a determined direction based on each bending amount ([0572]). A meter can be displayed that indicates the bending amount of the first joint in the UD or LR direction (see MT1 and MT2, figure 66). The needle (ND1 and ND2, figure 66) shows the current value of the bending amount, where the end portion of the range in which the needles can be displayed indicates the maximum value of the bending amount in that direction ([0611]).
It would have been obvious to modify the method of Miyayashi to use an endoscope system (1000G, figure 54) and drive device (200B, figure 54) as taught by Kambe. Doing so would display the current value and maximum value of bending amount in the up/down and left/right direction of the instrument (see MT1 and MT2, figure 6 | [0611]). The modified method would comprise determining a degree of articulation by the instrument (bending amount…[0572]); determining a direction of articulation by the instrument (MT1 or MT2, figure 66); and displaying an articulation indicator on the GUI having a length indicating the degree of articulation (location of ND1 or ND2, figure 66) and a position relative to the GUI indicating the direction of articulation (MT1 or MT2, figure 66; Kambe).
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over Miyayashiki (US 2012/0078043) and Kambe (US 2022/0330790) as applied to claim 11 above, and further in view of Morgan (US 2015/0054753).
Regarding claim 13, Miyayashiki and Kambe disclose all of the features in the current invention as shown above in claim 11. They are silent regarding the control system is further configured to: determine that the instrument is obstructed from further articulation; and change at least one property associated with the articulation indicator based on determining that the instrument is obstructed from further articulation.
Mogan teaches that if an articulation joint is obstructed or blocked, the symbolic representation (5254, figures 84-85) of the articulation angle can provide a visible indication of articulation to the user ([0407]).
It would have been obvious to modify the system of Miyayashiki and Kambe to provide an indication of an obstructed articulation to the user as taught by Morgan ([0407]). Doing so would provide a visible indication that the articulation is being obstructed or blocked ([0407]). The modified system would determine that the instrument is obstructed from further articulation (obstructed or blocked [0407]; Morgan); and change at least one property associated with the articulation indicator based on determining that the instrument is obstructed from further articulation (visible indication…to the user [0407]).
Claim(s) 14-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Miyayashiki (US 2012/0078043) as applied to claims 1 and 18 above, and in further view of Weeks (US 2022/0304559).
Regarding claim 14, Miyayashiki discloses all of the features in the current invention as shown above in claim 1. Miyayashiki is silent regarding the control system is further configured to: configure the images to depict an expanded view of the FOV or a restricted view of the FOV for display in the GUI, the expanded view representing the FOV in its entirety and the restricted view representing only a portion of the FOV.
Weeks teaches a system (figure 1) with an endoscope (100, figure 1) and an image processor (104, figure 1). The system may crop (644, figure 28) the image data to make it suitable for the intended display device ([0172] | see 682, figure 29E and 684, figure 29F). The parameters of the cropping operation may be pre-configured or may be determined by a user input ([0172]).
It would have been obvious to modify the system to crop the image as taught by Weeks ([0172]). Doing so would provide an image that is more suitable for display ([0172]). The modified system would configure the images to depict an expanded view of the FOV (684, figure 29F; Weeks) or a restricted view of the FOV for display in the GUI (682, figure 29E), the expanded view representing the FOV in its entirety and the restricted view representing only a portion of the FOV (see 682 vs. 684, figures 29E-F).
Regarding claim 15, Weeks further teaches the control system is further configured to toggle between the expanded view and the restricted view for display on the GUI responsive to user input (based upon a user input [0172]; Weeks).
Regarding claim 16, Weeks further teaches the control system is further configured to toggle between the restricted view and the expanded view in response to detecting one or more conditions associated with navigating the instrument within the anatomy (pre-configured…[0172]; Weeks | the modified pre-configuration can be based on a condition of navigation of the instrument).
Regarding claim 17, Weeks further teaches the expanded view includes one or more simulated image portions (post-processing image data [0172] | image processor…perform [0171]; Weeks).
Regarding claim 20, Miyayashiki discloses all of the features in the current invention as shown above in claim 18. Miyayashiki is silent regarding configuring the images to depict an expanded view of the FOV or a restricted view of the FOV for display in the GUI in response to user input or detecting one or more conditions associated with navigating the instrument within the anatomy, the expanded view representing the FOV in its entirety and the restricted view representing only a portion of the FOV.
Weeks teaches a system (figure 1) with an endoscope (100, figure 1) and an image processor (104, figure 1). The system may crop (644, figure 28) the image data to make it suitable for the intended display device ([0172] | see 682, figure 29E and 684, figure 29F). The parameters of the cropping operation may be pre-configured or may be determined by a user input ([0172]).
It would have been obvious to modify the method to crop the image as taught by Weeks ([0172]). Doing so would provide an image that is more suitable for display ([0172]). The modified method would comprise configuring the images to depict an expanded view of the FOV (684, figure 29F; Weeks) or a restricted view of the FOV for display in the GUI (682, figure 29E) in response to user input (user input [0172]) or detecting one or more conditions associated with navigating the instrument within the anatomy, the expanded view representing the FOV in its entirety and the restricted view representing only a portion of the FOV (see 682 vs. 684, figures 29E-F).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAMELA F WU whose telephone number is (571)272-9851. The examiner can normally be reached M-F: 8-4 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Carey can be reached at 571-270-7235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
PAMELA F. WU
Examiner
Art Unit 3795
September 5, 2026
/RYAN N HENDERSON/Primary Examiner, Art Unit 3795
Prosecution Insights