Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. Therefore, Applicant is advised to review all portions of the cited prior art if traversing a rejection based on the cited prior art.
Claims 1-5, 10, 15, and 17-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”).
Regarding Claim 1, Tognaccini discloses:
A robotic system (Tognaccini FIG. 1, medical robotic system 100), comprising:
a first robotic arm (Tognaccini FIG. 1, robotic arm assembly 130) coupled to a flexible scope (Tognaccini FIG. 1, entry guide 200; Tognaccini FIG. 3, camera 211; Tognaccini paragraph [0046], “entry guide 200 has…camera 211 extending out of its distal end”);
a viewer (Tognaccini FIG. 1, 3-D monitor 104 and/or auxiliary display screen 140) for displaying a field of view of a surgical site derived from the flexible scope (Tognaccini paragraph [0043], “a 3-D monitor 104 for displaying a 3-D image of a surgical site to the Surgeon”;
one or more processors (Tognaccini FIG. 1, processor 102); and
a memory configured to store instructions for execution by the one or more processors (Tognaccini paragraph [0050], “it is to be appreciated that the processor 102 may be implemented in practice by any combination of hardware, software and firmware”; Tognaccini paragraph [0073], “the method generates a three-dimensional computer model of the articulatable camera 211 and articulatable surgical tools 231, 241 extending out of the distal end of the entry guide 200 using the information received in 902”), the stored instructions including instructions for:
operating the flexible scope in a particular mode of a plurality of modes (Tognaccini FIG. 18, first, second, and third auxiliary viewing modes 1802, 1804, 1806); and
in accordance with a determination that the flexible scope is operating in the particular mode of the plurality of modes, providing electrical signals for presenting a first visual indicator, corresponding to the respective mode, on the viewer, the first visual indicator indicating at least one of: a position or an orientation of the flexible scope relative to a reference position (Tognaccini FIG. 30, anatomic structure 360, tools 241 and 231, views 2100, 3001, 3002; Tognaccini paragraph [0109], “FIG. 30 illustrates, as an example, a display screen of the monitor 104 in which an image 1501 captured by the camera instrument 211 is shown in a main window, an auxiliary view 2100 of articulatable instruments 211, 231, 241 extending out of the entry guide 200 is shown in a lower central window, and supplemental auxiliary views 3001, 3002 of the tools 241, 231 from a different perspective than that of the view 2100 are shown respectively in lower side windows.” See also Tognaccini FIG. 2, tool following modes T2 and T1, camera positioning modes C2 and C1, entry guide positioning modes G2 and G1, and Tognaccini paragraphs [0056] – [0058]).
Regarding Claim 2, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses wherein the flexible scope is inserted into a patient through a first port (Tognaccini FIG. 1, single entry aperture 150) and the reference position is determined based on a position of the first port (Tognaccini paragraph [0041], “entry guide (EG) 200 is inserted through a single entry aperture 150 into the Patient 40”. Examiner interprets the position/location of the single entry aperture 150 as determining the orientation/position of the EG 200 relative to a reference target/position.)
Regarding Claim 3, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses wherein:
the robotic system causes movement of a distal portion of the flexible scope from a first position or a first orientation to a second position distinct from the first position or a second orientation distinct from the first orientation while the robotic system is operating the flexible scope in a first mode (Tognaccini FIG. 18, camera positioning mode 1805; see also Tognaccini paragraphs [0056] – [0058] and [0079]), and
the stored instructions include instructions for updating the first visual indicator to indicate the second position or the second orientation of the distal portion of the flexible scope relative to the reference position (Tognaccini FIG. 14, bubble 1402 showing surgical tool portion 1402 moving too close to camera 211).
Regarding Claim 4, Tognaccini discloses the features of Claim 3, as described above.
Tognaccini further discloses wherein the first visual indicator (Tognaccini FIG. 14, image captured in window 1421) includes one or more reference axes (Tognaccini FIG. 14, overlaid reference marker 1410) and a graphical element (Tognaccini FIG. 14, overlaid bubble 1401) positioned relative to the one or more reference axes to identify at least one of a direction or an extent of the movement (Tognaccini FIG. 14, showing extent of movement of camera 211).
Regarding Claim 5, Tognaccini discloses the features of Claim 3, as described above.
Tognaccini further discloses wherein the stored instructions also include instructions for: in accordance with a determination that the distal portion of the flexible scope has moved at least one of: (i) from the first position to the second position or (ii) from the first orientation to the second orientation, providing electrical signals for presenting an updated graphical element that indicates a change in at least one of the position or orientation of the flexible scope associated with the movement (Tognaccini FIG. 14, window 1421 on display screen 140, colored bubble 1401 showing surgical tool 241 and camera 211; Tognaccini paragraph [0087], “FIG. 14 shows a semi-translucent sphere or bubble 1401 (preferably colored red) which is displayed by the method as part of the rendering process when a warning threshold is reached so as to indicate to the operator that the highlighted portions 1402, 1403 of the surgical tool 241 and camera 211 are close to colliding.” Examiner interprets this passage as disclosing that the distal portion of the camera 211 has moved from a first position to a second position, as indicated in window 1421.).
Regarding Claim 10, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses wherein the robotic system causes movement of the flexible scope about a target point in space while the robotic system is operating the flexible scope in a second mode of the plurality of modes (Tognaccini FIG. 2, input devices 108, 109; tool following modes T2 and T1; camera positioning modes C2 and C1; entry guide positioning modes G2 and G1; Tognaccini paragraph [0059], “the Surgeon depressing a button on one of the input devices 108, 109 or depressing the foot pedal 105, or using any other well known mode switching technique”. Examiner interprets this passage as teaching that movement of the robotic system can be changed into different modes (e.g., tool following mode, entry guide positioning mode, etc.).
Regarding Claim 15, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses:
wherein the robotic system causes the flexible scope to automatically insert and/or retract when the robotic system is operating the flexible scope in a third mode of the plurality of modes (Tognaccini paragraph [0042], “The entry guide manipulator is then used to robotically insert and retract the entry guide 200 into and out of the entry aperture 150”), wherein the stored instructions also include instructions for:
determining a shape of the flexible scope during activation of the third mode; and
providing an image that corresponds to the determined shape; and
updating the image while the flexible scope is automatically inserted or retracted (Tognaccini FIG. 12, showing on display screen 140 the shapes of camera 211 and tools 231, 241 of entry guide 200).
Regarding Claim 17, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses:
wherein the viewer is part of a surgeon console (Tognaccini FIG. 1, console 10), wherein:
the surgeon console includes a foot pedal (Tognaccini FIG. 1, foot pedal 105); and
the stored instructions include instructions for:
receiving an input on the foot pedal; and determining that the flexible scope is operating in the particular mode in accordance with the received input (Tognaccini paragraph [0080], “foot pedal 105 may also be used for Surgeon selection of viewing modes”).
Regarding Claim 18, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses wherein the first visual indicator includes a graphical representation of the flexible scope in a graphical projection (Tognaccini FIG. 12, image of camera 211 and tools 231, 241 of flexible scope 200 in display screen 140).
Regarding Claim 19, Tognaccini discloses the features of Claim 18, as described above.
Tognaccini further discloses wherein the stored instructions include instructions for displaying one or more reference user interface elements in accordance with a determination that the flexible scope is flexed (Tognaccini FIG. 12, showing flexed tools, 231, 241 in display screen 140).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”) in view of Daley et al. (US PGPUB 2021/0030497 – “Daley”).
Regarding Claim 6, Tognaccini discloses the features of Claim 3, as described above.
Tognaccini does not explicitly disclose:
wherein the first visual indicator includes a plurality of graphical elements, the plurality of graphical elements including a first graphical element and a second graphical element distinct from the first graphical element, wherein:
the first graphical element represents a first component of the movement of a distal end of the flexible scope along a first direction; and
the second graphical element represents a second component of the movement of the distal end of the flexible scope along a second direction, wherein the first direction is distinct from the second direction.
Daley is analogous art in the field of endoscopic imagery that teaches:
wherein the first visual indicator (Daley FIG. 3C and FIG. 3D, field of view (FOV 300 captured by an image sensor; see Daley Abstract) includes a plurality of graphical elements, the plurality of graphical elements including a first graphical element (Daley FIG. 3C, directional indicator 320-2) and a second graphical element (Daley FIG. 3D, directional indicator 320-3) distinct from the first graphical element, wherein:
the first graphical element represents a first component of the movement of a distal end of the flexible scope along a first direction (Daley FIG. 3C, directional indicator 320-2); and
the second graphical element represents a second component of the movement of the distal end of the flexible scope along a second direction (Daley FIG. 3D, directional indicator 320-3), wherein the first direction is distinct from the second direction.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Daley’s directional indicators for a surgical instrument with the robotic system disclosed by Tognaccini. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a robotic system having the capability of moving a medical instrument to a desired target location (see Daley paragraph [0088].
Regarding Claim 7, Tognaccini in view of Daley teaches the features of Claim 6, as described above.
Daley further teaches: wherein the stored instructions also include instructions for:
in accordance with a determination that the movement of the distal end of the flexible scope along the first direction has changed from a first magnitude to a second magnitude, providing electrical signals for updating a location of the first graphical element; and
in accordance with a determination that the movement of the distal end of the flexible scope along the second direction has changed from a third magnitude to a fourth magnitude, providing electrical signals for updating a location of the second graphical element (Daley FIG. 3, directional arrow 320-2 in FOV 300; Daley paragraph [0088], “FIG. 3C also shows example visual feedback 117 in the form of directional indicator 320-2, where the length L2 337 of the directional indicator may be: (a) proportional to the distance D2 335 between location V 310 and current FOV center U 303 (or a location estimated to be the center of current FOV 300)…In some embodiments, the visual feedback 117 may include progressively changing the color of directional indicator 320-2 (e.g. from green to yellow to orange to red) as the instrument is moved toward the boundary of current FOV 300 and/or other visual feedback 117 such as blinking the visual indicator, In the example above, when the instrument is outside current FOV 300 (or partially outside current FOV 300) the color of visual indicator 320-2 may be changed to red.” Examiner interprets this passage as teaching that the length and orientation of directional arrow 320-2 changes as instrument 120-5 changes position.).
Regarding Claim 8, Tognaccini in view of Daley teaches the features of Claim 6, as described above.
Daley further teaches:
wherein the stored instructions also include instructions for:
in accordance with a determination that the movement of the flexible scope in the first direction has changed from a first magnitude to a second magnitude, providing electrical signals for updating a length of the first graphical element; and
in accordance with a determination that the movement of the flexible scope in the second direction has changed from a third magnitude to a fourth magnitude, providing electrical signals for updating a length of the second graphical element (Daley FIG. 3, directional arrow 320-2 in FOV 300; Daley paragraph [0088], “FIG. 3C also shows example visual feedback 117 in the form of directional indicator 320-2, where the length L2 337 of the directional indicator may be: (a) proportional to the distance D2 335 between location V 310 and current FOV center U 303 (or a location estimated to be the center of current FOV 300)…In some embodiments, the visual feedback 117 may include progressively changing the color of directional indicator 320-2 (e.g. from green to yellow to orange to red) as the instrument is moved toward the boundary of current FOV 300 and/or other visual feedback 117 such as blinking the visual indicator, In the example above, when the instrument is outside current FOV 300 (or partially outside current FOV 300) the color of visual indicator 320-2 may be changed to red.” Examiner interprets this passage as teaching that the length and orientation of directional arrow 320-2 changes as instrument 120-5 changes position.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”) in view of Manzari et al. (US Patent 11,039,074 – “Manzari’).
Regarding Claim 9, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini further discloses wherein:
the field of view of the surgical site is derived from a camera coupled to the flexible scope (Tognaccini paragraph [0046], “images of the work site are being captured by the articulatable camera 211”).
Tognaccini does not explicitly disclose the stored instructions also include instructions for expanding the first visual indicator in accordance with a determination that the camera is active.
Manzari is analogous art in the field of endoscopic imagery that teaches expanding the first visual indicator (Manzari FIG. 10O, camera indicator 1044c) in accordance with a determination that the camera is active (Manzari FIG. 10P, camera indicator 1044c expanded in response to camera being activated).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Manzari’s camera indicator with the robotic system disclosed by Tognaccini. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a robotic system to indicate when a camera is turned on and off, in order to reduce wasted power when the camera is not needed.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”) in view of Inomata et al. (US PGPUB 2016/0345808 – “Inomata”).
Regarding Claim 14, Tognaccini discloses the features of Claim 1, as described above.
Tognaccini does not explicitly disclose:
wherein the first visual indicator includes:
one or more axes; and
a graphical element that represents an insertion direction of the flexible scope relative to the one or more axes.
Inomata is analogous art in the field of endoscopic imagery that teaches:
wherein the first visual indicator includes:
one or more axes (Inomata FIG. 6A, central axis of front view field image 18a captured by image pickup device 15a shown in Inomata FIG. 3); and
a graphical element (Inomata FIG. 6A, front view field image 18a) that represents an insertion direction of the flexible scope relative to the one or more axes (Examiner interprets the front view field image 18a as representing the insertion direction along the axis followed by image pickup device 15a).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Inomata’s front view field image with the robotic system disclosed by Tognaccini. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an endoscope device that affords a forward-looking view of the insertion device in order to navigate the insertion device.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”) in view of Itkowitz et al. (US PGPUB 2019/0047154 – “Itkowitz”).
Regarding Claim 16, Tognaccini discloses the features of Claim 1, as described above.
further comprising:
Tognaccini does not explicitly disclose a second robotic arm coupled to a surgical tool, wherein the stored instructions also include instructions for providing electrical signals for presenting a second visual indicator, corresponding to the surgical tool, on the viewer.
Itkowitz is analogous art in the field of endoscope robots that teaches a second robotic arm (Itkowitz FIG. 2, robotic arm assembly 48) coupled to a surgical tool (Itkowitz FIG. 2, tool 64), wherein the stored instructions also include instructions for providing electrical signals for presenting a second visual indicator, corresponding to the surgical tool, on the viewer (Itkowitz FIG. 8, showing images of instruments in view 202).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Itkowitz’s second robotic arm with the robotic system disclosed by Tognaccini. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a robotic system capable of simultaneously positioning and utilizing multiple instruments.
Allowable Subject Matter
Claim 11 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
No combination of identified prior art teaches or suggests “wherein the first visual indicator comprises a first graphical element representing the target point, wherein the stored instructions also include instructions for providing electrical signals for overlaying the first graphical element on the field of view, wherein the first visual indicator includes a second graphical element representing a line of sight from the target point to a port of the flexible scope”. The closest identified prior art is Tognaccini et al. (US PGPUB 2019/0298463 – “Tognaccini”), which teaches a visual indicator showing the position of an endoscopic tool (Tognaccini FIG. 30, tools 241 and 231, views 2100, 3001, 3002; Tognaccini paragraph [0109], “FIG. 30 illustrates, as an example, a display screen of the monitor 104 in which an image 1501 captured by the camera instrument 211 is shown in a main window, an auxiliary view 2100 of articulatable instruments 211, 231, 241 extending out of the entry guide 200 is shown in a lower central window, and supplemental auxiliary views 3001, 3002 of the tools 241, 231 from a different perspective than that of the view 2100 are shown respectively in lower side windows.”). However, neither Tognaccini nor any other identified prior art teaches or suggests “the first visual indicator includes a second graphical element representing a line of sight from the target point to a port of the flexible scope”. Furthermore, there is no reason or suggestion provided in the prior art to modify the above prior art to teach the limitations as claimed above, and the only reason to modify the references would be based on Applicant's disclosure, which is impermissible hindsight reasoning.
Claims 12-13 are deemed allowable for depending on Claim 11, and would be allowable if Claim 11 is rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 20 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
No combination of identified prior art teaches or suggests “wherein the stored instructions include instructions for ceasing to display the one or more reference user interface elements in accordance with a determination that the flexible scope is straight”. The closest identified prior art is Tognaccini, which disclosed in Tognaccini FIG. 12 a flexible scope camera 211 in a straight position. However, neither Tognaccini nor any combination of the identified prior art teaches or suggests stopping a display of the flexible scope when it is straight. Furthermore, there is no reason or suggestion provided in the prior art to modify the above prior art to teach the limitations as claimed above, and the only reason to modify the references would be based on Applicant's disclosure, which is impermissible hindsight reasoning.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure includes, but is not limited to:
Alvarez et al. (US Patent 9,763,741 – “Alvarez”), which teaches in Alvarez FIG. 7 a robotic endoscope system with multiple arms, and in Alvarez FIGs. 29A-30C images of an endoscopic tool in situ;
Gombert et al. (US PGPUB 2016/0235489 – “Gombert”), which teaches in Gombert FIG. 6 images of surgical instruments 15 and 19 oriented in 3-axes; and
Hunter et al. (US PGPUB 2004/0152970 – “Hunter”), which teaches in FIG. 5 an axis and slider bars indicating an orientation of an instrument 52 having a proximal end 106.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIM BOICE whose telephone number is (571)272-6565. The examiner can normally be reached Monday-Friday 9:00am - 5:00pm Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571)272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIM BOICE
Examiner
Art Unit 3795
/JAMES EDWARD BOICE/Examiner, Art Unit 3795
Prosecution Insights