DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/20/2026 has been entered.
Disposition of Claims
Claims 1, 3-17, and 19 are pending.
Claims 2 and 18 are canceled.
Response to Arguments
Applicant’s arguments, see Page 11, filed on 1/23/2026, with respect to the rejections under 35 U.S.C. § 112(b) of Claims 12-13 have been fully considered and are persuasive in light of amendments to the claims.
The rejections under 35 U.S.C. § 112(b) of Claims 12-13 have been withdrawn.
Applicant's arguments, see Pages 18-20, filed on 1/23/2026, with respect to the rejections under 35 U.S.C. § 101 of Claims 1, 15, and 16 have been fully considered but they are not persuasive.
The applicant argues that amended claims 1, 15, and 16 are directed to patent eligible subject matter under 35 USC § 101. The applicant’s argument is on the grounds that the additional limitations integrate the mental process into a practical application. The applicant draws attention to a benefit of the mental process such as avoiding excessive force to organs. However, claims 1, 15, and 16 do not include any limitations that specify the use of the mental process for avoiding exerting excessive force to organs.
Accordingly, the 101 rejections to claims 1, 15, and 16 stand. Please see section 35 USC § 101 below for further explanation.
Applicant’s arguments, see Pages 12-17, filed on 1/23/2026, with respect to the rejections under 35 U.S.C. § 103 of Claims 1, 15, and 16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Objections
Claims 1, 7, 9, and 15-16 are objected to because of the following informalities:
In claim 1, lines 14 and 17, claim 15, lines 14 and 17, and claim 16, lines 16 and 19, “determine the predetermined medical instrument” should read “determine that the predetermined medical instrument.”
In claim 7, line 6, “where the processor” should read “wherein the processor”
In claim 9, line 9, “the semi auto mode and the manual mode” should read “the semi auto mode, and the manual mode”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “a predetermined medical instrument configured to be electrically driven to engage in an instrument motion” in claims 1, 15, and 16.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The specification discloses on page 12, paragraph 4 that the “predetermined medical instrument” is an endoscope or treatment tool. The language in claims 1, 15, and 16 will be interpreted as requiring the disclosed structure on page 12, or equivalents.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites the limitation "the electrical driving of the endoscope” in line 6. There is insufficient antecedent basis for this limitation in the claim. For the purposes of examination, “the electrical driving of the endoscope” is being interpreted as “an electrical driving of the endoscope.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements and/or steps are well-understood, routine, and conventional in the art. They represent components and/or activities which would routinely be used in applying the abstract idea. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
A review of the instant claim(s) using the Alice two-part analysis is provided herein.
Step 1: The claims are one of the four statutory categories, in this instant, a machine.
Step 2 A:
Prong One: The claims are directed to an abstract idea without significantly more. The claims recite mental processes (MPEP § 2106.04(a)(2)(III)). The mental process is:
In response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned
in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned
Determine whether a predetermined condition regarding the medical instrument’s position is detected
Prong Two: This judicial exception is not integrated into a practical application because the additional elements: (a) merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea; and (b) the additional element does no more than generally link the use of the judicial exception to a particular technological environment or field of use.
Step 2 B: The claims do not contain additional elements which amount to significantly more than the abstract idea. The instant claims recite the additional elements of a medical instrument configured to be electrically driven in an instrument motion during a procedure and a processor comprising hardware. The instant claims recite the additional activities of controlling electrical driving of the medical instrument in a full auto mode and a semi auto mode and performing automatic control to intervene in the manual control.
These additional elements do not amount to significantly more because they represent components and/or activities which would routinely be used in applying the abstract idea, specifically generic tools for performing the abstract idea and insignificant extra-solution activities for performing the abstract idea. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
In accordance with MPEP § 2106.07(a)(III), Examiner notes that the machine is routine, conventional, and is recited at a high degree of generality. Specifically, Sokolov et al. (US 2025/0064296) and Gage et al. (US 2024/0252775) describe a well-known machine with automatic and semi-automatic motion modes of a medical instrument. Accordingly, these additional elements of the instant claims do not amount to significantly more than the abstract idea.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements and/or steps are well-understood, routine, and conventional in the art. They represent components and/or activities which would routinely be used in applying the abstract idea. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
A review of the instant claim(s) using the Alice two-part analysis is provided herein.
Step 1: The claims are one of the four statutory categories, in this instant, a process.
Step 2 A:
Prong One: The claims are directed to an abstract idea without significantly more. The claims recite mental processes (MPEP § 2106.04(a)(2)(III)). The mental process is:
In response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned
in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned
Determine whether a predetermined condition regarding the medical instrument’s position is detected
Prong Two: This judicial exception is not integrated into a practical application because the additional elements: (a) merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea; and (b) the additional element does no more than generally link the use of the judicial exception to a particular technological environment or field of use.
Step 2 B: The claims do not contain additional elements which amount to significantly more than the abstract idea. The instant claims recite the additional activities of controlling electrical driving of the medical instrument in a full auto mode and a semi auto mode and performing automatic control to intervene in the manual control.
These additional elements do not amount to significantly more because they represent components and/or activities which would routinely be used in applying the abstract idea, specifically extra-solution activities for performing the abstract idea. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
In accordance with MPEP § 2106.07(a)(III), Examiner notes that the process is routine, conventional, and is recited at a high degree of generality. Specifically, Sokolov et al. (US 2025/0064296) and Gage et al. (US 2024/0252775) describe a well-known process for using both a full auto mode and semi auto mode for controlling motion of an endoscope. Accordingly, these additional activities of the instant claims do not amount to significantly more than the abstract idea.
Claims 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements and/or steps are well-understood, routine, and conventional in the art. They represent components and/or activities which would routinely be used in applying the abstract idea. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
A review of the instant claim(s) using the Alice two-part analysis is provided herein.
Step 1: The claims are one of the four statutory categories, in this instant, a machine.
Step 2 A:
Prong One: The claims are directed to an abstract idea without significantly more. The claims recite mental processes (MPEP § 2106.04(a)(2)(III)). The mental process is:
In response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned
in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned
Determine whether a predetermined condition regarding the medical instrument’s position is detected
Prong Two: This judicial exception is not integrated into a practical application because the additional elements: (a) merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea; and (b) the additional element does no more than generally link the use of the judicial exception to a particular technological environment or field of use.
Step 2 B: The claims do not contain additional elements which amount to significantly more than the abstract idea. The instant claims recite the additional activities of controlling electrical driving of the medical instrument in a full auto mode and a semi auto mode and performing automatic control to intervene in the manual control.
These additional elements do not amount to significantly more because they represent components and/or activities which would routinely be used in applying the abstract idea, specifically generic tools for performing the abstract idea and insignificant extra-solution activities. As such, they do not meaningfully limit the claims, taken as a whole, to a particular application of the abstract idea; rather, the claims would tend to monopolize the abstract idea itself in practice.
In accordance with MPEP § 2106.07(a)(III), Examiner notes that the machine is routine, conventional, and is recited at a high degree of generality. Specifically, Sokolov et al. (US 2025/0064296) and Gage et al. (US 2024/0252775) describe a well-known machine with automatic and semi-automatic motion modes of a medical instrument. Accordingly, these additional elements of the instant claims do not amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. (See MPEP 2141.02(VI)) Therefore, Applicant is advised to review all portions of the cited prior art if traversing a rejection based on the cited prior art.
Claims 1, 4-9, and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2025/0064296 A1 to Sokolov et al. (“Sokolov”) in view of U.S. Patent Appl. Publ. No. 2023/0157768 A1 to Meguro (“Meguro”) and U.S. Patent Appl. Publ. No. 2024/0252775 A1 to Gage et al. (“Gage”).
Regarding claim 1, Sokolov discloses a medical system (an integrated robotic device or system 100; Fig. 1, paragraph 0111) comprising:
a predetermined medical instrument configured to be electrically driven to engage in an instrument motion during a procedure (actuating member 240 may be driven in multiple degrees of freedom to result in corresponding motions of the introducer (e.g., endoscope 250)… any suitable actuator 216 for driving actuating member 240 may be included in the device, such as drive electronic, one or more electrical motors; Fig. 2A, paragraph 0145-0146), the instrument motion being at least one of forward and backward movement of an insertion section of the predetermined medical instrument, bending of a bending section of the insertion section to a bending angle, and rolling rotation of the insertion section (the introducer may have multiple degrees of freedom controlled by driving an actuating member, including longitudinal forward-back movement, axial right-left rotation, and up-down, and right-left articulating motions at its distal tip; Fig. 5, paragraph 0152); and
a processor comprising hardware (one or more actuators may be under operational control by one or more processors 112 configured to analyze images or image data from one or both of first and second image sensors in the system using suitable AI; Fig. 1, paragraph 0116), the processor being configured to:
receive an operation input from an operator (additionally, or alternatively, actuator(s) 216 may be selectively engaged and/or disengaged in response to a user selection of one or more user interface elements. For example, actuator(s) 216 may be activated by selection of an AI operational button 280 or the like as shown in Fig. 2C and deactivated by selection of a STOP button 284 or the like as shown in Fig. 2C; Fig. 2C, paragraph 0132);
perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image (actuators 216 may be activated through one or more processors executing image recognition software instructions, in response to the processor(s) recognizing one or more anatomic landmarks through such image recognition techniques; Fig. 1, paragraph 0132) (the method may include automatically guiding the introducer (e.g. endoscope) (730) via an actuating member in the handheld housing based on one or more images from a laryngoscope and/or an endoscope; Fig. 7, paragraph 0172);
in response to determining that the predetermined medical instrument is being positioned, control electrical driving of the predetermined medical instrument in a full auto mode to perform automatic control of the electrical driving of the predetermined medical instrument to engage in the instrument motion (in a fully automated mode of exemplary device or assembly 100, the actuator may automatically maneuver the endoscope using AI or other robotic-assisted navigation, and electromechanical control of the actuating member; Fig. 7, #730, paragraph 0116 and 0172); and
in response to determining that the predetermined medical instrument is being used for the treatment after being positioned, control the electrical driving of the predetermined medical instrument in a semi auto mode (automated-manual assist mode; paragraph 0172) to:
perform operation input-based electrical driving of the predetermined medical instrument to engage in the instrument motion based on the operation input received (in such an automated-manual assist mode, for example, the actuating member and/or introducer may be controlled manually, such as with the use of a user interface device (e.g., joystick) or through the device display 118; Fig. 8A-10, paragraph 0117 and 0172).
However, Sokolov does not explicitly disclose wherein the processor is configured to perform image recognition processing of an image acquired by an endoscope to recognize whether the predetermined medical instrument is captured in the image and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned.
Meguro teaches a processor configured to perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image and whether the predetermined medical instrument is captured in the image (the image processing unit 204 includes … a region-of-interest recognizing unit 222 (a region-of-interest recognizing unit), a tool information recognizing unit 224 (a tool information recognizing unit); Fig. 3, paragraph 0055 and 0062). Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, the processor determines the predetermined medical instrument is being positioned, and Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, the processor determines the predetermined medical instrument is being used for a treatment after being positioned (on the basis of the result, the tool information recognizing unit 224 determines which of a treatment state, a pre-treatment state, and a non-treatment state the state of the endoscope 100 … is; Fig. 6, paragraph 0062 and 0077-0079). Meguro teaches that the tool information recognizing unit and the region-of-interest recognizing unit can be used to create a distinguishable display for the user based on the step of the procedure (paragraph 0083).
Meguro is considered to be analogous to the claimed invention because it is in the same field of endoscope systems capable of image recognition processing. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Meguro by programing the processor to recognize if the predetermined medical instrument is in the image and determine the step of the procedure based on that the recognition of the predetermined medical instrument. Doing so would help ensure a distinguishable display for the user based on the step of the procedure, as recognized by Meguro.
In addition, Sokolov fails to explicitly disclose a processor configured to determine whether a predetermined condition regarding the predetermined medical instrument's position is detected and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument.
Gage teaches a processor configured to control the electrical driving of the predetermined medical instrument in a semi auto mode to perform operation input-based electrical driving of the predetermined medical instrument to engage in instrument motion based on the operation input received, determine whether a predetermined condition regarding the predetermined medical instrument's position is detected, and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument (in the case of operating in a manual mode and in an automatic mode simultaneously, the intubation device may be configured to allow the user to operate the intubation device within certain set virtual boundaries, e.g., virtual spatial boundaries, wherein the intubation device may be configured to override the user’s manual operation if the user crosses the virtual boundaries; paragraph 0030-0034). Gage teaches that the use of automatic motion control to override the user’s manual operation may help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate (paragraph 0030).
Gage is considered to be analogous to the claimed invention because it is in the same field of endoscopes with automatic and manual motion controls. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Gage by programing the processor to allow automatic control of the endoscope motion to override manual control of the endoscope motion when the endoscope is controlled in the semi auto mode. Doing so would help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate, as recognized by Gage.
Regarding claim 4, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claim 1. Sokolov further discloses wherein the predetermined medical instrument comprises an endoscope configured to be electrically driven to engage in the instrument motion during an endoscopic operation and to capture the image (system 200 may include an introducer such as an endoscope 250 with an image sensor 252 disposed at a distal region of the introducer as shown; Fig. 2A, paragraph 0123), and wherein the processor is configured to, in response to determining that the endoscope is being positioned, control the electrical driving of the endoscope in the full auto mode to perform the automatic control of the electrical driving of the endoscope to engage in the instrument motion to position a distal end section of the endoscope (an automated mode may include automated image processing or recognition and automatic robotic scope control and/or navigation, which can optionally be performed in a continuous, closed loop manner; Fig. 8A and 9A, paragraph 0190 and 0172).
Regarding claim 5, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claim 1. Gage further teaches wherein in determining whether the predetermined condition regarding the predetermined medical instrument's position is detected, the processor is configured to determine whether the predetermined medical instrument contacts an organ or tissue or whether contact between the predetermined medical instrument and the organ or the tissue is expected in the semi auto mode, and wherein in performing the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument, the processor is configured to perform the automatic control to avoid the contact (in the case of operating in the manual mode and in the automatic mode simultaneously, the manipulating mechanism may be configured to allow the user to manipulate the stylet within certain set virtual boundaries, e.g., virtual spatial boundaries, wherein the manipulating mechanism may be configured to override the user’s manual operation if they cross said virtual boundaries. The intubation device may be configured to provide feedback e.g., haptic and/or visual and/or acoustic feedback, to the user when the virtual boundaries are crossed and/or just prior to the virtual boundaries being crossed, e.g.., when a section of the intubation device or the endotracheal tube is within a certain distance from a portion of the anatomy of the patent; paragraphs 0030-0034). The virtual boundaries taught by Gage can be placed around an organ or tissue to prevent or at least reduce the risk of severe injury to the patient caused by human error.
Regarding claim 6, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claim 1. Gage further teaches wherein in determining whether the predetermined condition regarding the predetermined medical instrument's position is detected, the processor is configured to determine whether the predetermined medical instrument is moved to a route different from an insertion route, and wherein in performing the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument, the processor is configured to perform the automatic control to restrict movement of the predetermined medical instrument to the route (in the case of operating in the manual mode and in the automatic mode simultaneously, the manipulating mechanism may be configured to allow the user to manipulate the stylet within certain set virtual boundaries, e.g., virtual spatial boundaries, wherein the manipulating mechanism may be configured to override the user’s manual operation if they cross said virtual boundaries. For instance, the manipulating mechanism may stop manipulation of the stylet; paragraph 0086 and 0031). The virtual boundaries taught by Gage can be placed around the planned insertion route to prevent or at least reduce the risk of severe injury to the patient caused by human error.
Regarding claim 7, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claim 1. Sokolov further discloses wherein the processor is configured to perform the operation input-based electrical driving of the predetermined medical instrument in a manual mode based on the operation input received where the automatic control to intervene in the operation input-based electrical driving is not performed (any of the devices herein may default to a manual mode or may not be adapted with AI or other automatic image processing and/or recognition; Fig. 9B, paragraph 0189), and
wherein the processor is configured to switch between the full auto mode, the semi auto mode, and the manual mode (in some instances, an automated mode may be default and may function in automated mode without needed user initiation….the fully automated mode may resume after a predetermined period of time (e.g., period of inactivity or non-movement of the actuating member 240, predetermined duration of time, etc.) while in the manual assist mode; Fig. 8A, paragraph 0190 and 0132).
Regarding claim 8, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claims 1 and 7. Sokolov further discloses wherein the processor is configured to:
determine that the predetermined medical instrument is inserted into a target; and
in response to determining that the predetermined medical instrument is inserted into the target, control the electrical driving of the predetermined medical instrument in the manual mode (in some instances, a manual mode may be initiated in response to a user action or triggering event, such as by pushing a button to initiate a manual mode, or by touching a touch screen to identify part of an image; Fig. 9B, paragraph 0186 and 0189). Sokolov teaches that an operator identifying a part of an image on a touch screen is used to determine whether the medical instrument has been inserted into a target.
Regarding claim 9, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claims 1 and 7. Sokolov further discloses wherein the procedure includes a plurality of steps, the plurality of steps including the positioning of the predetermined medical instrument and using the predetermined medical instrument for a treatment after being positioned (Fig. 7), and wherein the processor is configured to:
determine a step of the plurality of steps being performed based on input information and control the electrical driving of the predetermined medical instrument in one of the full auto mode, the semi auto mode and the manual mode corresponding to the step of the plurality of steps determined (the processor may guide the navigation, either automatically or via user prompts, in a predetermined sequence of steps. These steps may be triggered based on timing, force or resistance experienced by the components of the system, visual cues, motor feedback, message feedback, verbal feedback, touchscreen feedback, foot pedal feedback, etc.; paragraph 0362).
Regarding claim 15, Sokolov discloses a computer-implemented method for controlling a medical system comprising a predetermined medical instrument configured to be electrically driven to engage in an instrument motion during a procedure (actuating member 240 may be driven in multiple degrees of freedom to result in corresponding motions of the introducer (e.g., endoscope 250)… any suitable actuator 216 for driving actuating member 240 may be included in the device, such as drive electronic, one or more electrical motors; Fig. 2A, paragraph 0145-0146), the instrument motion being at least one of forward and backward movement of an insertion section of the predetermined medical instrument, bending of a bending section of the insertion section to a bending angle, and rolling rotation of the insertion section (the introducer may have multiple degrees of freedom controlled by driving an actuating member, including longitudinal forward-back movement, axial right-left rotation, and up-down, and right-left articulating motions at its distal tip; Fig. 5, paragraph 0152),
wherein the computer-implemented method comprises:
receiving an operation input from an operator (additionally, or alternatively, actuator(s) 216 may be selectively engaged and/or disengaged in response to a user selection of one or more user interface elements. For example, actuator(s) 216 may be activated by selection of an AI operational button 280 or the like as shown in Fig. 2C and deactivated by selection of a STOP button 284 or the like as shown in Fig. 2C; Fig. 2C, paragraph 0132);
perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image (actuators 216 may be activated through one or more processors executing image recognition software instructions, in response to the processor(s) recognizing one or more anatomic landmarks through such image recognition techniques; Fig. 1, paragraph 0132) (the method may include automatically guiding the introducer (e.g. endoscope) (730) via an actuating member in the handheld housing based on one or more images from a laryngoscope and/or an endoscope; Fig. 7, paragraph 0172);
in response to determining that the predetermined medical instrument is being positioned, control electrical driving of the predetermined medical instrument in a full auto mode to perform automatic control of the electrical driving of the predetermined medical instrument to engage in the instrument motion (in a fully automated mode of exemplary device or assembly 100, the actuator may automatically maneuver the endoscope using AI or other robotic-assisted navigation, and electromechanical control of the actuating member; Fig. 7, #730, paragraph 0116 and 0172); and
in response to determining that the predetermined medical instrument is being used for the treatment after being positioned, control the electrical driving of the predetermined medical instrument in a semi auto mode (automated-manual assist mode; paragraph 0172) to:
perform operation input-based electrical driving of the predetermined medical instrument to engage in the instrument motion based on the operation input received (in such an automated-manual assist mode, for example, the actuating member and/or introducer may be controlled manually, such as with the use of a user interface device (e.g., joystick) or through the device display 118; Fig. 8A-10, paragraph 0117 and 0172).
However, Sokolov does not explicitly disclose wherein the processor is configured to perform image recognition processing of an image acquired by an endoscope to recognize whether the predetermined medical instrument is captured in the image and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned.
Meguro teaches a processor configured to perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image and whether the predetermined medical instrument is captured in the image (the image processing unit 204 includes … a region-of-interest recognizing unit 222 (a region-of-interest recognizing unit), a tool information recognizing unit 224 (a tool information recognizing unit); Fig. 3, paragraph 0055 and 0062). Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, the processor determines the predetermined medical instrument is being positioned, and Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, the processor determines the predetermined medical instrument is being used for a treatment after being positioned (on the basis of the result, the tool information recognizing unit 224 determines which of a treatment state, a pre-treatment state, and a non-treatment state the state of the endoscope 100 … is; Fig. 6, paragraph 0062 and 0077-0079). Meguro teaches that the tool information recognizing unit and the region-of-interest recognizing unit can be used to create a distinguishable display for the user based on the step of the procedure (paragraph 0083).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Meguro by programing the processor to recognize if the predetermined medical instrument is in the image and determine the step of the procedure based on that the recognition of the predetermined medical instrument. Doing so would help ensure a distinguishable display for the user based on the step of the procedure, as recognized by Meguro.
In addition, Sokolov fails to explicitly disclose a processor configured to determine whether a predetermined condition regarding the predetermined medical instrument's position is detected and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument.
Gage teaches a processor configured to control the electrical driving of the predetermined medical instrument in a semi auto mode to perform operation input-based electrical driving of the predetermined medical instrument to engage in instrument motion based on the operation input received, determine whether a predetermined condition regarding the predetermined medical instrument's position is detected, and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument (in the case of operating in a manual mode and in an automatic mode simultaneously, the intubation device may be configured to allow the user to operate the intubation device within certain set virtual boundaries, e.g., virtual spatial boundaries, wherein the intubation device may be configured to override the user’s manual operation if the user crosses the virtual boundaries; paragraph 0030-0034). Gage teaches that the use of automatic motion control to override the user’s manual operation may help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate (paragraph 0030).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Gage by programing the processor to allow automatic control of the endoscope motion to override manual control of the endoscope motion when the endoscope is controlled in the semi auto mode. Doing so would help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate, as recognized by Gage.
Regarding claim 16, Sokolov discloses a control apparatus for controlling a predetermined medical instrument (an integrated robotic device or system 100; Fig. 1, #100, paragraph 0111), wherein the predetermined medical instrument is configured to be electrically driven to engage in an instrument motion during a procedure (actuating member 240 may be driven in multiple degrees of freedom to result in corresponding motions of the introducer (e.g., endoscope 250)… any suitable actuator 216 for driving actuating member 240 may be included in the device, such as drive electronic, one or more electrical motors; Fig. 2A, paragraph 0145-0146), the instrument motion being at least one of forward and backward movement of an insertion section of the predetermined medical instrument, bending of a bending section of the insertion section to a bending angle, and rolling rotation of the insertion section (the introducer may have multiple degrees of freedom controlled by driving an actuating member, including longitudinal forward-back movement, axial right-left rotation, and up-down, and right-left articulating motions at its distal tip; Fig. 5, paragraph 0152),
wherein the control apparatus comprises a processor comprising hardware (one or more actuators may be under operational control by one or more processors 112 configured to analyze images or image data from one or both of first and second image sensors in the system using suitable AI; Fig. 1, paragraph 0116), the processor being configured to:
receive an operation input from an operator (additionally, or alternatively, actuator(s) 216 may be selectively engaged and/or disengaged in response to a user selection of one or more user interface elements. For example, actuator(s) 216 may be activated by selection of an AI operational button 280 or the like as shown in Fig. 2C and deactivated by selection of a STOP button 284 or the like as shown in Fig. 2C; Fig. 2C, paragraph 0132);
perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image (actuators 216 may be activated through one or more processors executing image recognition software instructions, in response to the processor(s) recognizing one or more anatomic landmarks through such image recognition techniques; Fig. 1, paragraph 0132) (the method may include automatically guiding the introducer (e.g. endoscope) (730) via an actuating member in the handheld housing based on one or more images from a laryngoscope and/or an endoscope; Fig. 7, paragraph 0172);
in response to determining that the predetermined medical instrument is being positioned, control electrical driving of the predetermined medical instrument in a full auto mode to perform automatic control of the electrical driving of the predetermined medical instrument to engage in the instrument motion (in a fully automated mode of exemplary device or assembly 100, the actuator may automatically maneuver the endoscope using AI or other robotic-assisted navigation, and electromechanical control of the actuating member; Fig. 7, #730, paragraph 0116 and 0172); and
in response to determining that the predetermined medical instrument is being used for the treatment after being positioned, control the electrical driving of the predetermined medical instrument in a semi auto mode (automated-manual assist mode; paragraph 0172) to:
perform operation input-based electrical driving of the predetermined medical instrument to engage in the instrument motion based on the operation input received (in such an automated-manual assist mode, for example, the actuating member and/or introducer may be controlled manually, such as with the use of a user interface device (e.g., joystick) or through the device display 118; Fig. 8A-10, paragraph 0117 and 0172).
However, Sokolov does not explicitly disclose wherein the processor is configured to perform image recognition processing of an image acquired by an endoscope to recognize whether the predetermined medical instrument is captured in the image and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, determine the predetermined medical instrument is being positioned and in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, determine the predetermined medical instrument is being used for a treatment after being positioned.
Meguro teaches a processor configured to perform image recognition processing of an image acquired by an endoscope to recognize whether a predetermined anatomical structure is captured in the image and whether the predetermined medical instrument is captured in the image (the image processing unit 204 includes … a region-of-interest recognizing unit 222 (a region-of-interest recognizing unit), a tool information recognizing unit 224 (a tool information recognizing unit); Fig. 3, paragraph 0055 and 0062). Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is not captured in the image, the processor determines the predetermined medical instrument is being positioned, and Meguro teaches that in response to recognizing that the predetermined anatomical structure is captured in the image and recognizing that the predetermined medical instrument is captured in the image, the processor determines the predetermined medical instrument is being used for a treatment after being positioned (on the basis of the result, the tool information recognizing unit 224 determines which of a treatment state, a pre-treatment state, and a non-treatment state the state of the endoscope 100 … is; Fig. 6, paragraph 0062 and 0077-0079). Meguro teaches that the tool information recognizing unit and the region-of-interest recognizing unit can be used to create a distinguishable display for the user based on the step of the procedure (paragraph 0083).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Meguro by programing the processor to recognize if the predetermined medical instrument is in the image and determine the step of the procedure based on that the recognition of the predetermined medical instrument. Doing so would help ensure a distinguishable display for the user based on the step of the procedure, as recognized by Meguro.
In addition, Sokolov fails to explicitly disclose a processor configured to determine whether a predetermined condition regarding the predetermined medical instrument's position is detected and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument.
Gage teaches a processor configured to control the electrical driving of the predetermined medical instrument in a semi auto mode to perform operation input-based electrical driving of the predetermined medical instrument to engage in instrument motion based on the operation input received, determine whether a predetermined condition regarding the predetermined medical instrument's position is detected, and in response to determining that the predetermined condition is detected, perform the automatic control to intervene in the operation input-based electrical driving of the predetermined medical instrument (in the case of operating in a manual mode and in an automatic mode simultaneously, the intubation device may be configured to allow the user to operate the intubation device within certain set virtual boundaries, e.g., virtual spatial boundaries, wherein the intubation device may be configured to override the user’s manual operation if the user crosses the virtual boundaries; paragraph 0030-0034). Gage teaches that the use of automatic motion control to override the user’s manual operation may help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate (paragraph 0030).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the processor of Sokolov to incorporate the teachings of Gage by programing the processor to allow automatic control of the endoscope motion to override manual control of the endoscope motion when the endoscope is controlled in the semi auto mode. Doing so would help prevent or at least reduce the risk of injury to the patient caused by human error while allowing the user at least some freedom to operate, as recognized by Gage.
Regarding claim 17, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claim 1. Sokolov further discloses wherein the processor is configured to, in performing the image recognition processing, recognize whether an organ, as the predetermined anatomical structure, is captured in the image (suitable image recognition or processing algorithms may identify anatomical landmarks in images from the device such as that described herein; paragraph 0173).
Claims 3 and 10-13 are rejected under 35 U.S.C. 103 as being unpatentable over Sokolov in view of Meguro and Gage as applied to claim 1 above, and further in view of U.S. Patent Appl. Publ. No 2003/0078473 A1 to Richardson (“Richardson”).
Regarding claim 3, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claim 1. Sokolov discloses that the device may robotically assist navigation of the introducer during any suitable endoscopic procedure to provide for a faster and/or less traumatic endoscopic procedure compared to conventional manual techniques (paragraph 0113). However, Sokolov does not explicitly teach wherein in performing the image recognition processing, the processor is configured to perform the image recognition processing to determine whether the predetermined medical instrument is being positioned or being used for is a cannulation using the predetermined medical instrument as the treatment.
Richardson teaches wherein the treatment is a cannulation using the medical instrument (the biliary catheters of the present invention permit easy cannulation and radiological examination of the biliary and pancreatic ducts of a patient during ERCP procedures; Fig. 1, #10, paragraph 0012). Richardson teaches that papillary tissue involved in the cannulation procedure is extremely delicate and each attempt at cannulation increases trauma to the surrounding tissue, and subsequently, increases the discomfort experienced by the patient (paragraph 0008).
Richardson is considered to be analogous to the claimed invention because it is in the same field of endoscopes inserted through the esophagus. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the procedure of Sokolov, as modified by Meguro and Gage, to incorporate the teachings of Richardson by including a cannulation procedure. Doing so would provide less trauma to the extremely delicate papillary tissue and decrease patient discomfort, as recognized by Richardson and Sokolov.
Regarding claim 10, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claim 1. Sokolov further discloses wherein the device may robotically assist navigation of the introducer during any suitable endoscopic procedure to provide for a faster and/or less traumatic endoscopic procedure compared to conventional manual techniques (paragraph 0113). However, Sokolov does not explicitly teach wherein the procedure is endoscopic retrograde cholangiopancreatography.
Richardson teaches wherein the procedure is endoscopic retrograde cholangiopancreatography (the biliary catheters of the present invention permit easy cannulation and radiological examination of the biliary and pancreatic ducts of a patient during ERCP procedures; Fig. 1, paragraph 0012). Richardson teaches that papillary tissue involved in the ERCP procedure is extremely delicate and each attempt at cannulation increases trauma to the surrounding tissue, and subsequently, increases the discomfort experienced by the patient (paragraph 0008).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the procedure of Sokolov, as modified by Meguro and Gage, to incorporate the teachings of Richardson by including an ERCP procedure. Doing so would provide less trauma to the extremely delicate papillary tissue and decrease patient discomfort, as recognized by Richardson and Sokolov.
Regarding claim 11, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claims 1 and 10. Sokolov further discloses wherein the predetermined medical instrument includes an endoscope configured to be electrically driven to engage in the instrument motion during the procedure (actuating member 240 may be driven in multiple degrees of freedom to result in corresponding motions of the introducer (e.g., endoscope 250)… any suitable actuator 216 for driving actuating member 240 may be included in the device, such as drive electronic, one or more electrical motors; Fig. 2A, #250, paragraph 0145-0146), and wherein, in performing the image recognition processing of the image acquired by the endoscope (actuators 216 may be activated through one or more processors executing image recognition software instructions, in response to the processor(s) recognizing one or more anatomic landmarks through such image recognition techniques; Fig. 1, paragraph 0132), the processor is configured to determine the step of the procedure based on the position of the predetermined medical instrument (the processor may guide the navigation, either automatically or via user prompts, in a predetermined sequence of steps. These steps may be triggered based on timing, force or resistance experienced by the components of the system, visual cues, motor feedback, message feedback, verbal feedback, touchscreen feedback, foot pedal feedback, etc.; paragraph 0362).
However, Sokolov does not explicitly teach wherein the steps include positioning the endoscope to a papillary portion of duodenum and inserting a treatment tool into a biliary duct as the treatment after being positioned to the papillary potion of the duodenum.
Richardson teaches positioning the endoscope to a papillary portion of duodenum (the physician guides endoscope 10 through the patient’s esophagus, stomach, and stops when distal end 12 of endoscope 10 reaches the duodenum. More specifically, the physician stops the advancement of endoscope 10 when the endoscope reaches the papilla of Vater 22, where the ducts of the biliary tree and pancreas open into the duodenum; Fig. 1, paragraph 0032) and inserting a treatment tool into a biliary duct (a biliary catheter 16 is advanced through endoscope 10 until the distal tip of biliary catheter 16 emerges from the endoscope 10 side opening and is advanced to the papilla of Vater 22 leading to the common bile duct 26 and the pancreatic duct 28; Fig. 1, paragraph 0033).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the medical system of Sokolov, as previously modified by Gage and Richardson, to incorporate the processor recognition of anatomical landmarks through image recognition software in the ERCP procedure. Doing so would provide less trauma to the extremely delicate papillary tissue and decrease patient discomfort, as recognized by Richardson and Sokolov.
Regarding claim 12, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claims 1 and 10. Sokolov further discloses wherein the procedure includes a plurality of steps including the positioning of the predetermined medical instrument and using the predetermined medical instrument for a treatment after being positioned, wherein the processor is configured to perform the operation input-based electrical driving of the medical instrument in manual mode based on the operation input received where the automatic control to intervene in the operation input-based electrical driving is not performed (any of the devices herein may default to a manual mode or may not be adapted with AI or other automatic image processing and/or recognition; Fig. 9B, paragraph 0189), and the processor is configured to determine a step of the plurality of steps being performed based on input information and control the electrical driving of the predetermined medical instrument in one of the full auto mode, the semi auto mode, and the manual mode corresponding to the step of the plurality of steps determined (the processor may guide the navigation, either automatically or via user prompts, in a predetermined sequence of steps. These steps may be triggered based on timing, force or resistance experienced by components of the system (blade, laryngoscope, introducer scope etc.) visual cues, motor feedback (for example, force required to advance or retract or articulate the introducer scope), message feedback, verbal feedback, touchscreen feedback, foot pedal feedback, etc.; paragraph 0362). Sokolov, as previously modified by Richardson, teaches the procedure can be an endoscopic retrograde cholangiopancreatography.
Regarding claim 13, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claims 1, 10, and 12. Sokolov further discloses wherein the predetermined medical instrument comprises the endoscope configured to be electrically driven to engage in an instrument motion (actuating member 240 may be driven in multiple degrees of freedom to result in corresponding motions of the introducer (e.g., endoscope 250)… any suitable actuator 216 for driving actuating member 240 may be included in the device, such as drive electronic, one or more electrical motors; Fig. 2A, #250, paragraph 0145-0146), and wherein the processor is configured to determine the step of the plurality of steps being performed is a step of inserting the endoscope in a treatment area and in response to determining the step of the plurality of steps, control the electrical driving of the predetermined medical instrument in the manual mode (any of the devices herein may default to a manual mode or may not be adapted with AI or other automatic image processing and/or recognition; Fig. 9B, paragraph 0189). Sokolov shows the use of manual mode after landmark recognition of the treatment area in Fig. 9B.
Richardson teaches the treatment area is a papillary portion of the duodenum during an ERCP procedure (the physician guides endoscope 10 through the patient’s esophagus, stomach, and stops when distal end 12 of endoscope 10 reaches the duodenum. More specifically, the physician stops the advancement of endoscope 10 when the endoscope reaches the papilla of Vater 22, where the ducts of the biliary tree and pancreas open into the duodenum; Fig. 1, paragraph 0032).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the treatment area of Sokolov to incorporate the teachings of Richardson by specifying the treatment area as the papillary portion of the duodenum when performing the ERCP procedure.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Sokolov in view of Meguro and Gage as applied to claim 1 above, and further in view of U.S. Patent Appl. Publ. No. 2020/0281449 A1 to Yoshimura (“Yoshimura”).
Regarding claim 14, Sokolov, as previously modified by Meguro and Gage, discloses the medical system according to claim 1. However, Sokolov, as modified by Meguro and Gage, does not explicitly teach wherein in controlling the electrical driving of the predetermined medical instrument in the full auto mode, the processor is configured to not perform the operation input-based electrical driving of the predetermined medical instrument based on the operation input received.
Yoshimura teaches wherein in controlling the electrical driving of the medical instrument in the full auto mode, the processor is configured to not perform the operation input-based electrical driving of the medical instrument based on the operation input received (in the situation where the operation mode of the control portion 33 is the automatic mode, the operation of the operation input portion 51 is deactivated by the control portion 33 such that the joint 23 and the like of the arm 21 of the endoscope 2 cannot be operated; Fig. 2, paragraph 0079). Yoshimura teaches that the control portion determines the current position of the treatment device through the use of an image taken when the activate portion is turned on (Fig. 5, paragraph 0090-0091) thereby manual movement of the treatment device would interfere with the control portion’s ability to record a proper reference position.
Yoshimura is considered to be analogous to the claimed invention because it is in the same field of endoscopes with automatic and manual operation modes. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the automatic mode of Sokolov, as modified by Meguro and Gage, to incorporate the teachings of Yoshimura by instructing the processor to ignore manual instruction of the instrument motion from the operation device while in the full auto mode. Doing so would stop the manual movement of the medical instrument from interfering with the processor’s ability to record a proper reference position, as recognized by Yoshimura.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Sokolov in view of Meguro and Gage as applied to claims 1 and 17-18 above, and further in view of U.S. Patent Appl. Publ. No. 2020/0030575 A1 to Bogusky et al. (“Bogusky”).
Regarding claim 19, Sokolov, as previously modified by Meguro and Gage, teaches the medical system according to claims 1 and 17. Specifically, Sokolov discloses the use of image recognition software for determining the position of the predetermined medical instrument and thus the step of the procedure being performed. However, Sokolov, as modified by Meguro and Gage, does not explicitly teach wherein the processor is configured to recognize that the organ captured in the image is a stomach or a duodenum before a papilla, recognizing the organ captured in the image is the papilla and the medical instrument is not captured in the image, and recognizing the organ captured in the image is the papilla and the predetermined medical instrument is captured in the image for treatment after being positioned.
Bogusly teaches the use of image processing software (alternatively, or additionally, imaging techniques can be used to automatically identify target area of the anatomy; Fig. 2A, paragraph 0088) for recognizing that the organ captured in the image is a stomach or a duodenum before a papilla (at 718, navigational guidance may be provided on the display system 110 to direct the medical instrument 226 to the target anatomy, such as to the duodenum D of a patient, proximal to the major duodenal papilla; Fig. 7A, paragraph 0114), recognizing the organ captured in the image is the papilla and the medical instrument is not captured in the image (the elongated device 202 is positioned in the pose in a direction of the major duodenal papilla; Fig. 7A, paragraph 0115), and recognizing the organ captured in the image is the papilla and the medical instrument is captured in the image for treatment (at 726, the medical instrument 226 or another medical tool may be advanced through the lumen 241 of the elongate device 202 and on through the target anatomy; Fig. 7A, paragraph 0116). Bogusky teaches the use of image recognition software helps to increase consistent successful cannulation of the common bile duct BD (paragraph 0114)
Bogusky is considered to be analogous to the claimed invention because it is in the same field of endoscopes with autonomic control of the electrical driving of the endoscope. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the procedure of Sokolov, as modified by Meguro and Gage, to incorporate the teachings of Bogusky by including a cannulation procedure. Doing so would provide increase consistent successful cannulation of the common bile duct BD, thereby improving the outcome of the procedure, as recognized by Bogusky.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLIVIA G STARKEY whose telephone number is (571)272-3375. The examiner can normally be reached Monday-Friday 8:00-5:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Carey can be reached at 5712707235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLIVIA GRACE STARKEY/Examiner, Art Unit 3795
/MICHAEL J CAREY/Supervisory Patent Examiner, Art Unit 3795