DETAILED ACTION
Election/Restrictions
Applicant’s election without traverse of Group 1, claims 1-17 in the reply filed on 3/4/2026 is acknowledged. Applicant’s arguments with respect to the species requirement is persuasive. Therefore claims 1-17 will be examined and the restriction is made FINAL.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 7/8/2024 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) has/have been considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Treatment Instrument Recommendation for Endoscopic Systems.
Claim Objections
Claims 2, 4, and 14 are objected to because of the following informalities:
Claim 2 line 3-4 “matrix conversion processing, noise removal processing to the image” should read “matrix conversion processing, or noise removal processing to the image”.
Claim 4 line 3 “related to pixels to corresponding” should read “related to pixels corresponding”
Claim 14 line 3 “a high frequency a knife” should read “a high frequency knife,”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s)1-4, 9-10, and 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Juergens (Pub. No. US20230260114A1) in view of Nozaki (Pub. No. US20210256701 A1).
Regarding claim 1, Juergens discloses An endoscopic control apparatus, comprising: one or more processors configured to: (Juergens ¶72-73; an endoscopic system is disclosed which includes processors.) detect a treatment target based on the image feature obtained from the image matching, (Juergens ¶87 lesion identification is performed using the endoscopy video feed.) determine a treatment instrument based on the detected treatment target, and display guide information, (Juergens ¶89-92; treatment guideline is determined based on the image classification of the lesion. This can include tool recommendation, for example ¶90 “use cold forceps polypectomy or cold snare polypectomy.” The treatment is then presented to the operator ¶92. ¶44 specifically says the treatment guideline is displayed.) wherein the guide information includes information related to the treatment instrument. (Juergens ¶90; explains specific treatment guidelines which include specific tools, for example “use cold forceps polypectomy or cold snare polypectomy.”)
Juergens discloses image analysis to identify and classification of lesions (Juergens ¶87-88), but not explicitly process image matching with respect to a first image and a second image to determine an image feature in the first image and the second image, the first image obtained from an endoscope during an endoscopy, the image feature comprises information related to common pixels among the first image and the second image.
Nozaki, however discloses process image matching with respect to a first image and a second image to determine an image feature in the first image and the second image, the first image obtained from an endoscope during an endoscopy, the image feature comprises information related to common pixels among the first image and the second image. (Nozaki ¶204; comparing (matching) images to references images (second image) of various stages of gastric cancer is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of Juergens with teachings of Nozaki by including matching between incoming images and reference images in order to stage gastric cancer (Nozaki ¶204).
Regarding claim 2, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the one or more processors is configured to: process one or more color adjustment processing, matrix conversion processing, noise removal processing to the image. (Nozaki ¶55; noise removal is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki further with teachings of Nozaki by including noise removal in order to mitigate the effect of noise.
Regarding claim 3, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the second image is obtained from the endoscope during the endoscopy, and the image feature includes information related to pixels corresponding to a common feature between the first image and the second image, the common feature is obtained from image matching. (Nozaki ¶205;taking a difference image (matching) between images that are taken with different wavelengths to visualize the mucosa is disclosed. If feature patterns of cancer are present in the submucosa but not in the mucosa (matching) then Scirrhous stomach cancer is a possibility.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki further with teachings of Nozaki by including taking a difference between images taken at different wavelengths in order to determine the possibility of Scirrhous stomach cancer (Nozaki ¶205).
Regarding claim 4, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the second image is a template image, and the image feature includes information related to pixels to corresponding a common feature between the first image and the second image, the common feature is obtained from image matching. (Nozaki ¶204; comparing (matching) images to references images (second image) of various stages of gastric cancer is disclosed.)
Regarding claim 9, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the one or more processors is configured to determine the treatment instrument using an inference model, and wherein the inference model is obtained from learning training data obtained by annotating a treatment instrument appearing in an endoscopic image obtained from the endoscope during the endoscopy. (Juergens ¶94, 99; a neural network is trained using images that underwent the workflow with an appropriate treatment guideline. ¶35 and ¶39 detection of endoscopic tools is also disclosed.)
Regarding claim 10, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the one or more processors is configured to: determine the treatment instrument based on the detected treatment target and a database reference. (Juergens ¶90; the type of lesion and its size determine which the recommended tool is. ¶19 lesions can be classified using a neural network that uses pre-classified images of lesions (database reference).)
Regarding claim 13, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the image feature indicates one or more of a location of the treatment target, a shape of the treatment target, a size of the treatment target, a color of the treatment target, an edge of the treatment target, a focused area of the treatment target, concavity and convexity about the treatment target, a blood pattern about the treatment target, a blood arrangement about the treatment target or a distance between the endoscope and the treatment target. (Juergens ¶44; the treatment guideline is based on the size and classification of the lesion.)
Regarding claim 14, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose wherein the treatment instrument comprises one of a snare, a biopsy forceps, a grasping forceps, a basket, a high frequency a knife or a syringe needle. (Juergens ¶90; forceps and snare are specifically mentioned. ¶39 mentions snares, forceps, baskets, curettes or knives.)
Regarding claim 15, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above. They further disclose An endoscope system comprising: the endoscopic control apparatus according to claim 1. (Juergens ¶9; a system including the endoscope and controller is disclosed.)
Regarding claim 16, the combination of Juergens and Nozaki disclose the claim limitations of claim 15 as described above. They further disclose further comprising display equipment. (Juergens ¶9; a display is disclosed as part of the system.)
Regarding claim 17, the combination of Juergens and Nozaki disclose the claim limitations of claim 15 as described above. They further disclose A medical system, comprising: the endoscope system according to claim 15; and display equipment. (Juergens ¶9; a display is disclosed as part of the system.)
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Juergens (Pub. No. US20230260114A1) in view of Nozaki (Pub. No. US20210256701 A1) and Kimura (Pub. No. US 20240188799A1).
Regarding claim 5, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose wherein the image feature includes a blurring amount of a distal end of the endoscope.
Kimura, however, discloses wherein the image feature includes a blurring amount of a distal end of the endoscope. (Kimura ¶86; a detected lesion is only output if a reliability degree meets a threshold value. Where the reliability degree depends on degree of blurring of an endoscopic image.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Kimura by including a reliability threshold based on blur degree prior to output in order to only present reliable results to users.
Claim(s) 6-8 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Juergens (Pub. No. US20230260114A1) in view of Nozaki (Pub. No. US20210256701 A1) and Shelton (Pub. No. US20250014734A1).
Regarding claim 6, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose wherein the guide information includes a location of the treatment instrument.
Shelton, however, discloses wherein the guide information includes a location of the treatment instrument. (Shelton ¶240 systems that are within the OR are recommended for use, i.e. the system is located in the OR.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Shelton by including a recommendation of systems that are within the OR in order to keep the recommendation to readily available systems.
Regarding claim 7, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose wherein the one or more processors is configured to: determine whether the treatment instrument is present on a treatment instrument cart; and in response to the determining that the treatment instrument is present on the treatment instrument cart, output the guide information including the treatment instrument presented on the treatment instrument cart.
Shelton, however, discloses wherein the one or more processors is configured to: determine whether the treatment instrument is present on a treatment instrument cart; and (Shelton ¶99 and 102; devices are pulled to see if they are in the operating room. ¶110, smart carts can keep inventory of what is there.) in response to the determining that the treatment instrument is present on the treatment instrument cart, output the guide information including the treatment instrument presented on the treatment instrument cart. (Shelton ¶240 systems that are within the OR are recommended for use. )
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Shelton by including a recommendation of systems that are within the OR in order to keep the recommendation to readily available systems.
Regarding claim 8, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose wherein the one or more processors is configured to: in response to the determining that the treatment instrument is not present on a treatment instrument cart, determine whether a substitute treatment instrument is present on the treatment instrument cart, and wherein the guide information includes information related to the substitute treatment instrument.
Shelton, however, discloses wherein the one or more processors is configured to: in response to the determining that the treatment instrument is not present on a treatment instrument cart, (Shelton ¶99 and 103 determining objects that are not in the operating room that should be is disclosed. ¶110, smart carts can keep inventory of what is there.)) determine whether a substitute treatment instrument is present on the treatment instrument cart, and wherein the guide information includes information related to the substitute treatment instrument. (Shelton ¶151; a suitable alternative is proposed based on location. ¶240 alternatives for items not in the OR are recommended.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Shelton by including a recommendation of alternatives based on location in order to keep the recommendation to readily available systems.
Regarding claim 12, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose further comprising: a communication apparatus configured to wirelessly communicate with an electronic tag provided in the treatment instrument, wherein the one or more processors is configured to check for a presence of the treatment instrument by communication between the communication apparatus and the electronic tag.
Shelton, however, discloses further comprising: a communication apparatus configured to wirelessly communicate with an electronic tag provided in the treatment instrument, (Shelton ¶77; smart packaging that his tagged is used.) wherein the one or more processors is configured to check for a presence of the treatment instrument by communication between the communication apparatus and the electronic tag. (Shelton ¶99; smart packing is used to detect which surgical instruments are in the operating room.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Shelton by including smart tags for OR instruments in order to perform automatic inventory (Shelton ¶99).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Juergens (Pub. No. US20230260114A1) in view of Nozaki (Pub. No. US20210256701 A1) and Mino (Pub. No. US20260198756).
Regarding claim 11, the combination of Juergens and Nozaki disclose the claim limitations of claim 1 as described above.
The combination of Juergens and Nozaki does not explicitly disclose wherein the one or more processors is further configured to: calculate a treatment time period for treating a lesion part based on an endoscopic image obtained from the endoscope during the endoscopy; and determine a degree of difficulty based on the treatment time period and the treatment instrument, and wherein the guide information includes the degree of difficulty.
Mino, however, discloses ; and (Mino ¶93-94; estimating procedure time is disclosed, given a treatment target.) determine a degree of difficulty based on the treatment time period and the treatment instrument, and wherein the guide information includes the degree of difficulty. (Mino ¶94; a probability of success is determined using the estimated time and tool used.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the control apparatus of the combination of Juergens and Nozaki with teachings of Mino by including estimating procedure time and probability of success in order to give a clinician information so they can properly weigh the risk of a procedure.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights