DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants Amendments filed on February 6, 2026, has been entered and made of record.
Currently pending Claim(s): 1-12
Independent Claim(s): 1, 11, 12
Amended Claim(s): 1, 11, 12
This office action is responsive to the Applicant’s Arguments/Remarks Made in an Amendment
received on February 5, 2026.
In view of amendments filed on, the Applicant has amended independent Claim 1 to recite “capture an endoscopic image using the endoscope during an endoscopic examination; detect a lesion part from the endoscopic image, and display a detection result of the lesion part on a monitor during the endoscopic examination using the endoscope; and detect an abnormality using the endoscopic image to obtain an abnormal image during the endoscopic examination or after an end of the endoscopic examination, and store only the abnormal image in a first storage device for storage after determining the end of the endoscopic examination”. Originally, (in the claim set dated February 5, 2026) Claim 1 recited, “detect a lesion part from an endoscopic image captured by the endoscope, and display a detection result of the lesion part on a monitor during an endoscopic examination using the endoscope; and detect an abnormality that has occurred in the endoscopic image, and store the endoscopic image in which the abnormality has been detected in a first storage device for storage after an end of the endoscopic examination”, and was rejected by Takenouchi (JPWO 2020/054543) in view of Koch (US Pub No 2023/0010235).
As discussed in the following paragraphs, the combination of Takenouchi and Koch does not render obvious the new limitation of storing only the endoscopic image. However, the Applicant’s amendment necessitated the new grounds of rejection presented in this Office Action. Upon conducting a new search, the Examiner argues that the newly amended Claims 1, 11, and 12, are unpatentable over
Takenouchi and Koch, and Satoshi et al. (JP Pub No 2019/092702), hereinafter Satoshi.
In view of Applicant Arguments/Remarks filed on February 5, 2026, with respect to the claims the Applicant explained (on Remarks, page 9-10), that Takenouchi fails to teach feature A, ‘detect an abnormality using the endoscopic image to obtain an abnormal image during the endoscopic examination or after an end of the endoscopic examination’. The Applicant explained that although Takenouchi teaches identifying abnormal (or unsuitable) images for recognition, Takenouchi teaches that abnormal images are “excluded from further processing for the purpose of avoiding generating a classification result with low truthfulness. As such Takenouchi has no motivation to teach or suggest obtaining the image determined to be unsuitable for recognition for later use”.
The Examiner respectfully disagrees. Takenouchi teaches abnormalities during the endoscopic examination can be identified, and thus an ‘abnormal image’ is identified. Furthermore, in Claim 4 of the instant application, images identified as ‘abnormal’ are excluded from detection of lesions, just as Takenouchi teaches that unsuitable images are excluded from the detection of lesions. Thus, the Examiner maintains Takenouchi.
Next, the Applicant argued (on Remarks, page 11), that Koch fails to teach the limitation of ‘store only the abnormal image in a first storage device after determining an end of the endoscopic examination’. The applicant explained the memory unit in Koch stores whatever images the user chooses to record during the endoscopic procedure, and does not selectively store only abnormal images. The applicant further argued (on Remarks, page 11), that the combination of Takenouchi and Koch is based on impermissible hindsight bias. The Examiner agrees with this argument. Koch does not teach storing solely abnormal images; rather, when an abnormality is detected by a user, all images can be saved so that the cause of the error may be obtained. However, the newly found art teaches saving only abnormal images to a database.
Thus, the Applicant’s amendment necessitated the new grounds of rejection presented in this Office actions, and the independent claims 1, 11, and 12 are rejected under 36 USC 103 as being unpatentable over Takenouchi, Koch, and Satoshi.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 3-9, 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Takenouchi
(JPWO 2020/054543), hereinafter Takenouchi, in view of Koch et al. (US Pub No 2023/0010235)
hereinafter Koch, and further in view of Satoshi et al. (JP Pub No 2019/092702), hereinafter Satoshi.
As to Claim 1, Takenouchi teaches an endoscope system (see Fig. 1) comprising:
an endoscope configured to image an inside of a body cavity (see paragraph [0031], “an endoscope system including an electronic endoscope that captures an inside of a body cavity”) and; one or more processors (see paragraph [0031], “a processor device that processes an image signal obtained from the electronic endoscope”) configured to:
capture an endoscopic image using the endoscope during an endoscopic examination (see paragraph (see paragraph [0077], “When the inside of a body cavity is observed using the endoscope system …and an image of the inside of the body cavity captured by the imaging element 62”)
detect a lesion part from the endoscopic image (see paragraph [0025], “In the medical image processing device according to still another aspect of the present disclosure, the classification unit may be configured to recognize a region of a lesion from the image obtained from the image acquisition unit and perform classification of the lesion”),
and display a detection result of the lesion part on a monitor during the endoscopic examination (see paragraph [0036] “The display unit that displays the classification result of the classification unit can be one form of the notification unit”, where the classification unit detects lesions)
and detect an abnormality using the endoscopic image to obtain an abnormal image during the endoscopic examination or after an end of the endoscopic examination (see paragraph [0036], “an image acquisition unit that acquires a plurality of time-series images including a subject image captured using the electronic endoscope, a possibility determination unit that determines whether or not the image obtained from the image acquisition unit is an image unsuitable for recognition”, where the image unsuitable for recognition is the abnormal image).
Takenouchi fails to explicitly teach storing an endoscopic image in which the abnormality has been detected in a first storage device for storage after an end of the endoscopic examination. However, Koch teaches that an endoscope system in which abnormal endoscopic images can be stored in a first storage device (see paragraph [0009], “whereby a database is created in the one or more memory units storing for a plurality of previously performed endoscopic procedures one or more single images or stream of images together with the metadata , wherein the processing unit is configured to, when being set in the error state, overwrite with new data at least some of the metadata for the plurality of images”, where ‘a memory unit’ is interpreted as a first storage device). Koch also teaches that the images are stored after a user input, indicating the end of the examination (see paragraph [0072], “Then, at 503, in response to a user input, one or more single images or one or more streams of images received from the image sensor, together with the received metadata , are stored in one or more memory units, at 504 . The user input may be referred to as a “record” command. The record command can be, for example, an image record command or a video record command , which may comprise a start recording command and a stop recording command”). Koch is combinable Takenouchi because both are from the field of endoscopy. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the endoscopic system of Takenouchi with the storage methods taught Koch. The motivation for doing so would be to allow technicians to see abnormal endoscopic images without seeing patient metadata. Koch teaches in paragraph [0010], “Consequently, by overwriting at least some of the metadata with new data the state of the image recording unit may be kept at least closer to the state it had when a malfunction occurred. This may enable a technician more easily to determine the root cause of the problem”. Thus, after the patient has been examined, endoscopic images are stored while patient metadata pertaining to the images is overwritten. By storing endoscopic images with abnormalities after an end of an endoscopic examination, technicians are able to more easily determine the cause of malfunctions. Thus, it would have been obvious to combine the endoscopic system taught by Takenouchi with the endoscopic system taught by Koch to obtain the invention as claimed in Claim 1.
Takenouchi in view of Koch fails to explicitly each storing only the abnormal image. However, Satoshi teaches that diagnostic medical images and abnormal images can be obtained (see paragraph [0012]), “The image management apparatus 100 compresses the received diagnostic image, and transmits the compressed diagnostic image to the image management server 130 via the network 102. The image management apparatus 100 also compresses the received rejected image and transmits the compressed rejected image to the image management server 130. The image management apparatus 100 further transmits, to the rejected image information management server 140,”, where the ‘rejected’ image is interpreted as the abnormal image),
and that only abnormal images can be set to a first storage device (see paragraph [0024], “The rejected image storage unit 502 stores the rejected image received from the rejected image transmission / reception unit 501, and outputs the stored rejected image to the rejected image request reception unit 505 in response to a request from the rejected image request reception unit 505”).
Satoshi is combinable with Takenouchi and Koch as all three are from the analogous art of medical imaging. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the abnormal image storage unit taught by Satoshi with the teachings of Takenouchi and Koch. The motivation for doing so would be to retain abnormal images so that they can be referred to later (see paragraph [0003], “In addition, when a medical image is captured, re-imaging may be required due to the presence of a foreign object, the body movement of a patient, or the like. Such an image that cannot be used for diagnosis is called a rejected image. The rejected image is not used for diagnosis, but may be stored in the management server for recording, or may be referred to in a conference for improving the skill of a radiographer who performs imaging.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the endoscopic systems taught by Takenouchi and Koch with the abnormal storage unit taught by Satoshi in order to obtain the invention as claimed in in Claim 1.
As to Claim 3, Takenouchi in view of Koch and Satoshi teaches processors configured to detect the abnormality during the endoscopic examination. Takenouchi teaches a processor used to detect abnormalities during endoscopic examination (see paragraph [0033], “A processor device according to still another aspect of the present disclosure is a processor device that processes an image signal obtained from an electronic endoscope, and includes an image acquisition unit that acquires a plurality of time-series images including a subject image captured using the electronic endoscope, a possibility determination unit that determines whether or not the image obtained from the image acquisition unit is an image unsuitable for recognition”).
As to Claim 4, Takenouchi in view of Koch and Satoshi teaches an endoscope system configured to detect the abnormality before detecting the lesion part and exclude the endoscopic image in which the abnormality has been detected, from a detection target of the lesion part. Takenouchi teaches in paragraph [0131], “According to the medical image processing device 160, in a case in which an image inappropriate for recognition is temporarily included in images obtained in time series, the motion estimation process, the action determination process, and the classification process for the inappropriate image are omitted, and these processes are not performed. Therefore, it is possible to avoid a situation in which a classification result with low authenticity (an erroneous recognition result) is generated from an image inappropriate for recognition and provided to the user”).
As to Claim 5, Takenouchi in view of Koch and Satoshi teaches an endoscope system that can detect the lesion part for a thinned-out endoscopic image which is a part of a plurality of the endoscopic images. Takenouchi teaches that lesions can be detected from thinned-out endoscopic images (see paragraph [0090] and [0094], “The recognition unit 164A may calculate the feature amount of each frame image of the moving image or the thinned frame images at regular intervals, and determine whether or not the image is unsuitable for recognition using the calculated feature amount. By performing image analysis on each of the three frame images shown in FIG. 8, classification can be performed for each image”, where the classification unit determines whether a lesion is present in the image).
As to Claim 6, Takenouchi in view of Koch and Satoshi teaches an endoscope system which stores the endoscopic image in which the abnormality has been detected in the first storage device in a form accessible to an external device. Takenouchi teaches that endoscopic images can be saved in a form accessible to an external device (see paragraph [0086], “In addition, the image acquisition unit 162 may be a communication network terminal provided in the processor device 16, a media interface terminal for external storage media, a connection terminal of an external device, or an appropriate combination thereof”).
As to Claim 7, Takenouchi and Satoshi fails to teach an endoscope system wherein the endoscopic image includes personal information including at least any of subject person information regarding a subject person of the endoscopic examination or operator information regarding an operator of the endoscopic examination, and erases the personal information from the endoscopic image in which the abnormality has been detected and store the endoscopic image in the first storage device. However, Koch teaches that personal information can be overwritten (see paragraph [0010], “Consequently, by overwriting at least some of the metadata with new data the state of the image recording unit may be kept at least closer to the state it had when a malfunction occurred . This may enable a technician easily to determine the root cause of the problem”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Koch with the endoscope systems taught by Takenouchi and Satoshi. The motivation for doing so would be to allow technicians to access endoscopic images without compromising patient confidentiality. Koch teaches in paragraph [0006], “When an image recording unit malfunctions it may require a technician to investigate the problems and find a solution . This may be problematic as the technician may not be authorized to access patient's personal information that may be present in the metadata”. Thus, it would have been obvious to combine the teachings of Koch with the systems taught by Takenouchi and Satoshi to protect patient data and obtain the invention as claimed in Claim 7.
As to Claim 8¸ Takenouchi and Satoshi fail to teach that an endoscope system configured to set an allowable access range for the endoscopic image in which the abnormality has been detected to a range different from an allowable access range for an endoscopic image used for the endoscopic examination. However, Koch teaches a system which restricts access to images with abnormalities by encrypting patient data (see paragraph [0030], “Consequently, it may be secured that the overwritten data can only be accessed by a party having access to both the image recording unit and the encrypted safety copy”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Koch with the endoscope systems taught by Takenouchi and Satoshi. The motivation for doing so would be to allow technicians to access system errors without compromising patient confidentiality. Koch teaches in paragraph [0006], “When an image recording unit malfunctions it may require a technician to investigate the problems and find a solution . This may be problematic as the technician may not be authorized to access patient's personal information that may be present in the metadata”. Thus, it would have been obvious to combine the teachings of Koch with the systems taught by Takenouchi and Satoshi to protect patient data and obtain the invention as claimed in Claim 8.
As to Claim 9, Takenouchi in view of Satoshi fails to teach an endoscope system comprising setting an allowable editing type for the endoscopic image in which the abnormality has been detected to a type different from an allowable editing type for an endoscopic image used for the endoscopic examination. However, Koch teaches that images with abnormalities may be edited, see paragraph [0060] “The GUI may present various icons corresponding to actions selectable by the user with any of the above described user input devices , to for example store or record a copy of a live image, store or record a portion of video corresponding to live images , invert the views apply correction curves to the image data to reduce overexposure, etc.”). To access the GUI, Koch teaches in paragraph [0059] “The GUI can be responsive to user inputs received via a user interface , for example the display 204, when it is a touch screen , or other user interfaces having user input capability ties . The processing unit 210 or the memory 402 may comprise, embedded therein, an encryption key 426, e.g. a private encryption key , and encryption logic 428, which are known in the art.” Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Koch with the endoscope systems taught by Takenouchi and Satoshi. The motivation for doing so would be to allow technicians to access endoscopic images and edit them without compromising patient confidentiality. Koch teaches in paragraph [0006], “When an image recording unit malfunctions it may require a technician to investigate the problems and find a solution . This may be problematic as the technician may not be authorized to access patient's personal information that may be present in the metadata”. Thus, it would have been obvious to combine the teachings of Koch with the systems taught by Takenouchi and Satoshi to protect patient data and obtain the invention as claimed in Claim 9.
As to Claim 11, Claim 11 teaches an operational method of an endoscope system, the method comprising the same process disclosed in Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
As to Claim 12, Claim 12 claims one or more processors (see Takenouchi, paragraph [0031], “a processor device that processes an image signal obtained from the electronic endoscope”) that are connected to an endoscope system comprising the same limitations as disclosed in Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Takenouchi (JPWO 2020/054543A1) in view of Koch et al. (US Pub No 2023/0010235), in view of Satoshi et al. (JP Pub No 2019/092702), hereinafter Satoshi further in view of Oh et al. (US Pub No 2020/0268302A1) hereinafter Oh.
As to Claim 2, Takenouchi in view of Satoshi fails to teach an endoscope system configured to store the endoscopic image in a second storage device for temporary storage during the endoscopic examination and detect the abnormality using the endoscopic image stored in the second storage device after the end of the endoscopic examination. However, Koch teaches an endoscope system in which images are stored in temporary storage during examination (see paragraph [0071], “The processing unit 210 outputs video signals incorporating the GUI and the images. The video signals may be received by a memory buffer and the buffer may be read by the display or video output port to present the GUI and images”). A ‘memory buffer’ is a storage unit used for temporarily storing data and is interpreted as “the second storage device”. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the automatic abnormality detection taught by Takenouchi with the memory buffer taught by Koch. The motivation for doing so would be to allow the user to continuously see images as they are generated. Koch teaches in paragraph [0071], “images captured by the image sensor are continuously received and a display is controlled to show the received images. By continuously received it is meant that the images are live images , captured and transmitted during the procedure by the image sensor at a predetermined frame rate”. Koch also teaches that “Techniques for presenting images are well known, including techniques using buffers or mapped memory”. Thus, it would have been obvious to combine the teachings of Takenouchi with Koch and Satoshi in order to obtain the temporary storage as claimed in Claim 2.
Takenouchi in view of Satoshi and Koch teaches a temporary storage unit but fails to explicitly describe detecting the abnormality after the end of the endoscopic examination. However, Oh teaches that abnormalities can be found in medical images (see paragraph [0104], “The diagnostic engine 1108 may also receive x-ray images to extract features, such as bladder neck feature (open or closed) and leakage detection from the x-ray image”)
after the end of the examination (see paragraph [0041], “In general, the data obtained in the urodynamic study may include several errors or artifacts. These errors may lead to erroneous interpretation if not corrected quickly and accurately. The present invention may include artificial intelligence (AI) algorithm 192 that may be trained using a vast amount of inspection data accumulated in advance and have a function to auto-detect these errors. The AI algorithm 192 may detect some of the errors that can be avoided in advance, and accurately recognize the other errors that cannot be avoided in advance after the examination and suggest a countermeasure for the recognized errors”). Oh is combinable with Takenouchi, Koch, and Satoshi because they are from fields of medical imaging and analysis. Thus, it would have been obvious to one of ordinary skill in the art before ethe effective filing date of the claimed invention to combine the detection system taught by Oh with the teachings of Takenouchi, Koch, and Satoshi. The motivation for doing so would be to suggest countermeasures for the recognized errors, and to prevent errors from occurring in future examinations. Thus. it would have been obvious to combine the teachings of Oh with the teachings of Takenouchi, Koch, and Satoshi in order to obtain the invention as claimed in Claim 2.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Takenouchi (JPWO 2020/054543A1) in view of Koch et al. (US Pub No 2023/0010235), in view of Satoshi et al. (JP Pub No 2019/092702), hereinafter Satoshi , and further in view of Ogihara (US Pub No 2016/0262596A1) hereinafter Ogihara.
As to Claim 10, Takenouchi in view of Koch and Satoshi teaches an abnormality based on an external factor occurring outside the main body of the endoscope (see Takenouchi paragraph [0005], “However, in an endoscopic examination, a lesion site may be hidden or an observation image may be blurred due to an operation of a scope, a fetal movement, a residue, or the like, and it may be difficult to accurately recognize a lesion by image analysis”, where all the listed factors are factors that occur outside the main body of an endoscope. However, Takenouchi in view of Koch and Satoshi fails to teach an endoscope system configured to detect the abnormality based on the internal factor, where an internal factor includes a malfunction in a main body of the endoscope. However, Ogihara teaches an endoscope system that can recognize internal failures (see paragraph [0021], “The controller 114 has a failure detection unit 1141. The failure detection unit 1141 detects a failure in the image sensors 102 a and 102 b based on the image data (image data A) obtained through the image sensor 102a and the image data (image data B) obtained through the image sensor 102b”). Detecting failure of the image sensors would be an example of detecting an abnormality based on an internal factor. Ogihara is combinable with Takenouchi, Koch, and Satoshi because all four belong to the field of endoscopy. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the endoscope system taught by Takenouchi with the internal abnormality detection taught by Ogihara. The motivation for doing so would be to allow for the user to efficiently analyze the reason for failure and allow the operator to remove the endoscope from the patient if needed. Ogihara teaches in paragraph [0023] “that detecting the abnormality through images allows the user to According to the embodiment as explained above, in the case where it is difficult to continuously use the endoscope apparatus having a plurality of image sensors due to a failure in an image sensor, only the power supply of the image sensor in which a failure is detected is turned off. By this operation, it is possible to display a two-dimensional endoscope image by using a normal image sensor. Therefore, the operator can remove the endoscope apparatus 100 while looking at the images necessary for the remove.” Thus, it would have been obvious to combine the teachings of Takenouchi, Koch, Satoshi with the internal error taught by Ogihara to obtain the invention as claimed in Claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kiba et al (US Pub No 2019/0269298), describes an endoscope system with an abnormality detection unit used to identify abnormalities in a memory unit, and a temporary memory unit configured to temporarily store images. Kiba also discloses a ‘thinning out’ processing unit configured to perform thinning-out processing on the image signal output.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOUMYA THOMAS whose telephone number is (571)272-8639. The examiner can normally be reached M-F 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.T./Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664