Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Cited Prior Art
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. (See MPEP 2141.02(VI)) Therefore, Applicant is advised to review all relevant portions of the cited prior art if traversing a rejection based on the cited prior art.
Election/Restrictions
Applicant’s election without traverse of species i) reading on claims 1-13, 15-19, and 22-25 in the reply filed on 3/24/2023 is acknowledged.
Claims 14, 20-21 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected species, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 3/24/2026.
Claim Rejections - 35 USC § 112b
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 17, and any dependent claims are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites “the processor displays an item of hemostat as the item information”. It is unclear what an item of hemostat is since hemostat is an object and the prior recitations of “an item of …” in the claims has been with respect to procedures, i.e. treatment and diagnosis and findings. This recitation is therefore considered indefinite and should be amended to be consistent with respect to procedures, e.g. hemostasis.
Claim 17 recites “during a period with a length” considered to be indefinite. There exists an inconsistency between the terms “a period” and “a length” since one deals with time and the other deals with distance. Consistent terms should be used to avoid indefiniteness, e.g. “during a period with a duration according to …”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-13, 15-18, 22-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Enoki US20190125319 and further in view of Yoshida US20170337683 and Ono JP2016021216.
Enoki discloses for claim 1, “An endoscope system comprising:
an audio input device (microphone array 3709; fig 1; 0035, 0057);
an image sensor (endoscopic camera described at 0036, 0038) that images a subject; and
a processor (controller 3007 and control device 3009; 0034), wherein the processor is configured to:
set, in a case where a result of any one of the plurality of image recognitions is obtained, an audio recognition dictionary according to the obtained result of any one of the plurality of image recognitions (0108-0109 describes progress of a procedure or surgery is detected by the surgery-information-detecting unit 150 and in order to improve accuracy of voice recognition the sound-output-control unit 100B, by the voice-recognizing unit 140, may select a dictionary 160 to be used in voice recognition on the basis of the progress of the procedure; additionally, “For example, in the case where a procedure in which the use of a predetermined device is highly probable, a dictionary 160 having high accuracy of recognition of commands that can be received by that device may be selected. In addition, in the case where surgical instruments to be used differ according to the process of the surgery, a dictionary 160 having high accuracy of recognition of surgical instruments to be used during that progress may be selected.”);
perform audio recognition using the set audio recognition dictionary, in a case where audio input is performed for the item information displayed on the display device (0108-0109 describes progress of a procedure or surgery is detected by the surgery-information-detecting unit 150 and in order to improve accuracy of voice recognition the sound-output-control unit 100B, by the voice-recognizing unit 140, may select a dictionary 160 to be used in voice recognition on the basis of the progress of the procedure); and
display a result of the audio recognition in association with the item information, on the display device (fig 11; 0106)”.
Enoki does not disclose:
“acquire a plurality of medical images obtained by the image sensor imaging the subject in chronological order”;
Yoshida teaches in the same field of endeavor (0047-0049), discloses the file creation section 11a may create an image file of an endoscopic movie in a predetermined time period including a point of time of specimen acquisition as an image at the time of cell acquisition or create an image file of at least either of still images immediately before and immediately after specimen acquisition as an image at the time of cell acquisition; and
“perform a plurality of image recognitions on the acquired plurality of medical images”;
Additionally, Yoshida teaches in the same field of endeavor (0047), the specimen state determination section 11b may detect an image part of, e.g., a polyp by image recognition and determines that it is the time of specimen acquisition when a change in which an image feature of, e.g., the polyp disappears from the image part occurs.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Yoshida into the invention of Enoki in order to configure the image recognition function e.g. as claimed because it helps to provide an improved system for efficiently assisting in bio-related examination, treatment, work, and processing facilitating reliability of the examination (0030).
Enoki does not disclose:
“display, in a case where the audio recognition dictionary is set, item information indicating an item to be recognized using the audio recognition dictionary, on a display device”. Ono teaches in the same field of endeavor, the feature of storing auxiliary information, which is dictionary information including body part name data, disease name data, property data, etc., and, for example, by saying "body part selection" as voice input, body part names included in the auxiliary information are displayed together with the "body part selection" (fig. 19-20), and the feature of also making a similar display for disease name selection, property selection, etc. (fig. 21-24; 0085-0087). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Ono into the invention of Enoki in order to configure the endoscope system e.g. as claimed because it allows continued tracking and notification of a surgical procedure (Enoki: 0108-0109) with respect to a customized set of recognized audio prompts to be displayed (Ono: 0086-0087; fig 19-20).
Modified Enoki discloses for claim 2, “The endoscope system according to claim 1, wherein the processor performs image recognition for detecting a specific subject, as one of the plurality of image recognitions (Yoshida: 0047 describes detection of an image part/feature)”.
Modified Enoki discloses for claim 3, “The endoscope system according to claim 2, wherein the specific subject is at least one of a treatment tool and a hemostat (Yoshida: 0048 describes an image feature to be detected is a treatment instrument)”.
Modified Enoki discloses for claim 4, “The endoscope system according to claim 3, wherein, in a case where a result of image recognition indicating detection of the treatment tool (Yoshida: 0048 describes an image feature to be detected is a treatment instrument) is obtained, the processor displays an item of treatment as the item information on the display device (Ono: 0086-0087; fig 19-20), and/or in a case where a result of image recognition indicating detection of the hemostat is obtained, the processor displays an item of hemostat, as the item information, on the display device”.
Modified Enoki discloses for claim 5, “The endoscope system according to claim 2, wherein the processor further performs image recognition for discrimination of a lesion detected from the plurality of medical images, as one of the plurality of image recognitions (Yoshida: 0047 describes detection of an image part/feature and 0048 describes detection of polyps)”.
Modified Enoki discloses for claim 6, “The endoscope system according to claim 5, wherein, in a case where a result of the discrimination is obtained, the processor displays an item of diagnosis and findings, as the item information, on the display device (Yoshida: 0045, 0051, 0099, and 0102)”.
Modified Enoki discloses for claim 7, “The endoscope system according to claim 1, wherein, in the audio recognition, the processor recognizes only registered words that are registered in the set audio recognition dictionary, and displays the result of the audio recognition for the registered words on the display device (Yoshida: 0117-0118, discloses the clinician 51 says “biopsy” toward, e.g., the dictation microphone 34a, the dictation section 31a determines that the image that is being displayed is an image at the time of cell acquisition)”.
Modified Enoki discloses for claim 8, “The endoscope system according to claim 1, wherein, in the audio recognition, the processor recognizes registered words that are registered in the set audio recognition dictionary and specific words, and displays the result of the audio recognition for the registered words among the recognized words on the display device (Yoshida: 0117-0118, discloses the clinician 51 says “biopsy” toward, e.g., the dictation microphone 34a, the dictation section 31a determines that the image that is being displayed is an image at the time of cell acquisition)”.
Modified Enoki discloses for claim 9, “The endoscope system according to claim 1, wherein the processor performs the plurality of image recognitions, using an image recognizer generated by machine learning (Yoshida: fig 1; 0031, 0045-0048)”.
Modified Enoki discloses for claim 10, “The endoscope system according to claim 1, wherein the processor displays information indicating that the audio recognition dictionary is set, on the display device (Ono: 0085 describes displaying the selection as confirmation, i.e. the indication of the setting).
Modified Enoki discloses for claim 11, “The endoscope system according to claim 1, wherein the processor displays type information indicating a type of the set audio recognition dictionary, on the display device (Yoshida: 0120 describe providing a type information, specifically with respect to a biopsy and the type of detailed examination to be performed)”.
Modified Enoki discloses for claim 12, “The endoscope system according to claim 1, wherein the processor displays the item information (Ono: fig. 21-24; 0085-0087) and the result of the audio recognition (Enoki: fig 11; 0106) in chronological order (the order as provided in claim 1)”.
Modified Enoki discloses for claim 13, “The endoscope system according to claim 1, wherein the processor displays the item information and the result of the audio recognition on a same display screen as the plurality of medical images (Enoki: 0105)”.
Modified Enoki discloses for claim 15, “The endoscope system according to claim 1, wherein the processor displays the item information and the result of the audio recognition during a display period after the setting, and ends the display in a case where the display period has elapsed (when the display of the information and result is ended, e.g. by user command or the detection of the next stage of the surgery, the display period is also considered to have elapsed, i.e. ended, therefore when a display period ends, the display period is also considered to have elapsed)”.
Modified Enoki discloses for claim 16, “The endoscope system according to claim 15, wherein the processor displays the item information and the result of the audio recognition during a period in which the audio recognition dictionary is set, as the display period, and ends the display of the item information and the result of the audio recognition in a case where the display period ends (when the display of the information and result is ended, e.g. by user command or the detection of the next stage of the surgery, the display period is also considered to have elapsed, i.e. ended, therefore ending the display).
Modified Enoki discloses for claim 17, “The endoscope system according to claim 15, wherein the processor displays the item information and the result of the audio recognition during a period with a length according to the obtained result of any one of the plurality of image recognitions, as the display period, and ends the display of the item information and the result of the audio recognition in a case where the display period ends (when the display of the information and result is ended, e.g. by user command or the detection of the next stage of the surgery, the display period is also considered to have elapsed, i.e. ended, therefore ending the display)”.
Modified Enoki discloses for claim 18, “The endoscope system according to claim 15, wherein the processor ends the display of the item information and the result of the audio recognition in a case where a state in which a specific subject is recognized in the plurality of medical images ends (the display of the information and result are ended such that the next detected stage of the surgery with its corresponding image recognition, setting of a dictionary, displaying of item information, and audio detection results are provided, allowing for the case wherein the state in question ends)”.
Modified Enoki discloses for claim 22, “The endoscope system according to claim 1, wherein the processor records the plurality of medical images, the item information, and the result of the audio recognition in a recording device (Yoshida: 0031, 0045-0048) such that the plurality of medical images are associated with the item information, and the result of the audio recognition (Yoshida: fig 1; 0045)”.
Modified Enoki (as provided in claim 1) discloses for claim 23, “A medical information processing apparatus comprising
a processor (Enoki: controller 3007 and control device 3009; 0034) configured to:
acquire a plurality of medical images obtained by an image sensor imaging a subject in chronological order (Yoshida: 0047-0049);
perform a plurality of image recognitions on the acquired plurality of medical images (Yoshida: 0047-0049);
set, in a case where a result of any one of the plurality of image recognitions is obtained, an audio recognition dictionary according to the obtained result of any one of the plurality of image recognitions (Enoki: 0108-0109);
display, in a case where the audio recognition dictionary is set, item information indicating an item to be recognized using the audio recognition dictionary, on a display device (Ono: fig. 21-24; 0085-0087);
perform audio recognition using the set audio recognition dictionary, in a case where audio input is performed for the item information displayed on the display device (Enoki: 0108-0109); and
display a result of the audio recognition in association with the item information, on the display device (Enoki: fig 11; 0106)”.
Modified Enoki (as provided in claim 1) discloses for claim 24, “A medical information processing method executed by an endoscope system (Enoki: fig 1; 0035-0036 discloses an endoscopic surgical system) including an audio input device (Enoki: microphone array 3709; fig 1; 0035, 0057), an image sensor (Enoki: endoscopic camera described at 0036, 0038) that images a subject, and a processor (Enoki: controller 3007 and control device 3009; 0034), the medical information processing method comprising:
acquiring, by the processor, a plurality of medical images obtained by the image sensor imaging the subject in chronological order (Yoshida: 0047-0049);
performing, by the processor, a plurality of image recognitions on the acquired plurality of medical images (Yoshida: 0047-0049);
setting, by the processor, in a case where a result of any one of the plurality of image recognitions is obtained, an audio recognition dictionary according to the obtained result of any one of the plurality of image recognitions (Enoki: 0108-0109);
displaying, by the processor, in a case where the audio recognition dictionary is set, item information indicating an item to be recognized using the audio recognition dictionary, on a display device (Ono: fig. 21-24; 0085-0087);
performing, by the processor, audio recognition using the set audio recognition dictionary, in a case where audio input is performed for the item information displayed on the display device (Enoki: 0108-0109); and
displaying, by the processor, a result of the audio recognition in association with the item information, on the display device (Enoki: fig 11; 0106)”.
Modified Enoki (as provided in claim 1) discloses for claim 25, “A non-transitory, tangible recording medium (Enoki: fig 1; 0035-0036 discloses an endoscopic surgical system) which records thereon a computer-readable code of a program for causing a processor (Enoki: controller 3007 and control device 3009; 0034) of an endoscope system including an audio input device (Enoki: microphone array 3709; fig 1; 0035, 0057) and an image sensor (Enoki: endoscopic camera described at 0036, 0038) that images a subject, to implement functions comprising:
acquiring a plurality of medical images obtained by the image sensor imaging the subject in chronological order (Yoshida: 0047-0049);
performing a plurality of image recognitions on the acquired plurality of medical images (Yoshida: 0047-0049);
setting, in a case where a result of any one of the plurality of image recognitions is obtained, an audio recognition dictionary according to the obtained result of any one of the plurality of image recognitions (Enoki: 0108-0109);
displaying, in a case where the audio recognition dictionary is set, item information indicating an item to be recognized using the audio recognition dictionary, on a display device (Ono: fig. 21-24; 0085-0087);
performing, audio recognition using the set audio recognition dictionary, in a case where audio input is performed for the item information displayed on the display device (Enoki: 0108-0109); and
displaying, a result of the audio recognition in association with the item information, on the display device (Enoki: fig 11; 0106)”.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Enoki, Yoshida, and Ono as applied to claim 15 above, and further in view of Fujita US20070268280.
Enoki does not disclose for claim 19, “The endoscope system according to claim 15, wherein the processor displays a remaining time of the display period on a screen of the display device”. Yoshida teaches in the same field of endeavor, a clock section 19 which provides time information and display (0055, 0119-0121) for pertinent aspects of the procedure as provided by Enoki, i.e. displaying of the relevant item information and audio detection. Fujita teaches in the same field of endeavor, displaying a remaining time for pertinent functions 0238. It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Fujita into the invention of Yoshida in order to configure the system e.g. as claimed because it allows tracking progress of a procedure (0238).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANHTUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795
6/19/26
Prosecution Insights