DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Claim 27 is withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 11/18/2025.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-26 and 28 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception in the form of an abstract idea without significantly more.
In a test for patent subject matter eligibility, the claims pass Step 1 (see 2019 Revised Patent Subject Matter Eligibility), as they are related to a process, machine, manufacture, or composition of matter.
When assessed under Step2A, Prong I, Independent claims 1 and 28 are found to recite a judicial exception (i.e. abstract idea). In this instance, claims 1 and 28 recite the limitations “acquire a medical image”, “output observation target identification information indicating an observation support target part included in the medical image or indicating that the medical image is out of an observation support target, by inputting the medical image to an observation target identification algorithm”, “select, from a plurality of observation support algorithm, one specific observation support algorithm based on the observation target identification information”, “output observation support information by inputting the medical image to the specific observation support algorithm”, “perform a control of notifying of the observation support information”. The cited limitation(s), under their broadest reasonable interpretation, encompass a mental process (i.e. abstract idea) of acquiring, outputting, selecting, outputting, and notifying which can be performed in the mind or by a human using a pen and a paper (e.g. observation, evaluation, judgment, opinion). In other words, a person could reasonably acquire a medical image (via observation/evaluation), “output observation target identification information (via thought), select one specific algorithm (via evaluation), output observation support information (via thought), and perform a control of notifying (via thought). Examiner notes that with the exception of generic computer-implemented steps (e.g. one or more processors recited in claims 1 and 28), there is nothing in the claims that preclude the limitation from being performed by a human, mentally or with pen and paper, thus the cited limitation(s) recites a judicial exception (MPEP 2106.04(a)) and the claim must be reviewed under Step 2A, Prong II to determine patent eligibility.
Step 2A, Prong II determines whether any claim recites an additional element that integrates the judicial exception into a practical application. Independent claims recites the following additional element(s):
Acquire a medical image (alternatively)
Inputting the medical image to an observation target identification algorithm (alternatively)
Inputting the medical image to the specific observation support algorithm (alternatively)
A light source device (claim 28)
An endoscope (claim 28)
The additional element(s) in the cited independent claim(s) are not found to integrate the judicial exception into a practical application. In this case, acquiring a medical image is alternatively interpreted as an additional element amounting to mere data gathering and inputting the medical image into an observation target identification algorithm and a specific observation support algorithm is alternatively interpreted as an additional element recited with such high generality that it amounts to merely inputting data to a generic computer/algorithm. The light source and the endoscope are considered merely generic components of an endoscope system. These elements are seen as adding insignificant extra-solution activity to the judicial exception. They do no more than link the judicial exception to a particular technological environment or field of use. Therefore, under step 2A Prong II the Judicial exception is not integrated into a practical application by additional elements of independent claims 1 and 28 and the claims must be reviewed under Step 2B to determine patent eligibility.
Step 2B determines where a claim amounts to significantly more.
The additional element listed above do not amount to significantly more than the judicial exception for the same reasons listed above. Additionally there is no improvement in the functioning of the computer or technological field, and there is no transformation of subject matter into a different state. Therefore, under Step 2B in a test for patent subject matter eligibility, the judicial exception of the independent claim(s) do not amount to significantly more and the independent claim(s) remain patent ineligible.
Dependent claims 2-26 further limit the abstract idea of independent claim 1. When analyzed as a whole, these claims are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed towards an abstract idea and do not sufficiently integrate the subject matter into a practical application or recite elements which constitute significantly more than the abstract ideas identified. The dependent claims are directed toward additional elements which encompass abstract ideas
In this instance, dependent claims recite the following limitations:
Output the endoscope operation information or the observation support stop information, by inputting the medical image to the operation support algorithm (claim 8)
Output insertion region information indicating an upper route insertion region and/or a lower route insertion region, which is a region suitable for insertion of an endoscope, included in the medical image, by inputting the medical image to an operation support algorithm (claim 15)
calculate insertion route information that is an area, a width, and/or coordinate information of a center position of the upper route insertion region and/or the lower route insertion region based on the insertion region information (claim 15)
output insertion region information indicating an upper route insertion region and/or a lower route insertion region, which is a region suitable for insertion of the endoscope, included in the medical image, by inputting the medical image to the operation support algorithm (claim 17)
calculate insertion route information that is an area, a width, and/or coordinate information of a center position of the upper route insertion region and/or the lower route insertion region based on the insertion region information (claim 17)
output nasopharyngeal position information indicating that the position of the endoscope is in an appropriate position or in an inappropriate direction position that is an inappropriate right position, an inappropriate left position, an inappropriate upper position, an inappropriate lower position, an inappropriate back position, or an inappropriate front position, by inputting the medical image to the operation support algorithm (claim 18)
output the endoscope operation information based on the nasopharyngeal position information (claim 18)
output oropharyngeal region information indicating a glottis region and/or an epiglottis region included in the medical image, by inputting the medical image to the operation support algorithm (claim 20)
calculate oropharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region and the epiglottis region based on the oropharyngeal region information (claim 20)
output the endoscope operation information using the oropharyngeal region arithmetic information (claim 20)
output hypopharyngeal region information indicating a glottis region and/or a vocal fold region included in the medical image, by inputting the medical image to the operation support algorithm (claim 22)
calculate hypopharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region, and that is a length of the vocal fold region based on the hypopharyngeal region information (claim 22)
output the endoscope operation information using the hypopharyngeal region arithmetic information (claim 22)
output the endoscope operation information for providing an instruction to pull out the endoscope by inputting the medical image to the operation support algorithm (claim 24)
output the observation support stop information, by inputting the medical image to the operation support algorithm (claim 25)
output that the medical image is out of the observation support target as the observation target identification information (claim 26)
The cited limitation(s), under their broadest reasonable interpretation, encompass mental processes (i.e. abstract idea) which can be performed in the mind or by a human using a pen and a paper (e.g. observation, evaluation, judgment, opinion). In other words, a human could reasonably output information via thought and perform calculations via observation/evaluation. Examiner notes that with the exception of generic computer-implemented steps (e.g. the one or more processors), there is nothing in the claims that preclude the limitation from being performed by a human, mentally or with pen and paper, thus the claimed limitation is considered to be directed towards a judicial exception (MPEP 2106.04(a)).
Under Step 2A, Prong II for dependent claims 2-26, present additional elements which only further narrow the judicial exceptions (e.g. claim 2 merely narrows the observation support target part, claim 3 merely narrows the notification to now be considered an additional element which amounts to merely insignificant extra-solution activity of displaying results/outputting results via voice, claim 3 further narrows the observation support information, claim 5 further narrows the endoscope operation information, claim 6 further recites displaying an endoscope operation support diagram which amounts to merely insignificant extra-solution activity of displaying results, claim 7 recites switching a display of the endoscope operation information with such high generality that it amounts to merely insignificant post-solution activity of displaying different results of the information, claims 9 and 12 which recite merely insignificant extra-solution activity of inputting data, claim 10 which further narrows the operation support algorithm such that it is a trained model which is recited with such high generality that it amounts to merely a generic computer, claim 11 which merely further narrows the specific observation support algorithm and recites trained models which are considered merely generic computer, claim 13 which further narrows the observation target identification algorithm such that it is a trained model that has been trained… such a recitation amounts to merely a generic computer, claim 14 which recites merely insignificant extra-solution activity of switching between providing the notification and not providing the notification, claim 15 which further recites displaying the insertion route information on the guide image amounting to merely insignificant extra-solution activity of displaying results, claim 16 which recites control of displaying insertion region by changing a display mode thereof based on the insertion route information which amounts to insignificant extra-solution activity of changing display data without any specificity as to how the display mode is changed or how it is based on the insertion route information, claims 19, 21, and 23, which further narrows the operation support algorithm/position determination module to be a trained model which is trained… amounting to merely use of a generic computer) and provide no additional element which are found to integrate the judicial exception into a practical application.
These dependent claims include no additional claims that are sufficient to amount to significantly more than the judicial exception. Additionally, there is no improvement in the functioning of the computer or technological field, and there is no transformation of subject matter into a different state. As discussed above with respect to integration of the abstract idea into a practical application, the additional claims do not provide any additional elements that would amount to significantly more than the judicial exception. Under Step 2B, these claims are not patent eligible.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-26 and 28 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation “out of an observation support target”. It is unclear if the observation support target is the same as the observation support target part or if this is a different element. If they are different, it is unclear what an observation support target encompasses and the difference between the observation support target part and the observation support target is.
Claim 5 recites the limitation “wherein the endoscope operation information is a moving direction and/or a moving amount of a distal end part of the endoscope”. Examiner notes that claim 4 does not positively set forth the endoscope operation information in that the observation support information may not include the endoscope operation information. It is therefore unclear whether the endoscope operation information is required of the claim in an instance where the observation support information does not include or is not the endoscope operation information. For examination purposes, it has been interpreted that the observation support information at least includes the endoscope operation information, however, clarification is required.
Claim 7 recites the limitation “a display of the endoscope operation information based on the endoscope position determination information”. Examiner first notes that neither the endoscope operation information nor the endoscope position determination information are positively recited in claim 4 and the observation support information may not include the endoscope operation information nor the endoscope position determination information. It is further unclear if the display of the endoscope operation information is the same as the notifying of the observation support information in claim 1 or if this is a different display of information. For examination purposes, it has been interpreted that the observation support information at least includes the endoscope operation information and the endoscope position determination information and that notifying includes displaying the endoscope operation information, however, clarification is required.
Claim 8 recites the limitation “wherein the specific observation support algorithm includes an operation support algorithm”. The limitations are unclear as claim 1 sets forth that the specific observation support algorithm is selected from a plurality of operation support algorithms, therefore, it is unclear if the introduction of the operation support algorithm included in the specific observation support algorithm is different from the plurality of operation support algorithms previously recited or if this is a different/distinct operation support algorithm. For examination purposes, it has been interpreted that it may be one of the plurality of operation support algorithms or a different/distinct operation support algorithm, however, clarification is required.
Claim 8 recites the limitation “inputting the medical image to the operation support algorithm”. It is unclear whether the operation support algorithm is one of the previously recited plurality of operation support algorithms or the operation support algorithm included in the specific observation support algorithm. For examination purposes, it has been interpreted to mean any of the previously recited operation support algorithms, however, clarification is required.
Claim 9 recites the limitation “input a latest medical image to the operation support algorithm”. It is unclear if the latest medical image is the same as the previously recited medical image and is attempting to further narrow the medical image to be a latest medical image or if this is a different input of a different latest medical image. It is further unclear which of the previously recited operation support algorithms the claim is referring to. For examination purposes, it has been interpreted to mean any latest medical image input to any of the previously recited operation support algorithms, however, clarification is required.
Claims 10 and 17-19 recite the limitation “the operation support algorithm”. It is unclear whether the operation support algorithm is one of the previously recited plurality of operation support algorithms or the operation support algorithm included in the specific observation support algorithm. For examination purposes, it has been interpreted to mean any of the previously recited operation support algorithms, however, clarification is required.
Claim 10 recites the limitation “the endoscope operation information and/or the subject position information”. It is unclear if the claim is attempting to further narrow the observation support information of claim 4 to be the endoscope operation information and/or the subject position information or if the limitation is merely defining that the trained model outputs the endoscope operation information and/or the subject position information which is not the observation support information. For examination purposes, it has been interpreted that the observation support information at least includes one of the endoscope operation information and the subject position information, however, clarification is required.
Claim 12 recites “a latest medical image”. It is unclear if the latest medical image is the same as the previously recited medical image and is attempting to further narrow the medical image to be a latest medical image or if this is a different input of a different latest medical image. For examination purposes, it has been interpreted to mean any latest medical image, however, clarification is required.
Claim 15 recites the limitation “an operation support algorithm”. The limitation is unclear as to whether this is one of the previously recited plurality of observation support algorithms, the specific observation support algorithm, or a different/distinct support algorithm. In other words, claim 1 sets forth that inputting the medical image to the specific observation support algorithm and now the medical image is input to an operation support algorithm making it unclear if the medical image is input to both algorithms or if the operation support algorithm is the same as or included as part of the specific observation support algorithm. This is further made unclear as the insertion rout information is recited as being displayed on the guide image as the observation support information. For examination purposes, it has been interpreted that the operation support algorithm may be the same as or included in the specific observation support algorithm or may be different, however, clarification is required.
Claims 15 and 17 recite the limitation “which is a region suitable for insertion of an endoscope”. It is unclear what “which” is referring to (i.e. the upper route insertion region or the lower route insertion region) therefore unclear which of the regions is suitable for insertion of an endoscope or if both are. For examination purposes, it has been interpreted to mean either of the regions, however, clarification is required.
Claim 20 recites the limitation “calculate oropharyngeal region arithmetic information that is an area, a width and/or coordinate information of a center position of the glottis region and the epiglottis region”. The limitation is unclear as to what is encompassed by the “and/or” limitation (e.g. an area of the glottis region, a width of the glottis region and/or a coordinate information of a center position of the glottis region or if the area and width are any area and width and not necessarily of the glottis region and further if the area, width and/or coordinate information of the center position of the glottis region and epiglottis region means that an area width and/or coordinate information is calculated for both the glottis region and the epiglottis region or if the coordinate information is of the center position between the glottis region and the epiglottis region. Finally it is unclear because the claim previously recites indicating a glottis region and/or an epiglottis region making it unclear if the claim is further narrowed to include both the glottis region and the epiglottis region being indicated and if not it is unclear how the calculations are of both the glottis region and the epiglottis region if both are not indicated. For examination purposes, it has been interpreted that the claim merely requires an area or a width (of any feature) or coordinate information of a center position of either the glottis region or epiglottis region (whichever is indicated) or both or between the glottis region and the epiglottis region if both are indicated by the region information.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 7-14, 18-19, 24, and 28 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by WIPO Kamon (WO 2020162275 A1), hereinafter Kamon.
Regarding claim 1,
Kamon discloses an image processing device comprising:
one or more processors (at least fig. 2 (200) and corresponding disclosure in at least pg. 2 second to last full paragraph and/or at least fig. 12 (204A) and corresponding disclosure in at least pg. 12) configured to:
acquire a medical image (at least fig. 16 (S110) and corresponding disclosure in at least pg. 9 second to last full paragraph which discloses medical image acquisition unit 220 acquires an endoscopic image (medical image)));
output observation target identification information indicating an observation support target part included in the medical image or indicating that the medical image is out of an observation support target, by inputting the medical image to an observation target identification algorithm (at least fig. 16 (S120) and corresponding disclosure in at least pg. 9 last paragraph which discloses the part information acquisition part 222 acquires part information indicating the part in the living body where the endoscopic image was captured and can acquire site information by analyzing the endoscopic image and further discloses when analyzing an endoscopic image, the part information acquisition unit 222 (part information acquisition unit) (i.e. an observation target identification algorithm) can perform analysis using a feature amount such as the color of the subject. A trained model (CNN, SVM, etc.) for analysis may be used. Examiner notes that such output would indicate an observation support target part is included in the medical image and indicate that a medical image is out of an observation support target (i.e. in a case where the observation support target is a stomach and the site information is the esophagus, it is indicates that the medical image is out of the stomach and vice versa);
select, from a plurality of observation support algorithms, one specific observation support algorithm based on the observation target identification information (at least fig. 16 (S130) and corresponding disclosure in at least pg. 13 last full paragraph which discloses the section unit 226 selects a recognizer (i.e. one specific observation support algorithm) corresponding to the part indicated by the part information (i.e. based on the observation target identification information) from the plurality of recognizers);
output observation support information by inputting the medical image to the specific observation support algorithm (at least fig. 16 (S140) and corresponding disclosure in at least pg. 10 first full paragraph);
and perform a control of notifying of the observation support information (at least fig. 16 (S150) and corresponding disclosure in at least pg. 10 second full paragraph which discloses the display control unit causes the monitor to display the endoscopic image and the recognition result on the monitor).
Regarding claim 2,
Kamon further discloses wherein the observation support target part is an esophagus (pg. 14 first paragraph which discloses the control unit 228 activates the esophageal detector 240 (the upper esophageal first detector 240A, the lower esophageal second detector 2408) when the part information indicates the esophagus).
Regarding claim 3,
Kamon further discloses wherein the notification of the observation support information is performed by a guide image for displaying the observation support information (at least fig. 16 (S150) and corresponding disclosure in at least pg. 10 second full paragraph which discloses the display control unit causes the monitor to display the endoscopic image and the recognition result on the monitor. See at least figs. 8-11 depicting a guide image for displaying the observation support information (i.e. recognition result)) and/or a voice for notifying of the observation support information (pg. 5 which discloses the voice processing unit 209 output a message (voice) regarding the recognition process and the recognition result from the speaker 209A under the control of the CPU 210 and the image processing unit 204.
Regarding claim 4,
Kamon further discloses wherein the observation support information is endoscope position determination information indicating whether a position of an endoscope is appropriate or inappropriate (pg. 10 first full paragraph which discloses the recognizer 224 grasps the position of the attention area shown in the image by the "feature map" at the pixel level (that is, the attention area is determined for each pixel of the endoscopic image. Whether or not it belongs) can be output and the detection result can be output. Examples of the region of interest (region of interest) to be detected include polyps, cancer, large intestine diverticulum, inflammation, treatment scars (EMR: Endoscopic Mucosal Resection), ESD scars (ESD: Endoscopic Submucosal Dissection), clip locations, etc.). Examples include bleeding points, perforations, and vascular atypia). Examiner notes that any of the recognition results would indicate whether a position of an endoscope is appropriate or inappropriate in its broadest reasonable interpretation. In other words, output of such information would indicate to a user of the system whether the endoscope is appropriate or inappropriate), endoscope operation information indication an operation method of the endoscope (Examiner notes that any of the recognition results would indicate an operation method of the endoscope in its broadest reasonable interpretation. In other words, output of such information would indicate to a user of a method (e.g. imaging, position, location, etc.) of the endoscope), subject position information for prompting change or configuration of a position of a subject (Examiner notes that “for prompting change” is considered intended use, where examiner notes that a person having ordinary skill in the art would have recognized that the recognition results of pg. 10 could be used for prompting change or configuration of a position of a subject depending on the location of the region of interest/attention area in the image).
Regarding claim 7,
Kamon further discloses wherein the one or more processors are configured to perform a control of switching a display of the endoscope operation information based on the endoscope position determination information (pg. 10 last paragraph to pg. 11 first paragraph which discloses further, the recognition result may be displayed or hidden depending on a predetermined condition (elapsed time or the like) other than the region. Furthermore, the display mode (changing the figure, changing the color and brightness, etc.) may be changed according to the recognition result and/or its certainty. Thus the one or more processors are configured to perform a control of switching a display of information (including recognition result information such as endoscope operation information) based on the recognition result including the endoscope position determination information)
Regarding claim 8,
Kamon further discloses wherein the specific observation support algorithm includes an operation support algorithm (examiner notes that the recognizer is considered an operation support algorithm), and
The one or more processors are configured to output the endoscope operation information, by inputting the medical image to the operation support algorithm (Examiner notes that the recognition result is considered the endoscope operation information as noted above).
Regarding claim 9,
Kamon further discloses wherein the one or more processors are configured to input a latest medical image to the operation support algorithm (Examiner notes that the endoscopic image input into the recognizer is considered the latest medical image in its broadest reasonable interpretation. See also pg. 9 last full paragraph which discloses the medical image acquisition unit 220 sequentially takes images of the inside of a living body, which is the subject, at a predetermined frame rate by the imaging unit (the imaging lens 132, the imaging element 134, the AFE 138, etc.) of the endoscope 100 (medical device). Then, the endoscopic image can be acquired in real time).
Regarding claim 10,
Kamon further discloses wherein the operation support algorithm is a trained model that outputs the endoscope operation information and/or the subject position information (pg. 13 second to last full paragraph which discloses the recognizers 224A and 225 can be configured by using a plurality of learned models such as CNN or SVM).
Regarding claim 11,
Kamon further discloses wherein the specific observation support algorithm (at least fig. 15 (225) and corresponding disclosure in at least pg. 13 second full paragraph) includes a position determination algorithm (at least fig. 14 (240A) and corresponding disclosure in at least pg. 12 last paragraph to pg. 13 first paragraph, see also 241A and pg. 13 first paragraph which discloses the case where two (upper and lower) detectors are provided for each organ) and an operation support algorithm (at least fig. 14 (240B) and corresponding disclosure in at least pg. 12 last paragraph to pg. 13 first paragraph see also 241B and pg. 13 first paragraph which discloses the case where two (upper and lower) detectors are provided for each organ)),
The position determination algorithm (240A) is a trained model (pg. 13 second to last full paragraph) that outputs the endoscope position determination information in response to an input of the medical image (Examiner notes that the upper esophageal recognizer outputs a recognition result which is considered to be the endoscope position determination information as noted above), and
The operation support algorithm (240B) is a trained model (pg. 13 second to last full paragraph) that outputs the endoscope operation information and/or the subject position information in response to an input of the medical image (Examiner notes that the upper esophageal recognizer outputs a recognition result which is considered to be endoscope operation information and/or subjection position information as noted above).
Regarding claim 12,
Kamon further discloses wherein the one or more processors are configured to input a latest medical image to each of the position determination algorithm and the operation support algorithm (Examiner notes that the endoscopic image input into the recognizer is considered the latest medical image in its broadest reasonable interpretation. See also pg. 9 last full paragraph which discloses the medical image acquisition unit 220 sequentially takes images of the inside of a living body, which is the subject, at a predetermined frame rate by the imaging unit (the imaging lens 132, the imaging element 134, the AFE 138, etc.) of the endoscope 100 (medical device). Then, the endoscopic image can be acquired in real time)..
Regarding claim 13,
Kamon further discloses wherein the observation target identification algorithm is a trained model that has been trained using a learning image including the medical image in which an esophagus is included in an observation target (pg. 10 which discloses when analyzing an endoscopic image, the part information acquisition unit 222 (part information acquisition unit) can perform analysis using a feature amount such as the color of the subject. A trained model (CNN, SVM, etc.) for analysis may be used and pg. 14 first paragraph which discloses when the part information indicates the esophagus. Examiner notes that a person having ordinary skill in the art would have recognized a trained model for analysis of the endoscopic image resulting in a part determination of the esophagus is necessarily trained using a learning image including the medical image in which an esophagus is included in an observation target)
Regarding claim 14,
Kamon further discloses wherein the one or more processors are configured to perform a control of switching between presence and absence of the notification of the observation support information (pg. 10 last paragraph which discloses It should be noted that the recognition result may be displayed or hidden depending on the region by the above-mentioned condition setting. Further, when the recognition result is set to be non-display, a mode in which "recognition is performed but the result is not displayed" is possible. Further, the recognition result may be displayed or hidden depending on a predetermined condition (elapsed time or the like) other than the region).
Regarding claim 18,
Kamon further discloses wherein the one or more processor are configured to, in a case in which the observation support target output by the observation target identification algorithm is a nasopharynx:
Output nasopharyngeal position information indicating that the position of the endoscope is in an appropriate position or an inappropriate direction position that is an inappropriate right position, an inappropriate left position, an inappropriate upper position, an inappropriate lower position, an inappropriate back position, or an inappropriate front position, by inputting the medical image to the operation support algorithm (Examiner notes that the recognition result of a lower organ detector (e.g. 240B/241B) is considered to indicate that the position of the endoscope is in an appropriate position or an inappropriate direction. The processor would function in any case including a case in which the observation support target output by the observation target identification algorithm is the nasopharynx in which case such position information is considered nasopharyngeal position information)
Output the endoscope operation information based on the nasopharyngeal position information (S150).
Regarding claim 19,
Kamon further discloses wherein the operation support algorithm is a trained model (pg. 13 second to last full paragraph) that has been trained using a learning image including the medical image associated with the nasopharyngeal position information (pg. 13 second to last full paragraph).
Regarding claim 24,
Kamon further discloses wherein the one or more processors are configured to, in a case in which the observation support target part output by the observation target identification algorithm is an esophagus, output the endoscope operation information for providing an instruction (i.e. the recognition result) to pull out the endoscope by inputting the medical image to the operation support algorithm (Examiner notes that to pull out the endoscope is considered an intended use where the recognition result is considered an instruction which may be used to pull out the endoscope).
Regarding claim 28,
Kamon discloses an endoscope system (at least fig. 1 (10) and corresponding disclosure in at pg. 2 last full paragraph) comprising:
the image processing device according to claim 1 (see rejection of claim 1 above);
A light source device (at least fig. 1 (300) and corresponding disclosure in at least pg. 4 first full paragraph) that emits illumination light (see pg. 4 first full paragraph which discloses illumination light source and illuminance of the observation light from the light source 310 is controlled); and
An endoscope (at least fig. 1 (100) and corresponding disclosure in at least pg. 2 last full paragraph) that images the medical image (pg. 9 last full paragraph which discloses the medical image acquisition unit 220 (medical image acquisition unit) acquires an endoscopic image (medical image) taken in the living body of the subject (step S110: medical image acquisition step) and The medical image acquisition unit 220 sequentially takes images of the inside of a living body, which is the subject, at a predetermined frame rate by the imaging unit (the imaging lens 132, the imaging element 134, the AFE 138, etc.) of the endoscope 100 (medical device))
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Kamon in view of WIPO Takenouchi (WO 2021157487 A1), hereinafter Takenouchi. Examiner notes that citations to Takenouchi are with respect to the translated copy provided herein.
Regarding claim 5,
Kamon teaches the elements of claim 4 as previously stated. Kamon fails to explicitly teach wherein the endoscope operation information is a moving direction and/or a moving amount of a distal end part of the endoscope, and the moving direction is a right direction, a left direction, an upward direction, a downward direction, a backward direction, a forward direction, a right turn, or a left turn.
Takenouchi, in a similar field of endeavor involving endoscopic image processing, teaches outputting observation support information including endoscope operation information indicating whether a position of an endoscope is appropriate or inappropriate, wherein the endoscope operation information is a moving direction and/or a moving amount of a distal end part of the endoscope (pg. 8 second-third paragraphs which discloses operation support information (first information to fifth information and the direction or amount of movement of the endoscope 100 is first information)
Wherein the moving direction is a right direction, a left direction, an upward direction, a downward direction, a backward direction, a forward direction, a right turn, or a left turn (see at least fig. 11/17 depicting an arrow and pg. 12 which discloses the user moves the endoscope scope 100 forward/ backward, bends, etc. according to the arrow 906 (one aspect of the operation support information))
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon to include endoscope operation information as observation support information as taught by Takenouchi in order to improve determination results of the region of interest (Takenouchi Abstract). Such a modification would provide guidance to a user in order to make it possible to make the region of interest appear in the endoscopic image in a state suitable for estimating the size (Takenouchi pg. 8).
Regarding claim 6,
Kamon, as modified, teaches the elements of claim 8 as previously stated. Takenouchi, as applied to claim 5 above, further teaches wherein one or more processor are configured to perform a control of displaying an endoscope operation support diagram on a guide image as the endoscope operation information.
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon, as currently modified, to include an endoscope operation support diagram as the endoscope operation information as taught by Takenouchi in order to improve determination results of the region of interest (Takenouchi Abstract). Such a modification would provide guidance to a user in order to make it possible to make the region of interest appear in the endoscopic image in a state suitable for estimating the size (Takenouchi pg. 8)
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Kamon in view of Ben Hassen et al. (US 20220286602 A1), hereinafter Ben Hassen.
Regarding claims 15 and 17,
Kamon teaches the elements of claims 3 and 17 as previously stated. Kamon fails to explicitly teach wherein the one or more processors are configured to, in a case in which the observation support target part output by the observation target identification algorithm is a nasal cavity:
Output insertion region information indicating an upper route insertion region and/or a lower route insertion region, which is a region suitable for insertion of an endoscope., included in the medical image, by inputting the medical image to an operation support algorithm;
Calculate insertion route information that is an area, a width, and/or coordinate information of a center position of the upper route insertion region and/or the lower route insertion region based on the insertion region information; and
Perform a control of displaying the insertion route information on the guide image as the observation support information.
Ben Hassen, in a similar field of endeavor involving endoscope guidance, teaches wherein one or more processors are configured to, in a case in which a target part is a nasal cavity:
Output insertion region information indicating an upper route insertion region and/or a lower route insertion region, which is a region suitable for insertion of an endoscope., included in the medical image, by inputting the medical image to an operation support algorithm (at least fig. 5 (422) and corresponding disclosure in at least [0093] which discloses a visible lumen determining component 522 for determining the visible lumen by identifying the darkest pixels in the image frame. See at least fig. 2B in which the vessel determination information is output between steps 222 and 223);
Calculate insertion route information that coordinate information of a center position of the upper route insertion region and/or the lower route insertion region based on the insertion region information (at least fig. 5 (525) and corresponding disclosure in at least [0093]); and
Perform a control of displaying the insertion route information on the guide image as the observation support information (see at least fig. 3A-3B and figs. 4A-4C (414, 424, and 434) and corresponding disclosure in at least [0053] and [0058]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon to include outputting insertion region information, calculating insertion route information and performing a control of displaying the insertion route information as taught by Ben Hassen in order to improve guidance of the endoscopic device to decrease contact of the endoscopic device with walls of the cavity of the patient, reducing pain to the patient and damage to the walls of the cavity of the patient (Ben Hassen [0024]).
Examiner notes that the one or more processors in the modified system would function in the manner as recited in any instance including a case in which the observation support target part output is the nasal cavity.
Regarding claim 16,
Kamon, as modified, teaches the elements of claim 15 as previously stated. Ben Hassen further teaches wherein the one or more processors are configured to perform a control of displaying the upper route insertion region or the lower route insertion region on the guide image by changing a display mode thereof based on the insertion route information (See at least figs. 3A and 3B and 4A-4C in which the insertion region (either upper or lower) are displayed and a display mode thereof is changed with each movement of the endoscope and therefore based on the insertion route information in its broadest reasonable interpretation)
Claims 20-23 are rejected under 35 U.S.C. 103 as being unpatentable over Kamon in view of Sun et al. (US 20250017460 A1), hereinafter Sun.
Regarding claim 20,
Kamon teaches the elements of claim 11 as previously stated. Kamon fails to explicitly teach wherein the one or more processors are configured to, in a case in which the observation support target part output by the observation target identification algorithm is an oropharynx, and the position determination algorithm to which the medical image is input outputs that the position of the endoscope is inappropriate as the endoscope position determination information:
output oropharyngeal region information indicating a glottis region and/or an epiglottis region included in the medical image, by inputting the medical image to the operation support algorithm;
calculate oropharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region and the epiglottis region based on the oropharyngeal region information; and output the endoscope operation information using the oropharyngeal region arithmetic information.
Sun, in a similar field of endeavor involving endoscopic guidance, teaches wherein one or more processors are configured to output oropharyngeal region information indicating a glottis region included in the medical image, by inputting the medical image to the operation support algorithm ([0031] which discloses one or more anatomical features are identified in the image signal including the glottis and epiglottis);
calculate oropharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region based on the oropharyngeal region information ([0050] which discloses the anatomy model can estimate passage size and feature size based on the image signal and extrapolate position s and sizes of other features and the direction of the arrow can be determined using the parameters (e.g. size and relative position estimates) determined based on the image signal that are provided to the anatomy model));
and output the endoscope operation information using the oropharyngeal region arithmetic information (at least fig. 6 (46a) and corresponding disclosure in a least [0043] and/or fig. 7 (46b) and corresponding disclosure in at least [0046]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon to include outputting oropharyngeal region information, arithmetic information, and outputting endoscope operation information as taught by Sun in order to provide guidance for orienting the endoscope towards an identified anatomical feature (Sun Abstract), thereby permitting improved visualization of an airway during intubation or other procedures (Sun [0025]). Such a modification would provide for enhanced navigation guidance through airways accordingly.
Examiner notes that in the modified system, the processors would function accordingly in any case including in a case in which the observation support target part output by the observation target identification algorithm is an oropharynx, and the position determination algorithm to which the medical image is input outputs that the position of the endoscope is inappropriate as the endoscope position determination information.
Regarding claim 21,
wherein the position determination algorithm is a trained model that has been trained using a learning image in which the medical image and the endoscope position determination information are associated with each other (pg. 13 second to last full paragraph which discloses the recognizers 224A and 225 can be configured by using a plurality of learned models such as CNN or SVM).
Regarding claim 22,
Kamon teaches the elements of claim 11 as previously stated. Kamon fails to explicitly teach wherein the one or more processors are configured to, in a case in which the observation support target part output by the observation target identification algorithm is a hypopharynx, and the position determination algorithm to which the medical image is input outputs that the position of the endoscope is inappropriate as the endoscope position determination information: output hypopharyngeal region information indicating a glottis region and/or a vocal fold region included in the medical image, by inputting the medical image to the operation support algorithm; calculate hypopharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region, and that is a length of the vocal fold region based on the hypopharyngeal region information; and output the endoscope operation information using the hypopharyngeal region arithmetic information.
Sun, in a similar field of endeavor involving endoscopic guidance, teaches wherein one or more processors are configured to output hypopharyngeal region information indicating a glottis region and/or a vocal fold region included in the medical image, by inputting the medical image to the operation support algorithm ([0031] which discloses one or more anatomical features are identified in the image signal including the glottis and vocal cords);
calculate hypopharyngeal region arithmetic information that is an area, a width, and/or coordinate information of a center position of the glottis region, and that is a length of the vocal fold region based on the hypopharyngeal region information ([0050] which discloses the anatomy model can estimate passage size and feature size based on the image signal and extrapolate position s and sizes of other features and the direction of the arrow can be determined using the parameters (e.g. size and relative position estimates) determined based on the image signal that are provided to the anatomy model));
and output the endoscope operation information using the hypopharyngeal region arithmetic information (at least fig. 6 (46a) and corresponding disclosure in a least [0043] and/or fig. 7 (46b) and corresponding disclosure in at least [0046])
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon to include outputting hypopharyngeal region information, arithmetic information, and outputting endoscope operation information as taught by Sun in order to provide guidance for orienting the endoscope towards an identified anatomical feature (Sun Abstract), thereby permitting improved visualization of an airway during intubation or other procedures (Sun [0025]). Such a modification would provide for enhanced navigation guidance through airways accordingly.
Examiner notes that in the modified system, the processors would function accordingly in any case including in a case in which the observation support target part output by the observation target identification algorithm is a hypopharynx, and the position determination algorithm to which the medical image is input outputs that the position of the endoscope is inappropriate as the endoscope position determination information.
Regarding claim 23,
Kamon further teaches wherein the position determination algorithm is a trained model that has been trained using a learning image in which the medical image and the endoscope position determination information are associated with each other (pg. 13 second to last full paragraph which discloses the recognizers 224A and 225 can be configured by using a plurality of learned models such as CNN or SVM).
Claims 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Kamon in view of Aladahalli et al. (US 20210145411 A1), hereinafter Aladahalli.
Regarding claim 25,
Kamon teaches the elements of claim 1 as previously stated. Kamon fails to explicitly teach wherein the one or more processors are configured to, in a case in which the observation target identification algorithm outputs that the medical image is out of the observation support target as the observation target identification information, output the observation support stop information, by inputting the medical image to the operation support algorithm.
Aladahalli, in a similar field of endeavor involving medical image analysis, teaches wherein one or more processors are configured to, in a case in which an observation target identification algorithm outputs that a medical image is out of an observation support target as observation target identification information, output an observation support stop information indicating that an observation support is stopped, by inputting the medical image to an operation support algorithm (at least fig. 3A (320) and corresponding disclosure in at least [0046] where it is noted that observation support stop information is information that there is turbulents/low image quality and the one or more desired image interpretation protocols are stopped (i.e. not deployed)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have modified Kamon to include outputting an observation support stop information as taught by Aladahalli in order to provide guidance to a user that the image quality is low and to hold the probe steady or move the probe to a position where the quality is improved so that the operation support may be continued.
Regarding claim 26,
Kamon, as modified, teaches the elements of claim 25, as previously stated. Kamon further teaches wherein the observation target identification algorithm is a trained model that has been trained to output that the observation support target is in the medical image, in a case in which the medical image including a foreign substance, which is food or saliva, shake, blurriness, or halation is input, output that the medical image is out of the observation support target as the observation target identification information (at least fig. 16 (S120) and corresponding disclosure in at least pg. 9 last paragraph which discloses the part information acquisition part 222 acquires part information indicating the part in the living body where the endoscopic image was captured and can acquire site information by analyzing the endoscopic image and further discloses when analyzing an endoscopic image, the part information acquisition unit 222 (part information acquisition unit) (i.e. an observation target identification algorithm) can perform analysis using a feature amount such as the color of the subject. A trained model (CNN, SVM, etc.) for analysis may be used. Examiner notes that outputting the esophagus as the part information is considered an output that the medical image is out of the observation support target (e.g. stomach) and vice versa and would occur in any instance including an instance in which the medical image includes a foreign substance, which is food or saliva, shake, blurriness, or halation)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kamon (US 20200320702 A1) teaches selecting a specific operation support algorithm ([0066]), of a plurality of operation support algorithms (46A-46H), based on an observation target identification information output by an observation target identification algorithm (48) ([0099])
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BROOKE L KLEIN whose telephone number is (571)270-5204. The examiner can normally be reached Mon-Fri 7:30-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at 5712700552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BROOKE LYN KLEIN/Primary Examiner, Art Unit 3797
Prosecution Insights