DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Applicant is advised that should claim 1 be found allowable, claim 21 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a step of acquiring an image obtained by imaging the observation target;”, “ a step of setting a plurality of regions of interest in the image;”, “ a step of performing determination of, with respect to each of the plurality of regions of interest, whether or not correction of data using the region of interest is possible;” and “a step of correcting the data using the region of interest for which it is determined that the correction of the data is possible” in claim 22.
The ”a step” is a general place holder and a non-structural term that does not impose any specific structural limitation. The step is modified by functional language, “of”, and is not further modified by sufficient structure, material or acts for performing the claim function. Accordingly, this limitation meets the 3-prong analysis, and therefore invoke 112(f).
The applicant’s specification discloses “The series of procedures of the processing in the endoscope system or the like will be described using a flowchart” (Paragraph 0170), and further detail steps (ST110-ST210) of the processing in the endoscope system in Paragraph 0171-0173). Therefore, the step of-functions provides a sufficient actions or structures of the processing in the endoscope system as recited in the specification.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-23 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process for determining the oxygen saturation of the object) without significantly more.
Step 1:
The claimed invention of claim 1-8, 13, 16-18, and 21-23 are directed to statutory subject matter as the claims recite an endoscope system and a method for calculating oxygen saturation of an observation target.
Step 2A, Prong One:
Regarding claim 1-8, 13, 16-18, and 21-23, the recited steps are directed to mental process, i.e., concepts that can be performed in a human mind (see MPEP 2106.04(a)(2) subsection (III)). The courts have found that concepts performed in a human mind falls within the judicial exceptions, often labelled as “abstract ideas”.
Specifically, the following limitations recite mental process;
Regarding claim 1, the limitation of “calculates oxygen saturation of an observation target used in calculating the oxygen saturation” (mental process: pen and pencil),
Regarding claim 1, the limitation of “ acquire an image obtained by imaging the observation target” (mental process: observation)
Regarding claim 1, the limitation of “set a plurality of regions of interest in the image” (mental process: observation and evaluation or with pen and paper)
Regarding claim 1, the limitations of “perform determination of, with respect to each of the plurality of regions of interest, whether or not correction of the data” and “which it is determined that the correction of the data is possible”(mental process: opinion, judgement, evaluation),
Regarding claim 1, the limitations of “whether or not correction of the data using the region of interest is possible”, and “correct the data” (mental process: opinion, judgement, evaluation)
Regarding claim 2, the limitation of “determination processing of determining a representative region” (mental process: opinion, judgement, evaluation),
Regarding claim 2, the limitation of “by comparing the region reliability degrees” (mental process: opinion, judgement, evaluation),
Regarding claim 3, the limitation of “the region reliability degree is calculated, using a pixel reliability degree calculated for each pixel included” ( mental process: pen and pencil),
Regarding claim 4, the limitation of “comparing the region reliability degree of the representative region, determined through the representative region determination processing” (mental process: opinion, judgement, evaluation),
Regarding claim 5, the limitation of “correct the pixel reliability degree” (mental process: opinion, judgement, evaluation),
Regarding claims 5-7, the limitation of “the region reliability degree is calculated” (mental process: pen and pencil),
Regarding claims 8, the limitation of “set the regions of interest such that each of the plurality of regions of interest has substantially the same shape” (mental process: observation and evaluation or with pen and paper)
Regarding claim 9, the limitation of “set the regions of interest such that each of the plurality of regions of interest has substantially the same area” (mental process: observation and evaluation or with pen and paper)
Regarding claim 10, the limitation of “set the regions of interest that have shapes different from each other and/or areas different from each other based on positions of the regions of interest in the image” (mental process: observation and evaluation or with pen and paper)
Regarding claim 11, the limitation of “set the regions of interest such that at least two of the plurality of regions of interest include the same pixel of the image” (mental process: observation and evaluation or with pen and paper)
Regarding claim 12, the limitation of “set the regions of interest such that each of the plurality of regions of interest includes pixels of the image different from each other” (mental process: observation and evaluation or with pen and paper)
Regarding claim 13, the limitation of “perform control of giving a notification to a user, which it is determined that the correction of the data is possible” (mental process: opinion, judgement, evaluation),
Regarding claim 14 and 15 the limitations of “perform control of displaying the region of interest for which it is that determined that the correction of the data is possible on a display”, and “perform control of displaying a pixel having a pixel reliability degree” (extra-solution activities)
Regarding claim 16, the limitation of “calculate, in a case where the data is corrected, the oxygen saturation of the observation target using the corrected data” (mental process: pen and pencil)
Regarding claim 17, the limitations of “calculate, the oxygen saturation of the observation target using the uncorrected data ,” and “it is determined that the correction of the data is possible” (mental process: opinion, judgement, evaluation and pen and pencil).
Regarding claim 18, the limitation of “correct the data using image information of the region of interest for which it is determined that the correction of the data is possible.” (mental process: opinion, judgement, evaluation)
Regarding claim 19, the limitation of “the image information is a concentration of a specific coloring agent other than blood hemoglobin” (mental process: opinion, judgement, evaluation)
Regarding claim 20, the limitation of “the specific coloring agent is a yellow coloring agent” (mental process: opinion, judgement, evaluation)
Regarding claim 21, the limitation of “calculates oxygen saturation of an observation target” (mental process: pen and pencil)
Regarding claim 21, the limitation of “ acquire an image obtained by imaging the observation target” (mental process: observation)
Regarding claim 21, the limitation of “set a plurality of regions of interest in the image” (mental process: observation and evaluation or with pen and paper)
Regarding claim 21, the limitation of “it is determined that the correction of the data is possible” (mental process: opinion, judgement, evaluation),
Regarding claim 21, the limitation of “correct the data” (mental process: opinion, judgement, evaluation),
Regarding claim 22, the limitation of “calculating oxygen saturation of an observation target” (mental process: pen and pencil),
Regarding claim 22, the limitations of “performing determination of”, “it is determined that the correction of the data is possible.” (mental process: opinion, judgement, evaluation)
Regarding claim 23, the limitation of “calculates oxygen saturation of an observation target” (mental process: pen and pencil),
Regarding claim 23, the limitations of “performing determination of”, and “it is determined that the correction of the data is possible” (mental process: opinion, judgement, evaluation),
Furthermore, the “performing determination” step reasonably encompasses a user mentally evaluating whether correction is possible or manually performing that evaluation using pen and paper. The “acquire an image” and “ set a plurality of regions” reasonably encompasses a human mentally performing observation after the step of taking the image. So a user just acquires an image that was obtained and set a plurality of regions. The “correct pixel reliability degree”, and ” correct the data” reasonably encompasses a user mentally performing the opinion, judgment, and evaluation
“The image information is a concentration of a specific coloring agent other than blood hemoglobin” reasonably encompasses a user characterizing or identifying a coloring agent can be treated as abstract idea, performing opinion, judgement, evaluation. “The specific coloring agent is a yellow coloring agent” reasonably encompasses a user specifying the type of coloring agent, performing opinion, judgement, evaluation.
Step 2A, Prong Two:
The claims, including dependent claims, are analyzed as a whole to determine whether additional limitations are recited such that the claims amount to significantly more than the abstract idea.
Regarding claim 1, the additional limitation of “a processor” amounts to mere instructions to implement the abstract idea on generic computer components.
Regarding claim 5 and 8-18, the additional limitation, the “processor”, amounts to a generic computer component.
Regarding claim 21, the additional limitations, “a processor” and the “processor”, amounts to mere instructions to implement the abstract idea on generic computer components,
Regarding claim 23, the additional limitation, “a computer”, amounts to a mere generic computer.
Regarding claim 23, the additional limitation, “a non-transitory computer readable medium”, amounts to mere instructions to implement the abstract idea on generic computer components,
The applicant’s specification does not sufficiently describe a concrete technological improvement attributable to the claimed processing.
Merely including instructions to implement an abstract idea on “a computer” and “a non-transitory computer readable medium” do not integrate a judicial exception. Regarding Claim 1-23, the judicial exception is not integrated into a practical application.
Step 2B:
The claim 1-8, 13, 16-18, and 21-23 do not include additional elements that are sufficient to amount significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements, including “a computer” and “a non-transitory computer readable medium” are recited at a high level of generality and are used for extra-solution activity, such as data gathering necessary to perform the abstract idea as well as part of generic computer equipment used to perform well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984 (see MPEP § 2106.05(d)). Therefore, these elements do not provide an inventive concept sufficient to transform the claimed abstract idea into patent eligible subject matter.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-23 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, the term “it” in line 9 renders the claim indefinite because “it” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 1, the phrase “calculates oxygen saturation of an observation target using data used in calculating the oxygen saturation” renders the claim indefinite because the meaning is unclear and it is unclear where the data comes from.
Claim 1 recites the limitation “the region of interest” in lines 8 and 9. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Claim 2 recites the limitation “the region of interest” in line 4. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Claim 6 recites the limitation “the region of interest” in line 3. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Claim 7 recites the limitation “the region of interest” in line 3. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Regarding claim 14, the term “it” in line 3 renders the claim indefinite because “it” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 17, the term “it” in line 3 renders the claim indefinite because “it” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 18, the term “it” in line 3 renders the claim indefinite because “it” is undefined or unclear what the term “it” meant to refer to. “it” should be defined.
Regarding claim 21, the term “it” in line 9 renders the claim indefinite because “it” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 21, the term, "data" in line 8 renders the claim indefinite because it is unclear and where the term, “data” comes from.
Claim 21 recites the limitation “the region of interest” in lines 8 and 9. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Regarding claim 22, the term, "data" in line 6 renders the claim indefinite because it is unclear and where the term, “data” comes from.
Claim 22 recites the limitation “the region of interest” in lines 6 and 7. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Regarding claim 22, the term “it” in line 7 renders the claim indefinite because “It” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 23, the term “it” in line 8 renders the claim indefinite because “It” is undefined or unclear what the term “it” meant to refer to. “It” should be defined.
Regarding claim 23, the term, "data" in line 7 renders the claim indefinite because it is unclear and where the term, “data” comes from.
Claim 23 recites the limitation “the region of interest” in lines 7 and 8. There is insufficient antecedent basis for this limitation in the claim. [07.34.05]. The limitation should be “each of the plurality of regions of interest”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8-9, 12, 16, 18-20, and 21-23 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Shigeta” (WO2023119856A1).
Regarding claim 1,
Shigeta teaches
An endoscope system that calculates oxygen saturation of an observation target using data used in calculating the oxygen saturation, the endoscope system comprising (Paragraph, 0001: “”, Paragraph, 0022: “As shown in FIG. 3, in the oxygen saturation mode, the oxygen saturation of the observation target is calculated, and the oxygen saturation image obtained by imaging the calculated oxygen saturation is displayed on the extension display 18”):
a processor, wherein the processor is configured to (Paragraph, 0033: ” The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”)
acquire an image obtained by imaging the observation target (Paragraph, 0030: “The imaging sensor 36 is a color imaging sensor that captures an image of an observation target during illumination with illumination light”, and Paragraph, 0029: “The light from the observation target irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, the image of the observation target is formed on the imaging sensor 36”);
set a plurality of regions of interest in the image (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Paragraph, 0125: “Based on the above-described effective pixel determination, the sum of squares of the number of pixels of the effective pixel, the total pixel value of the effective pixel, and the pixel value of the effective pixel is calculated for each ROI”, and Figure 59);
perform determination of, with respect to each of the plurality of regions of interest (Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”, Paragraph, 0081: “the reliability of the G2 image signal outside the fixed range Rx has a lower luminance value of the G2 image signal than the reliability in the fixed range Rx”, and Paragraph, 0129: “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”),
whether or not correction of the data using the region of interest is possible (Paragraph, 0129: “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”, and Paragraph, 0088: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region. In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed. On the other hand, when the number of effective pixels for the pixels in the specific region is less than the predetermined value, it is determined that the correction processing cannot be appropriately performed”); and
correct the data using the region of interest for which it is determined that the correction of the data is possible (Paragraph, 0130: “In the specific dye concentration calculation, the specific dye concentration is calculated for each of the 16 ROIs”, “As a result, the total 16 specific dye concentration PG 1 is calculated for each ROI”, Paragraph, 0132: “When there is no error in the error determination related to the movement, one specific dye concentration is calculated from the specific dye concentration PG 1 of the total 32 and the specific dye concentration PG 2 by using a specific estimation method (for example, a robust estimation method). The calculated specific pigment concentration is used in the correction processing of the correction mode.”, Paragraph, 0073: “The table correction unit 63 performs a table correction process for correcting the oxygen saturation degree calculation table 73 on the basis of the specific dye concentration as the correction process performed in the correction mode”, “The table correction unit 63 corrects the oxygen saturation degree calculation table 73 so as to become the generated contour line EL”).
Regarding claim 8, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above),
Shigeta teaches wherein the processor is configured to: set the regions of interest such that each of the plurality of regions of interest has substantially the same shape (Paragraph, 0124: “The effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Fig. 59 illustrates 16-uniformly sized and shaped rectangular ROIs in a regular 4 by 4 grid).
Regarding claim 9, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above)
Shigeta teaches wherein the processor is configured to: set the regions of interest such that each of the plurality of regions of interest has substantially the same area (Paragraph, 0124: “The effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Fig. 59 illustrates 16-uniformly sized and shaped rectangular ROIs in a regular 4 by 4 grid).
Regarding claim 12, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above)
Shigeta teaches wherein the processor is configured to: set the regions of interest such that each of the plurality of regions of interest includes pixels of the image different from each other (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Fig. 59 illustrates 16-uniformly sized and shaped rectangular ROIs in a regular 4 by 4 grid, not overlapping each other. Each ROI appears to contain a different set of pixels.).
Regarding claim 16, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above)
Shigeta teaches wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
in a case where the data is corrected (Paragraph, 0086: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region. In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed”, and paragraph, 0089: “as shown in FIG. 34, the message MS 1 such as "correction processing is appropriately performed" is displayed on the extension display 18. In this case the table correction unit 63 performs the table correction processing on the basis of the specific dye concentration in the specific region as the correction processing”),
calculate, the oxygen saturation of the observation target using the corrected data (Paragraph, 0132: “When there is no error in the error determination related to the movement, one specific dye concentration is calculated from the specific dye concentration PG 1 of the total 32”, “The calculated specific pigment concentration is used in the correction processing of the correction mode”, and Paragraph, 0073: “The table correction unit 63 performs a table correction process for correcting the oxygen saturation degree calculation table 73 on the basis of the specific dye concentration as the correction process performed in the correction mode. In the table correction process, the signal ratio B 1 stored in the oxygen saturation degree calculation table 73／ G2, R2／ The correlation between G 2 and the oxygen saturation is corrected”).
Regarding claim 18, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above)
Shigeta teaches wherein the processor is configured to: correct the data using image information of the region of interest (Paragraph, 0087: “It is preferable that the display mode control unit 65 makes the display mode of the specific area when it is determined that the correction processing can be properly performed and the display mode of the specific area when it is determined that the correction processing cannot be properly performed in the first correction determination unit 67. For example, as illustrated in FIG. 33A, when it is determined that the correction processing can be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the first color. On the other hand, as shown in FIG. 33B, when it is determined that the correction processing cannot be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the second color different from the first color. In order to make it easier to grasp whether or not the first color and the second color are appropriate, the first color and the second color are preferably opposite colors”) for
which it is determined that the correction of the data is possible (Paragraph, 0086: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region. In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed. On the other hand, when the number of effective pixels for the pixels in the specific region is less than the predetermined value, it is determined that the correction processing cannot be appropriately performed. It is preferable that the first correction determination unit 67 performs the determination each time the image is acquired and the reliability is calculated until the correction operation is performed.”).
Regarding claim 19, Shigeta teaches the endoscope system of claim 18 (See rejection of claim 18 above)
Shigeta teaches wherein the image information is a concentration of a specific coloring agent (Paragraph, 0089: “In this case, the table correction unit 63 performs the table correction processing on the basis of the specific dye concentration in the specific region as the correction processing”)
other than blood hemoglobin (Paragraph, 0022: “In the correction mode, a correction process related to calculation of the oxygen saturation is performed based on a specific dye concentration of a specific dye other than blood hemoglobin such as a yellow dye”).
Regarding claim 20, Shigeta teaches the endoscope system of claim 19 (See rejection of claim 19 above),
Shigeta teaches wherein the specific coloring agent is a yellow coloring agent (Paragraph, 0022: “In the correction mode, a correction process related to calculation of the oxygen saturation is performed based on a specific dye concentration of a specific dye other than blood hemoglobin such as a yellow dye”).
Regarding claim 21,
Shigeta teaches an image processing apparatus that calculates oxygen saturation of an observation target, the image processing apparatus (Abstract: “The endoscope system includes: an image signal acquisition unit acquiring first image signal in first wavelength range where the amount of light absorption changes according to the concentration of yellow dye”, Paragraph, 0001: “The present invention relates to an endoscope system for calculating the oxygen saturation of an observation target and a method for operating the endoscope system”, Paragraph, 0022: “As shown in FIG. 3, in the oxygen saturation mode, the oxygen saturation of the observation target is calculated, and the oxygen saturation image obtained by imaging the calculated oxygen saturation is displayed on the extension display 18”) comprising:
a processor, wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
acquire an image obtained by imaging the observation target (Paragraph, 0030: “The imaging sensor 36 is a color imaging sensor that captures an image of an observation target during illumination with illumination light”, and Paragraph, 0029: “The light from the observation target irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, the image of the observation target is formed on the imaging sensor 36”);
set a plurality of regions of interest in the image (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion, and Fig. 59 illustrates the pixel determination within 16 region of interest ROI, and Paragraph, 0125: “Based on the above-described effective pixel determination, the sum of squares of the number of pixels of the effective pixel, the total pixel value of the effective pixel, and the pixel value of the effective pixel is calculated for each ROI, and Figure 59);
perform determination of, with respect to each of the plurality of regions of interest (Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”, “For example, it is preferable that the luminance value of the G2 image signal is outside the fixed range Rx”, and “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”),
whether or not correction of data using the region of interest is possible (Paragraph, 0129: “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”, and Paragraph, 0086: “In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed. On the other hand, when the number of effective pixels for the pixels in the specific region is less than the predetermined value, it is determined that the correction processing cannot be appropriately performed”); and
correct the data using the region of interest for which it is determined that the correction of the data is possible (Paragraph, 130: “In the specific dye concentration calculation, the specific dye concentration is calculated for each of the 16 ROIs”, “As a result, the total 16 specific dye concentration PG 1 is calculated for each ROI”).
Regarding claim 22,
Shigeta teaches an operation method of an endoscope system of calculating oxygen saturation of an observation target, the operation method (Paragraph, 0001: “The present invention relates to an endoscope system for calculating the oxygen saturation of an observation target and a method for operating the endoscope system”, Paragraph, 0022: “As shown in FIG. 3, in the oxygen saturation mode, the oxygen saturation of the observation target is calculated, and the oxygen saturation image obtained by imaging the calculated oxygen saturation is displayed on the extension display 18”) comprising:
a step of acquiring an image obtained by imaging the observation target (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion, and Fig. 59 illustrates the pixel determination within 16 region of interest ROI, Paragraph, 0125: “Based on the above-described effective pixel determination, the sum of squares of the number of pixels of the effective pixel, the total pixel value of the effective pixel, and the pixel value of the effective pixel is calculated for each ROI”, Paragraph, 0030: “The imaging sensor 36 is a color imaging sensor that captures an image of an observation target during illumination with illumination light”, Paragraph, 0029: “The light from the observation target irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, the image of the observation target is formed on the imaging sensor 36”)
a step of setting a plurality of regions of interest in the image (Paragraph, 0024: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion, and Fig. 59 illustrates the pixel determination within 16 region of interest ROI);
a step of performing determination of, with respect to each of the plurality of regions of interest, whether or not correction of data using the region of interest is possible (Paragraph, 0087: “It is preferable that the display mode control unit 65 makes the display mode of the specific area when it is determined that the correction processing can be properly performed and the display mode of the specific area when it is determined that the correction processing cannot be properly performed in the first correction determination unit 67. For example, as illustrated in FIG. 33A, when it is determined that the correction processing can be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the first color. On the other hand, as shown in FIG. 33B, when it is determined that the correction processing cannot be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the second color different from the first color. In order to make it easier to grasp whether or not the first color and the second color are appropriate, the first color and the second color are preferably opposite colors.”); and
a step of correcting the data using the region of interest for which it is determined that the correction of the data is possible (Paragraph, 0089: “as shown in FIG. 34, the message MS 1 such as "correction processing is appropriately performed" is displayed on the extension display 18. In this case the table correction unit 63 performs the table correction processing on the basis of the specific dye concentration in the specific region as the correction processing”).
Regarding claim 23,
Shigeta teaches A non-transitory computer readable medium for storing a computer-executable program (Paragraph, 0033: “A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory”) for
causing a computer to function as an endoscope system that calculates oxygen saturation of an observation target, the computer-executable program causing the computer to realize (Paragraph, 0064: “Programs related to various processes are incorporated in the program memory (not shown) in the extended processor device 17. By executing a program in the program memory by a central control unit (not shown) configured by the processor, functions of the oxygen saturation image generation unit 61, the specific dye concentration calculation unit 62, the table correction unit 63, the mode switching unit 64, the display mode control unit 65, the reliability calculation unit 66, the first correction determination unit 67, the second correction determination unit 68, and the determination notification unit 69 are realized.”, and Paragraph, 0134: “One processing unit may be configured by one of the various processors, or may be configured by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Further”):
a function of acquiring an image obtained by imaging the observation target (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion, and Fig. 59 illustrates the pixel determination within 16 region of interest ROI, Paragraph, 0125: “Based on the above-described effective pixel determination, the sum of squares of the number of pixels of the effective pixel, the total pixel value of the effective pixel, and the pixel value of the effective pixel is calculated for each ROI”, Paragraph, 0029: “The light from the observation target irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, the image of the observation target is formed on the imaging sensor 36”, and Paragraph, 0030: “The imaging sensor 36 is a color imaging sensor that captures an image of an observation target during illumination with illumination light”);
a function of setting a plurality of regions of interest in the image (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion, and Fig. 59 illustrates the pixel determination within 16 region of interest ROI);
a function of performing determination of, with respect to each of the plurality of regions of interest, whether or not correction of data using the region of interest is possible (Paragraph, 0087: “It is preferable that the display mode control unit 65 makes the display mode of the specific area when it is determined that the correction processing can be properly performed and the display mode of the specific area when it is determined that the correction processing cannot be properly performed in the first correction determination unit 67. For example, as illustrated in FIG. 33A, when it is determined that the correction processing can be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the first color. On the other hand, as shown in FIG. 33B, when it is determined that the correction processing cannot be appropriately performed in the first correction determination unit 67, the specific region 81 is highlighted in the second color different from the first color. In order to make it easier to grasp whether or not the first color and the second color are appropriate, the first color and the second color are preferably opposite colors”); and
a function of correcting the data using the region of interest for which it is determined that the correction of the data is possible (Paragraph, 0089: “as shown in FIG. 34, the message MS 1 such as "correction processing is appropriately performed" is displayed on the extension display 18. In this case the table correction unit 63 performs the table correction processing on the basis of the specific dye concentration in the specific region as the correction processing”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of “Bakker” (Bakker, "Edge preserving orientation adaptive filtering", IEEE Computer Society Conference On Computer Vision & Pattern Recognition, 1999, V1, P535-540).
Regarding claim 2, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above),
Shigeta teaches wherein the determination processing includes region reliability degree calculation processing of calculating a region reliability degree of each of the plurality of regions of interest based on the region of interest (Paragraph, 0130: “In the specific dye concentration calculation, the specific dye concentration is calculated for each of the 16 ROIs”, Paragraph, 0081: “the reliability of the G2 image signal outside the fixed range Rx has a lower luminance value of the G2 image signal than the reliability in the fixed range Rx”, and Paragraph, 0129: “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”) and
However, Shigeta is silent on representative region determination processing of determining a representative region by comparing the region reliability degrees of the plurality of respective regions of interest with each other.
Bakker teaches representative region determination processing of determining a representative region by comparing the region reliability degrees (Page 536, Section 2, below Figure 1: “Each of these windows yields an estimate and a confidence value”) of the plurality of respective regions of interest (Window) (Page 536, Section 2, below Figure 1: “Our filter evaluates each fixed size window that contains the current pixel”) with each other (Page 536, Section 2, below Figure 1: “The estimate from the window with the highest confidence value is taken as the result”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Bakker because comparing the region reliability degrees of multiple candidate ROIs and select the representative ROI having the highest reliability would have avoided filtering across region boundaries and preserved sharp region boundaries during subsequent image processing ((Bakker, Page 536, Section 2, (above Figure 1) and Page 536, Section 2, (below Figure 1)).
Regarding claim 4, Shigeta in view of Bakker teaches the endoscope system of claim 2 (See rejection of claim 2 above),
Shigeta teaches wherein the determination processing includes region reliability degree comparison processing (Paragraph, 0086: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region”)
of comparing the region reliability degree of the representative region (the specific region) (Paragraph, 0086: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region. In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed”, “On the other hand, when the number of effective pixels for the pixels in the specific region is less than the predetermined value, it is determined that the correction processing cannot be appropriately performed. It is preferable that the first correction determination unit 67 performs the determination each time the image is acquired and the reliability is calculated until the correction operation is
performed”)
However, Shigeta is silent on determined through the representative region determination processing with a region reliability degree threshold value set in advance.
Bakker teaches determined through the representative region determination processing with a region reliability degree threshold value set in advance (Page 536, Section 2, below Figure 1: “Each of these windows yields an estimate and a confidence value”, “Our filter evaluates each fixed size window that contains the current pixel”, and “The estimate from the window with the highest confidence value is taken as the result”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Bakker’s window (ROI) selection process because selecting the highest-confidence candidate region would have facilitated Shigeta’s threshold comparison process performed on the most reliable representative region, rather than an arbitrary region for Shigeta’s subsequent determination of whether correction processing can be appropriately performed (Shigeta, Paragraph, 0086).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of Bakker, and “Tian” (WO 2022107595 A1)
Regarding claim 3, Shigeta in view of Bakker teaches the endoscope system of claim 2 (See rejection of claim 2 above),
Shigeta teaches wherein the region reliability degree (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”, “in the case of the pair of the effective pixel data Gr 2 and the effective pixel data W 3, the reliability of the total 32 is calculated. In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed”)
However, Shigeta does not teach “is calculated using a pixel reliability degree using a pixel reliability degree calculated for each pixel included in each of the plurality of regions of interest”.
Tian teaches is calculated using a pixel reliability degree using a pixel reliability degree calculated for each pixel (Page 8, Paragraph 8, lines 23-25: “the recognition unit 331 recognizes a predetermined recognition target for each pixel of an image (hereinafter referred to as a captured image) captured by the camera 51 (image sensor) in FIG. 1, and determines the reliability of the recognition result”, Page 13, Paragraph 7,lines 27-29: “the recognition result image consisting of the recognition value of each pixel and the reliability image consisting of the reliability of each pixel are generated”)
included in each of the plurality of regions of interest (P) (Page 13, Paragraph 10, lines 34-35: “the verification image collecting unit 362 calculates an average value (hereinafter, referred to as an average reliability) of the reliability of each pixel of the reliability image”, Page 13, Paragraph 11, lines 39-40: “the verification image collecting unit 362 compares the reliability of each pixel of the reliability image with the reliability threshold τ”, and Page 13, Paragraph 11, lines 41-44: “The verification image collecting unit 362 regards each pixel of the reliability image as a pixel having a reliability greater than the reliability threshold τ (hereinafter referred to as a high reliability pixel) and a pixel having a reliability equal to or less than the reliability threshold τ (hereinafter referred to as a high reliability pixel). It is classified as a low-reliability pixel)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Tian’s Pixel reliability selection because determining reliable and unreliable pixels would have enabled region reliability to be calculated from reliable pixel information, thereby improving the reliability of the subsequent region reliability determination and correction processing (Tian, Page 13, Paragraph 11, lines 39-44, Page 13, Paragraph 7, lines 27-29).
Claim 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of Bakker, Tian, and “Takeshima” (US 20210174501 A1),
Regarding claim 5, Shigeta in view of Bakker and Tian teaches the endoscope system of claim 3 (See rejection of claim 3 above),
Shigeta teaches wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
the region reliability degree is calculated (Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”, “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed. The result of the error determination on the reliability is notified to the user by display on the extension display 18 or the like”, and Paragraph, 0079: “the correction image may change the display mode according to the reliability. Specifically, the display mode control unit 65 changes the display mode of the correction image 80 so that the difference between the low reliability region having a low reliability relating to the calculation of the oxygen saturation and the high-reliability region having a high reliability is emphasized
The reliability represents the calculation accuracy of the oxygen saturation in each pixel, and the larger the reliability, the better the calculation accuracy of the oxygen saturation. The low reliability region is a region in which the reliability is less than the reliability threshold. The high reliability region is a region in which the reliability is equal to or greater than the reliability threshold. By emphasizing the difference between the low reliability region and the high reliability region in the correction image, it is possible to enter the high reliability region inside the specific region while avoiding the low reliability region.”, Paragraph, 0086: “the first correction determination unit 67 determines whether or not the correction processing can be appropriately performed on the basis of the reliability in the specific region. In the first correction determination unit 67, when the number of effective pixels for which the reliability is equal to or greater than the reliability threshold value is equal to or greater than a certain value for the pixels in the specific region, it is determined that the correction processing can be appropriately performed”, “that the first correction determination unit 67 performs the determination each time the image is acquired and the reliability is calculated until the correction operation is performed. The cycle in which the determination is performed may be changed as appropriate”)
However, Shigeta does not teach correct the pixel reliability degree according to a position of each pixel in the image, and using the corrected pixel reliability degree.
Takeshima teaches correct the pixel reliability degree according to a position of each pixel in the image (Paragraph, 0073:“ Since the g factor is determined in accordance with the positional relationship between each pixel and a coil, it has different values depending on the pixel positions. Thus, the g factor can be understood as a reliability for the pixel value of each pixel; therefore, it may be used as a correct reconstruction reliability. By performing machine learning using the g factor as a correct reconstruction reliability, a reconstruction reliability similar to the g factor may be output by using the reconstruction”, Paragraph, 0068: “The series of provisional reconstruction images IAk are obtained by applying provisional reconstruction DNNs having different elements to the same raw data. Thus, in the DNN reconstruction according to the present embodiment, the DNN reconstruction image is obtained from the raw data by way of the series of provisional reconstruction images IAk. In the DNN reconstruction image generated through the process described above, the variation of the pixel values of the series of provisional reconstruction images IAk is relatively small for the pixel P1 in which the DNN reconstruction is relatively stable. Accordingly, the reliability of the pixel P1 is considered to be high. The variation of the pixel values of the series of provisional reconstruction images IAk is relatively large for the pixel P2 in which the DNN reconstruction is relatively unstable. Accordingly, the reliability of the pixel P2 is considered to be low. Since the reconstruction reliability, as described above, represents the variation of the pixel values of the same image area of the series of provisional reconstruction images IAk, the reliability of the DNN reconstruction image can be accurately evaluated”, and Paragraph, 0075: “the reconstruction reliability is obtained by using the DNN. Therefore, a bias of the reconstruction reliability can be corrected’),
Takeshima teaches the region using the corrected pixel reliability degree (Paragraph, 0073:“ Since the g factor is determined in accordance with the positional relationship between each pixel and a coil, it has different values depending on the pixel positions. Thus, the g factor can be understood as a reliability for the pixel value of each pixel; therefore, it may be used as a correct reconstruction reliability. By performing machine learning using the g factor as a correct reconstruction reliability, a reconstruction reliability similar to the g factor may be output by using the reconstruction”, Paragraph, 0068: “The series of provisional reconstruction images IAk are obtained by applying provisional reconstruction DNNs having different elements to the same raw data. Thus, in the DNN reconstruction according to the present embodiment, the DNN reconstruction image is obtained from the raw data by way of the series of provisional reconstruction images IAk. In the DNN reconstruction image generated through the process described above, the variation of the pixel values of the series of provisional reconstruction images IAk is relatively small for the pixel P1 in which the DNN reconstruction is relatively stable. Accordingly, the reliability of the pixel P1 is considered to be high. The variation of the pixel values of the series of provisional reconstruction images IAk is relatively large for the pixel P2 in which the DNN reconstruction is relatively unstable. Accordingly, the reliability of the pixel P2 is considered to be low. Since the reconstruction reliability, as described above, represents the variation of the pixel values of the same image area of the series of provisional reconstruction images IAk, the reliability of the DNN reconstruction image can be accurately evaluated”, and Paragraph, 0075: “the reconstruction reliability is obtained by using the DNN. Therefore, a bias of the reconstruction reliability can be corrected”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Takeshima because Incorporating Takeshima’s position-dependent Pixel reliability into Shigeta’s ROI reliability calculation would have improved the accuracy of the region reliability determination by accounting for pixel-based reliability during the ROI reliability calculation, while using each reference according to its established function (Takeshima, Paragraph, 0073, 0075, 0076, Figures 2-5).
Regarding claim 15, Shigeta in view of Tian and Bakker teaches the endoscope system of claim 3 (See rejection of claim 3 above),
Shigeta teaches wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
However, Shigeta does not teach perform control of displaying a pixel having a pixel reliability degree equal to or higher than a pixel reliability degree threshold value set in advance on a display in an aspect of being distinguishable from other pixels.
Takeshima teaches perform control of displaying a pixel having a pixel reliability degree equal to or higher than a pixel reliability degree threshold value set in advance (Paragraph, 0078: “the processing circuitry 11 may extract a region in which the reconstruction reliability is higher or lower than an arbitrary threshold value, and may cause the reconstruction reliability map I12 to be superimposed on the DNN reconstruction image I11 only in the extracted region”, Paragraph, 0073: “Since the g factor is determined in accordance with the positional relationship between each pixel and a coil, it has different values depending on the pixel positions. Thus, the g factor can be understood as a reliability for the pixel value of each pixel; therefore, it may be used as a correct reconstruction reliability. By performing machine learning using the g factor as a correct reconstruction reliability, a reconstruction reliability similar to the g factor may be output by using the reconstruction”, Paragraph, 0076: “the processing circuitry 11 causes the DNN reconstruction image and the reconstruction reliability, which have been output in step SA3, to be displayed (step SA4). The processing circuitry 11 may display the reconstruction reliability in a tabular format, in which a pixel and a numerical value of the reconstruction reliability are associated, or in a format of a spatial distribution of reconstruction reliabilities (hereinafter referred to as the reconstruction reliability map)”, Paragraph 00079: “the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is lower than the threshold value”, and see Figures 2-6)
on a display in an aspect of being distinguishable from other pixels (Paragraph, 0079: “the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is higher than the threshold value, and emphasize the specified region as a reliable region with a visual effect, such as a color, a flag, etc”, and Paragraph, 0080: “ if a pixel on the DNN reconstruction image I11 or the reconstruction reliability map I12 is designated by the input interface 14 or the like, the processing circuitry 11 may display a basis of the reconstruction reliability of the designated pixel. For example, as the basis, the processing circuitry 11 may display a numerical value of the reconstruction reliability, or a graph showing the variation of the pixel value of the designated pixel over the series of provisional reconstruction images as shown in the lower part of FIG. 6.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Takeshima’s display processing because displaying the reliability distribution and specifying image regions having reconstruction reliability above or below a threshold value and emphasizing the specified regions with visual effects would have enabled reliability information to be displayed in a visually distinguishable manner, thereby allowing a user to readily recognize reliable and unreliable image regions (Takeshima, Paragraph, 0076, 0079).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of Bakker, and “Wang” (CN 106910207 B),
Regarding claim 6, Shigeta in view of Bakker teaches the endoscope system of claim 2 (See rejection of claim 2 above),
Shigeta teaches wherein in the region reliability degree calculation processing, the region reliability degree is calculated (Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”),
However, Shigeta is silent on after performing weighting according to an area of the region of interest.
Wang teaches after performing weighting according to an area of the region of interest (Page 5, lines 33-40: “in step S540, respectively performing second matching calculation on the at least one region of interest and the at least one primary matching region. FIG. 12 shows a result diagram of performing a second matching calculation with the primary matching area according to a first care area, where a credibility thereof is 0.984, and is denoted as C1. FIG. 13 shows a result diagram of performing second matching calculation with the primary matching area according to a second care area, and a credibility thereof is 0.984, which is denoted as C2; and FIG. 14 shows a result diagram of performing second matching calculation according to a third care area and the primary matching area as 0.981, which is denoted as C3”, and Page 5, lines 41-42: “The area of the first region of interest is S 1, the confidence level of the first region of interest is C 1, the second region of interest is S 2, the calculated credibility is C 2, the third region of interest is S 3, the obtained credibility is C 3, and the final credibility C is obtained as follows”, see the weighting equation of the credibility:
C
=
S
1
*
C
1
+
S
2
*
C
2
+
S
3
+
C
3
S
1
+
S
2
+
S
3
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta’s region reliability degree calculation with Wang’s area-based credibility (region reliability degree) calculation because weighting the credibility (reliability degree) of the plurality of regions of interest according to the areas of the respective ROIs would have provided an aggregated value that reflects the relative contribution of each RIO based on its area, thereby providing an aggregated credibility value that reflects the relative contribution of each regions of interest based on its area and providing a more reliable credibility determination (Page 5, lines 33-40; Page 5, lines 41-41; the formula defining
C
=
S
1
*
C
1
+
S
2
*
C
2
+
S
3
+
C
3
S
1
+
S
2
+
S
3
).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of Bakker, Wang , and “Ishikawa” (JP 2015112418 A),
Regarding claim 7, Shigeta in view of Bakker teaches the endoscope system of claim 2 (See rejection of claim 2 above),
Shigeta teaches wherein in the region reliability degree calculation processing, the region reliability degree is calculated (Paragraph, 0129: “In the calculation of the reliability, the reliability is calculated for each of the 16 ROIs”)
However, Shigeta is silent on after performing weighting according to a distance of the region of interest from a center of the image.
Wang teaches the region reliability degree is calculated after performing weighting (Page 5, lines 41-42: “The area of the first region of interest is S 1, the confidence level of the first region of interest is C 1, the second region of interest is S 2, the calculated credibility is C 2, the third region of interest is S 3, the obtained credibility is C 3, and the final credibility C is obtained as follows”, see the weighting equation of the credibility: C=(S1*C1+S2*C2+S3+C3)/(S1+S2+S3)”)
Ishikawa teaches according to a distance of the region of interest from a center of the image (Paragraph, 0061: “In step S7, the data calculation unit 14e (see FIG. 4) determines the distance d between the position coordinates of the brain region of interest ROI and the estimated measurement point position E or the estimated measurement point position E to the brain region of interest ROI. Based on the belonging probability, the weighting factor P (n) of the extracted measurement channel 1c is determined”, Paragraph, 0062: “the data calculation unit 14e performs weighted averaging (weighted averaging) processing of the calculated weighting coefficient P (n) and the measurement value X (n) of the extracted measurement channel 1c (the above equation 2) Calculate the region measurement data 42 of the brain region of interest ROI by reference)”, and Paragraph, 0067: “the data calculation unit 14e is configured to calculate the region measurement data 42 of the brain region of interest ROI. Thereby, the reliability of the region measurement data 42 of the calculated brain region of interest ROI can be improved”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta’s region reliability degree calculation with Wang’s credibility (region reliability degree) calculation because weighting the credibility (reliability degree) of the plurality of regions of interest would have provided an aggregated value that reflects the relative contribution of each RIO based on its area, thereby providing an aggregated credibility value that reflects the relative contribution of each regions of interest based on its area and providing a more reliable credibility determination (Page 5, lines 33-40; Page 5, lines 41-41; the formula defining
C
=
S
1
*
C
1
+
S
2
*
C
2
+
S
3
+
C
3
S
1
+
S
2
+
S
3
).
Furthermore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta in view of Wang further with Ishikawa’s distance-based weighting function because incorporating Ishikawa’s distance-based weighting technique into Wang’s weighted credibility calculation would have caused the credibility values to be weighted according to distances. Such modification would have predictably reduced the contribution of measurement information associated with position farther from the relevant ROI reference position, thereby improving the reliability of the calculated regional result (Paragraph, 0010, 0013, 0068)
Claim 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of “Fruchter” (WO 2022216558 A1)
Regarding claim 10, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above),
Shigeta teaches wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
set the regions of interest (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Figure 59 illustrates the pixel determination within 16 region of interest ROI).
However, Shigeta silent on the regions of interest that have shapes different from each other and/or areas different from each other based on positions of the regions of interest in the image.
Fruchter teaches the regions of interest that have shapes different from each other and/or areas different from each other (Paragraph, 0061: “determine, generate, calculate, or define two or more different ROIs for different areas of a field of view of the camera 114”, Paragraph, 0071: “the ROIs can have any shape (e.g., rectangular, circular, irregular, etc.)”, and Paragraph, 0116: “ step 1206 includes determining a size or dimensions of one or more regions of new image data”)
based on positions of the regions of interest in the image (Paragraph, 0061: “determine, generate, calculate, or define two or more different ROIs for different areas of a field of view of the camera 114”, Paragraph, 0082: “the new regions 602 may include only a region along the right edge of the new field of view 504”, and Paragraph, 0116: “the one or more regions of the new image data can be along a horizontal and/or vertical edge of a field of view of the camera”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Fruchter’s ROI selection because selecting ROI according to the lesion location and adjusting the dimensions of the selected ROI would have enabled an appropriately positioned and sized ROI analysis (Fruchter, Paragraph, 0059,0061,0082, and 0116).
Regarding claim 11, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above),
Shigeta teaches wherein the processor is configured to (Paragraph, 0033: “The processor device 14 includes a DSP (Digital Signal Processor) 45, an image processing unit 50, a display control unit 52, and a central control unit 53. A program related to various processes is incorporated in the program memory (not shown) in the processor device 14. By the central control unit 53 configured by the processor executing a program in the program memory, functions of the DSP 45, the image processing unit 50, the display control unit 52, and the central control unit 53 are realized”):
: set the regions of interest (Paragraph, 0124: “As shown in FIG. 59, the effective pixel determination is performed based on the pixel values in the 16 regions of interest ROI provided in the image center portion”, and Figure 59 illustrates the pixel determination within 16 region of interest ROI”)
However, Shigeta is silent on such that at least two of the plurality of regions of interest include the same pixel of the image.
Fruchter teaches such that at least two of the plurality of regions of interest include the same pixel of the image (Paragraph, 0063: “ the image 204 can be divided, segmented, etc., into various ROIs 202”, “The ROIs 202 can overlap each other, thereby sharing similar image or pixel information” and Paragraph, 0118: “the one or more regions at least partially overlap, or are nested within each other”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Fruchter’s ROI overlapping because generating a plurality of overlapping candidate regions of interest would have enabled evaluation of multiple candidate regions , thereby facilitating selection of an appropriate region of interest for subsequent image analysis (Fruchter, Paragraph, 0063, 0118).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of Takeshima,
Regarding claim 14, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 1 above),
Shigeta teaches wherein the processor is configured to: perform control of displaying the region of interest (Paragraph, 0020: “The processor device 14 performs system control of the endoscope system 10, and further generates an endoscopic image by performing image processing or the like on the image signal transmitted from the endoscope 12”, and Paragraph, 0022: “As shown in FIG. 3, in the oxygen saturation mode, the oxygen saturation of the observation target is calculated, and the oxygen saturation image obtained by imaging the calculated oxygen saturation is displayed on the extension display 18”) for,
The correction of the data (Paragraph, 0129: “In a case where the reliability is calculated, when there is an ROI having a low reliability, or when the reliability average value of each ROI is less than a predetermined value, an error determination regarding the reliability is performed. The result of the error determination on the reliability is notified to the user by display on the extension display 18 or the like”)
However, Shigeta is silent on which it is determined that the correction of the data is possible on a display.
Takeshima teaches which it is determined that the correction of the data is possible on a display (Paragraph, 0079: “Furthermore, the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is lower than the threshold value, and emphasize the specified region as an unreliable region with a visual effect, such as a color, a flag, etc. Contrarily, the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is higher than the threshold value, and emphasize the specified region as a reliable region with a visual effect, such as a color, a flag, etc.”, Paragraph, 0076:” the processing circuitry 11 causes the DNN reconstruction image and the reconstruction reliability, which have been output in step SA3, to be displayed (step SA4). The processing circuitry 11 may display the reconstruction reliability in a tabular format, in which a pixel and a numerical value of the reconstruction reliability are associated, or in a format of a spatial distribution of reconstruction reliabilities (hereinafter referred to as the reconstruction reliability map)”, Paragraph, 0079:” the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is lower than the threshold value, and emphasize the specified region”, “the processing circuitry 11 may specify a region of the DNN reconstruction image I11 in which the reconstruction reliability is higher than the threshold value”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Takeshima’s display processing because displaying the reliability distribution and low-reliability regions in a distinguishable manner would have enabled a user to readily identify the reliability of image regions and visually recognize regions requiring attention (Takeshima, Paragraph, 0076, 0079).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Shigeta in view of “Baker” (JP 4748854 B2).
Regarding claim 17, Shigeta teaches the endoscope system of claim 1 (See rejection of claim 17 above),
Shigeta does not teach wherein the processor is configured to: calculate, in a case where there is no region of interest for which it is determined that the correction of the data is possible, the oxygen saturation of the observation target using the uncorrected data.
Baker teaches wherein the processor is configured to: calculate, in a case where there is no region of interest for which it is determined that the correction of the data is possible, the oxygen saturation of the observation target using the uncorrected data (Paragraph, 0004: “Older generations of oximeters attempt to detect a sensor off ( SENSOR OFF ) condition, but the reliability of this detection scheme is not well proven. Some oximetry systems address the problem by providing for a time out period during which no pulse is detected or a pulse is detected that is not acceptable for oximetry calculations. If a timeout occurs, it is assumed that the sensor is not in contact with the patient, and the oximetry system ceases to display saturation and pulse. However, more recent oximetry algorithms are more sophisticated and use pulse (or pulse as an alternative to the notification of a time-out, previously unusable date, for example date corrupted by motion artefacts ( motion artifact ), can be used or
optionally ignored during the determination of the blood component concentration. As a result, the pulse timeout algorithm for determining whether the sensor is in contact with the patient is no longer adequate.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shigeta with Baker because using uncorrected observation-garget data to be inappropriate during the determination of the blood component concentration would have enabled the system to continue using uncorrected observation-target data when correction is determined to be inappropriate (Baker, Paragraph, 0004).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MORGAN S SHIM whose telephone number is (571)272-9032. The examiner can normally be reached Mon-Fri 7:30AM-4:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert(Tse) Chen can be reached at (571) 272-3672. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MORGAN SANGJO SHIM/Examiner, Art Unit 3791
/PATRICK FERNANDES/Primary Examiner, Art Unit 3791
Prosecution Insights