DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 1/28/25, 3/4/25, and 10/8/25 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: “Ps” in Figure 3. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 5 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5 recites the limitation "the monitor”. There is insufficient antecedent basis for this limitation in the claim. This is being interpreted as “a monitor”.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of determining colors based on pixel values, without significantly more.
The claim recites: “An endoscope apparatus comprising:
one or more processors being configured to:
normalize a sum of a red pixel value and a green pixel value by a blue pixel value for each of a plurality of pixels in at least a partial region of an image;
calculate an index of each of the pixels;
select a threshold value set according to a combination of endoscope information and processor information from a plurality of threshold value sets corresponding to a plurality of combinations of a plurality pieces of endoscope information and a plurality pieces of processor information;
identify a symptom level of each of the pixels based on the selected threshold value set;
acquire an identification color corresponding to the symptom level of each of the pixels; and
generate an identification image using the identification color of each of the pixels.”
The limitations, as drafted, are processes that, under their broadest reasonable interpretation, cover performance of the limitation in the mind and by mathematical calculations. The sum normalization and index calculation amount to mathematical calculations/formulas. The human mind is capable of performing the selection of a threshold value set based on endoscope information and processor information combinations, symptom identification, and color acquisition. The identification image generation amounts to insignificant extra-solution activity.
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of a processor. This is recited at a level of generality such that amounts to no more than a generic processor. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements are recited at a high-level of generality. It is therefore a judicial exception that is not integrated into a practical application, and does not include additional elements that are sufficient to amount to significantly more than the judicial exception. This claim is not patent eligible.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of generically describing the endoscope information and processor information, without significantly more. The claim is not patent eligible.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of describing the threshold value sets as correction sets and a standard set, without significantly more. The claim is not patent eligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of describing the mental process of updating a data set, without significantly more. The claim is not patent eligible.
Claims 5 and 11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a generically recited monitor for performing the insignificant extra-solution activity of image display, without significantly more. The claims are not patent eligible.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mathematical formula, without significantly more. The claim is not patent eligible.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a generically recited endoscope, without significantly more. The claim is not patent eligible.
Claims 7-9 and 12-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a method and non-transitory computer readable storage medium (generically recited additional element) analogous to the abstract idea recited in claims 1-6 and 10-11
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-14, and 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ariyoshi (US20200107699A1) in view of Kobayashi (JP 2003290130 A) further in view of Matsui (US20190133426A1).
Regarding claim 1, Ariyoshi teaches “An endoscope apparatus comprising: one or more processors” (Ariyoshi, Figure 1.)
“being configured to: normalize a sum of a red pixel value and a green pixel value by a blue pixel value for each of a plurality of pixels in at least a partial region of an image; calculate an index of each of the pixels;” (Ariyoshi, Paragraphs 66 and 78-79 “The predetermined index calculation expression P1 includes the index calculation expression (Vr+Vg)/Vb obtained by normalizing a sum of the red pixel value Vr and the green pixel value Vg by the blue pixel value Vb. The index calculation expression (Vr+Vg)/Vb is preferably used for the mucosa in the nose and paranasal sinuses. Although the index calculation expression (Vr+Vg)/Vb may be used by the illumination light L that is either the ordinary light or the special light, the index calculation expression (Vr+Vg)/Vb is preferably used by the special light.”; FIG. 7 shows an example of the endoscope image A. In the example in FIG. 7, an area A1 is white, an area A2 is pale red, an area A3 is dark red, and an area A4 is almost black. Although FIG. 7 shows the areas A1 to A4 in a simplified manner, complicated colors and patterns are seen in the actual endoscope image A. Based on the endoscope image A, the calculation section 33 b calculates the indexes by using the predetermined index calculation expression P1 read from the storage section 33 a. Here, for the predetermined index calculation expression P1, (Vr+Vg)/Vb is used. The calculation section 33 b decomposes the endoscope image A into RGB components, and calculates the indexes by performing an operation of (Vr+Vg)/Vb for each calculation-target pixel on the endoscope image A. The calculated indexes are outputted to the image generation section 33 c.” Note that the whole image amounts to “at least a partial region of an image”.)
While Ariyoshi teaches selecting a threshold value set from a plurality of threshold value sets (Ariyoshi, Figure 8 shows threshold value sets associated with degrees of abnormality indicated by index ranges; Paragraph 81 recites, “In the example in FIG. 8, the indexes are set in a range of 0 to 69, and the predetermined threshold values P2 are set at 10, 20, 30, 40, 50, and 60. For the identifiable display information P3, the indexes of 0 to 9 are set to a white color, the indexes of 10 to 19 are set to a red color, the indexes of 20 to 29 are set to an orange color, the indexes of 30 to 39 are set to a yellow color, the indexes of 40 to 49 are set to a green color, the indexes of 50 to 59 are set to a blue color, and the indexes of 60 to 69 are set to a black color.” Paragraph 43 further describes, “The predetermined threshold value P2 is set so as to separate the indexes calculated by using the predetermined index calculation expression P1 into grades and stored in the storage section 33 a. A plurality of the predetermined threshold values P2 may be set according to the type of the observation mode and the type of the observation site and stored in the storage section 33 a.”), Ariyoshi does not describe that the threshold value set and plurality of threshold value sets are according to different combinations of endoscope information and processor information.
Kobayashi (JP 2003290130 A) teaches a disease detection threshold value set according to an endoscope imager information from a plurality of threshold value sets corresponding to a plurality of endoscope information. (Kobayashi, Table 3 (see translation below), Page 8 Paragraph 5, “In step S205, the system control 301 recognizes the CCD type from the received CCD type signal and substitutes the CCD type data corresponding to the CCD type into the variable D.”; Page 10 last paragraph – first full Paragraph of Page 11, “A comparison is made with the threshold look-up table built in the memory 303. As described above, the affected area is a portion “bright in the reference image and dark in the fluorescent image”, but in order to determine whether a certain pixel is the affected area (luminance value of pixel of reference image / pixel of fluorescent image) Brightness value) is calculated, and this value is compared with a predetermined threshold value. In this embodiment, (Luminance value of pixel of reference image / luminance value of pixel of fluorescent image) Is more than the threshold value a, it is judged to be a particularly dangerous affected part, and if it is less than the threshold value a and not less than the threshold value b, it is judged to be a slightly dangerous affected part. Since the appropriate threshold values a and b differ depending on the characteristics of the CCD of the electronic endoscope, step S31 In 4, the thresholds a and b corresponding to the contents of the variable D are extracted from the threshold reference table. The threshold reference table is shown in Table 3 below.” Note that the CCD is part of the endoscope and therefore amounts to endoscope information.)
PNG
media_image1.png
145
463
media_image1.png
Greyscale
However, Kobayashi does not expressly disclose incorporating processor information.
Matsui (US20190133426A1) teaches setting endoscope image processing parameters based on processor information from a plurality of endoscope image processing parameters corresponding to a plurality of processor information. (Matsui, Paragraphs 45-47, “The processor identification unit 137 identifies a type of the processor 2 connected to the scope 1′. For example, in a case where the processor 2 is given an ID and the type of the processor 2 can be identified by the ID information, when the processor 2 is connected to the scope 1′, the processor identification unit 137 receives the ID information from the processor 2 through the communication circuits 25, 136.” … “The ROM 138 is a storage unit composed of a nonvolatile memory, and a gain parameter by which a video signal is multiplied is registered for each type of a processor 2 having a possibility of being connected to the scope 1′.” … “The gain adjustment unit 132 refers to the ROM 138 when information on the type of the processor 2 is inputted from the processor identification unit 137. Then, a gain parameter corresponding to the received type of the processor 2 is extracted from the ROM 138, and outputted to the gain circuit 131. The gain circuit 131 uses the gain parameter set in the gain adjustment unit 132 to adjust an output level of a video signal received from the video signal processing circuit 122.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application for the threshold value set and plurality of threshold value sets of Ariyoshi to be based in accordance with endoscope information of a plurality pieces of endoscope information, as taught by Kobayashi. It would have further been obvious to modify this combination by additionally basing the threshold value set and plurality of threshold value sets on processor information of a plurality of pieces of processor information, as taught by Matsui, in combination with the endoscope information of Kobayashi.
The motivation for doing so would have been to account for both factors (endoscope and processor) for more accurate image analysis. Different endoscopes will pick up pixel value signals differently, making such adaptive thresholding strategic for classifications based on pixel value calculations. Similarly, different processors will output different pixel values signals for a same imaging target, motivating adaptive thresholding based on processor type. A person skilled in the art would understand that different endoscopes may be used in combination with different processers, necessitating the thresholds to correspond to different combinations of endoscopes and processors. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ariyoshi with the above teachings of Kobayashi and Matsui to fully disclose “select a threshold value set according to a combination of endoscope information and processor information from a plurality of threshold value sets corresponding to a plurality of combinations of a plurality pieces of endoscope information and a plurality pieces of processor information;”.
Ariyoshi in view of Kobayashi further in view of Matsui further disclose, “identify a symptom level of each of the pixels based on the selected threshold value set; acquire an identification color corresponding to the symptom level of each of the pixels; and generate an identification image using the identification color of each of the pixels.” (Ariyoshi, Figures 8-9 show symptom level (normal vs. abnormal) for each pixel based on the threshold value sets (index ranges), wherein each symptom level has a color (red, orange, blue, etc.) corresponding to the symptom level of each pixel, and a generated identification image using the respective identification color.; Paragraph 47 further describes that the indices indicate degrees of abnormality: “In other words, the display control section 33 is configured to calculate indexes indicating degrees of abnormality of the object, according to colors included in the object image acquired by the image pickup portion 24 picking up an image of the object, and to generate the display image B in which the indexes are identifiably displayed based on the predetermined threshold value P2 set independently of the object image.”)
Regarding claim 2, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 1,”
“wherein the endoscope information is at least one of endoscope model data and endoscope individual data, and the processor information is at least one of processor model data or processor individual data. (Kobayashi, as described in the rejection of claim 1, recites “In step S205, the system control 301 recognizes the CCD type from the received CCD type signal and substitutes the CCD type data corresponding to the CCD type into the variable D.”; Matsui, as described in the rejection of claim 1, recites a processor ID. Note that these teachings were incorporated with motivation and rationale in the rejection of claim 1.)
Regarding claim 3, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 2,”
“wherein the plurality of threshold value sets include a plurality of correction data sets according to the plurality of combinations of the plurality pieces of endoscope information and the plurality pieces of processor information, and a standard threshold value set.” (One of the plurality of threshold value sets is mapped to the standard threshold value set, and the remaining threshold value sets are mapped to the plurality of correction data sets according to the plurality of combinations of the plurality pieces of endoscope information. The claim does not provide meaningful limitations on the corrected data set and standard threshold value set.)
Regarding claim 4, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 2,”
“wherein the one or more processors is configured to correct the threshold value set according to a change of a parameter of an image processing.” (As the references are combined in the rejection of claim 1, the threshold value set used is modified (corrected) in accordance with each different endoscope information and processor information. Each of the endoscope information and processor information are features of the image processing pipeline and are therefore ‘parameters of an image processing’.)
Regarding claim 5, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 1,”
“further comprising the monitor configured to display the identification image.” (Ariyoshi, Figure 1 and Paragraph 48, “The display unit 41 is configured by using, for example, a monitor capable of displaying a color image, and configured to display the display image B inputted from the image generation section 33 c.”)
Regarding claim 10, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 1,”
“further comprising: an endoscope configured to pick up the image and output an image pickup signal.” (Ariyoshi, Figure 1.)
Regarding claim 11, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 1,”
“wherein the one or more processors is configured to control a monitor to display the identification image.” (Ariyoshi, Figure 1 and Paragraph 48, “The display unit 41 is configured by using, for example, a monitor capable of displaying a color image, and configured to display the display image B inputted from the image generation section 33 c.”)
Regarding claims 7, 8, 12-14, and 16, these claims recite a method with steps corresponding to the elements of the system recited in claims 1-4, 6, and 10-11. Therefore, the recited steps of these claims are mapped to the analogous elements in the corresponding system claims. Additionally, the rationale and motivation to combine the references apply here.
Regarding claims 9 and 17-19 these claims recite a non-transitory computer readable storage medium storing a program with instructions corresponding to the elements of the system recited in claims 1-4 and 6. Therefore, the recited programming instructions of these claims are mapped to the analogous elements in the corresponding system claims. Additionally, the rationale and motivation to combine the references apply here. Finally, the references disclose a non-transitory computer readable storage medium storing a program and a computer (Ariyoshi, Paragraph 104, “Each “section/portion/unit” in the present description does not necessarily have a one-to-one correspondence with a specific hardware piece or software routine. Order in which individual procedures in the embodiment are performed may be changed, or a plurality of procedures may be performed at the same time, or the individual procedures in the embodiment may be performed in different order each time the procedures are performed, unless inconsistency occurs in the nature of the procedures. Moreover, all or part of the individual procedures in the embodiment may be implemented by using software executed by a computer.” Note that a person skilled in the art would understand that such software instructions would be stored in a non-transitory computer readable storage medium for repeated use.)
Claim(s) 6, 15, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ariyoshi in view of Kobayashi further in view of Matsui further in view of design choice.
Regarding claim 6, Ariyoshi in view of Kobayashi further in view of Matsui teach “The endoscope apparatus according to claim 1,”
While Ariyoshi in view of Kobayashi further in view of Matsui disclose index calculation by normalizing a sum of a red pixel value and a green pixel value by a blue pixel value (see rejection of claim 1), they do not expressly describe using twice the blue pixel value in this calculation.
Using twice the blue pixel value in this calculation would have been a matter of design choice based in trivial, routine experimentation.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use twice the blue pixel value, based on a design choice based in trivial routine experimentation, in the index calculation of Ariyoshi in view of Kobayashi further in view of Matsui.
The motivation for doing so would have been to scale and compress the data. Further, one skilled in the art could have substituted one known element for another (i.e., 2x the blue pixel value for 1x the blue pixel value), and the substitution would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ariyoshi in view of Kobayashi further in view of Matsui with the additional teaching obtained through design choice to fully disclose, wherein the one or more processors being configured to calculate the index by normalizing a sum of a red pixel value and a green pixel value by a value which is twice of a blue pixel value.”
Regarding claims 15, this claim recites a method with steps corresponding to the elements of the system recited in claim 6. Therefore, the recited steps of this claim are mapped to the analogous elements in the corresponding system claim. Additionally, the rationale and motivation to combine the references apply here.
Regarding claim 20 this claim recites a non-transitory computer readable storage medium storing a program with instructions corresponding to the elements of the system recited in claim 6. Therefore, the recited programming instructions of this claim are mapped to the analogous elements in the corresponding system claim. Additionally, the rationale and motivation to combine the references apply here.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Peirce (US20070150745A1) discloses user authentication based on biometric data, including adaptive thresholding based on changes of various factors including devices and algorithms.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON JOSEPH SORRIN whose telephone number is (703)756-1565. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON JOSEPH SORRIN/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights