Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of JP2022/037418, on 10/06/2022.
Preliminary Amendment
Applicant submitted a preliminary amendment on 11/15/2023. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 11/15/2023 and 9/25/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 7, and 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Kamon (U.S. Patent Pub. No. 2021/0150277) in view of Fram (U.S. Patent No. 8867807).
Regarding Claim 1, Kamon teaches an image processing device comprising:
at least one memory configured to store instructions; and at least one processor configured to execute the instructions to (¶74 A read only memory (ROM) 211 is a nonvolatile storage element (a non-transitory recording medium) and stores a computer-readable code of a program that causes the CPU 210 and/or the image processing unit 204 (an image processing apparatus, a computer) to execute various image processing methods:)
acquire an endoscopic image obtained by photographing an examination target by a camera provided in an endoscope (¶42The first image database 601 and the second image database 602 are each constituted by a recording medium such as a hard disk, and endoscopic images acquired by the endoscope systems 10 are recorded therein; ¶61 As illustrated in FIG. 4 and FIG. 5, the imaging lens 132 (an imaging unit) is disposed on a distal-end-side surface 116A of the tip rigid part 116.)
detect a lesion (¶57 The output layer 562C may execute discrimination of a lesion and output a discrimination result) based on a selection model which is selected from a first model and a second model (¶48 The image processing system 1 (the image processing apparatus 500) according to the first embodiment includes the first CNN 562 and the second CNN 563 each of which is a convolutional neural network (a hierarchical network),)
change a parameter to be used for detection of the lesion based on a non-selection model that is the first model or the second model other than the selection model (¶46 the control unit 530 back-propagates an error (loss) calculated by an error calculating unit 568, which will be described below, to a first convolutional neural network (CNN) 562 and/or a second CNN 563, thereby updating the weight parameters of these CNNs.)
Kamon does not explicitly disclose the first model being configured to make an inference based on a predetermined number of endoscopic images, the second model being configured to make an inference based on a variable number of endoscopic images.
Fram is in the same field of art of image analysis. Further, Fram teaches the first model being configured to make an inference based on a predetermined number of endoscopic images, the second model being configured to make an inference based on a variable number of endoscopic images (Col 20 Lines 60-68: Depending on the embodiment, a processing model may be selected based on various criteria. For example, a particular type of image series (e.g., an image series of a particular modality and/or having one or more particular characteristics) may be associated with a mapping that indicates which processing model to use for different portions of the image series. For example, for an image series of a first type, the mapping may indicate that a first processing model is used for a first 5% of the images of the image series, while a second processing model is used for the remaining 95% of the images of the image series.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kamon by determining which model to use based on the number of images that is taught by Fram; thus, one of ordinary skilled in the art would be motivated to combine the references for efficient processing of medical images (Fram Background).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 3, Kamon in view of Fram discloses the image processing device according to claim 1, wherein the first model is a deep learning model whose architecture includes a convolutional neural network (Kamon, ¶17 in any one of the first to seventh aspects, the hierarchical network is a convolutional neural network.)
Regarding Claim 7, Kamon in view of Fram discloses the image processing device according to claim 1, wherein the at least one processor is configured to execute the instructions to determine the selection model from the first model and the second model, based on a degree of variation between the endoscopic images (Fram, Col 21 Lines 4-16: In one embodiment, the mapping is associated with anatomical features (e.g., organs) that should be present in images of a particular image series. Thus, such a mapping may be used to associate a particular processing model with a particular portion of an image series having an associated anatomical feature. In the example above, the system may determine that in an abdomen MRI image series, images of the kidneys are in a particular portion of a image series. Thus, when processing of images within that portion of the image series is requested, the system may use the particular processing model associated with the kidney portion of image series.)
Regarding Claim 9, Kamon in view of Fram discloses the image processing device according to claim 1, wherein the at least one processor is configured to further execute the instructions to display or output, by audio, information regarding a detection result of the lesion by the lesion detection means (Kamon, ¶47 A display unit 540 includes a monitor 542 (a display apparatus) and displays an endoscopic image, a learning result, a recognition result, a processing condition setting screen, and so forth.)
Regarding Claim 10, Kamon in view of Fram discloses the image processing device according to claim 9, wherein the at least one processor is configured to execute the instructions to output the information regarding the detection result of the lesion and information regarding the selection model to assist in decision making by an examiner (Kamon, ¶47 A display unit 540 includes a monitor 542 (a display apparatus) and displays an endoscopic image, a learning result, a recognition result, a processing condition setting screen, and so forth.)
Regarding claim 11, claim 11 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kamon further teaching on: An image processing method executed by a computer Kamon (¶2 The present invention relates to an image processing method and an image processing apparatus, and specifically relates to an image processing method and an image processing apparatus for performing machine learning using a hierarchical network)
Regarding claim 12, claim 12 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kamon further teaching on: A non-transitory computer readable storage medium storing a program executed by a computer (Kamon ¶74 A read only memory (ROM) 211 is a nonvolatile storage element (a non-transitory recording medium) and stores a computer-readable code of a program that causes the CPU 210 and/or the image processing unit 204 (an image processing apparatus, a computer) to execute various image processing methods.)
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Kamon (U.S. Patent Pub. No. 2021/0150277) in view of Fram (U.S. Patent No. 8867807) in view of Freese (U.S. Patent No. 9443002).
Regarding Claim 5, Kamon in view of Fram teaches the image processing device according to claim 1.
Kamon in view of Fram does not explicitly disclose wherein the second model is a model based on SPRT.
Freese is in the same field of art of image analysis. Further, Freese teaches wherein the second model is a model based on SPRT (Col 15 lines 4-7: Probability evaluator 425 can analyze the physician responses provided through data selector 418 using a hypothesis test, such as a sequential probability ratio test (“SPRT”) based on the SPRT developed by Abraham Wald; Col 15 Lines 29-31: Using the SPRT approach allows for a certain level of confidence to be reached while only requiring the number of responses necessary to reach that conclusion.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kamon in view of Fram by using a model based on SPRT that is taught by Freese; thus, one of ordinary skilled in the art would be motivated to combine the references in order to determine an outcome with a certain level of confidence only using the required amount of data (Freese Col 15).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allowable Subject Matter
Claims 2, 4, 6, and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 2, no prior art teaches wherein the parameter is a parameter defining a condition for determining that the lesion is detected, and wherein the at least one processor is configured to execute the instructions to change the parameter so that, the higher a degree of confidence of presence of the lesion indicated by a score calculated by the non-selection model is, the more the condition is relaxed.
Regarding claim 4, no prior art teaches wherein the at least one processor is configured to execute the instructions to determine that the lesion is detected if a consecutive number of times a degree of confidence of presence of the lesion exceeds a predetermined threshold value is larger than a predetermined number of times,
the degree of confidence being indicated by a score calculated by the first model from the endoscopic images acquired in time series,
wherein the parameter is at least one of the predetermined number of times and/or the predetermined threshold value, and
wherein the at least one processor is configured to execute the instructions to change at least one of the predetermined number of times and/or the predetermined threshold value, based on a score calculated by the second model.
Regarding claim 6, no prior art teaches wherein the selection model is the second model,
wherein the at least one processor is configured to execute the instructions to determine that the lesion is detected if a degree of confidence of presence of the lesion exceeds a predetermined value,
the degree of confidence being indicated by a score calculated by the second model,
wherein the parameter is the predetermined threshold value, and
wherein the at least one processor is configured to execute the instructions change the predetermined threshold value based on a score calculated by the first model.
Regarding claim 8, no prior art teaches wherein the at least one processor is configured to execute the instructions to start calculating a score based on the non-selection model if it is determined that a predetermined condition based on a score calculated by the selection model is satisfied.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664