Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 6, 13-15, 17 and 19 is/are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Hirokazu WO2019012586.
With respect to claim 1, Hirokazu WO2019012586 teaches an image processing apparatus which is a processing apparatus for processing medical images, comprising one or more processors (CPU described at the top of page 4. Note also the 9th para. which states that the user inserts the insertion portion into the subject and performs an endoscopic examination while viewing various endoscopic images. The processor is configured to capture a plurality of images via an endoscope, see the 6th full paragraph at page 4 where it states in part: “… the image data of the endoscope image generated by the image processing unit 12 of the video processor 3….” Hirokazu teaches recognizing a region of interest which is a region suspected of having a lesion (see the last full para. of page 4, see also page 5, para. 10).
Hirokazu teaches a “predetermined mark” which is superimposed information over the medical image as claimed. Hirokazu also teaches displaying the superimposed information with the medical image by means of display apparatus 5.
Hirokazu teaches starting the control of the superimposed information by an operation information or setting information of the video processor to an arrow pointer display state, described at the 7th para. of page 4.
As to the claimed limitation addressing stoppage of control when a predetermined time has elapsed. Hirokazu teaches a reporting control unit (display determination unit 23 that performs a control to bring reporting information, supplied from the video processor, recorded in the image, either as a reporting state in which the information is reported or a non-reporting state in which the information is not reported, wherein the reporting control unit 23 brings the reporting information in the non-reporting state when non-reporting condition is satisfied. This non-reporting condition is the arrow pointer or a cursor described at page 4, 9th full para. In addition to the non-reporting of the arrow or cursor, the round mark 35 may also not be displayed or reported on the display. The video processor determines the health professional’s use of the endoscope in normal light observation mode or special light observation mode (beginning at para. 5 of page 6). It appears that when the special light observation mode is selected, the cursor is not displayed. The Hirokazu reference teaches brining the reporting information in to the reporting state after a non-reporting state has elapsed. The elapsed time corresponds with the time designated by the user of the endoscope when not operating in the special light observation. The reporting state corresponds with the display of the cursor while the health professional does not require the special operation mode or while in the normal light observation mode.
With respect to claim 2, Hirokazu teaches the medical imaging processing apparatus, according to claim 1, wherein the information mark 33 indicating the position of the region of interest (region where the legion is present) is frame shaped (substantially rectangular shaped) and around the region of interest of the lesion, see figure 3.
With respect to claim 3, Hirokazu teaches wherein the CPU, described at the top of page 4, is configured to display in an area, outside of the medical image, having the lesion, information displayed indicative of the control superimpose information, by means of the “YES” command illustrated as icon 34 illustrated in figure 3. See also the bottom of page 6 which states wherein an image analysis unit 21 detects a region suspected of being a lesion in the image, as shown in Fig. 3, a square frame image (indicated by a dotted line) surrounding the region LS suspected of being a lesion is shown as a predetermined mark 33…”
With respect to claim 5, Hirokazu teaches wherein at least one processor, as described at the top of page 4, starts the control of the superimpose information by means of image generation unit 22 which superimposes a predetermined mark if a region of an image is detected of having a lesion.
With respect to claim 6, Hirokazu teaches wherein processor, described at the top of page 4, is configured to display information to display apparatus 5 indicative of the control of the superimposed information, illustrated by figure 3, wherein mark 33 is the superimposed information.
With respect to claim 13, Hirokazu teaches wherein a processor, described at the top of page 4, is configured to recognize a region of interest 32 after stopping control to superimpose the information, for example, when a specialized lighting condition is selected, see page 6 beginning at the 5th full para. Medical images are displayed by display unit 5 comprising a display screen 31.
With respect to claim 14, Hirokazu teaches the medical imaging processing apparatus comprising an endoscope for producing the medical images. The endoscope system is described as system 1 with endoscope 2. See also page 4, full paragraphs 9 and 10.
With respect to claim 15, Hirokazu teaches a medical imaging process executed by at least one processor, described at the top of page 4, comprising:
acquiring a plurality of images captured by an endoscope 2 of endoscope system 1, an image analysis unit 21 for recognizing regions of interest of medical images wherein the regions of interest include lesions that are displayed on a monitor (display screen 31).
Hirokazu teaches the claimed limitation addressing stoppage of control when a predetermined time has elapsed. Hirokazu teaches a reporting control unit (display determination unit 23 that performs a control to bring reporting information, supplied from the video processor, recorded in the image, either as a reporting state in which the information is reported or a non-reporting state in which the information is not reported, wherein the reporting control unit 23 brings the reporting information in the non-reporting state when non-reporting condition is satisfied. This non-reporting condition is the arrow pointer or a cursor described at page 4, 9th full para. In addition to the non-reporting of the arrow or cursor, the round mark 35 may also not be displayed or reported on the display. The video processor determines the health professional’s use of the endoscope in normal light observation mode or special light observation mode (beginning at para. 5 of page 6). It appears that when the special light observation mode is selected, the cursor is not displayed. The Hirokazu reference teaches brining the reporting information in to the reporting state after a non-reporting state has elapsed. The elapsed time corresponds with the time designated by the user of the endoscope when not operating in the special light observation. The reporting state corresponds with the display of the cursor while the health professional does not require the special operation mode or while in the normal light observation mode.
With respect to claim 17, Hirokazu teaches the medical imaging processing method as claimed, wherein the information mark 33 indicating the position of the region of interest (region where the legion is present) is frame shaped (substantially rectangular shaped) and around the region of interest of the lesion, see figure 3.
With respect to claim 19, Hirokazu teaches an image processing method wherein a processor, is configured to recognize a region of interest 32 after stopping control to superimpose the information, for example, when a specialized lighting condition is selected, see page 6 beginning at the 5th full para. Medical images are displayed by display unit 5 comprising a display screen 31.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 7, 8 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hirokazu in view of KSR v. Teleflex 550 U. S. 398 (2007).
With respect to claim 4, Hirokazu teaches all of the subject matter upon which the claim depends except for allowing a user to stop a superimposed image ahead of an earlier designation of the stoppage of the superimposed image.
Hirokazu clearly teaches that a user has direct input to stop superimposed images at will. For example, a user may have desired to look at one region of the body to determine if a lesion was present in the normal mode. While in this mode, the superimposed image would stay engaged until a region no longer determines a region where a lesion is present. However, in the specialized lighting mode, a user may activate this mode and will not see the special mark, simply by changing lighting modes.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try different methods of scanning the body for lesions, which include the use of different lighting conditions which affect the present of the superimposed data as taught by Hirokazu.
It would have been obvious to try one method under normal lighting and subsequently use different light which has the effect of stopping the superimposed data before a time in which the medical person was working in normal lighting and switched to specialized lighting.
With respect to claim 7, Hirokazu teaches the apparatus as claimed, wherein the region of interest is a “lesion”, see the first 6 lines of the Description, at page 2. Lesions may be cancerous or may be caused by inflammation. Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to replace the lesion, taught by Hirokazu for a specific type of lesion such as cancer or inflammation.
With respect to claim 8, Hirokazu teaches a mark display determination unit that determines whether or not do display a predetermined mark indicating a presence on a medical image, see the top of page 3. The mark display determination unit interfaces with image analysis unit that detects a medical image to determine the size of an area of a lesion (see the second full para. of page 3). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to program the mark display determination unit so as not to superimpose a mark upon determination that the size of the lesion is larger than a threshold size. The second para. of page 3 clearly sets forth a condition as to whether the information mark is to be superimposed or not.
With respect to claim 9, the claim recites a center region of a medical image and determining if a detected region is greater than a certain distance from the center of the image and if so, then not superimposing information.
Hirokazu teaches a medical image processing method that determines whether a notification is to be sent to a display device regarding whether a predetermined mark is to be displayed or not.
The Hirokazu reference teaches an arrow pointer, described by at least paras. 3, 5, 8 and 9 of page 4 for designating any portion of the medical image, such as a center portion thereof. The Hirokazu reference further teaches the mark display determination unit that determines whether or not the specified mark will be superimposed on the image or not, see the third para. at page 3). The HIrokazu reference further teaches the image analysis unit 21 for determining the presence of a lesion. Hence, the dimensions of the lesion are determined by the image analysis unit 21.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have been motivated to try different calibrations of the display condition determining unit 23 so as not to superimpose a mark upon determining the position of the lesion is closer to at a center than a preset distance condition as a matter of design choice.
With respect to claim 16, Hirokazu teaches the method as claimed, wherein the region of interest is a “lesion”, see the first 6 lines of the Description, at page 2. Lesions may be cancerous or may be caused by inflammation.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to replace the lesion, taught by Hirokazu for a specific type of lesion such as cancer or inflammation.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hirokazu in view of TW201706890.
Hirokazu teaches all of the subject matter upon which the claim depends. However, what is not specifically taught by Hirokazu is determining a temporal change in the region of interest that is smaller than a predetermined condition so that medical information is not superimposed on the image.
The 890 reference teaches a control system 112 which includes a processor 114 and memory 116 for repositioning surgical data overlay 9 (claimed information). A temporal change is determined by the “tool tracking system” (page 7, second full para.) or “eye tracking module” ( page 7, fourth full para.). The temporal change is small enough to be detected as defined by a time determined by the processor 114 of control system 112. The control system detects the surgeon identifying a new area of interest, a different area than previously under view relating to the surgical procedure.
Within the temporal change, there can be a “bad delay” for example, one five or any range of seconds between 0 and 20 (see page 7, bottom of the second full para.). Within this time, identified as “bad delay”, no surgical data overlay is presented in an orientation designated by the operator. Hence, the surgical data overlay (claimed information) is not superimposed until after the delay has expired.
Since Hirokazu and the 890 reference are both directed to tracking during a surgical procedure, the purpose of not superimposing medical information would have been recognized by Hirokazu as set forth by the 890 reference.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the processing of the CPU taught by Hirokazu so that it performs in the manner as the control system 112 of the 890 reference. For the purpose of not superimposing medical information along with surgical images.
Claim(s) 11, 12 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hirokazu in view of Usuda US Pat.12, 029,384
With respect to claim 11, Hirokazu teaches at least one processor, taught at the top of page 4, wherein the processor is configured perform all of the steps upon which claim 1 depends.
Hirokazu teaches a display unit 5 for recognition of whether the superimposed information will be combined with the medical image.
Hirokazu teaches an image analysis unit 21 for identifying the lesion within the region of interest.
Hirokazu does not teach that the imaging unit uses artificial intelligence.
Usuda teaches image diagnosis using medical images has been employing a way of analyzing a medical image and automatically detecting a region of interest to be carefully observed, such as a lesion or a tumor in an organ. In particular, execution of machine learning such as deep learning dramatically increases the accuracy of detecting a region of interest.
Since Usuda and Hirokazu are both directed to the detection of lesions in medical imaging, the purpose of using artificial intelligence would have been recognized by Hirokazu as set forth by Usuda.
It would have been obvious to one of ordinary skill in the art to replace the imaging analyzing unit 21, taught by Hirokazu, for a processor, using artificial intelligence, for the purpose of identifying whether within a region of interest, the presence of cancer, lesion, polyp, inflammation, angiodysplasia or any other known cellular abnormalities.
With respect to claim 12, Hirokazu teaches at least one processor, taught at the top of page 4, wherein the processor is configured perform all of the steps upon which claim 1 depends.
Hirokazu teaches a display unit 5 for recognition of whether the superimposed information will be combined with the medical image.
Hirokazu teaches an image analysis unit 21 for identifying the lesion within the region of interest.
Hirokazu does not teach that the imaging unit uses artificial intelligence.
Usuda teaches image diagnosis using medical images has been employing a way of analyzing a medical image and automatically detecting a region of interest to be carefully observed, such as a lesion or a tumor in an organ. In particular, execution of machine learning such as deep learning dramatically increases the accuracy of detecting a region of interest. The use of artificial intelligence is the means for discriminating the region of interest such that the medical images are processed using artificial intelligence for performing the image recognition of lesions.
Since Usuda and Hirokazu are both directed to the detection of lesions in medical imaging, the purpose of using artificial intelligence would have been recognized by Hirokazu as set forth by Usuda.
It would have been obvious to one of ordinary skill in the art to replace the imaging analyzing unit 21, taught by Hirokazu, for a processor, using artificial intelligence, for the purpose of identifying whether within a region of interest, the presence of cancer, lesion, polyp, inflammation, angiodysplasia or any other known cellular abnormalities.
With respect to claim 18, HIrokazu teaches an image processing method wherein a processor is configured to perform all of the steps upon which claim 16 depends.
Hirokazu teaches a display unit 5 for recognition of whether the superimposed information will be combined with the medical image.
Hirokazu teaches an image analysis unit 21 for identifying the lesion within the region of interest.
Hirokazu does not teach wherein the method of displaying images obtained by an imaging unit using artificial intelligence.
Usuda teaches image diagnosis using medical images has been employing a way of analyzing a medical image and automatically detecting a region of interest to be carefully observed, such as a lesion or a tumor in an organ. In particular, execution of machine learning such as deep learning dramatically increases the accuracy of detecting a region of interest. The use of artificial intelligence is the means for discriminating the region of interest such that the medical images are processed using artificial intelligence for performing the image recognition of lesions.
Since Usuda and Hirokazu are both directed to the detection of lesions in medical imaging, the purpose of using artificial intelligence would have been recognized by Hirokazu as set forth by Usuda.
It would have been obvious to one of ordinary skill in the art to replace the imaging analyzing unit 21, taught by Hirokazu, for a processor, using artificial intelligence, for the purpose of identifying whether within a region of interest, the presence of cancer, lesion, polyp, inflammation, angiodysplasia or any other known cellular abnormalities.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEROME GRANT II whose telephone number is (571)272-7463. The examiner can normally be reached M-F 9:00 a.m. - 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEROME GRANT II/Primary Examiner, Art Unit 2664
Prosecution Insights