DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 14, 23–25, 28, 29, 34, 35, 39, 41, and 42 are rejected under 35 U.S.C. 103 as being unpatentable over Miyai (US 2017/0251901 A1), Wright (US 2020/0175719 A1), and Rhead (US 2018/0172820 A1).
Examiner notes these claims represent the most basic of concepts. The Applicant seeks to patent enhancement of an image by conventional enhancements like smoothing the image using a low-pass filter and sharpening the image using a high-pass filter. Preprocessing an image to enhance the image is basic and obvious to one skilled in the art.
Regarding claim 14, the combination of Miyai, Wright, and Rhead teaches or suggests an image processing apparatus comprising: circuitry configured to: receive an input image; enhance a low-frequency component of the input image that is below a predetermined spatial frequency, wherein enhancing the low-frequency component includes (Examiner finds Applicant’s low-pass filter is a smoothing filter; Miyai, ¶ 0008: teaches, in an endoscope application, performing low-pass filtering to remove high-frequency components of an image and then enhancing the low-frequency image): reducing the input image according to a predetermined reduction ratio (Miyai, ¶ 0059: teaches the low-frequency image is reduced in size (i.e. resolution) prior to smoothing (reducing noise) of the image; See also Coleman, cited under the Conclusion Section of this Office Action, explaining it doesn’t matter much whether you downsample before or after; Miyai, ¶ 0075: explaining the image size is reduced and then returned to its original input size); applying a smoothing process to the reduced input image (Miyai, ¶ 0059: teaches the low-frequency image is reduced in size (i.e. resolution) prior to smoothing (reducing noise) of the image); generating a difference image by calculating a difference between the reduced input image and the smoothed reduced input image (Wright, ¶ 0239: teaches, in an endoscope application, that a smoothed image is subtracted from the previous (unsmoothed) image to achieve a residual image); enlarging the difference image according to a predetermined enlargement ratio (Wright, ¶ 0239: teaches the downsampled images are recombined by upsampling the downsampled images; see also Miyai, ¶ 0075: explaining the image size is reduced and then returned to its original input size); applying a gain factor to the enlarged difference image (Wright, ¶ 0239: teaches certain frequencies subjected to amplification to enhance the image by improving the SNR for that band); and combining the input image with the gain-adjusted difference image to produce an enhanced output image (Neither Miyai nor Wright is relied upon for teaching that a composite image can be the result of superposing the input image with one or more frequency-enhanced versions of the input image; Rhead, ¶ 0216: explains that the skilled artisan knows well that composite images can be formed by combining an input image with one or more additional images produced from the input image using conventional image processing techniques, such as high frequency (or low frequency) enhancement).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, with those of Wright, because both references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because finding a difference between an enhanced image and the input image allows for further processing on a reduced set of data that can be added back later to reconstruct the enhanced image. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai and Wright used in this Office Action unless otherwise noted.
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai and Wright, with those of Rhead, because all three references are drawn to image enhancement and at least Rhead and Miyai are drawn to producing composite images of an input image and an enhanced version of an input image such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because the skilled artisan has readily available in her toolkit the plethora of image pre-processing techniques such that the most basic of smoothing and sharpening filters (i.e. low-pass and high-pass filtering) would be obvious to implement when taking medical images. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, and Rhead used in this Office Action unless otherwise noted.
Regarding claim 23, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 14, wherein the input image is a medical image (This is unpatentable subject matter; You can’t get a patent on the content of the image; Miyai, Abstract: teaches the images are from an endoscope used in endoscopic surgery).
Regarding claim 24, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 23, wherein the medical image includes at least one of: a white light image captured under white light illumination, a narrow band image captured under narrow band light illumination, and a fluorescence image captured under excitation light illumination (Miyai, ¶ 0071: teaches the white light and “special light” have different high frequency noise characteristics such that one type of illumination may benefit from high-pass filtering noise out of the image; Miyai, ¶ 0002: teaches the light sources can be white light and narrow band light).
Regarding claim 25, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 24, wherein when two or more types of input images are received, the circuitry is configured to control enhancement processing of one type of input image based on information derived from another type of input image (Miyai, ¶ 0050–0052, 0064–0066, and 0071: teaches the white light and “special light” have different high frequency noise characteristics such that one type of illumination may create a high-frequency image component that has a large amount of noise and in such a case the processing of the low-frequency image component is influenced).
Regarding claim 28, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 14, wherein the predetermined enlargement ratio corresponds to the predetermined reduction ratio (Miyai, ¶ 0075: explaining the image size is reduced and then returned to its original input size).
Regarding claim 29, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 14, wherein the gain factor is adjustable to control an enhancement level of the low-frequency component (Wright, ¶ 0239: teaches certain frequencies subjected to amplification to enhance the image by improving the SNR for that band).
Regarding claim 34, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 14, wherein the input image includes a sequence of video frames (Miyai, ¶ 0033: teaches the endoscope is envisaged to be capable of providing video images).
Regarding claim 35, the combination of Miyai, Wright, and Rhead teaches or suggests the image processing apparatus according to claim 14, wherein the circuitry is configured to receive the input image from an endoscope device (Miyai, ¶ 0033: teaches the endoscope is envisaged to be capable of providing video images).
Claim 39 lists the same elements as claim 14, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 14 applies to the instant claim.
Claim 41 lists the same elements as claim 14, but includes an endoscope having generic endoscopic components. Therefore, the rationale for the rejection of claim 14 applies to the instant claim in view of Miyai, ¶ 0008, explaining the image processing is for an endoscopic application.
Claim 42 lists the same elements as claim 14, but in CRM form rather than apparatus form. Therefore, the rationale for the rejection of claim 14 applies to the instant claim.
Claims 15–17, 19, 36, and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, and Takahira (US 2020/0175719 A1).
Regarding claim 15, the combination of Miyai, Wright, Rhead, and Takahira teaches or suggests the image processing apparatus according to claim 14, wherein the circuitry is further configured to suppress excessive enhancement in the difference image before enlarging the difference image (Consistent with Applicant’s Specification, Examiner finds the suppressed excessive enhancement means not enhancing dark or bright luminance; Takahira, ¶ 0061: teaches applying gain to an image based on luminance or chrominance components wherein the luminance component is subjected to sharpness processing and the chrominance components are subjected to low-pass filtering; see also Thaler, cited under the Conclusion Section of this Office Action; Examiner finds it is an obvious image processing technique to not over-brighten bright areas, termed “over-saturation” or raise the luminance of dark areas for fear of losing contrast).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, and Rhead, with those of Takahira, because all four references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because, as Takahira evidences, the skilled artisan had in their possession the ability to apply gain to color components of an image that was subjected to other pre-processing such as low- and high-pass filtering. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, and Takahira used in this Office Action unless otherwise noted.
Regarding claim 16, the combination of Miyai, Wright, Rhead, and Takahira teaches or suggests the image processing apparatus according to claim 15, wherein the circuitry is further configured to: separate the input image into a luminance component image and a color component image; identify pixels to be enhanced and pixels not to be enhanced based on the luminance component image and the color component image; and adjust, on a pixel by pixel basis, the gain factor based on whether a pixel is identified as a pixel to be enhanced or a pixel not to be enhanced (Takahira, ¶ 0061: teaches applying gain to an image based on luminance or chrominance components wherein the luminance component is subjected to sharpness processing and the chrominance components are subjected to low-pass filtering; see also Thaler, cited under the Conclusion Section of this Office Action; Examiner finds it is an obvious image processing technique to not over-brighten bright areas, termed “over-saturation” or raise the luminance of dark areas for fear of losing contrast).
Regarding claim 17, the combination of Miyai, Wright, Rhead, and Takahira teaches or suggests the image processing apparatus according to claim 16, wherein adjusting the gain factor includes reducing the gain factor for pixels having a luminance value outside of a predetermined luminance range (Wright, ¶ 0239: teaches certain frequencies subjected to amplification to enhance the image by improving the SNR for that band; see also Vanderby, cited under the Conclusion Section of this Office Action, wherein weights can be applied to frequency components of an image to control enhancement; Takahira, ¶ 0061: teaches applying gain to an image based on luminance or chrominance components wherein the luminance component is subjected to sharpness processing and the chrominance components are subjected to low-pass filtering; see also Thaler, cited under the Conclusion Section of this Office Action; Examiner finds it is an obvious image processing technique to not over-brighten bright areas, termed “over-saturation” or raise the luminance of dark areas for fear of losing contrast (i.e. maintain best contrast ratio)).
Regarding claim 19, the combination of Miyai, Wright, Rhead, and Takahira teaches or suggests the image processing apparatus according to claim 15, wherein suppressing excessive enhancement includes reducing an enhancement level for pixel regions of the difference image having a gradient above a predetermined threshold (Wright, ¶ 0239: teaches certain frequencies subjected to amplification to enhance the image by improving the SNR for that band; see also Vanderby, cited under the Conclusion Section of this Office Action, wherein weights can be applied to frequency components of an image to control enhancement; Takahira, ¶ 0061: teaches applying gain to an image based on luminance or chrominance components wherein the luminance component is subjected to sharpness processing and the chrominance components are subjected to low-pass filtering; see also Thaler, cited under the Conclusion Section of this Office Action; Examiner finds it is an obvious image processing technique to not over-brighten bright areas, termed “over-saturation” or raise the luminance of dark areas for fear of losing contrast (i.e. maintain best contrast ratio)).
Regarding claim 36, the combination of Miyai, Wright, Rhead, and Takahira teaches or suggests the image processing apparatus according to claim 20, wherein the circuitry is configured to reduce or suspend enhancement of the high-frequency component when the input image contains a high proportion of noise (Consistent with Applicant’s Specification, Examiner finds the suppressed excessive enhancement means not enhancing dark or bright luminance; Takahira, ¶ 0061: teaches applying gain to an image based on luminance or chrominance components wherein the luminance component is subjected to sharpness processing and the chrominance components are subjected to low-pass filtering; see also Thaler, cited under the Conclusion Section of this Office Action; Examiner finds it is an obvious image processing technique to not over-enhance image noise, which the skilled artisan knows can be presented as one or more of low-frequency or high-frequency noise patterns).
Claim 40 lists the same elements as claim 15, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 15 applies to the instant claim.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, Takahira, and Sakurai (US 2007/0097267 A1).
Regarding claim 18, the combination of Miyai, Wright, Rhead, Takahira, and Sakurai teaches or suggests the image processing apparatus according to claim 16, wherein adjusting the gain factor includes reducing the gain factor for pixels having a chrominance value indicative of chromatic aberration (Sakurai, ¶ 0004 and Title: teaches chromatic aberration suppression by reducing a gain factor when chromatic aberration, represented as a large absolute value of G, is large).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, Rhead, and Takahira, with those of Sakurai, because all five references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because, as Sakurai evidences, the skilled artisan had in their possession the ability to apply gain control to reduce gain for pixels of an image exhibiting characteristics indicative of chromatic aberration. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, Takahira, and Sakurai used in this Office Action unless otherwise noted.
Claims 20–22, 26, 27, 32, and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, and Mitani (US 2020/0412959 A1).
Regarding claim 20, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 14, wherein the circuitry is further configured to enhance a high-frequency component of the input image that is above the predetermined spatial frequency (Examiner finds the high-frequency components correspond to edge enhancement or sharpening or raising contrast; Miyai does not appear to teach high-pass filtering to sharpen the image; Mitani, ¶ 0093: teaches, in an endoscopic application, pre-processing for enhancement can include high frequency enhancement).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, and Rhead, with those of Mitani, because all four references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings, because Mitani teaches it is common to subject endoscopic images to high-frequency sharpening enhancement such that one skilled in the art would be led to modify Miyai’s high-frequency image to include sharpening enhancement, and because the skilled artisan has readily available in her toolkit the plethora of image pre-processing techniques such that the most basic of smoothing and sharpening filters (i.e. low-pass and high-pass filtering) would be obvious to implement when taking medical images. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, and Mitani used in this Office Action unless otherwise noted.
Regarding claim 21, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 20, wherein the circuitry is configured to: first enhance the low-frequency component of the input image to produce a low-frequency enhanced image; combine the low-frequency enhanced image with the input image; and subsequently enhance the high-frequency component of the combined image (see treatment of claim 14; Examiner finds the high-frequency components correspond to edge enhancement or sharpening or raising contrast; Miyai does not appear to teach high-pass filtering to sharpen the image; Mitani, ¶ 0093: teaches, in an endoscopic application, pre-processing for enhancement can include high frequency enhancement; Examiner notes the order of processing and combining with a given image is demonstrated to be arbitrary as evidenced by Applicant’s original claim 22).
Regarding claim 22, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 20, wherein the circuitry is configured to: enhance the low-frequency component of the input image to produce a low-frequency enhanced image (Examiner finds this low-pass filter is a smoothing filter; Miyai, ¶ 0008: teaches, in an endoscope application, performing low-pass filtering to remove high-frequency components of an image and then enhancing the low-frequency image); enhance the high-frequency component of the input image to produce a high-frequency enhanced image (Examiner finds the high-frequency components correspond to edge enhancement or sharpening or raising contrast; Miyai does not appear to teach high-pass filtering to sharpen the image; Mitani, ¶ 0093: teaches, in an endoscopic application, pre-processing for enhancement can include high frequency enhancement); and combine the low-frequency enhanced image and the high-frequency enhanced image with the input image (Miyai, ¶ 0050: teaches that the output unit can output the combination of the low-frequency enhanced image and the high-frequency image (i.e. input image); Neither Miyai nor Mitani is relied upon for teaching that a composite image can be the result of superposing the input image with one or more frequency-enhanced versions of the input image; Rhead, ¶ 0216: explains that the skilled artisan knows well that composite images can be formed by combining an input image with one or more additional images produced from the input image using conventional image processing techniques, such as high frequency (or low frequency) enhancement).
Regarding claim 26, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 25, wherein the circuitry is configured to, based on detecting a fat tissue region in the white light image, increase an enhancement level of a corresponding region in the fluorescence image (Miyai, ¶ 0050–0052, 0064–0066, and 0071: teaches the white light and “special light” have different high frequency noise characteristics such that one type of illumination may create a high-frequency image component that has a large amount of noise and in such a case the processing of the low-frequency image component is influenced; Mitani, ¶ 0195: teaches wavelength properties of irradiation sources can be tailored to tissues such as superficial tissues; See also plethora of prior art cited under the Conclusion Section of this Office Action demonstrating the well-known correlation between frequency and depth of tissue imaging; Examiner notes there is no technological feature represented by this claim that is not a law of nature or natural phenomenon; The claim merely lists an intended use or context in which the camera operates; Deep tissue being represented by low-frequency components of the image (or, likewise superficial tissue being represented by high-frequency components) has to do with the laws of nature of the imaging radiation source and how that source interacts with the properties of the bodily tissue; Such subject matter, without more, is not patent-eligible subject matter; Therefore, less patentable weight is given to this claim limitation).
Regarding claim 27, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 25, wherein the circuitry is configured to, based on identifying an organ in the white light image known to accumulate fluorescent dye, decrease an enhancement level of a corresponding region in the fluorescence image (Miyai, ¶ 0050–0052, 0064–0066, and 0071: teaches the white light and “special light” have different high frequency noise characteristics such that one type of illumination may create a high-frequency image component that has a large amount of noise and in such a case the processing of the low-frequency image component is influenced; Mitani, ¶ 0195: teaches wavelength properties of irradiation sources can be tailored to tissues such as superficial tissues; See also plethora of prior art cited under the Conclusion Section of this Office Action demonstrating the well-known correlation between frequency and depth of tissue imaging; Examiner notes there is no technological feature represented by this claim that is not a law of nature or natural phenomenon; The claim merely lists an intended use or context in which the camera operates; Deep tissue being represented by low-frequency components of the image (or, likewise superficial tissue being represented by high-frequency components) has to do with the laws of nature of the imaging radiation source and how that source interacts with the properties of the bodily tissue; Such subject matter, without more, is not patent-eligible subject matter; Therefore, less patentable weight is given to this claim limitation).
Regarding claim 32, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 14, wherein the circuitry is further configured to increase an enhancement level to enhance image features having spatial frequency characteristics of deep tissues covered by biological membranes or fat (Mitani, ¶ 0195: teaches wavelength properties of irradiation sources can be tailored to tissues such as superficial tissues; See also plethora of prior art cited under the Conclusion Section of this Office Action demonstrating the well-known correlation between frequency and depth of tissue imaging; Examiner notes there is no technological feature represented by this claim that is not a law of nature or natural phenomenon; The claim merely lists an intended use or context in which the camera operates; Deep tissue being represented by low-frequency components of the image (or, likewise superficial tissue being represented by high-frequency components) has to do with the laws of nature of the imaging radiation source and how that source interacts with the properties of the bodily tissue; Such subject matter, without more, is not patent-eligible subject matter; Therefore, less patentable weight is given to this claim limitation).
Regarding claim 33, the combination of Miyai, Wright, Rhead, and Mitani teaches or suggests the image processing apparatus according to claim 32, wherein the circuitry is configured to enhance image features having spatial frequencies characteristic of surface blood vessels (Mitani, ¶ 0195: teaches wavelength properties of irradiation sources can be tailored to tissues such as superficial tissues; See also plethora of prior art cited under the Conclusion Section of this Office Action demonstrating the well-known correlation between frequency and depth of tissue imaging; Examiner notes there is no technological feature represented by this claim that is not a law of nature or natural phenomenon; The claim merely lists an intended use or context in which the camera operates; Deep tissue being represented by low-frequency components of the image (or, likewise superficial tissue being represented by high-frequency components) has to do with the laws of nature of the imaging radiation source and how that source interacts with the properties of the bodily tissue; Such subject matter, without more, is not patent-eligible subject matter; Therefore, less patentable weight is given to this claim limitation).
Claims 30 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, and Otsuka (US 2010/0322502 A1).
Regarding claim 30, the combination of Miyai, Wright, Rhead, and Otsuka teaches or suggests the image processing apparatus according to claim 14, wherein the circuitry is configured to automatically adjust a filter size used in the smoothing process based on a zoom magnification of the input image (Otsuka, ¶ 0134 and Claim 15: teaches adjusting a smoothing filter size based on magnification of the image).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, and Rhead, with those of Otsuka, because all four references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings, because Otsuka teaches it is common to subject medical images to enhancement processing such that one skilled in the art would be led to modify Miyai’s medical image to include smoothing enhancement, and because, as evidenced by Otsuka, the skilled artisan has readily available in her toolkit the plethora of image pre-processing techniques such as adjusting smoothing filter size according to magnification when taking medical images. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, and Otsuka used in this Office Action unless otherwise noted.
Regarding claim 31, the combination of Miyai, Wright, Rhead, and Otsuka teaches or suggests the image processing apparatus according to claim 30, wherein the circuitry is configured to increase a size of a filter used in the smoothing process in response to an increase in the zoom magnification of the input image (Otsuka, ¶ 0134 and Claim 15: teaches adjusting a smoothing filter size based on magnification of the image).
Claim 37 is rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, Takahira, and Ijiri (US 2021/0004954 A1).
Regarding claim 37, the combination of Miyai, Wright, Rhead, Takahira, and Ijiri teaches or suggests the image processing apparatus according to claim 16, wherein the circuitry is configured to identify the pixels to be enhanced and the pixels not to be enhanced using a machine learning model (Ijiri, ¶ 0011: teaches using machine learning to process images including providing a smoothing filter).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, Rhead, and Takahira, with those of Ijiri, because all five references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because, as Ijiri evidences, the skilled artisan had in their possession the ability to apply image enhancement techniques such as smoothing filters on images using machine learning. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, Takahira, and Ijiri used in this Office Action unless otherwise noted.
Claim 38 is rejected under 35 U.S.C. 103 as being unpatentable over Miyai, Wright, Rhead, Takahira, Ijiri, and Yates (US 2019/0201140 A1).
Regarding claim 38, the combination of Miyai, Wright, Rhead, Takahira, Ijiri, and Yates teaches or suggests the image processing apparatus according to claim 37, wherein the machine learning model is trained to identify at least one of artifacts, biological tissues, deep blood vessels, and superficial blood vessels (Yates, ¶ 0337: teaches a medical imaging device being supplemented with a machine learning system to classify organs or tissue types).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Miyai, Wright, Rhead, Takahira, and Ijiri, with those of Yates, because all six references are drawn to the same field of endeavor such that one wishing to practice image pre-processing on frequency components of an image obtained by an endoscope would be led to their relevant teachings and because, as Yates evidences, the skilled artisan had in their possession the ability to lean on machine learning to assist in classifying tissues in medical images. Therefore, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Miyai, Wright, Rhead, Takahira, Ijiri, and Yates used in this Office Action unless otherwise noted.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Meglan (US 2018/0310811 A1) teaches pyramid decomposition filtering in which an image is subjected to repeated smoothing and subsampling (¶ 0042).
Thaler (US 6,313,883 B1) teaches image enhancement of endoscopic images by using gain on pixels of luminance and chrominance such that only a subset of the pixels are subjected to a gain (col. 3, ll. 54–67).
Michielin (US 2018/0075612 A1) teaches basic preprocessing of images for an endoscope includes low-pass and high-pass filtering to enhance the image (¶ 0041).
Urushiya (US 2009/0257559 A1) teaches preprocessing includes noise removal (low-pass filtering) and sharpening (high-pass filtering). (¶ 0023).
Niijima (US 2016/0088267 A1) teaches edge enhancement by isolating the high-frequency components from the low-frequency components of an image and enhancing edges of the luminance signal to be added to a high-frequency signal wherein those signals are added to signals having already passed through a low-pass filter and further teaches reducing aliasing noise (see e.g. ¶¶ 0174 and 0181).
Sonnenschein (US 2018/0014811 A1) teaches high-pass filtering to sharpen the image (¶ 0157).
Mitani (US 2020/0412959 A1) demonstrates Sony (Applicant) defines high-frequency enhancement as sharpness enhancement (¶ 0093).
Coleman (US 2019/0350484 A1) teaches downsampling can take place before low-pass filtering because down-sampling first can be the most computationally efficient method (¶ 0079).
Pelissier (US 2018/0220995 A1) teaches higher frequency ultrasound signals for shallower tissue imaging and lower frequency signals for deep tissue imaging (¶ 0050), wherein, “the image data from the multiple ultrasound signals are then combined into a single image frame that has improved imaging clarity at the different depths.” (¶ 0053).
Feferberg (US 2020/0139160 A1) teaches higher frequencies for shallower tissue and lower frequencies for deeper tissue (¶ 0096).
Yarmush (US 2021/0378627 A1) teaches “deeper tissue structures may require a lower ultrasound frequency, whereas more superficial structures may warrant a higher frequency.” (¶ 0059).
Vanderby (US 2014/0371587 A1) teaches combining images obtained at different ultrasound frequencies (Abstract), wherein it is a known property that lower frequencies are suited to deep tissue imaging at lower resolution and higher frequencies are suited to superficial tissue yet achieving higher resolution (e.g. ¶ 0008), and wherein during the combination of frequency images, weights can be applied to emphasize or deemphasize certain frequencies at certain depths of interest (e.g. ¶ 0027).
Nair (US 2014/0180078 A1) teaches combining enhanced image quality of multiple frequencies in a single image to facilitate analysis of tissue (¶ 0007) by forming a composite image (Abstract), wherein higher frequencies correspond to shallower tissue and increased resolution and lower frequencies correspond to deeper tissue imaging (¶ 0072).
Matsuda (US 2007/0003159 A1) teaches filter sizes determined based on magnification (¶ 0070).
Shen (US 2019/0295223 A1) teaches using machine learning to enhance images (¶ 0019).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael J Hess whose telephone number is (571)270-7933. The examiner can normally be reached Mon - Fri 9:00am-5:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8933.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J HESS/Examiner, Art Unit 2481
Prosecution Insights