Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1, 9-15, 18-19 and 25-27 are pending. Claims 9-15 are withdrawn.
Response to arguments
On pages 2 of the remarks, applicant points to paragraph [0137] for support of the claim language which was pointed to for containing written description issues:
“As described above, in the imaging apparatus10, the first image75D can be obtained by processing the RAW image 75A2 for inference by using the Al method using the trained NN82. Further, in the imaging apparatus10, the second image75E can be obtained by processing the RAW image 75A2 for inference without using the Al method. Here, in a case where the noise that is included in the RAW image75A is removed as the characteristics of the trained NN82, there is a possibility that the microstructure is reduced accordingly. On the other hand, in the second image75E, the microstructure that is reduced by the trained NN82 from the RAW image 75A2 for inference also remains. Therefore, in the imaging apparatus10, the composite image75F is generated by combining the first image75D and the second image75E. As a result, it is possible to achieve both of the suppression of excess and deficiency of noise included in the image and the suppression of excess and deficiency of sharpness of the microstructure of the subject reflected in the image, as compared with the case where the image is processed by using only the Al method that uses the trained NN82. Therefore, according to the present configuration, it is possible to obtain an image where the image quality is adjusted as compared with the case where the image is processed by using only the Al method that uses the trained NN82”
This cited language seems to be directed to a different embodiment.
PNG
media_image1.png
640
952
media_image1.png
Greyscale
Paragraph [0137] most likely related to figure 7 due to many of the numbers present in the paragraph being present in the figure. Specifically that the second image is labeled as 75E, and is combined with 75D to generate 75F. The description supports this idea of by processing the RAW image 75A2 for inference without using the Al method, because they use a digital filter instead. This means the image is actually processed in some way. The claim language requires “the second image being obtained by processing the captured image” and “wherein the second image is an image in which the noise adjustment for the captured image is not performed”. The language seem to be directed to 2 separate embodiments, the first being “the second image being obtained by processing the captured image” (supported by paragraph [0137] and Fig. 7) and the second being “wherein the second image is an image in which the noise adjustment for the captured image is not performed” (paragraph [0237] and Fig. 25). There is no example of these embodiments being combined in the disclosure.
Applicant the further points to [0237] for support of this idea that the image is processed in some way and also does not have its noise adjusted:
For example, as an example shown in Fig. 25, the image that is obtained without performing the noise adjustment on the RAW image 75A2 for inference, that is, the second weight106 may be applied to the RAW image 75A2 for inference. In this case, the RAW image 75A2 for inference is an example of a "second image" according to the present disclosed technology.
While this does provide support for obtaining an image without adjusting the noise, it does not provide support for processing a captured image to obtain the second image. The Raw image is not processed and is used as a second image, as the specification explicitly says “the RAW image 75A2 for inference is an example of a "second image"”.
Applicant further asserts:
“Next, in paragraph [0237], the RAW image 75A2 for inference is positioned as "an example shown in Fig. 25" and "an example of a 'second image' according to the present disclosed technology". Accordingly, a person skilled in the art would understand from this disclosure that it is not intended to limit the second image to a specific RAW-format image, but rather that any image obtained without adjusting noise for the captured image can be included as the "second image" of the present disclosure. Hence, the recitation in claim 1 that "the second image is an image in which the noise adjustment for the captured image is not performed" generalizes the "RAW image 75A2 for inference" exemplified in paragraph [0237]”.
The examiner respectfully disagrees. One of ordinary skill in the art would see Fig. 25 and recognize a Raw image being used for the second image, not a processed image. Moving from a “raw image” to “any image that doesn’t have its noise processed” is unrealistic with to current disclosure, and one would not make that move simply by reading that the raw image is “an example” of the second image. The same numerical denotation (#75A2) is used for the Raw image and the second image that gets input into the weight applying unit. The Examiner recognizes there is support for processing the Raw image to get the second image ([0137] and Fig. 7 describe using a digital filter to process the raw image to produce the image) and support for a second image that is obtained without adjusting the noise ([0237] and Fig. 25 describe a raw image being used which has not had its noise adjusted), but these are separate embodiments and the language seems to be combining them when they do not seem combinable. This does not correct the standing 112(a) issues. The examiner recommends fixing the claim language to correct this issue, either keep the embodiment which processes the image and the noise is adjusted (example 1) or remove the language is which the captured image is processed (example 2). Example claim language is provided below:
Example 1: An information processing apparatus comprising:
a processor; and
a memory connected to or built into the processor,
wherein the processor is configured to:
perform AI method noise adjustment processing of adjusting noise included in a captured image, by using an AI method that uses a neural network, and
perform composition processing of compositing a first image obtained by processing the captured image by using the AI method, and a second image obtained by processing the captured image without using the AI method, to adjust the noise, and
wherein the second image is an image in which the noise adjustment for the captured image is weaker than that of the first image
Example 2: An information processing apparatus comprising:
a processor; and
a memory connected to or built into the processor,
wherein the processor is configured to:
perform AI method noise adjustment processing of adjusting noise included in a captured image, by using an AI method that uses a neural network, and
perform composition processing of compositing a first image obtained by processing the captured image by using the AI method, and a second image obtained by using as the second image
wherein the second image is an image in which the noise adjustment for the captured image is not performed.
With respect to the arguments of the 103 rejection of claims 1 and 25-27, on pages 6-9, applicant asserts:
“The purpose of Mao is to obtain a high signal-to-noise ratio comparable to that
of a conventional technique of averaging a plurality of images, from a single image
capture, and to that end, Mao is based on a concept of combining outputs of multiple
types of noise reduction filters. Thus, in Mao, images to be composited are consistently
noise-reduced images to which some kind of noise reduction processing has been applied,
and there is no disclosure of using an input image (a captured image) that still includes
noise as a compositing target. In contrast, if an input image including noise were mixed
into the composition, the noise suppression effect comparable to averaging a large
number of images sought by Mao would be degraded. A person skilled in the art
reading Mao would understand that images to be composited should be noise-reduced
images, and an input image including noise should not be used for composition. That
is, Mao teaches away from such a direction of modification”
The examiner respectfully disagrees, MPEP 2145(D)(1) states:
Furthermore, “the prior art’s mere disclosure of more than one alternative does not constitute a teaching away from any of these alternatives because such disclosure does not criticize, discredit, or otherwise discourage the solution claimed….” In re Fulton, 391 F.3d 1195, 1201, 73 USPQ2d 1141, 1146 (Fed. Cir. 2004). See also UCB, Inc. v. Actavis Labs, UT, Inc., 65 F.4th 679, 692, 2023 USPQ2d 448 (Fed. Cir. 2023) (“a reference does not teach away if it merely expresses a general preference for an alternative invention but does not criticize, discredit or otherwise discourage investigation into the invention claimed.”) (internal quotations omitted) (quoting DePuy Spine, Inc. v. Medtronic Sofamor Danek, Inc., 567 F.3d 1314, 1327 (Fed. Cir. 2009)); and Schwendimann v. Neenah, Inc., 82 F.4th 1371, 1381, 2023 USPQ2d 1173 (Fed. Cir. 2023) (“Although Oez [the prior art] used a white pigment with a cross-linking polymer, it does not discourage a skilled artisan from using the white pigment without a cross-linking polymer or lead the skilled artisan in a direction divergent from the path taken in the Appealed Patents. Thus, Oez's disclosure is substantial evidence that supports the Board's finding that Oez does not teach away from the proposed combination.”).
Mao taches a way to reduce noise by combining images that have had their noise reduced, this is just an alternative to a method that uses image with some noise in combination with images with no noise to reduce overall noise, in this way Mao does not teach away, merely reciting an alternative.
Applicant further asserts:
As described above, in Iwase, the image in which noise adjustment is not performed is the first image (input image), while the second image is rather a high-quality image to which noise reduction, contrast enhancement, or the like has been applied. Further, the purpose of composition in Iwase is to mitigate unnaturalness that may be caused by strong image quality improvement processing such as GAN processing and to display an image suitable for interpretation and observation. This is clearly different from claim 1 of the present application, which prepares a first image obtained by processing the captured image by using the AI method, and a second image obtained without processing the captured image by using the AI method (and obtained without adjusting noise for the captured image), and adjusts noise characteristics themselves by composition based on an asymmetric two-path configuration.
PNG
media_image2.png
700
304
media_image2.png
Greyscale
The examiner respectfully disagrees. Underlining will be used to better highlight the connecting language
PNG
media_image3.png
468
772
media_image3.png
Greyscale
The disclosed invention uses an AI method to remove noise from a raw image, and uses the same original raw image as a second image in composition processing to generate a composite image in which the noise is reduced.
Iwase describes uses an AI method to produce a high quality image from a raw image, and uses the same original raw image as a second image in composition processing to generate a composite image in which the noise is reduced ([0146]: “can output a high quality image in which noise is reduced and/or contrast is enhanced from an input image”).
The two inventions do not seem clearly different, they seem clearly related by the named operations. It also should be noted that the applicants recognized Iwase’s purpose is to “mitigate unnaturalness”. This is the very definition of noise reduction, to mitigate unnaturalness. This is another similarity between the two art.
Applicant further asserts:
A configuration based on Mao is a composition of an AI filter output image and
a non-AI noise reduction filter output image, both of which are noise-reduced images and
thus does not satisfy the condition that the second image is an image obtained without
adjusting noise for the captured image. Also in Iwase, the second image is a high-
quality image to which noise reduction or contrast enhancement has been applied, and
Iwase does not teach a second image obtained without adjusting noise for the captured
image. Accordingly, even by combining both references, one would not arrive at the
configuration and effects unique to the present application, in which noise is adjusted by
compositing a first image obtained by processing the captured image by using the AI
method and a second image obtained without processing the captured image by using the AI method for the captured image.
The examiner respectfully disagrees. Mao teaches the composition of image generated from AI filters and non-AI filters, Mao is not relied upon for obtaining an image in which noise is not adjusted, Iwase is used for that aspect. Applicant is pointing to arbitrary numbering of the images in Iwase detailing that the second image in Iwase differs from the second image in the disclosed invention. However, the examiner is not pointing to the second image for this limitation, the examiner is pointing to the original captured image which is combined with the high quality image, so Iwase does teach a second image obtained without adjusting noise for the captured image, specifically it is the original image used for merging. Combining these art in this would result in a configuration in which noise is adjusted by compositing a first image obtained by processing the captured image by using the AI method (Mao’s AI filter) and a second image obtained without processing the captured image by using the AI method for the captured image (Iwase’s original image). It is for these reasons that the 103 rejection of the claims is maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 25-27 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “perform composition processing of compositing a first image obtained by processing the captured image by using the AI method, and a second image obtained by processing the captured image without using the AI method, to adjust the noise, and wherein the second image is an image in which the noise adjustment for the captured image is not performed”. For support, applicants point to claim 2 of the original claims:
“The information processing apparatus according to claim 1, wherein the processor is configured to perform Al method noise adjustment processing of adjusting noise included in the captured image, by using the Al method, and adjust the noise by performing the composition processing”.
and paragraph [0237] of the specification:
“For example, as an example shown in Fig. 25, the image that is obtained without performing the noise adjustment on the RAW image 75A2 for inference, that is, the second weight106 may be applied to the RAW image 75A2 for inference. In this case, the RAW image 75A2 for inference is an example of a "second image" according to the present disclosed technology.”
PNG
media_image4.png
622
922
media_image4.png
Greyscale
Fig. 25 details this idea of the second image being an image that does not have its noise adjusted, however, this second image is the raw image input into the weight applying unit, not a processed version of the raw image which is claimed earlier for the second image: “a second image obtained by processing the captured image without using the AI method”. There is no support for a second image which is processed in some way and also does not have its noise adjusted. For examination purposes, since Fig. 25 was pointed to for support, the concept of the second image being the raw or original image will be the interpretation (any recitation of the processing of the second image before weight application will be ignored). For the additional support provided by the applicant in the remarks, refer to the response to arguments section regarding the analysis of the new language that is the alleged support.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over Mao et al. (US 20200279352 A1 Hereinafter “Mao”) in further view of Iwase et al. (US 20210390696 A1 Hereinafter “Iwase”).
Regarding claim 1, Mao discloses an information processing apparatus comprising: a processor ([0050]: “Such a system may include a computer having one or more processors”); and
a memory connected to or built into the processor ([0050]: “Such a system may include a computer having one or more processors (e.g., in the form of an integrated circuit(s), discrete circuitry, or the like) for executing the method, storage (such as a hard disk, memory, RAM, or the like)”,
wherein the processor is configured to
perform Al method noise adjustment processing of adjusting noise included in a captured image, by using the Al method that uses a neural network (Fig. 2A-2B, [0032]: “Referring back to FIGS. 2A and 2B, a deep learning system trained to suppress random noise may correspond to noise reduction filter 1 (a sharp filter) 202-1, 212-1”. This filter can be seen processing the original noisy image (200) in Fig. 2A.), and
perform composition processing of compositing a first image obtained by processing the captured image by using the Al method, and a second image (Fig. 2A, [0035]: “The noise-reduced images output by each filter can be combined according to any statistical combination method”. One of the filters was defined as being a deep learning system which is used to generate the first image (204: Noise reduced image-1), while one of the other filters is defined as being a non-AI filter “In still other embodiments, any type of filter capable of reducing a desired type of noise may be used. Such filters may include median filters, Gaussian filters, spectral domain filters, and the like”[0034]. This non-AI filter is used to generate the second image (204: Noise reduced image-2). These images are combined in a weighted manner to reduce the noise from the original image ([0036]: “By way of comparison, combining the outputs of each filter can produce a final noise-reduced image comparable to an image produced by averaging 128 images taken at the same location (a traditional technique for suppressing noise)”), and
Mao does not expressly disclose performing combination operations wherein the second image is an image in which the noise adjustment for the captured image is not performed.
However, Iwase teaches performing combination operations wherein the second image is an image in which the noise adjustment for the captured image is not performed ([0010]: “a display controlling unit configured to cause a composite image obtained by combining the first image and the second image according to a ratio”. The second image is a processed version of the input image “The image quality improving unit 404 uses an image quality improving engine that includes a machine learning engine to generate, from the input image, a high quality image (second image) which has undergone at least one of noise reduction and contrast enhancement relative to the input image”[0145]. The first image is the input image “The obtaining unit 401 obtains an input image (first image) that is an image of a predetermined site of the subject”[0145]. So Iwase teaches a combination process in which a first image processed to reduce noise (second image) is combined with a second image in which the noise has not been adjusted (input image).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to substitute Mao’s second image obtaining method (non-AI noise processing to generate a second image) with Iwase’s second image obtaining method (using the input image) because such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, with Iwase’s second image obtaining method (using the input image) teaches that when performing a weighted combination of two images to generate an improved version of the original image, one can use the original image and the processed image in the weighted combination, and one of ordinary skill in the art would expect similar effects if substituted for Mao’s second image obtaining method (non-AI noise processing to generate a second image).
Regarding claim 25, the content of claim 25 is similar to the content of claim 1, with the additional teachings of an image sensor. Mao also discloses this information([0041]: “For example, the input may be a single optical coherence tomography (OCT) (or like coherent imaging modality) B-scan, a 3D volumetric scan, an en face image or C-scan, an angiographic B-scan image, an angiography en face image or C-scan, or like images from other imaging modalities”. Since the input is images captured from an imaging modality, there must be an image sensor present, and it is explicitly disclosed that Mao’s method is performed for OCT imaging in [0051]: “OCT ophthalmological imaging is used as an example modality and application”). Therefore, claim 1 is rejected for the same reasons of obviousness as claim 1, along with the additional teachings above.
Regarding claim 26, the content of claim 26 is similar to the content of claim 1, therefore it is rejected for the same reasons of obviousness as claim 1.
Regarding claim 27, the content of claim 27 is similar to the content of claim 1, with the additional teachings of a non-transitory computer readable medium. Mao also discloses this information ([0050]: “Such a system may include a computer having one or more processors (e.g., in the form of an integrated circuit(s), discrete circuitry, or the like) for executing the method, storage (such as a hard disk, memory, RAM, or the like)”. Therefore, claim 1 is rejected for the same reasons of obviousness as claim 1, along with the additional teachings above.
Allowable Subject Matter
Claims 18-19 are allowed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Nett et al. (US 20210279847 A1) teaches denoises images using filter or CNN, then blends the denoised with the original, the blends them together.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEFANO A DARDANO whose telephone number is (703)756-4543. The examiner can normally be reached Monday - Friday 11:00 - 7:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEFANO ANTHONY DARDANO/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights