Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 07/27/2026 have been fully considered but they are not persuasive.
Applicant argues that “Although Kikuchi teaches acquiring a point spread function (PSF) or an optical transfer function (OTF), Kikuchi does not teach obtaining information about at least one of a spread of a point spread function or a modulation transfer function, which represents a resolution performance of an optical apparatus used for the image capturing, and generating an output image having a smaller sampling pitch than that of the captured image based on the captured image and the information. For example, in Kikuchi, a point spread function (PSF) or an optical transfer function (OTF) is used to adjust the blur of the high resolution training image. This blur, however, has nothing to do with sampling pitch of an image. Thus, Kikuchi does not teach, for example, using a point spread function (PSF) or an optical transfer function (OTF) for adjusting the sampling pitch of an image.”
In response, it is noted that “an output image having a smaller sampling pitch that of the captured image” is nothing more than an upscaled or upsampled image having a higher resolution than that of the captured image.
KIKUCHI discloses, at paragraph 0050, that “The training processing section acquires a point spread function (PSF) or an optical transfer function (OTF) under these conditions. That is, the training processing section acquires the PSF of the first imaging system and the PSF of the second imaging system, or the OTF of the first imaging system and the OTF of the second imaging system.” KIKUCHI further discloses, at paragraph 0057, that “the training processing section uses the PSF of the first imaging system that performs imaging at a high resolution and the PSF of the second imaging system that performs imaging at a low resolution to add a blur to the high resolution training image 30. The PSF is a function representing a unit impulse response to the optical system, i.e., a function representing image distribution when the optical system uses a point light source to form an image.” Thus, at least one of “the PSF of the first imaging system that performs imaging at a high resolution and the PSF of the second imaging system that performs imaging at a low resolution” disclosed by KIKUCHI corresponds to the claim limitation “information about at least one of a spread of a point spread function or a modulation transfer function, which represents a resolution performance of an optical apparatus used for the image capturing”.
KIKUCHI further discloses generating an output image by reducing having a smaller sampling pitch than that of the captured image based on the captured image and the information resolution (Fig. 1, item 3, para. 0034, e.g., “the inference in the present embodiment is processing of resolution recovering a high resolution image from the processing target image 10”).
Applicant’s arguments with respect to claim(s) 1, 19, 20, 21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: Each “unit” claims 19, 21.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5-7, 9-14, 16, 18-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kokura (US 2020/0396415) in view of KIKUCHI (US 20250299297).
As to claim 1, Kokura discloses an image processing method comprising:
obtaining a captured image by image capturing using an optical apparatus (para. 0025, e.g., an image 108 obtained by capturing an athlete (object 104) is obtained from the image capturing apparatus 101), and obtaining resolution performance information about a resolution performance of the optical apparatus (para. 0025, e.g., the image capturing apparatus 105 captures a supervisory image for learning a parameter to be used for estimation processing for increasing the resolution of the image obtained by image capturing of the image capturing apparatus 101); and
generating an output image by having a smaller sampling pitch than that of the captured image based on the captured image and the resolution performance information (para. 0026, e.g., the image processing apparatus 102 may generate a plurality of processed images by performing resolution
increase processing for each of a plurality of images based on image capturing by the plurality of image capturing apparatuses 101 to 107; para. 0034, e.g., Based on the weight parameter obtained by learning, a resolution increase unit 310 increases the resolution of a low-resolution input image 505 to output an output image 506, and increases the resolution of a low-resolution input image 507 to output an output image 508),
wherein the information indicating the resolution performance is a map, and each pixel of the map indicates the resolution performance of a corresponding pixel of the captured image (para. 0031, e.g., in a learning type resolution increase method, a plurality of pairs of high-resolution supervisory images and deteriorated images obtained by reducing the resolutions of the supervisory images are prepared, and a function of mapping the supervisory image and the deteriorated image is learned; para. 0034, e.g., the resolution of a captured supervisory image is reduced to obtain a deteriorated image. Next, the learning unit 309 is made to learn a function of mapping the low-resolution deteriorated image and the high-resolution supervisory image. A parameter of a neural network obtained as a result of learning will be referred to as a weight parameter hereinafter. FIG. 5 shows, as pairs of deteriorated images and supervisory images, a pair of a deteriorated image 501 and a supervisory image 502 and a pair of a deteriorated image 503 and a supervisory image 504 ).
Kokura is silent regarding a spread of a point spread function of the optical apparatus or a modulation transfer function of the optical apparatus.
KIKUCHI teaches obtaining information about at least one of a spread of a point spread function or a modulation transfer function, which represents a resolution performance of an optical apparatus used for the image capturing (para. 0050, 0052, 0057).
It would have been obvious to one of ordinary skill in the art to incorporate KIKUCHI’s teaching into Kokura since doing so would merely combine prior art elements according to known methods to yield predictable results, and enhance the blur processing as suggested by lower KIKUCHI at paragraph 0052).
As to claim 2, the combination of Kokura KIKUCHI discloses the image processing method according to claim 1, wherein the output image is an image obtained by enlarging or demosaicing the captured image (Kokura, para. 0053, 0057, e.g., the resolution increase unit 310 inputs the input data pair (the pair of the input image and the resolution of the object) obtained by the association unit 307 to the neural network set with the learned weight parameter, and generates and outputs a corresponding high-resolution image).
As to claim 3, the combination of Kokura KIKUCHI discloses the image processing method according to claim 1, wherein the resolution performance information includes information about a degree of blur occurring in the optical apparatus (Fig. 5, item 501, 503, para. 0034, e.g., a parameter of a neural network obtained as a result of learning will be referred to as a weight parameter hereinafter. FIG. 5 shows, as pairs of deteriorated images and supervisory images, a pair of a deteriorated image 501 and a supervisory image 502 and a pair of a deteriorated image 503 and a supervisory image 504).
As to claim 5, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 1, wherein the resolution performance information includes different pieces of information for each pixel of the captured image (para. KIKUCHI, para. 0038) .
As to claim 6, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 1, wherein the resolution performance information is a map having a number of pixels corresponding to the number of pixels of the captured image (para. KIKUCHI, para. 0038, 0039).
As to claim 7, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 6, wherein a value of each pixel of the map is based on a frequency at which a modulation transfer function of the optical apparatus has a predetermined value (para. KIKUCHI, para. 0052).
As to claim 9, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 1, wherein the resolution performance information is obtained using information about at least one of a type of the optical apparatus or a state of the optical apparatus in the image capturing, and wherein the state is at least one of a focal length, an F-number, or a focus distance (KIKUCHI, para. 0050).
As to claim 10, the combination of Kokura KIKUCHI discloses the image processing method according to claim 1, wherein the optical apparatus includes an image sensor, and wherein the resolution performance information is obtained using information about a pixel pitch of the image sensor (Kokura, para. 0025, 0026, 0027, 0034).
As to claim 11, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 1, wherein the output image is obtained by correcting blur in the captured image due to the optical apparatus (KIKUCHI, para. 0052, 0053).
As to claim 12, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 1, wherein in the generation of the output image, the output image is generated based on the captured image, the resolution performance information, and information about noise in the captured image (KIKUCHI, para. 0058).
As to claim 13, the combination of Kokura and KIKUCHI discloses the image processing method according to claim 12, wherein the information about the noise includes at least one of information about an intensity of noise generated in the image capturing, or information about denoising executed on the captured image (KIKUCHI, para. 0080-0082).
As to claim 14, the combination of Kokura KIKUCHI discloses the image processing method according to claim 1, wherein in the generation of the output image, the output image is generated by inputting the captured image and the resolution performance information to a machine learning mode (Kokura, Fig. 10, para. 0024, 0034).
As to claim 16, the combination of Kokura KIKUCHI discloses the image processing method according to claim 14, wherein the machine learning model includes one or more residual blocks (Kokura, Fig. 10, para. 0060-0065, e.g., neural network inherently includes one or more residual blocks).
As to claims 18-19, these claims recite features similar to those discussed above. Therefore, they are rejected for reasons similar to those discussed above.
As to claim 20, Kokura discloses a method for producing a trained machine learning model comprising:
obtaining a first image, resolution performance information about a resolution performance corresponding to the first image, and a second image with a smaller sampling pitch than a sampling pitch of the first image (Fig. 3, items 301, 302, para. 0025, 0039);
generating an output image by inputting the first image and the resolution performance information to a machine learning model and reducing the sampling pitch of the first image (Fig. 3, item 310, para. 0034); and
updating weights of the machine learning model using the output image and the second image (para. 0034, 0037),
wherein the information indicating the resolution performance is a map, and each pixel of the map indicates the resolution performance of a corresponding pixel of the captured image (para. 0031, e.g., in a learning type resolution increase method, a plurality of pairs of high-resolution supervisory images and deteriorated images obtained by reducing the resolutions of the supervisory images are prepared, and a function of mapping the supervisory image and the deteriorated image is learned; para. 0034, e.g., the resolution of a captured supervisory image is reduced to obtain a deteriorated image. Next, the learning unit 309 is made to learn a function of mapping the low-resolution deteriorated image and the high-resolution supervisory image. A parameter of a neural network obtained as a result of learning will be referred to as a weight parameter hereinafter. FIG. 5 shows, as pairs of deteriorated images and supervisory images, a pair of a deteriorated image 501 and a supervisory image 502 and a pair of a deteriorated image 503 and a supervisory image 504).
Kokura is silent regarding a spread of a point spread function of the optical apparatus or a modulation transfer function of the optical apparatus.
KIKUCHI teaches obtaining information about at least one of a spread of a point spread function or a modulation transfer function, which represents a resolution performance of an optical apparatus used for the image capturing (para. 0050, 0052, 0057).
It would have been obvious to one of ordinary skill in the art to incorporate KIKUCHI’s teaching into Kokura since doing so would merely combine prior art elements according to known methods to yield predictable results, and enhance the blur processing as suggested by lower KIKUCHI at paragraph 0052).
As to claim 21, the claim recites features similar to those discussed above. Therefore, claim 21 is rejected for reasons similar to those discussed above.
As to claim 22, the combination of Kokura KIKUCHI discloses an image processing system comprising: the image processing apparatus according to claim 19; and a control apparatus configured to communicate with the image processing apparatus (Kokura, Fig. 2, para. 0026, 0028-0030),
wherein the control apparatus includes a unit configured to transmit a request for executing processing on the captured image (Kokura, Fig. 2, para. 0026, 0028-0030), and
wherein the image processing apparatus includes a unit configured to execute processing on the captured image in response to the request (Kokura, Fig. 2, para. 0026, 0028-0030).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 14-15, 17-21 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by KIKUCHI (US 20250299297).
As to claim 1, KIKUCHI discloses an image processing method comprising:
obtaining a captured image by image capturing (para. 0039);
obtaining information about at least one of a spread of a point spread function or a modulation transfer function, which represents a resolution performance of an optical apparatus used for the image capturing (para. 0050, 0052, 0057; e.g., “the PSF of the first imaging system that performs imaging at a high resolution and the PSF of the second imaging system that performs imaging at a low resolution”);
generating an output image by reducing having a smaller sampling pitch than that of the captured image based on the captured image and the information resolution (Fig. 1, item 3, para. 0034, e.g., “the inference in the present embodiment is processing of resolution recovering a high resolution image from the processing target image 10”).
As to claim 14, KIKUCHI discloses the image processing method according to Claim 1, wherein in the generation of the output image, the output image is generated by inputting the captured image and the performance information to a machine learning model (Fig. 1, para. 0033, 0034).
As to claim 15, KIKUCHI discloses the image processing method according to Claim 14,
wherein in the generation of the output image, input data obtained by concatenating the
captured image and the resolution performance information in a channel direction is input
to the machine learning model (Figs. 5, 6, para. 0066-0069, e.g., the machine learning model uses concatenated input image, PSFs, color channels).
As to claim 17, KIKUCHI discloses the image processing method according to Claim 1,
wherein in the generation of the output image, the output image is generated by adding a
first intermediate image and a second intermediate image, the first intermediate image being obtained by reducing the sampling pitch of the captured image without using the performance information, the second intermediate image being obtained by reducing the sampling pitch of the captured image using the captured image and the performance information (Figs. 11, 12, para. 0097-0101).
As to claim 8, KIKUCHI discloses an image processing method comprising:
obtaining a captured image by image capturing (para. 0039);
obtaining information about a resolution performance of an optical apparatus used
for the image capturing (para. 0050, 0052, 0057): and
generating an output image by reducing a sampling pitch of the captured image
based on the captured image and the information (Fig. 1, item 3, para. 0034, e.g., “the inference in the present embodiment is processing of resolution recovering a high resolution image from the processing target image 10”),
wherein the information is a map, and each pixel of the map indicates the
resolution performance of a corresponding pixel of the captured image (para. 0050, 0052, 0057), and
wherein the resolution performance information includes a plurality of channel components representing different resolution performance components for same pixel of the captured image (para. 0066-0069).
As to claims 18-21, these claims recite features similar to those discussed above. Therefore, they are rejected for reasons similar to those discussed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUOC TRAN whose telephone number is (571)272-7399. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUOC TRAN/Primary Examiner, Art Unit 2668
Prosecution Insights