DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-15 were pending in the application filed for examination of May 5, 2023. As of the remarks and amendments received January 7, 2026, Claims 1, 4, and 6 are amended, no claims are added, and claims 12-15 are cancelled. Accordingly, claims 1-11 are currently pending in the application for examination.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Lu et al. ("Global-local fusion network for face super-resolution." Neurocomputing 387 (2020): 309-320, IDS).
Regarding claim 1, (Currently Amended) Lu discloses a recognition system comprising:
a storage device for storing a learned model; and an arithmetic circuit accessible to the storage device (Abstract and sections 5.2 and 5.8, Storage device and processor are basic components of a computer-based system such as Lu’s. Lu specifically used “Intel Core i7-6700K CPU at 4.00 GHz and 8 GB RAM”), the learned model including:
a first model part (Fig. 3, the Reconstruction Sub-network and Section 3.2.1) learned to, in response to input of a first resolution image showing a target object at a first resolution (the low resolution input image x showing a target face), output a second resolution image (the synthetic high resolution (SHR) image PR(x)) and a difference image (equation (3), the residual image fR(x)), the second resolution image being corresponding to an image resulting from conversion of the first resolution image into a second resolution higher than the first resolution (Fig. 3, the “Reconstruction Sub-network” converts low resolution image x to high resolution image PR(x)), the difference image being corresponding to a difference between the first resolution image and the second resolution image (section 3.2.1, equation (4)); and
a second model part learned to receive both the second resolution image and the difference image as inputs and output a feature amount of the target object based on the inputs (Fig. 3 and sections 3.2.2 - 3.2.4: the final high resolution image PF(x) defines a feature amount of the target face. PF(x) is generated in response to the SHR image and the residual image which is generated by the two residual enhancement sub-networks. Also see section 5.10, features used for face recognition also defines “a feature amount”), and the arithmetic circuit being configured to perform:
obtainment processing of obtaining the first resolution image as a target image; and inference processing of providing the target image obtained by the obtainment processing to the learned model to allow the learned model to calculate a feature amount of a target object shown in the target image (Section 5.10: low resolution images are obtained from Yale-B database. The images are input to the super-resolution model (Fig. 3) to allow the learned model to calculate the final high- resolution images).
Regarding claim 2, (Original) Lu discloses the recognition system of claim 1, wherein the inference processing includes recognizing the target object based on a feature amount of a target object shown in the target image (Section 5.10: the output of the super-resolution model is used for face recognition).
Regarding claim 3, (Previously Presented) Lu discloses the recognition system of claim 1, wherein the arithmetic circuit is configured to execute output processing of outputting a result of the inference processing (Fig. 3 and section 5.9: output/display the final high- resolution images).
Claims 4-5 have been analyzed and are rejected for the same reasons as outlined above in the rejection of claim 1. Lu’s model can be seen as a distillation model because it is generated by distillation of a learned model (Abstract: “a novel global-local fused network (GLFSR) to refine HF information for recovering fine details of facial images. In contrast to existing methods that often increase the depth of network, we enhance the residual HF information from local to global levels through the networks”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-11 are rejected under 35 U.S.C. 103 as being unpatentable over Lu ("Global-local fusion network for face super-resolution." Neurocomputing 387 (2020)).
Regarding claim 6, (Currently Amended) Lu discloses a learning method comprising:
a preparation step of preparing a model (Section 3.2.1 : "divide the training sample into two parts … The first part … is used for the reconstruction network. The remaining part … is used for the residual enhancement network". The reconstruction network training step is preparation step); and
a learning step of performing machine learning using the model prepared by the preparation step (Section 3.2.1, the residual enhancement networks are trained using the output of the trained reconstruction network), the model including
a first model part, a second model part, and a third model part, the first model part being a model for, in response to input of a first resolution image showing a target object at a first resolution, outputting a second resolution image and a difference image, the second resolution image being corresponding to an image resulting from conversion of the first resolution image into a second resolution higher than the first resolution, the difference image being corresponding to a difference between the first resolution image and the second resolution image (Fig. 3, the Reconstruction Sub-network and Section 3.2.1. Please refer to analysis of claim 1), the second model part being a model for receiving both the second resolution image and the difference image as inputs and outputting a feature amount of the target object based on the inputs (Fig. 3 and sections 3.2.2 - 3.2.4. Please refer to analysis of claim 1), and the third model part being a model for outputting a result of recognition of the target object in response to input of a feature amount of the target object from the second model part, and the learning step including training the model to learn a relationship between the first resolution image and a feature amount of a target object shown in the first resolution image, by machine learning using a learning dataset (Section 5.10: " 11 random images are used for the training task of the face recognition and the rest for testing. We use the model, which is trained with the CASIA-Webface database, to test those test images. Then we use the classic algorithm kernel partial- least-squares discrimination (KPLSD) [57] to verify the identity information of SR facial test images").
Lu does not expressly disclose that learning the third model includes the first resolution image as input and a result of recognition of a target object shown in the first resolution image as ground truth.
However, it is well known and common practice in the art to use end-to-end training to improve performance and accuracy (see e.g., Lu, section 6). Lu’s face recognition system is composed of the reconstruction model, the residual enhancement models and the recognition model. To improve face recognition performance, it would have been obvious to a person having ordinary skill in the art to choose end-to-end learning, with the low-resolution images as input and the corresponding face recognition result as output.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to which the claimed invention pertains to yield the invention as described in claim 6 from the teachings of Lu.
Regarding claim 7, (Original) Lu discloses the learning method of claim 6, wherein the preparation step includes:
a generation step of generating a learning dataset including the first resolution image as input and a set of the second resolution image and the difference image as ground truth; and a pre-learning step of training the first model part to learn a relationship between the first resolution image and the set of the second resolution image and the difference image, by using the leaning dataset generated by the generation step (see analysis of claim 6)
Regarding claim 8, (Original) Lu discloses the learning method of claim 7, wherein the generation step includes: a first step of obtaining the second resolution image; a second step of generating the first resolution image by converting the second resolution image obtained by the first step into an image at the first resolution; a third step of generating the difference image from the second resolution image obtained by the first step and the first resolution image generated by the second step; and a fourth step of generating a learning dataset including the first resolution image generated by the second step as input and the set of the second resolution image prepared by the first step and the difference image generated by the third step as ground truth (Section 2.1: "the observed LR images are generated by the following model: xi = DByi + v , where B is the blurring operation … D is the downsampling operation…"; Section 2.2 and section 3.1 "we define the residual image ri = yi - xi". In Lu, a low resolution image is generated by blurring, down-sampling and adding noise to a high resolution image and a difference image is generated using the high resolution image and an image generated by interpolating the low resolution image. Please also refer to analysis of claim 6 on end-to-end learning. It would have been obvious to a person having ordinary skill in the art to choose end-to-end learning of the reconstruction model and the residual enhancement models, with the low-resolution image as input and the corresponding high-resolution image and the difference image as output).
Regarding claim 9, (Original) Lu discloses the learning method of claim 8, wherein the third step enlarges the first resolution image generated by the second step to a size same as the second resolution image obtained by the first step and generates the difference image based on differences of pixels between the first resolution image enlarged and the second resolution image obtained by the first step (Section 2.2: "Considering that the input and output images are very similar, we define the residual image ri = yi – x~i, and its pixel value is mostly zero or small. Here x~i is the interpolation version of xi").
Regarding claim 10, (Original) Lu discloses the learning method of claim 9, wherein a range of each pixel in the difference image is narrower than a range of each pixel of the first resolution image enlarged and the second resolution image obtained by the first step (Section 2.2: "Considering that the input and output images are very similar, we define the residual image ri = yi – x~i, and its pixel value is mostly zero or small. Here x~i is the interpolation version of xi").
Regarding claim 11, (Previously Presented) Lu discloses the learning method of claim 6, but fails to further teach the remaining limitations of claim 11, which relates to using a discriminator to discriminate between real and generated high-resolution image.
However, using generative adversarial networks (GANs) for face super-resolution is well known in the art (see e.g., Lu, section 1). Using a discriminator to improve the accuracy of Lu’s generator (i.e., the GLFSR model that generates a HR image from a LR image) would be obvious for a person of ordinary skill in the art.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to which the claimed invention pertains to yield the invention as described in claim 11 from the teachings of Lu.
Response to Arguments
I. The anticipation rejections of claims 1-5 and 12-14 based on Lu ("Global-local fusion network for face super-resolution." Neurocomputing 387 (2020): 309-320), as noted on page 3 of the Office Action
The Examiner most respectfully disagrees with Applicants’ assertion that the citations do not meet the limitations of the claimed invention because Figure 3 and Equation (8) on page 312 of Lu show that the residual (difference) image(fF(x)) is added to the coarse image (PR(x)) within a Fusion Module to construct the final high-resolution image (PF). The Examiner presents Figure 3 below and Equation 8:
PNG
media_image1.png
214
813
media_image1.png
Greyscale
Figure 3, Lu
PNG
media_image2.png
542
690
media_image2.png
Greyscale
Section 3.2.4 Fusion Module, Lu
As can be seen from the prior art Figure 3, the second model part being a model for receiving both the second resolution image and the difference image as inputs and outputting a feature amount of the target object based on the inputs is shown in the Fig. 3 and sections 3.2.2 - 3.2.4. The prior art provides identical functionality to that of the present invention, and therefore, the rejection is kindly sustained.
The Examiner most respectfully disagrees with Applicants’ assertion that the recognition model never receives the residual image as a separate or independent input. Applicant is directed to Figure 3 above where this is shown as a separate and independent input. The prior art provides identical functionality to that of the present invention, and therefore, the rejection is respectfully sustained.
The Examiner most respectfully disagrees with Applicants’ assertion that Lu does not disclose "a second model part learned to receive both the second resolution image and the difference image as inputs and output a feature amount of the target object based on the inputs," as claimed in amended claim 1.
Again, Applicants are kindly directed to Figure 3, and the separate and distinct inputs into the second and distinct network, as shown in the Figure. The prior art provides identical functionality to that of the present invention, and therefore, the rejection is respectfully maintained.
The Examiner greatly appreciates the statements concerning advantages over conventional devices and encourages Applicants to claim the corresponding technical aspects as stated in the original specification into the claims.
Thus, it is respectfully asserted that the cited reference meets the presently claimed limitations of independent claims 1 and 4. Therefore, the Examiner most respectfully maintains the rejection.
Dependent claims
As noted above, it is respectfully asserted that independent claims 1 and 4 are not allowable, and therefore it is further respectfully maintained that dependent claims 2, 3 and 5 are also not allowable.
III. The obviousness rejections of claims 6-11 and 15 based on Lu as noted on page 6 of the
Office Action
Please refer to the response to arguments set forth above, for the maintained rejection of claims 6-11.
Conclusion
The prior art made of reference:
Porikli et al. US 20150269708 A1:
PNG
media_image3.png
564
504
media_image3.png
Greyscale
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emily C Terrell whose telephone number is (571)270-3717. The examiner can normally be reached Monday - Thursday 7 a.m.-4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights