Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5, 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “high-resolution” in claim 5 and in claim 13 is a relative term which renders the claim indefinite. The term “high-resolution” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. For example, some of ordinary skill in the art may define a high-resolution image as having a pixel density above a specific threshold, while others may define it based on a threshold of pixel dimensions (i.e. size) of the image; additional confusion may arise as to what threshold value is appropriate for classifying an image as "high-resolution" instead of low-resolution. Additionally, applicant's disclosure does not provide an explicit definition, but it merely provides examples of what may constitute applicant's intended "high-resolution image", as in [0033] of the specification.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5, 8-9, 13, 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Alshina (US20260113470) in view of Ahmed (US 20240414448 A1).
*Examiner note: Alshina was effectively filed before the effective filing date of the present invention because it claims priority to PCT/EP2023067440, filed 06/27/2023, which discloses the subject matter relied upon in this office action (For example, see at least pages 3, 8-9, 76-77, 80-81, 97 and Figures 28-29).
Regarding claim 1, Alshina discloses an image processing system (Fig. 28, [0493] a method for processing a picture using a neural network; [0770] a computer program product comprising computer executable instructions that when executed on a computing system cause the computing system to execute a method), comprising:
an image processing module, comprising a preprocessing module, a neural network module, and an upsampling module (Fig. 28, [0493] the NN comprises a multi-stage context model (MCM) which comprises a plurality of MCM.sub.k models, a first down-shuffle layer, a second down-shuffle layer, and an up-shuffle layer),
wherein the preprocessing module is configured to receive an image, and downsample the image to obtain a downsampled tensor (Fig. 28, [0494] operation 2810: obtaining a first tensor; [0497] operation 2840: down-shuffling, based on the first down-shuffle layer, the padded first tensor to obtain a re-shuffled first tensor; [0013] down-sampling/down-shuffle layers (i.e. the MCM down-shuffle layers are down-sampling layers)),
the neural network module is configured to process the downsampled tensor based on a plurality of first parameters, and generate an output tensor (Fig. 28, [0500] operation 2870: processing based on the plurality of MCM.sub.k models, the reshuffled first tensor and the re-shuffled second tensor to obtain a latent space tensor; [0561]-[0564] four multistage context modeling MCM.sub.k, k=0…3 models parameters), and
the upsampling module is configured to upsample the output tensor to generate an upsampled tensor having same dimensions as the image (Fig. 28, [0502] operation 2880: up-shuffling, based on the up-shuffle layer, the latent space tensor to obtain a reshuffled latent space tensor; [0013] up-sampling/up-shuffle layers (i.e. the MCM up-shuffle layers are up-sampling layers); see also Fig. 29 and [0561], [0565] wherein the input of this process has dimensions [C, h.sub.4, w.sub.4] and the output of this process has dimensions [C, h.sub.4, w.sub.4]).
Alshina fails to disclose an addition module, configured to perform element-by-element addition on the upsampled tensor and the image to obtain an output image.
Ahmed, in a related system from the same field of endeavor of image processing including neural networks (Abstract), discloses an addition module, configured to perform element-by-element addition on the upsampled tensor and the image to obtain an output image (Fig. 4, [0026] the architecture performs feature fusion using element-wise addition; [0054] operate on an input image using a U-shaped hierarchical network to produce an output image, wherein an encoder of the network contributes to a decoder of the network using a skip connection based on element-wise addition (e.g. see 419 of Fig. 4)).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Ahmed with Alshina including an addition module configured to perform element-by-element addition on the upsampled tensor and the image to obtain an output image, as disclosed by Ahmed, as part of an image processing system including downsampling, a neural network, and upsampling, as disclosed by Alshina, for the purpose of an efficient image processing system that maintains quality of output images such as for image restoration (see Ahmed: [0005], [0026]-[0027]).
Regarding claim 5, Alshina in view of Ahmed discloses the image processing system according to claim 1 as applied above. Alshina fails to disclose wherein the image is a high-resolution image.
Ahmed, in a related system from the same field of endeavor of image processing including neural networks (Abstract), discloses wherein the image is a high-resolution image ([0005] the proposed model works efficiently with high resolution cameras; [0072] the input image has a resolution of 1024 by 1024 pixels).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Ahmed with Alshina wherein the image is high-resolution image, as disclosed by Ahmed, as part of an image processing system including downsampling, a neural network, and upsampling, as disclosed by Alshina, for the purpose of an efficient image processing system that maintains quality of output images such as for image restoration (see Ahmed: [0005], [0026]-[0027]).
Regarding claim 8, Alshina in view of Ahmed discloses the image processing system of claim 1 as applied above. Alshina further discloses wherein the neural network module comprises a plurality of residual network layers connected in series, and the residual network layers are configured to receive the downsampled tensor and generate the output tensor ([0057] the NN comprises a synthesis transform net…comprises a concatenation layer configured to concatenate a main tensor and an auxiliary tensor as an input tensor…the synthesis transform net comprises a light weight residual block that is followed by a first transposed convolution layer combined with a first cropping layer and a first residual activation unit…a second residual activation unit…a third residual activation unit; [0062]-[0063] processing the input tensor using the base operating point and outputting a processed tensor).
Regarding claim 9, Alshina in view of Ahmed discloses everything claimed as applied above (see rejection of claim 1).
Regarding claim 13, Alshina in view of Ahmed discloses the image processing method according to claim 9 as applied above. Alshina in view of Ahmed further discloses everything claimed as applied above (see rejection of claim 5).
Regarding claim 16, Alshina in view of Ahmed discloses the image processing method according to claim 9 as applied above. Alshina in view of Ahmed further discloses everything claimed as applied above (see rejection of claim 8).
Regarding claim 17, Alshina discloses a processing module (Fig. 28, [0493] a method for processing a picture using a neural network; [0770] a computer program product comprising computer executable instructions that when executed on a computing system cause the computing system to execute a method);
a preprocessing module, configured to receive an input training image and downsample the input training image to obtain a downsampled tensor (Fig. 28, [0494] operation 2810: obtaining a first tensor; [0497] operation 2840: down-shuffling, based on the first down-shuffle layer, the padded first tensor to obtain a re-shuffled first tensor; [0013] down-sampling/down-shuffle layers (i.e. the MCM down-shuffle layers are down-sampling layers));
a neural network module, configured to process the downsampled tensor based on a plurality of first training parameters, and generate an output tensor (Fig. 28, [0500] operation 2870: processing based on the plurality of MCM.sub.k models, the reshuffled first tensor and the re-shuffled second tensor to obtain a latent space tensor; [0561]-[0564] four multistage context modeling MCM.sub.k, k=0…3 models parameters);
an upsampling module, configured to upsample the output tensor to generate an upsampled tensor having same dimensions as the input training image (Fig. 28, [0502] operation 2880: up-shuffling, based on the up-shuffle layer, the latent space tensor to obtain a reshuffled latent space tensor; [0013] up-sampling/up-shuffle layers (i.e. the MCM up-shuffle layers are up-sampling layers); see also Fig. 29 and [0561], [0565] wherein the input of this process has dimensions [C, h.sub.4, w.sub.4] and the output of this process has dimensions [C, h.sub.4, w.sub.4]).
Alshina fails to disclose a training system comprising a to-be-trained image processing module, an addition module configured to perform element-by-element addition on the upsampled tensor and the input training image to obtain an output training image, and wherein the processing module is configured to train the to-be-trained image processing module by using a plurality of training images in a training set and a plurality of target images corresponding to the training images, to obtain a trained parameter value of each of a plurality of image processing training parameters of the to-be-trained image processing module, and the image processing training parameters comprise the first training parameters.
Ahmed, in a relates system from the same field of endeavor of image processing including neural networks (Abstract), discloses a training system comprising a to-be-trained image processing module ([0085] the trainable parameters of the U-shaped network 400 are trained offline before deployment in the device (see also Fig. 4)), an addition module configured to perform element-by-element addition on the upsampled tensor and the input training image to obtain an output training image ((Fig. 4, [0026] the architecture performs feature fusion using element-wise addition; [0054] operate on an input image using a U-shaped hierarchical network to produce an output image, wherein an encoder of the network contributes to a decoder of the network using a skip connection based on element-wise addition (e.g. see 419 of Fig. 4))), and wherein the processing module is configured to train the to-be-trained image processing module by using a plurality of training images in a training set and a plurality of target images corresponding to the training images ([0086] training is performed, for example, using supervised learning using backpropagation given batches of raw data images and corresponding ground truth restored images (see also Figs. 2A-2B and [0040]-[0042])), to obtain a trained parameter value of each of a plurality of image processing training parameters of the to-be-trained image processing module, and the image processing training parameters comprise the first training parameters ([0085] the trainable parameters of the U-shaped network 400 are trained offline before deployment in the device; [0075] parameters are obtained from batch statistics during training).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to combine Ahmed with Alshina including a training system comprising a to-be-trained image processing module, an addition module configured to perform element-by-element addition on the upsampled tensor and the input training image to obtain an output training image, and wherein the processing module is configured to train the to-be-trained image processing module by using a plurality of training images in a training set and a plurality of target images corresponding to the training images to obtain a trained parameter value of each of a plurality of image processing training parameters of the to-be-trained image processing module, and the image processing training parameters comprise the first training parameters, disclosed by Ahmed, as part of an image processing system including downsampling, a neural network, and upsampling, as disclosed by Alshina, for the purpose of an efficient image processing system that maintains quality of output images such as for image restoration (see Ahmed: [0005], [0026]-[0027]).
Allowable Subject Matter
Claims 2-4, 6-7, 10-12, 14-15, 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, Alshina in view of Ahmed discloses the image processing according to claim 1 as applied above. Alshina fails to disclose wherein the preprocessing module performs pixel unshuffling on the image based on a zoom-out factor to downsample the image, and the downsampled tensor retains pixel information of the image.
Similar reasoning applies to claim 10 which discloses similar subject matter to claim 2.
Regarding claim 3, Alshina in view of Ahmed discloses the image processing according to claim 1 as applied above. Alshina fails to disclose wherein the upsampling module is configured to perform pixel shuffling on the output tensor based on a zoom-in factor to upsample the output tensor.
Similar reasoning applies to claim 11 and claim 18 which discloses similar subject matter to claim 3.
Regarding claim 4, Alshina in view of Ahmed discloses the image processing according to claim 1 as applied above. Alshina fails to disclose wherein the upsampling module comprises: an amplification module, configured to amplify the output tensor to generate an amplified output tensor; and a convolution module, comprising at least one convolutional layer, wherein the convolution module is configured to process the amplified output tensor based on a plurality of second parameters to generate the upsampled tensor.
Similar reasoning applies to claim 12 and claim 19 which discloses similar subject matter to claim 4.
Regarding claim 6, Alshina in view of Ahmed discloses the image processing according to claim 1 as applied above. Alshina fails to disclose comprising an image quality detection module and a loading module, wherein the image quality detection module is configured to generate, based on the image, an index corresponding to an image quality classification of a film to which the image belongs, and the loading module is configured to obtain a plurality of image processing parameter values corresponding to the image quality classification from a memory module based on the index, and load the image processing parameter values into the image processing module.
Claim 7 is dependent on claim 6 and thus similar reasoning applies. Similar reasoning applies to claims 14-15 which disclose similar subject matter to claims 6-7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Appalaraju (US 10909728 B1) discloses lossy image compression including encoding and decoding to reconstruct an image, further including shuffle operations and training a system for image processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE DEPALMA whose telephone number is (571)270-0769. The examiner can normally be reached Mon-Thurs 9:00am-4pm Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE E. DEPALMA/Examiner, Art Unit 2675
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights