DETAILED ACTION
Notice of Pre-AIA or AIA Status.
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 1-20 filed on 11/22/2024 are pending and being examined. Claims 1, 9, and 13 are independent form.
Priority
3. Acknowledgment is made of applicant's claim for foreign priority under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Claim Interpretation
4. The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
4-1. Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function.
Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
4-2. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
4-3. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are: “a first acquisition module...” and “a processing module...” in claim 9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
5. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to non-statutory subject matter (an abstract idea without significantly more).
6-1. Regarding independent claim 1, the claim recites an image restoration method, the method comprising:
[1] acquiring a to-be-restored image, target text information corresponding to the to-be-restored image, and a target restoration type, wherein the target text information is used to describe the to-be-restored image; and
[2] inputting the to-be-restored image, the target text information and the target restoration type into an image restoration model for image restoration processing to obtain a restored target image corresponding to the to-be-restored image,
[3] wherein the image restoration model is obtained by training sub-restoration models corresponding to different restoration types.
Step 1:
With regard to step (1), claim 1, is directed to an image restoration method. The claim 1 therefore is one of statutory categories of invention, i.e., a process.
Step 2A-1:
With regard to 2A-1, The elements recited in claim 1, as drafted, under their broadest reasonable interpretation, encompass a process(es) which fall(s) within mathematical concepts. For example, “inputting the to-be-restored image, the target text information and the target restoration type into an image restoration model for image restoration processing to obtain a restored target image corresponding to the to-be-restored image” in step [2] in the context of this claim, encompasses mathematical calculations and fall within the “mathematical concepts” grouping of abstract ideas, even though “the image restoration model is obtained by training sub-restoration models corresponding to different restoration types” as recited in step [3]. In other words, even though the image restoration model is recited by training sub-restoration models corresponding to different restoration types, the image restoration model mere generally encompasses mathematical calculations (i.e., abstract idea) without limiting how the trained model functions. Claim 1 therefore recites an abstract idea. If a claim limitation is directed to organizing human activity, can be practically performed in human mind, or falls within mathematical concepts, then the claim recites an abstract idea. See MPEP 2106.04(a)(2).
Step 2A-2:
The 2019 PEG defines the phrase "integration into a practical application" to require an additional element or a combination of additional elements in the claim to apply, rely on, or use the judicial exception. In the instant case, the additional elements of “acquiring a to-be-restored image, target text information corresponding to the to-be-restored image, and a target restoration type” in step [1] under their broadest reasonable interpretation, are mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. Therefore, claim 1 as a whole does not integrate the judicial exception into a practical application.
Step 2B:
As explained above, the “acquiring a to-be-restored image, target text information corresponding to the to-be-restored image, and a target restoration type” in step [1] was considered insignificant extra-solution activity. These conclusions should be reevaluated in Step 2B. The limitations are mere data gathering and/or output recited at high level of generality and amount to receiving (i.e., acquiring), accessing, or transmitting data over a network, which is well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II. The limitations remain insignificant extra-solution activity even upon reconsideration. Even when considered in combination, the additional elements present mere instructions to apply an exception and insignificant extra-solution activity, which cannot provide an inventive concept. The claim therefore is ineligible.
6-2. Regarding dependent claims 2-8, they are dependent from claim 1 and viewed individually, these additional elements are under its broadest reasonable interpretation, either covers performance of the limitation in the mind, performing a mathematical algorithm or extra solution activity for data gathering and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. And, when the claims are viewed as a whole, they do not improve a technology by allowing the technology to perform a function that it previously was not capable of performing; and they do not provide any limitations beyond generally linking the use of the abstract idea to a broad technological environment (i.e., computer-based analysis of generic data). Hence, the claimed invention does not constitute significantly more than the abstract idea, so the claims are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
6-3. Regarding independent claims 9 and 13, the claims recite a device using generic unit(s)-plus-function language (claim 9) and an electronic device comprising a processor and a memory (claim 13) and each of which is analogous to apparatus claim 1, grounds of rejection analogous to those applied to claim 1 are applicable to claims 9 and 13. Furthermore, the claim is a method that does not recite any additional elements, and according to step 2A-2 does not integrate the abstract idea into a practical application because it does not recite any additional elements that impose any meaningful limits on practicing the abstract idea. The claim recites an abstract idea.
Because the claim fails under (2A), the claim is further evaluated under (2B). The claim herein does not include any additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
6-4. Regarding dependent claims 10-12 and 14-20, they are either dependent from claim 9 or dependent from claim 13 and viewed individually, these additional elements are under its broadest reasonable interpretation, either covers performance of the limitation in the mind, performing a mathematical algorithm or extra solution activity for data gathering and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. And, when the claims are viewed as a whole, they do not improve a technology by allowing the technology to perform a function that it previously was not capable of performing; and they do not provide any limitations beyond generally linking the use of the abstract idea to a broad technological environment (i.e., computer-based analysis of generic data). Hence, the claimed invention does not constitute significantly more than the abstract idea, so the claims are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 112
7. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
8. Claim 8 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre- AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Specifically, it is not clear to which claim 8 is dependent from. However, for purpose of examination, the examiner is interpreting that claim 8 is dependent from claim 5.
Claim Rejections - 35 USC § 102
9. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
11. Claims 1, 9, and 13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Qi et al (“TIP: Text-Driven Image Processing with Semantic and Restoration Instructions”, arXiv, 2023, hereinafter “Qi”).
Regarding claim 1, Qi discloses an image restoration method (see the title and fig.2), the method comprising:
acquiring a to-be-restored image (see the “input image y” of fig.2), target text information corresponding to the to-be-restored image (see the “semantic prompt” input of fig.2), and a target restoration type (see the “restoration prompt” input of fig.2), wherein the target text information is used to describe the to-be-restored image (wherein the input image is about “a very large giraffe eating leaves” described by the semantic prompt); and
inputting the to-be-restored image, the target text information and the target restoration type into an image restoration model for image restoration processing to obtain a restored target image corresponding to the to-be-restored image (input the degraded image y, semantic prompt cs, and restoration prompt cr into the text-driven image processing (TIP) framework (i.e., the text-driven image restoration model
p
z
t
y
,
c
s
,
c
r
, see sec.3.2 and Sec. 3.3) and then the TIP frameworks outputs the restoration image
x
^
corresponding to the degraded image y; see fig.2 and Sec. 3.2, para.1), wherein the image restoration model is obtained by training sub-restoration models corresponding to different restoration types (wherein the text-driven image restoration model can trained using paired training data (x or z0, {y, cs, cr}), see Sec. 3.2, the 2nd paragraph in right col. on page 4; wherein degradation (or restoration) types can be 4 different types shown by the second row in Table 2, see “Our parameterized degradation pipeline” in the left col. on page 6).
Regarding claim 9, 13, each of them is an inherent variation of claim 1, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 1.
Claim Rejections - 35 USC § 103
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. Claim 2-8, 10-12, and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Qi in view of Marivani et al (“Designing CNNs for Multimodal Image Restoration and Fusion via Unfolding the Method of Multipliers”, 2022, hereinafter “Marivani”).
Regarding claim 2, 10, 14, Qi discloses, wherein inputting the to-be-restored image, the target text information and the target restoration type into the image restoration model for image restoration processing to obtain the restored target image corresponding to the to-be-restored image comprises: inputting the to-be-restored image, the target text information and the target restoration type into the image restoration model; if a quantity of target restoration types is one, inputting the to-be-restored image and the target text information into the sub-restoration model corresponding to the target restoration type to obtain the restored target image corresponding to the to-be-restored image (see fig.6, wherein each of the text-driven image restoration models was trained to remove one single target degradation type. For example, the text-driven image restoration model shown in the top row was trained to “remove all degradation” while that shown in the second row was trained to “denoise with sigma 0.006”); and if the quantity of target restoration types is more than one, inputting the to-be-restored image and the target text information respectively into a sub-restoration model corresponding to each target restoration type to obtain multiple restored images respectively output by sub-restoration models (see fig.9, wherein each of the text-driven image restoration models was trained at the same time to remove two different target degradation types with two different strengths. For example, the restoration image shown in the second row and the second column was the fusing result outputted by the model trained by both denoising degradation and deblurring degradation with the different weights sigma 0.06 and sigma 1.5, respectively).
Qi does not explicitly disclose “fusing the multiple restored images according to a weight corresponding to each target restoration type to obtain the restored target image corresponding to the to-be-restored image” as recited by claim 2. However, in the same field of endeavor, Marivani teaches a method for fusing two images with two different weights to obtain a high quality image. See sec. V-D on page 5837. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Marivani into the teachings of Qi and fuse the output after the denoising degradation and the output after the deblurring degradation. Suggestion or motivation for doing so would have been to reconstruct “a high quality image” as taught by Marivani, cf., sec. V-D, 1st paragraph. Therefore, the claim is unpatentable over Qi in view of Marivani.
Regarding claim 3, 11, 15, the combination of Qi and Marivani discloses, wherein fusing the multiple restored images according to the weight corresponding to each target restoration type to obtain the restored target image corresponding to the to-be-restored image comprises: for each pixel at a same pixel position in the multiple restored images, obtaining a weighted target pixel value according to a pixel value of the pixel and a weight corresponding to the pixel; and obtaining the restored target image corresponding to the to-be-restored image according to the target pixel value at each pixel position (Marivani, see Sec. V-D—“Image Fusion”).
Regarding claim 4, 12, 16, the combination of Qi and Marivani discloses, wherein the image restoration model is trained by: acquiring a sample image, sample text information corresponding to the sample image, a quantity of sample restoration type, and a sample restoration image, and the sample text information is used to describe the sample image; if the quantity of sample restoration type corresponding to the sample image is one, inputting the sample image and the sample text information into a sub-restoration model corresponding to a sample restoration type to obtain a restored target sample image, and training the sub-restoration model according to the restored target sample image and the sample restoration image to obtain the image restoration model (Qi, see fig.6, wherein each of the text-driven image restoration models was trained to remove one single target degradation type. For example, the text-driven image restoration model shown in the top row was trained to “remove all degradation” while that shown in the second row was trained to “denoise with sigma 0.006”); and if the quantity of sample restoration type is more than one, inputting the sample image and the sample text information respectively into a sub-restoration model corresponding to each sample restoration type to obtain multiple restored target sample images respectively output by sub-restoration models (Qi, see fig.9, wherein each of the text-driven image restoration models was trained at the same time to remove two different target degradation types with two different strengths. For example, the restoration image shown in the second row and the second column was the fusing result outputted by the model trained by both denoising degradation and deblurring degradation with the different weights sigma 0.06 and sigma 1.5, respectively), fusing the multiple restored target sample images according to a weight corresponding to each sample restoration type to obtain a fused image, and based on a difference between the fused image and the sample restoration image, adjusting weights corresponding to multiple sample restoration types to obtain the image restoration model (Marivani, see Sec. V-D, fusing two images with two different weights to obtain a high quality image.).
Regarding claim 5, 17, the combination of Qi and Marivani discloses, wherein the sub-restoration model is trained by: inputting the sample image into an encoder network contained in an initial sub-restoration model to extract image features to obtain an image feature sequence corresponding to the sample image; inputting the sample text information into a trained word vector model to obtain a semantic feature sequence corresponding to the sample text information; performing stitching and fusing on the image feature sequence and the semantic feature sequence to obtain a target feature sequence; sending the target feature sequence to a decoder network for image restoration processing to obtain the restored target sample image; and iteratively training the initial sub-restoration model based on a loss function until a function value of the loss function is less than a loss threshold, thereby obtaining the trained sub-restoration model, wherein the loss function is used to represent a difference between the restored target sample image and the sample restoration image (Qi, see fig.2 and section 3: the TIP framework is shown by fig.2, wherein the TIP framework is trained by minimizing the Gaussian noise”).
Regarding claim 6, 18, the combination of Qi and Marivani discloses, wherein inputting the sample image into the encoder network contained in the initial sub-restoration model to extract image features to obtain the image feature sequence corresponding to the sample image comprises: performing convolution processing on the sample image through a convolution layer contained in the encoder network to obtain a sample feature map corresponding to the sample image; performing a linear transformation on the sample feature map based on an activation function contained in the encoder network to obtain a linearly transformed sample feature map; and performing pooling on the linearly transformed sample feature map to obtain the image feature sequence corresponding to the sample image (Qi, see fig.2 and section 3, wherein the TIP framework comprises encoder and decoder which are trained by image samples).
Regarding claim 7, 19, the combination of Qi and Marivani discloses the image restoration method according to claim 5, wherein sending the target feature sequence to the decoder network for image restoration processing to obtain the restored target sample image comprises: performing image restoration processing on the target feature sequence by using a fully connected layer contained in the decoder network to obtain the restored target sample image (Qi, see fig.2 and section 3, wherein the TIP framework comprises encoder and decoder which are trained by image samples).
Regarding claim 8, 20, the combination of Qi and Marivani discloses, wherein restoration types include at least one of image noise, image blur, image occlusion and image loss, and the target text information is information obtained after a user describes the to-be-restored image (Qi, see “Restoration prompt: Deblur with sigma 0.4...” in fig.2).
Conclusion
14. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Li et al, US 12530872 B2.
15. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. The examiner can normally be reached 8:30am--5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached on (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov; https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center, and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUIPING LI/Primary Examiner, Ph.D., Art Unit 2676
Prosecution Insights