Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendment have persuasively overcome all but one of the 112b rejections and the 102 rejection.
The remaining issues are addressed below.
Specification Objections
Applicant argues:
Applicant will submit a substitute title and a substitute abstract in a separate paper, or upon indication by the Examiner of an allowable claim set
Examiner responds:
This is not responsive. Additionally, the Office does not act on “preview” arguments.
101
Applicant argues:
These operations cannot practically be performed in the human mind. See MPEP § 2106.04(a)(2)(III)(C).
Examiner responds:
Applicant has not addressed the reasoning in the non-final.
Applicant argues:
That a person can look at two images and subjectively judge which appears sharper bears no meaningful resemblance to the specific, mathematically-defined training operations recited in the claims.
Examiner responds:
The various models are also mental processes, see example 47, claim 2, element (d) (from the July 2024 AI subject matter eligibility examples).
Applicant argues:
Here, by contrast, the encoder, generator, and discriminator are specifically-trained neural networks whose parameters …
Examiner responds:
These are software. As above (and as per the non-final), such neural networks are mental processes.
Applicant argues:
to generate synthetic images that require less storage space
Examiner responds:
See the 112(b)
Applicant argues:
and are generated without decoding or decompressing
Examiner responds:
This is not claimed.
Claim Objections
While the legal analysis below focuses on dependency, the underlying issue is whether Applicant needs to redraft claims such that they are correctly charged for additional independent claims.
Claim 9 references claim 1, but attempts to only import certain limitations (such as the generator and encoder, but not the discriminator), but not fully depend, and thus is not a proper dependent claim.
Claim 11 references claim 9, but only refers to the output of claim 9, rather than incorporating all of the limitations of claim 9.
Claim 31 references claim 12, but does not properly depend from claim 12 because the instructions can exist without performance of any of the method steps. Here, claim 12 is a method but claim 31 is an apparatus, and the apparatus claim can be met without necessarily practicing the method.
Claim 34 references claim 32, but does not properly depend from claim 32 because the instructions can exist without performance of any of the method steps. Here, claim 32 is a method but claim 34 is an apparatus, and the apparatus claim can be met without necessarily practicing the method.
Claim 10 references claim 9.
MPEP 608.01(n)(III) addresses the “test for proper dependency.”
MPEP 607(III) states:
Any claim which is in dependent form but which is so worded that it, in fact, is not a proper dependent claim, as for example it does not include every limitation of the claim on which it depends, will be required to be canceled as not being a proper dependent claim; and cancellation of any further claim depending on such a dependent claim will be similarly required. The applicant may thereupon amend the claims to place them in proper dependent form, or may redraft them as independent claims, upon payment of any necessary additional fee.
Claims 9-11, 31, and 34 are such claims. MPEP 608.01(n)(III). While, in the interest of compact prosecution, claims 9-11, 31, and 34 has been examined, claims 9-11, 31, and 34 are required to be cancelled.
Claim 7 is objected to for having a period after “is inferred.”
Specification
The abstract of the disclosure is objected to because it does not “enable the Office and the public generally to determine quickly from a cursory inspection the nature and gist of the technical disclosure.” 37 CFR 1.72(b). Specifically, the abstract appears to describe basic Generative Adversarial Networks. See, e.g., https://en.wikipedia.org/w/index.php?title=Generative_adversarial_network&oldid=1141855224 (Generative adversarial networks, Wikipedia, February 27, 2023), and note the discussion of approximations (e.g., universal approximation teaches the claimed compression).
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 2, 4, 7, 9-13, 16, 17, 21, 24-27, 29, 31, 32, and 34 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, recite “wherein the generated synthetic image data is a pixel representation in an image space requiring less storage space than the original image data.” First, it is not grammatically clear what “requiring less storage space” modifies (e.g., is it the image data, the pixel representation, or the image space that requires less storage space). Second, whether a pixel representation requires less storage space depends on which pixel representation is used. For example, the size of a bitmap file is the number of bits per pixel times the number of pixels, and would not benefit from removing redundancy (as compared to the analysis in PG Pub [0003]). Similarly, representing an image as a JPEG versus a GIF results in different compression for a given image. Thus, specifying that less storage space is needed requires an objective determination of the amount of storage space, e.g., specifying a compression/storage algorithm.
Claim 11 is directed to a stored synthetic image. If this is a product by process limitation, it is not clear what structural limitations are implied. MPEP 2111.05. Otherwise, the scope of the claim is unclear.
Claim 12 recites details of how the training is performed, but it is unclear if the intent is that these limitations apply to all of the training, or just any portion of the training. For example, claim 12 recites “wherein the training comprises the steps of: receiving original image data.” Is the intent that at least one piece of original image data is used (and the rest is synthetic) or is the intent that the training uses original image data (and not synthetic). This issue applies to all of the recitations regarding training, such as dependent claim 29’s “training the system minimizes the loss function” – is the intent that the loss function is minimized at least once, or is the intent that this is the loss function for all of the training?
Claim 24 recites “images generated by the generator based on the latent variable.” It is unclear how to determine whether an image was generated based on a latent variable or not.
Dependent claims are likewise rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, 4, 7, 9-13, 16, 17, 21, 24-27, 29, 31, 32, and 34 (all claims) are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more.
Step 1: As above, many of the claims have improper dependencies, and thus their status for Step 1 is unclear.
Claims 1 and 9 recite systems, and machines satisfy Step 1 of the eligibility test.
Claim 12 (and its dependents) recite a method, and processes satisfy Step 1 of the eligibility test.
Step 2A, prong one: All of the elements of the claims are a mental process because a person can imagine a fuzzier or sharper version of an image, and decide if it’s real. Further, the various models are also mental processes, see example 47, claim 2, element (d) (from the July 2024 AI subject matter eligibility examples). MPEP 2106.04(a)(2)(III)(C) explains that use of a generic computer or in a computer environment is still a mental process. In particular, this section begins by citing Gottschalk v. Benson, 409 US 63 (1972). “The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea.” In Benson the Supreme Court did not separately analyze the computer hardware at issue; the specifics of what hardware was claimed is only included in an appendix to the decision.
Because there are no additional elements, no further analysis is required for Step 2A, prong two or Step 2B.
Note that MPEP 2106.05(f)(2) states “TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. … The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information).”
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 9-12, 25, 26, 31, 32, and 34 (the remaining claims re rejected as obvious, below) are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wu L, Huang K, Shen H. A gan-based tunable image compression system. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) 2020 Mar 1 (pp. 2323-2331). IEEE. (“Wu”)
1. (Currently Amended) A training system for training computer implemented generation of synthetic image data that is a compressed version of original image data, comprising a hardware processing unit configured to execute: (Wu, title, “A GAN-based Tunable Image Compression System”)
- an encoder configured to encode original image data into a latent space representation, (Wu, section 3.1, subheading “Encoder”)
- a generator configured to generate synthetic image data based on a latent variable describing a distribution of the latent space representation, and (Wu, section 3.1, subheading “Decoder” and “In fact, the decoder is also the generator in our GAN based system.”)
- a discriminator configured to evaluate the generated synthetic image data as to its authenticity, (Wu, section 3.1, subheading “Discriminator” and “The discriminator D(·) is able to identify the authenticity of the input image, i.e., whether it is the original image or the reconstructed image”)
wherein the generated synthetic image data is a pixel representation in an image space requiring less storage space than the original image data, thereby constituting a compressed version of the original image data. (Wu, abstract, “A tunable compression scheme is also proposed in this paper to compress an image to any specific compression ratio without retraining the model.”)
2. (Currently Amended) The training system according to claim 1, wherein the encoder comprises a machine learning model configured to encode the original image data into a latent space representation having a vector dimensionality that is smaller than a vector dimensionality of the original image data, (Wu, section 3.1, subheading “Encoder” and Fig. 3. Wu’s changing scales to further compress the image teaches the claimed lower vector dimensionality.)
wherein the discriminator comprises a machine learning model configured to determine if image data received by the discriminator is original image data or is image data generated by the generator, (Wu, section 3.1, subheading “Discriminator” and “The discriminator D(·) is able to identify the authenticity of the input image, i.e., whether it is the original image or the reconstructed image”)
wherein the discriminator is configured to feed a result of the evaluation back to the generator, wherein the generator comprises a machine learning model, (Wu, section 3.2, subheading “Adversarial Loss.” Minimizing the overall loss teaches the claimed feed a result of the evaluation.)
wherein the generator is configured to adapt parameters of its machine learning model dependent on the result of the evaluation. (Wu, section 3.1, subheading “Decoder” and “It improves its performance during the alternating training with the discriminator and generates images that the discriminator cannot identify the authenticity.”)
Claim 9 is rejected as per claim 1.
10. (Previously Presented) The system according to claim 9, comprising one or more of
- a storage, wherein the generator is configured to store the generated image data in the storage, and (Wu, section 5, Fig. 10. Having these images teaches that they were stored.)
- a transmission network, wherein the generator is configured to transmit the generated image data over the transmission network.
Claim 11 is rejected as per claim 10.
12. (Currently Amended) A computer-implemented method for training generation of synthetic image data that is a compressed version of original image data, in a training system according to claim 1, wherein the training comprises the steps of:
receiving original image data, and (Wu, section 3.2, “Now we train the model on a batch of B, that is XB={X(1),X(2),···,X(B)}, containing high-resolution images.”)
training the discriminator by the received original image data. (Wu, section 3.2, “Now we train the model on a batch of B, that is XB={X(1),X(2),···,X(B)}, containing high-resolution images.”)
25. (Currently Amended) The training method according to claim 12, comprising:
training the discriminator concurrently with training the encoder, and (Wu, section 3.1, subheading “Discriminator” and “As an important part of GAN, the discriminator D(·) is trained in parallel with the generator G(·) [3,20] to improve the performance of generating images.”)
training the generator concurrently with training the encoder. (Wu, section 3.1, subheading “Discriminator” and “As an important part of GAN, the discriminator D(·) is trained in parallel with the generator G(·) [3,20] to improve the performance of generating images.”)
26. (Previously Presented) The training method according to claim 12, wherein training the generator and discriminator comprises minimizing the loss functions:LG = Error (D(G(z)), 1),LD = Error (D(x), 1) + Error (D(G(z)), 0),wherein:LG is the generator loss,LD is the discriminator loss,Error is the error function,x is the original image data,z is the latent variable,G(z) is the generated synthetic data from the generator,D(G(z)) is the discriminator's evaluation of the generated synthetic image data,D(x) is the discriminator's evaluation of the original image data. (Wu, section 3.2, subheading “Overall Loss” equation 9)
Claim 31 is rejected as per claim 12.
Claim 32 is rejected as per claim 10.
Claim 34 is rejected as per claim 10.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 7, 13, 16, 17, 21, 24, 27, and 29 (the claims not rejected above) are rejected under 35 U.S.C. 103 as being unpatentable over Wu L, Huang K, Shen H. A gan-based tunable image compression system. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) 2020 Mar 1 (pp. 2323-2331). IEEE. (“Wu”) in view of Chen Y, Gao Q, Wang X. Inferential Wasserstein generative adversarial networks. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2022 Feb;84(1):83-11 (“Chen”).
4. (Currently Amended) The training system according to claim 2, wherein the processing unit is further configured
- to execute a generative adversarial network including the generator and the discriminator, (Wu, title, “A GAN-based Tunable Image Compression System”)
wherein the discriminator is a discriminator trained to evaluate the authenticity of image data received, to discriminate between original image data and synthetic image data, (Wu, section 3.1, subheading “Decoder” and “It improves its performance during the alternating training with the discriminator and generates images that the discriminator cannot identify the authenticity.”)
wherein the generator and the discriminator are entities trained to minimize a plurality of loss components of a loss function to improve a realism of the synthetic image data, (Wu, section 3.2, subheading “Adversarial Loss,” equation 7)
wherein the processing unit is configured to execute the encoder producing the latent space representation from the original image data, (Wu, section 5 (end of section), “On the Kodak dataset, to achieve MI-SSIM of 0.95, the average time to encode and decode the image is 21 ms and 29 ms, running on the GeForce GTX 1080 Ti.”)
Wu is not relied on for the below claim language.
However, Chen teaches wherein the generative adversarial network is a Wasserstein generative adversarial network. (Chen, title, “Inferential Wasserstein generative adversarial networks”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Chen to the teachings of Wu such that Chen’s iWGAN is used with Wu GAN for the purpose of achieving the benefits listed in Chen’s abstract.
Based on the above, this is an example of “combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
7. (Currently Amended) The training system according to claim 4, comprising
- an estimator configured to estimate parameters of the distribution of the latent space representation and configured to determine the latent variable based on the estimated parameters, (Chen, section 4, p. 93, “The iWGAN has proposed an efficient framework to stably and automatically estimate both the encoder and the generator.”)
- wherein the processing unit is configured to execute the estimator producing the parameters of the distribution from which the latent variable is inferred. (Chen, section 4, p. 95, equation 16)
wherein the estimator is configured to estimate the parameters of the distribution by applying a maximum likelihood estimation. (Chen, section 4, p. 95, “Therefore, the training of the iWGAN is to seek the MLE.”)
Claim 13 is rejected as per claim 7.
Claims 16, 17 and 21 are rejected as per claim 7.
24. (Currently Amended) The training method according to claim 17, comprising training the encoder independent from the discriminator, (Wu, section 3.1, subheading “Encoder” and Fig. 3.)
training the discriminator with images generated by the generator based on the latent variable describing a noise distribution. (Wu, section 3.1, subheading “Discriminator” and “The discriminator D(·) is able to identify the authenticity of the input image, i.e., whether it is the original image or the reconstructed image”)
27. (Currently Amended) The training method according to claim 12, comprising
applying a perceptual loss function configured to capture a similarity between the original image and the generated synthetic image, (Wu, section 4, “PSNR and MS-SSIM are used in this paper because they are more consistent with the actual perception of human vision [19].”)
wherein the perceptual loss function determines a loss between the activation of the original image data and the activation of generated synthetic image data. (Wu, section 5, “In terms of MS-SSIM and PSNR, our method is superior to conventional compression algorithms such as JPEG, JPEG2000 and BPG”)
Wu is not relied on for the below claim language.
However, Chen teaches applying a regularization loss function between the distribution of the latent representation produced by the encoder referred to as the posterior p(zlx) and the distribution obtained by the latent variable, referred to as the prior p(z), (Chen, section 4, p. 95, equation 17. Note the above discussion of regularization.)
wherein the regularization loss function includes the Wasserstein distance, (Chen, section 4, p. 95, equation 17. The discussion of the gradient teaches the claimed loss (because the gradient is used for training).)
Chen and Wu are combined as per claim 4.
Claim 29 is rejected as per claims 26 and 27.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200272903 A1 – title, “Machine-Learning Based Video Compression”
US 11166014 B2 – title, “Image Encoding And Decoding Method And Device Using Prediction Network”
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/Primary Examiner, Art Unit 2663
Prosecution Insights