DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to communications filed on 12/07/2022.
Claims 1-20 have been canceled.
Claims 21-40 are pending and have been examined.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Information Disclosure Statement
The information disclosure statement (IDS) submitted was filed on 10/14/2022. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21-40 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per claim 21, there is lack of antecedent basis for “the one or more discriminator parameters” in line 8. There is lack of antecedent basis for “the discriminator module” in lines 9-10. Independent claims 28 and 34 also recite the same limitations and therefore have the same issues. Due at least to their dependency upon claims 21, 28, or 34, dependent claims 22-27, 29-33, and 35-40 are also indefinite.
As per claim 22, there is lack of antecedent basis for “the refinement function” in line 2. This similarly applies to claims 23, 29-30, and 35-36.
As per claim 24, there is lack of antecedent basis for “said receiving” in line 2. This similarly applies to claims 25, 31-32, and 37-38.
As per claim 26, there is lack of antecedent basis for “the unrefined synthetic data” in line 4. The term “better” in line 3 is a relative term which renders the claim indefinite. The term “better” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. What is considered “better” varies depending on context, person, etc. As such, claim 26 is indefinite. The above similarly applies to claims 39.
As per claim 27, there is lack of antecedent basis for “the synthesized data” in lines 3-4. This similarly applies to claims 33 and 40.
Further as per claim 28, it is unclear whether “the processors” in lines 2 and 3 are referring to the “one or more processors”, or is different. Due at least to their dependency upon claim 28, dependent claims 29-33 are also indefinite (note that “the processors” is used in these claims as well).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21-40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a method, system, and medium comprising modifying, classifying, and adjusting.
The limitations “modifying… classifying… adjusting…” as recited in claim 21 are each a process, under the broadest reasonable interpretation, covering performance of the limitations in the mind or by pen and paper (See Berkheimer v. HP, Inc., 881 F.3d 1360, 1366, 125 USPQ2d 1649 (Fed. Cir. 2018)) but for the recitation of generic computer components. That is, other than reciting “according to a refiner neural network” and “according to a neural network”, the limitation “modifying synthetic data that approximates real data to produce refined synthetic data, wherein the synthetic data is modified…and based at least in part on one or more refinement parameters” in the context of the claim encompasses the user making evaluations using the mind or pen and paper. The limitation “classifying the refined synthetic data as either synthetic or real…and based in part on one or more parameters” in the context of the claim encompasses the user making judgements in the mind. The limitation “adjusting, based on said classifying, at least one of the one or more refinement parameters and at least one of the one or more discriminator parameters to improve realism of the synthetic data and to improve the classifying by the discriminator module” in the context of the claim encompasses the user making evaluations in the mind or pen and paper. If a claimed limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “mental processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements. The claim recites “according to a refiner neural network” and “according to a neural network”. The elements are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. see MPEP 2106.05(f)), and/or amounts to generally linking the use of the judicial exception to a particular technological environment or field of use (e.g. see MPEP 2106.05(h)). Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements are no more than mere instructions to apply the exception using a generic computer component, and/or amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. Therefore, the claim is not patent eligible.
Claims 28 and 34 also recite similar claim language as claim 21, and thus have the same issues. It is noted, with respect to claim 28, that the claim recites “one or more processors” and “a memory coupled to the processors” to perform the method. The elements are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. See MPEP 2106.05(f)). It is noted, with respect to claim 34, that the claim further recites “non-transitory, computer-readable storage medium storing program instructions” to perform the method. The elements are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. See MPEP 2106.05(f)). Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea and do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 22, the claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception. For example, the claim merely further describes the adjusting, which is part of the mental steps and do not include additional elements. This similarly applies to claims 29 and 35.
Regarding claim 23, the claim does not include any additional elements that are sufficient to amount to significantly more than the judicial exception. For example, the claim merely further describes the adjusting, which is part of the mental steps and do not include additional elements. This similarly applies to claims 30 and 36.
Regarding claim 24, the claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception. For example, the claim recites training the refiner neural network, which is recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. see MPEP 2106.05(f)), and/or amounts to generally linking the use of the judicial exception to a particular technological environment or field of use (e.g. see MPEP 2106.05(h)). This similarly applies to claims 31 and 37.
Regarding claim 25, the claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception. For example, the claim merely further describes the training, which amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. see MPEP 2106.05(f)), and/or amounts to generally linking the use of the judicial exception to a particular technological environment or field of use (e.g. see MPEP 2106.05(h)). This similarly applies to claims 32 and 38.
Regarding claim 26, the claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception. For example, the claim further recites applying the refined synthetic data as training input to a machine learning system, which amounts to no more than mere instructions to apply the exception using a generic computer component (e.g. see MPEP 2106.05(f)), and/or amounts to generally linking the use of the judicial exception to a particular technological environment or field of use (e.g. see MPEP 2106.05(h)). This similarly applies to claim 39.
Regarding claim 27, the claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception. For example, the claim merely further describes how the synthetic data is generated, which is part of the mental steps and do not include additional elements. This similarly applies to claims 33 and 40.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 21-40 are rejected under 35 U.S.C. 103 as being unpatentable over Okanohara et al. (US 20180365089 A1) in view of Shi et al. (US 20180075581 A1).
As per independent claim 1, Okanohara teaches a computer implemented method, comprising:
modifying synthetic data that approximates real data to produce refined synthetic data (e.g. in paragraphs 106 and 112, “expression z.sub.0 of the input data is inferred based on the latent variable model, and on the other hand, sampling is performed from the sample generator (Sampler) to produce a false expression z.sub.1… Based on z.sub.0 and z.sub.1, training data (z.sub.0, 0), (z.sub.1, 1) to be input to the discriminator are created… probability that the input is real data… the parameters of the encoder are updated and learned…improves the accuracy of inference in the encoder… learning is repeated so that the training data x and restored data x˜ coincide with each other”, i.e. approximates real data), wherein the synthetic data is modified according to a refiner neural network and based at least in part on one or more refinement parameters (e.g. in paragraphs 83 and 106, “learning…at the encoder and…at the decoder, a neural network is used… expression z.sub.0 of the input data is inferred based on the latent variable model, and on the other hand, sampling is performed from the sample generator (Sampler) to produce a false expression z.sub.1… Based on z.sub.0 and z.sub.1, training data (z.sub.0, 0), (z.sub.1, 1) to be input to the discriminator are created… the parameters of the encoder are updated and learned [i.e. refinement parameters]… improves the accuracy of inference in the encoder”);
determining data related to the refined synthetic data as synthetic or real according to a neural network and based in part on one or more parameters (e.g. in paragraphs 45, 106, and 116, “updates the parameter of the discriminator... The learned discriminator outputs the probability that the input is real data” and figure 8 showing “neural network”); and
adjusting, based on said determining, at least one of the one or more refinement parameters and at least one of the one or more discriminator parameters to improve realism of the synthetic data and to improve the classifying by the discriminator module (e.g. in paragraphs 105-106, “calculate a reconstruction error. Then, by referring to the reconstruction error, the parameters of the encoder and decoder are updated, for example, by the stochastic gradient descent method so as to reduce the reconstruction loss… discriminator first updates the parameter of the discriminator. The discriminator then updates the parameters of the encoder to confuse the network for discrimination at the discriminator… regularization error is obtained in the process of distinguishing between normal data and false data in the discriminator, and not only the discriminator but also the parameters of the encoder are updated and learned using the regularization error, so that this improves the accuracy of inference in the encoder and improves the discrimination accuracy of the discriminator”),
but does not specifically teach wherein determining data includes classifying as either synthetic or real.
However, Shi teaches determining including classifying data as either synthetic or real (e.g. in paragraphs 13, 26, and 48, “discriminating of the super-resolved image data from real image data can include using a binary classifier that discriminates between the super-resolved image data and reference data… goal of fooling a differentiable discriminator D that was trained to distinguish super-resolved images from real images”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Okanohara to include the teachings of Shi because one of ordinary skill in the art would have recognized the benefit of determining whether the discriminator is fooled.
As per claim 22, the rejection of claim 21 is incorporated and the combination further teaches wherein said adjusting comprises applying an adversarial cost value to the refinement function, wherein the adversarial cost value is based at least in part on said classifying (e.g. Okanohara, in paragraphs 105-106, “calculate a reconstruction error. Then, by referring to the reconstruction error, the parameters of the encoder and decoder are updated, for example, by the stochastic gradient descent method so as to reduce the reconstruction loss… regularization error is obtained in the process of distinguishing between normal data and false data in the discriminator, and not only the discriminator but also the parameters of the encoder are updated and learned using the regularization error”; Shi, in paragraph 26, “an adversarial loss… differentiate between the super-resolved images and original photo-realistic images”).
As per claim 23, the rejection of claim 21 is incorporated and the combination further teaches wherein said adjusting comprises applying a self-regularization term to the refinement function, wherein the self-regularization term comprises a difference value based on a comparison between the synthetic data and refined synthetic data (e.g. Okanohara, in paragraphs 18 and 105-108, “a deviation calculation step of calculating deviation between the training data and the restored data; and a parameter updating step of updating parameters of the latent variable model and the joint probability model based on the deviation… learning at regularization stage”; Shi, in paragraphs 26, 41, 54, and 63-64, “capable of recovering photo-realistic images from 4X downsampled images… content loss… Euclidean distance between the last convolutional feature maps… regularization loss”).
As per claim 24, the rejection of claim 21 is incorporated and the combination further teaches training the refiner neural network, wherein said training comprises repeating said receiving, said modifying, said classifying and said adjusting (e.g. Okanohara, in paragraphs 106 and 112, “learning at regularization stage, discriminators are learned, and at that time, learning is also performed for encoders… learning is repeated so that the training data x and restored data x˜ coincide with each other”; Shi, in paragraph 85, “method may iterate, seeking to reduce the differences between the plurality of characteristics determined from the high-quality visual data 730 and the estimated enhanced quality visual data”).
As per claim 25, the rejection of claim 24 is incorporated and the combination further teaches wherein said training comprises repeating said receiving, said modifying, said classifying and said adjusting until the refined synthetic data is classified as real (e.g. Okanohara, in paragraphs 106 and 112, “learning at regularization stage, discriminators are learned, and at that time, learning is also performed for encoders… probability that the input is real data… learning is repeated so that the training data x and restored data x˜ coincide with each other”; Shi, in paragraph 85, “method may iterate, seeking to reduce the differences between the plurality of characteristics determined from the high-quality visual data 730 and the estimated enhanced quality visual data”, i.e. causing classified as real).
As per claim 26, the rejection of claim 21 is incorporated and the combination further teaches applying the refined synthetic data as training input to a machine learning system, wherein the machine learning system performs better when trained using the refined synthetic data than when trained using the unrefined synthetic data (e.g. Okanohara, in paragraph 106, “discriminators are learned, and at that time, learning is also performed for encoders… Based on z.sub.0 and z.sub.1, training data (z.sub.0, 0), (z.sub.1, 1) to be input to the discriminator are created… parameters of the encoder are updated and learned using the regularization error, so that this improves the accuracy of inference in the encoder”, i.e. performs better; Shi, in paragraphs 26, 72, and 85, “perceptually better the result of the GAN system”).
As per claim 27, the rejection of claim 21 is incorporated and the combination further teaches generating the synthetic data based on one or more key attributes describing the synthesized data (e.g. Okanohara, in paragraph 106, “expression z.sub.0 of the input data is inferred based on the latent variable [i.e. key attributes] model”; Shi, in paragraph 59, “feature space representation… important details (e.g., textures, high frequency information)”).
Claims 28-33 are the system claims corresponding to method claims 21-25 and 27, and the combination further teaches one or more processors and a memory coupled to the processors, wherein the memory comprises program instructions configured to cause the processors to perform the method (e.g. Okanohara, in paragraph 142, “CPU 100 reads the program for each function of the monitoring control system stored in the auxiliary storage device 105 to the RAM 101, the control unit executes the program read to the RAM”).
Claims 34-40 are the medium claims corresponding to method claims 21-27, and the combination further teaches a non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to perform the method (e.g. Okanohara, in paragraphs 142-143, “RAM… optical disk such as a CD-ROM”, etc.).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 21-40 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 11475276. Although the claims at issue are not identical, they are not patentably distinct from each other because they recite similar subject matter that is obvious over one another. Correspondence is shown in the following table:
Present application
U.S. Patent No. 11475276
Claims 21, 28, and 34
Claims 1, 8, and 14
Claims 22, 29, and 35
Claims 2, 9, and 15
Claims 23, 30, and 36
Claims 3, 10, and 16
Claims 24, 31, and 37
Claims 4, 11, and 17
Claims 25, 32, and 38
Claims 5, 12, and 18
Claims 26 and 39
Claims 6 and 19
Claims 27, 33, and 40
Claims 7, 13, and 20
For example, the claims of ‘276 recite features of the claims of the present application, but does not specifically describe “adjusting… at least one of the one or more discriminator parameters… to improve the classifying by the discriminator module”. However, prior art (e.g. see Okanohara above) teaches adjusting at least one of one or more discriminator parameters to improve classifying by a discriminator module (e.g. Okanohara, in paragraphs 105-106, “calculate a reconstruction error. Then, by referring to the reconstruction error, the parameters of the encoder and decoder are updated, for example, by the stochastic gradient descent method so as to reduce the reconstruction loss… discriminator…updates the parameter of the discriminator… regularization error is obtained in the process of distinguishing between normal data and false data in the discriminator, and not only the discriminator but also the parameters of the encoder are updated and learned using the regularization error, so that this improves the accuracy of inference in the encoder and improves the discrimination accuracy of the discriminator”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of ‘276 to include the teachings of the prior art (e.g. see Okanohara above) because one of ordinary skill in the art would have recognized the benefit of improving accuracy.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For example,
SALIMANS, et al. ("Improved Techniques for Training GANs", arXiv:1606.03498v1 [cs.LG], June 10, 2016, Pages 1-10, cited in IDS dated 10/14/2022) teaches “goal of GANs is to train a generator network G(z; θ(G)) that produces samples from the data distribution, pdata(x), by transforming vectors of noise z as x = G(z; θ(G)). The training signal for G is provided by a discriminator network D(x) that is trained to distinguish samples from the generator distribution pmodel(x) from real data. The generator network G in turn is then trained to fool the discriminator into accepting its outputs as being real… trained using gradient descent techniques that are designed to find a low value of a cost function” (e.g. page 1).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM WONG whose telephone number is (571)270-1399. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TAMARA KYLE can be reached at (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.W/Examiner, Art Unit 2144 09/20/2025
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2144