Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
(b) CROSS-REFERENCES TO RELATED APPLICATIONS: See 37 CFR 1.78 and MPEP § 211 et seq.
The parent application of the instant invention, has now pass to issue, as a US Patent. Please update the specification. Correction is require.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of USPatent 12,17,096 in further view of the combination of Paraskevopoulos (20200335086) in view of Sargsyan et al (20200066296). Although the claims at issue are not identical, they are not patentably distinct from each other because the alternative language in the disclosed steps are not necessary to realize the functionality of the instant claims. As to the extra claim steps of details toward the type of transform/inverse transform, as well as the detailed frequency ranges, and differing applications to the end user, found in the claims in the instant invention, the combination of Paraskevopoulos (20200335086) in view of Sargsyan et al (20200066296) teaches ( Paraskevopoulos (20200335086), transforming the audio signal data into spectrograms – para 0009, 0010, as well as using the ‘general approach’ may be adapted to produce synthesized audio signals –para 0027; examiner notes that it is very old and notoriously well known, in the art of signal transformation, to use short-time Fourier transforms, and it’s inverses, in generating spectrograms from audio signals and likewise, generating audio signals from spectrograms -- see prior art listing at the end of the office action); further, Sargsyan et al (20200066296), as sending to the user to review/hear the processed audio signal – para 0025, in an end-user service, or in para 0026, showing the ability of the user to use the application to cancel noise in their audio/video before uploading; and further, Sargsyan et al (20200066296), teaching the expansion of low bitrate audio of 8kHz up to 44.1 kHz – para 0027. See mapping below.
As per claims 7,8,18, examiner notes that the combination of the ‘096 patent in view of Paraskevopoulos/Sargyan references meet the claim scope of these claims, as well as, mappings of the ‘096 patent to claims 1-6,9-16. See table below.
18/934075
12,170,096
1. A non-transitory medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising: identifying a first discrete audio signal to be upsampled from a first sampling rate to a second sampling rate; applying a transform to the first discrete audio signal to produce a first magnitude spectrogram; providing the first magnitude spectrogram to a generative adversarial network that is associated with the second sampling rate and that produces, as output, a second magnitude spectrogram, wherein the generative adversarial network generates the second magnitude spectrogram from the first magnitude spectrogram by adjusting a characteristic learned, during training, from analysis of multiple magnitude spectrograms, each of which is associated with a different discrete audio signal having the second sampling rate; and applying an inverse transform to the second magnitude spectrogram to produce a second discrete audio signal having the second sampling rate.
2. The non-transitory medium of claim 1, wherein the transform is a short-time Fourier transform (STFT), and wherein the inverse transform is an inverse short-time Fourier transform (ISTFT).
3. The non-transitory medium of claim 1, wherein the generative adversarial network includes a pair of neural networks that are trained using the multiple magnitude spectrograms that correspond to different discrete audio signals that have the second sampling rate.
4. The non-transitory medium of claim 1, wherein the operations further comprise: receiving, through an interface, first input that is indicative of a selection of the first discrete audio signal; and receiving, through the interface, second input that is indicative of a selection of the second sampling rate.
5. The non-transitory medium of claim 1, wherein the operations further comprise: posting, to an interface, an indication that the second discrete audio signal having the second sampling rate has been produced.
6. The non-transitory medium of claim 5, wherein the operations further comprise: allowing a user to initiate playback of the second discrete audio signal through the interface.
7. The non-transitory medium of claim 1, wherein said applying further produces a phase for the first discrete audio signal, and wherein the inverse transform is applied to the second magnitude spectrogram and the phase to produce the second discrete audio signal.
8. The non-transitory medium of claim 1, wherein the second sampling rate is at least double the first sampling rate.
9. The non-transitory medium of claim 1, wherein the operations further comprise: receiving, through an interface, first input that is indicative of a selection of the first discrete audio signal by a user; and storing the second discrete audio signal in a database that is associated with a media compilation on which the user is working through the interface.
10. The non-transitory medium of claim 1, wherein the operations further comprise: storing the second discrete audio signal in a database that is associated with a media compilation with which the first discrete audio signal is determined to be associated.
11. The non-transitory medium of claim 10, wherein the operations further comprise: determining that the first discrete audio signal is associated with the media compilation by— comparing words uttered in the first discrete audio signal to transcripts that are associated with different media compilations, and establishing that the words most closely match a transcript that is associated with the media compilation.
12. A method comprising: determining that (i) a first discrete audio signal that has a first sampling rate is to be upsampled to a given sampling rate, and (ii) a second discrete audio signal that has a second sampling rate is to be upsampled to the given sampling rate; applying a Fourier transform to the first and second discrete audio signals to produce a first magnitude spectrogram for the first discrete audio signal and a second magnitude spectrogram for the second discrete audio signal; providing the first magnitude spectrogram to a first neural network that is associated with the first sampling rate and that produces, as output, a third magnitude spectrogram; providing the second magnitude spectrogram to a second neural network that is associated with the second sampling rate and that produces, as output, a fourth magnitude spectrogram; and applying an inverse Fourier transform to the third and fourth magnitude spectrograms to produce a third discrete audio signal having the given sampling rate based on the third magnitude spectrogram and a fourth discrete audio signal having the given sampling rate based on the fourth magnitude spectrogram.
13. The method of claim 12, further comprising: receiving, via an interface, input that is indicative of a selection of the first and second discrete audio signals.
14. The method of claim 12, further comprising: acquiring the first discrete audio signal from a first source; acquiring the second discrete audio signal from a second source that is different than the first source; and discovering that the first sampling rate of the first discrete audio signal and the second sampling rate of the second discrete audio signal fall beneath a threshold; wherein said determining is performed in response to said discovering.
15. The method of claim 12, further comprising: generating an interface through which playback of the third and fourth discrete audio signals is initiable.
16. The method of claim 12, wherein the first and second neural networks are part of generative adversarial networks that are trained in an unsupervised manner to output magnitude spectrograms corresponding to the given sampling rate.
17. A non-transitory medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising: applying a transform to a first discrete audio signal that is to be upsampled from a first sampling rate to a second sampling rate, so as to produce a first magnitude spectrogram and a phase; providing the first magnitude spectrogram to a generative model that is associated with the second sampling rate and that produces, as output, a second magnitude spectrogram, wherein the generative model generates the second magnitude spectrogram from the first magnitude spectrogram by adjusting a characteristic learned, during training, from analysis of multiple magnitude spectrograms, each of which is associated with a different discrete audio signal having the second sampling rate; and applying an inverse transform to the second magnitude spectrogram and the phase that corresponds to the first magnitude spectrogram, so as to produce a second discrete audio signal that has the second sampling rate.
18. The non-transitory medium of claim 17, wherein the second sampling rate is at least 40,000 hertz.
19. The non-transitory medium of claim 17, wherein the operations further comprise: examining a database to identify the generative model from among multiple generative models that are associated with different sampling rates.
20. The non-transitory medium of claim 17, wherein the operations further comprise: posting a visualization of the second discrete audio signal to an interface, through which playback of the second discrete audio signal is initiable.
. A method for implementing a generative adversarial network, the method comprising: receiving, by a processor, input that is indicative of a selection of a first discrete audio signal to be upsampled from a first sampling rate to a second sampling rate; examining, by the processor, a database that includes multiple generative adversarial networks associated with different sampling rates, so as to identify the generative adversarial network that is associated with the second sampling rate; applying, by the processor, a transform to the first discrete audio signal to produce a first magnitude spectrogram; providing, by the processor, the first magnitude spectrogram to the generative adversarial network as input so as to produce a second magnitude spectrogram, wherein the second magnitude spectrogram is generated from the first magnitude spectrogram by the generative adversarial network by adjusting a characteristic learned, during training, from analysis of multiple magnitude spectrograms, each of which is associated with a different discrete audio signal having the second sampling rate; and applying, by the processor, an inverse transform to the second magnitude spectrogram to produce a second discrete audio signal having the second sampling rate.
5. The method of claim 1, wherein the transform is a short-time Fourier transform (STFT).
6. The method of claim 5, wherein the inverse transform is an inverse short-time Fourier transform (ISTFT).
2. The method of claim 1, wherein the generative adversarial network includes a pair of neural networks that are trained using the multiple magnitude spectrograms that correspond to different discrete audio signals that have the second sampling rate.
The method of claim 1, further comprising: receiving, by the processor, second input indicative of a selection of the second sampling rate.
4. The method of claim 1, further comprising: receiving input indicative of an identification of a media compilation with which the first discrete audio signal is associated; and storing the second discrete audio signal in a database that is associated with the media compilation.
8. The method of claim 7, wherein playback of the second discrete audio signal is initiable through the network-accessible interface.
7. The method of claim 1, further comprising: posting a visual representation of the second discrete audio signal to a network-accessible interface.
8. The method of claim 7, wherein playback of the second discrete audio signal is initiable through the network-accessible interface.
4.The method of claim 1, further comprising: receiving input indicative of an identification of a media compilation with which the first discrete audio signal is associated; and storing the second discrete audio signal in a database that is associated with the media compilation.
9. A method comprising: determining, by a processor, that (i) a first discrete audio signal having a first sampling rate and acquired from a first source is to be upsampled to a given sampling rate, and (ii) a second discrete audio signal having a second sampling rate and acquired from a second source is to be upsampled to the given sampling rate; applying, by the processor, a Fourier transform to the first and second discrete audio signals to produce a first magnitude spectrogram for the first discrete audio signal and a second magnitude spectrogram for the second discrete audio signal; acquiring, by the processor, a first generative model that includes a first neural network trained to facilitate upsampling from the first sampling rate to the given sampling rate, and a second generative model that includes a second neural network trained to facilitate upsampling from the second sampling rate to the given sampling rate; providing, by the processor, the first magnitude spectrogram to the first generative model as input so as to produce a third magnitude spectrogram; providing, by the processor, the second magnitude spectrogram to the second generative model as input so as to produce a fourth magnitude spectrogram; and applying, by the processor, an inverse Fourier transform to the third and fourth magnitude spectrograms to produce a third discrete audio signal having the given sampling rate for the third magnitude spectrogram and a fourth discrete audio signal having the given sampling rate for the fourth magnitude spectrogram.
10. The method of claim 9, further comprising: receiving, by the processor, input indicative of an instruction to upsample the first and second discrete audio signals to the given sampling rate; wherein said determining is performed responsive to said receiving.
11. The method of claim 9, further comprising: discovering, by the processor, that the first sampling rate of the first discrete audio signal and the second sampling rate of the second discrete audio signal fall beneath a predetermined threshold; wherein said determining is performed responsive to said discovering.
12. The method of claim 9, further comprising: generating, by the processor, an interface through which playback of the third and fourth discrete audio signals is initiable.
13. The method of claim 9, wherein the first and second generative models are generative adversarial networks that are trained in an unsupervised manner to output magnitude spectrograms corresponding to the given sampling rate.
Allowable Subject Matter
Claims 1-20 are allowed over the prior art of record. As per the independent claims, the claim elements in combinations, toward the first and second generative models with first and second upsampling rates, along with first and second spectrograms (or more), are not explicitly taught by the prior art of record. Representative prior art, Paraskevopoulos (20200335086) teaches obtaining data samples that are spectrograms, using GAN’s, abstract, operating on spectrograms to generate a representative set of spectrograms – (para 0027) and providing the series of magnitude spectrograms using batches of real and generated spectrograms (para 0034) to generate synthesized spectrograms – abstract, para 0029, 0034); further adapting the general approach to produce synthesized audio signal for the class of signals – para 0027 – examiner notes, that by definition, in producing synthesized audio signals from a spectrogram, an inverse transform is applied to the spectrogram, to convert frequency domain data back to discrete time based data; being represented by the synthesize spectrograms -- abstract, para 0029, 0034). Although Paraskevopoulos (20200335086) discusses the altering of sampling rate of the data (including spectrograms), and altering the sampling rate feature – see para 0029, wherein upsampling layers is used to generate more realistic samples, Paraskevopoulos (20200335086) does not explicitly teach closer tracking/examination of the sampling rate to judge the category pertaining to a particular sampling rate and result. Sargsyan et al (20200066296) teaches the use of upsampling and downsampling layers (para 0038) and altering the sampling rate to maximize signal (noise reduction – para 0040, 0041-0043). Sargsyan et al (20200066296), further teaches the discovery of different SR (sampling rates) – para 0072, selects the calculated features for that particular sampling rate, that maximizes the NoiseModelApproximation – para 0072)). Kawahara et al (20110015931) teaches the use of short term fourier transforms to generate spectrograms (para 0082) as well as, an inverse fourier transform operating on the same parameters and hence, short term as well – para 0137). Germain et al (20160007130) teaches an inverse short term fourier transform operating on spectrograms (para 0040). Arik (20190355347) teaches the spectrogram output of a neural network to be inverse transformed to generate a synthesized waveform (para 0047). Higurashi (20190392802) teaches a neural network generating a synthesized spectrogram and comparing to audio data based spectrogram (para 0057). Thomson et al (20200175961), and related apps, teaching the use of upsampling of audio magnitudes and gan’s/neural networks (para 0249) Uhle et al (20110191101) teaches sampling rate variation for the neural network measurement of snr using spectrograms (para 0088, 0094, reflecting back on para 0048). Huang et al (20190043507) teaches preprocessing according to sampling rate – para 0037, and using machine learning, neural networks, to extract a speaker selection – para 0041, wherein the features can also be spectrograms (para 0060). However, none of the prior art of record explicitly teaches the claim limitations of the independent claims, as noted above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see related art, listed on the PTO-892 form.
The following reference were found, pertinent to applicants spec (and partially described above):
Kawahara et al (20110015931) teaches the use of short term fourier transforms to generate spectrograms (para 0082) as well as, an inverse fourier transform operating on the same parameters and hence, short term as well – para 0137)
Germain et al (20160007130) teaches an inverse short term fourier transform operating on spectrograms (para 0040).
Arik (20190355347) teaches the spectrogram output of a neural network to be inverse transformed to generate a synthesized waveform (para 0047).
Higurashi (20190392802) teaches a neural network generating a synthesized spectrogram and comparing to audio data based spectrogram (para 0057)
Thomson et al (20200175961), and related apps, teaching the use of upsampling of audio magnitudes and gan’s/neural networks (para 0249)
Uhle et al (20110191101) teaches sampling rate variation for the neural network measurement of snr using spectrograms (para 0088, 0094, reflecting back on para 0048).
Huang et al (20190043507) teaches preprocessing according to sampling rate – para 0037, and using machine learning, neural networks, to extract a speaker selection – para 0041, wherein the features can also be spectrograms (para 0060).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Opsasnick, telephone number (571)272-7623, who is available Monday-Friday, 9am-5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Mr. Richemond Dorvil, can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/Michael N Opsasnick/Primary Examiner, Art Unit 2658 06/05/2026
Prosecution Insights