DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 101 that form the basis for the rejections under this section made in this Office action:
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
3. Claims 1-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Under the 2019 PEG (now been incorporated into MPEP 2106), the revised procedure for determining whether a claim is "directed to" a judicial exception requires a two-prong inquiry into whether the claim recites: (1) any judicial exceptions, including certain groupings of abstract ideas (i.e., mathematical concepts, certain methods of organizing human interactions such as a fundamental economic practice, or mental processes); and (2) additional elements that integrate the judicial exception into a practical application (see MPEP § 2106.05(a)-(c), (e)-(h)).
Only if a claim (1) recites a judicial exception and (2) does not integrate that exception into a practical application, do we then look to whether the claim: (3) adds a specific limitation beyond the judicial exception that is not "well-understood, routine, conventional" in the field (see MPEP § 2106.0S(d)); or (4) simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.
Claims 1-14 are directed to an abstract idea of music synthesis.
Specifically, representative claim 1 recites:
A computer-implemented audio processing method comprising:
(S1) for each time step of a plurality of time steps on a time axis, acquiring encoded data that reflects current musical features of a tune for a current time step and musical features of the tune for succeeding time steps succeeding the current time step;
(S2) for each time step of the plurality of time steps on a time axis, acquiring control data according to a real-time instruction provided by a user; and
(S3) for each time step of the plurality of time steps on a time axis, generating acoustic feature data representative of acoustic features of a synthesis sound in accordance with first input data including the acquired encoded data and the acquired control data.
The claim limitations in the abstract idea have been highlighted in bold above; the remaining limitations are “additional elements”.
The highlighted portion of the claim constitutes an abstract idea under the 2019 Revised Patent Subject Matter Eligibility Guidance and the additional elements are NOT sufficient to amount to significantly more than the judicial exceptions, as analyzed below:
Step
Analysis
1. Statutory Category ?
Yes.
Method
2A - Prong 1: Judicial Exception Recited?
Yes.
See the bolded portion listed above.
Under its broadest reasonable interpretation (BRI), the limitation S3 encompasses mental processes, i.e., data analysis, evaluation, judgement and/or concepts that can be performed in the human mind with the aid of pen and paper.
The limitation S4 is recited at a high level of generality. Under its BRI and in light of the USPTO’s July 2024 Subject Matter Eligibility Examples (e.g., Example 47, claim 2), a generic recitation of training an AI model, by a computer, based on input of existing training data involve optimizing the AI model using a series of mathematical calculations to iteratively adjust the algorithms and/or parameter values of the AI model, therefore encompasses mathematical concepts (see Applicant’s Spec. para. [0061]-[0064]).
Nothing in the bolded portion precludes the limitation S3 from practically being performed in the mind and/or using a pen and paper. As such, the bolded portion of instant claim 1 falls within a combination of the “Mental Process” grouping of Abstract Ideas defined by the 2019 PEG.
2A - Prong 2: Integrated into a Practical Application?
No.
Under the BRI, each of the limitations S1 and S2 encompasses an insignificant pre-solution activity (i.e., necessary data gathering). According to MPEP 2106.05(g)(3): … that were described as mere data gathering in conjunction with a law of nature or abstract idea. See also Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 13863, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015) (presenting offers and gathering statistics amounted to mere data gathering).
The limitation of “current musical features of a tune for a current time step and musical features of the tune for succeeding time steps succeeding the current time step” is considered merely data characterization that generally links the use of the judicial exception to the relevant technological environment or field of use.
The limitation of “computer” is recited at a high level of generality. Under the BRI, it encompasses a general-purpose computer and related computing components, e.g., one or more processors and memory modules. According to the MPEP 2106.04(a)(2), if a claim limitation, under its broadest reasonable interpretation, covers mental processes except for the mention of generic computer components performing computing activities via basic function of the computer, then the claim is likely considered to be directed to an ineligible abstract idea, as it essentially describes a mental process that could be performed by a human without the computer components adding any significant practical application beyond the abstract concept itself.
In general, the claim as a whole does not meet any of the following criteria to integrate the abstract idea into a practical application:
An additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Various considerations are used to determine whether the additional elements are sufficient to integrate the abstract idea into a practical application. However, in all of these respects, the claim fails to recite additional elements which might possibly integrate the claim into a particular practical application. Instead, based on the above considerations, the claim would tend to monopolize the algorithm across a wide range of applications.
2B: Claim provides an Inventive Concept?
No.
See analysis given in 2A - Prong 2 above.
Focusing on what the inventors have invented exactly, it is considered that the “heart” of pending claim 1 is directed to an algorithm of music synthesis. The claim does not recite any additional element that is qualified for “significantly more” or reflects an “inventive concept” (see also discussion of prior art as set forth in sections 4-5 below).
The claim is therefore ineligible under 35 USC 101.
The dependent claims 2-10 and 13-14 inherit attributes of the independent claim 1, but do not add anything which would render the claimed invention a patent eligible application of the abstract idea. These claims merely extend (or narrow) the abstract idea which do not amount for "significant more" because they merely add details to the algorithm which forms the abstract idea as discussed above.
In particular, claim 3 recites: “wherein the acoustic feature data is generated by inputting the first input data to a trained first generative model”. In light of the USPTO’s July 17, 2024 Subject Matter Eligibility Examples (e.g., Examples 47-49), a prediction or generation of outputs using a machine learning model is considered an "abstract idea" if the claim focuses solely on the concept of making predictions using a generic machine learning algorithm, without any specific technical improvements or applications that go beyond the basic idea of using a computer to analyze data and generate predictions; essentially, if the claim is too high-level and does not describe a concrete, inventive implementation of the machine learning process. In the instant case, “a trained first generative model” is recited at a high level of generality; and the generation of “the acoustic feature data” “by inputting the first input data to a trained first generative model” generally applies the abstract idea without placing any limits on how the “trained first generative model” functions. Rather, the claim only recites the outcome of the generation/prediction but does not include any details about how the “generation” is accomplished. See MPEP 2106.05(f). Further, under the BRI, the “trained first generative model” may be interpretated as software, hardware or combinations thereof. An element directed to functional descriptive material, including computer programs, per se, amounts to no more than mere instructions to apply the exception using a generic computer as a tool commonly known in the art. Claims 7, 13 and 14 recite ineligible subject matter for the same reasons as for claim 3.
Claim 4 recites the additional limitation: “generating an audio signal representative of a waveform of the synthesis sound based on the generated time series of acoustic feature data”. Under the BRI, this limitation is treated as an insignificant extra-solution activity to the judicial exception that does not amount to the recitation of significantly more than the abstract idea itself. Such a post solution activity encompasses merely instructions to apply a judicial exception for an intended use or to link the use of the judicial exception to the relevant technological environment but without improving, e.g., the performance of a musical instrument, a computer, or network communications. It does not integrate the judicial exception into a practical application under Step 2A nor provides an inventive concept under Step 2B. See Accenture Global Servs., GmbH v. Guidewire Software, Inc., 728 F.3d 1336, 1347-48 (Fed. Cir. 2013) (claims to generating tasks based on rules to be completed upon the occurrence of an event recited an abstract idea and its implementation on a generic computer without any meaningful limitations to the concept did not transform the abstract idea into a patent-eligible application). See also MPEP 2106.04(d) and 2106.05(g).
Claim 8 recites the additional limitation “wherein the second input data further includes: position data representing which temporal position, in the unit period, each of the two or more intermediate data corresponds to; and pitch data representing a pitch in each of the two or more time steps”. Under the BRI, this limitation encompasses merely data characterization which can be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of musical synthesis for creating music by combining different audio signals or waveforms by manipulating various parameters such as pitch, amplitude, and timbre, etc.
Claim 10 recites the additional limitation: “generating the control data based on a series of indication values provided by the user”. Under the BRI, this limitation encompasses an insignificant pre-solution activity for gathering the data/information necessary for performing the abstract idea which does not amount to the recitation of significantly more than the abstract idea itself.
Claims 11-12 are treated as ineligible subject matter under 35 U.S.C. § 101 for the same reasons as for claim 1 discussed above. Claim 11 recites the additional limitation “one or more memories storing instructions; and one or more processors that implements the instructions to perform a plurality of tasks”. The “one or more memories storing instructions” and “one or more processors” are all recited at a high level of generality. Under the BRI, it encompasses a general-purpose computer and related computing components. According to the MPEP 2106.04(a)(2), if a claim limitation, under its broadest reasonable interpretation, covers mental processes except for the mention of generic computer components performing computing activities via basic function of the computer, then the claim is likely considered to be directed to an ineligible abstract idea, as it essentially describes a mental process that could be performed by a human without the computer components adding any significant practical application beyond the abstract concept itself.
Claim Rejections - 35 USC § 102
4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention; or
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
5. Claims 1-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by KIM et al. (Neural Music Synthesis for Flexible Timbre Control, ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 1 November 2018).
Regarding claims 1, 11 and 12, KIM discloses a computer-implemented audio processing method/system, and the computer program instructions for implementing the method/system (section 2, the method is implemented on a computer, see also section 2.2, 4th paragraph), wherein the system comprising: one or more memories storing instructions; and one or more processors that implements the instructions to perform a plurality of tasks of the method (section 2.2: “For the WaveNet vocoder, we used nv-wavenet2, a real-time open-source implementation of autoregressive WaveNet by NVIDIA. This implementation limits the recurrent channel size at 64 and the skip channels at 256, because of the GPU memory capacity. A 20-layer WaveNet model was trained with the maximum dilation of 512, and the Mel spectrogram input is upsampled using two transposed convolution layers of window sizes 16 and 32 with strides of 8 and 16, respectively. An Adam optimizer with the initial learning rate of 0.001 is used, and the learning rate is halved every 100,000 iterations, for one million iterations in total. Each iteration takes a mini-batch of 4 sequences of length 16,384, i.e. 1.024 seconds …”), the method comprising, for each time step of a plurality of time steps on a time axis (section 2, 1st paragraph: "quantizing the note timings to the nearest time step''): acquiring encoded data that reflects current musical features of a tune for a current time step and musical features of the tune for succeeding time steps succeeding the current time step (Fig. 1 and section 2, 3rd paragraph: "the use of bidirectional LSTM ... , making it non-causal''); acquiring control data (see discussion of the “instrument embedding” in Fig. 1: the note sequences and instrument embeddings are combined …) according to a real-time instruction provided by a user (section 3.4, last paragraph: "using a user-selected point in the embedding space''); and generating acoustic feature data representative of acoustic features of a synthesis sound in accordance with first input data including the acquired encoded data and the acquired control data (see Fig. 1 and related text; section 2.2, 2nd paragraph: "For Mel spectrogram prediction …''; see also discussions of the convolution layer for generating the Mel-Spectrogram).
Regarding claim 2, KIM discloses: wherein the first input data of the current time step includes one or more acoustic feature data generated at one or more preceding time steps preceding the current time step, from among a plural pieces of acoustic feature data generated at the plurality of time steps (see discussion of the prediction by the bi-directional LSTM in section 2, last paragraph; see also Fig. 1 and related text).
Regarding claim 3, KIM discloses: wherein the acoustic feature data is generated by inputting the first input data to a trained first generative model (section 2: “The neural network shown in Figure 1, dubbed Mel2Mel, concerns the task of synthesizing music corresponding to given note sequences and timbre”; section 2.2, 2nd paragraph: “The model is trained for 100,000 iterations, ...”; see also Fig. 1 and related text).
Regarding claim 4, KIM discloses: wherein: the generating generates a time series of acoustic feature data at the plurality of time steps, the method further comprises generating an audio signal representative of a waveform of the synthesis sound based on the generated time series of acoustic feature data (section 2, last paragraph: “The resulting Mel spectrogram is then fed to a WaveNet vocoder to produce the music”; see also discussion of the “Generated Audio” in Fig. 1).
Regarding claim 5, KIM discloses: generating, from music data, a plurality of symbol data (section 2, 1st paragraph: “a piano roll representation” reads on the claimed symbol data) corresponding to a plurality of symbols in the tune, the music data representing a series of symbols that constitute the tune, wherein each symbol data of the plurality of symbol data reflects musical features of a symbol (“note timing and corresponding force information for the piano keys” read on the claimed musical features of a symbol) corresponding to the symbol data and musical features of another symbol succeeding the symbol in the tune (section 2, 1st paragraph); and converting the symbol data for each symbol into the encoded data for each time step (section 2: “the piano roll representation is encoded as a matrix by quantizing the note timings to the nearest time step”; section 2.1: “For each instrument to model, its timbre is represented in an embedding vector t, implemented as a learned matrix multiplication on one-hot encoded instrument labels”).
Regarding claim 6, KIM discloses: generating, from music data, a plurality of symbol data corresponding to a plurality of symbols in the tune, the music data representing a series of symbols that constitute the tune, wherein each symbol of the plurality of symbol data reflects musical features of a symbol corresponding to the symbol data; converting the symbol data for each symbol into intermediate data for one or more time steps (see discussion of claim 5 above); and generating the encoded data at the current time step based on second input data including two or more intermediate data corresponding to two or more time steps including the current time step and another time step succeeding the current time step (section 2; see also Fig. 1 and related text).
Regarding claim 7, KIM discloses: wherein the encoded data is generated by inputting the second input data to a trained second generative model (section 2: “The neural network shown in Figure 1, dubbed Mel2Mel, concerns the task of synthesizing music corresponding to given note sequences and timbre”; section 2.2, 2nd paragraph: “The model is trained for 100,000 iterations, ...”; see also Fig. 1 and related text).
Regarding claim 8, KIM discloses: wherein the converting of the symbol data to the intermediate data for one or more time steps is based on each of the plurality of symbol data, the one or more time steps constituting a unit period during which a symbol corresponding to the symbol data is sounded, wherein the second input data further includes: position data representing which temporal position, in the unit period, each of the two or more intermediate data corresponds to; and pitch data representing a pitch in each of the two or more time steps (section 2; see also Fig. 1 and related discussion about the input to the neural network).
Regarding claim 9, KIM discloses: generating intermediate data, at the current time step, reflecting musical features of a symbol that corresponds to the current time step among a series of symbols that constitute the tune; and generating the encoded data based on second input data including two or more pieces of intermediate data corresponding to, among the plurality of time steps, two or more time steps including the current time step and another time step succeeding the current time step (section 2; see also Fig. 1 and related discussion about the input to the neural network: the features processed by the linear convolution layer and the FiLM layer read on the claimed intermediate data, the output feature of the bi-LSTM layer read on the claimed encoded data).
Regarding claim 10, KIM discloses: generating the control data based on a series of indication values provided by the user (section 3.4: The control data is generated from a time series of indicator values corresponding to indicators from the user).
Regarding claims 13 and 14, KIM discloses the claimed invention (section 2; section 2.2, 2nd paragraph; see also Fig. 1 and related text).
Contact Information
6. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANCHUN QIN whose telephone number is (571)272-5981. The examiner can normally be reached 9AM-5:30PM EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei Hammond can be reached at (571)270-7938. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIANCHUN QIN/Primary Examiner, Art Unit 2837
Prosecution Insights