DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 1/22/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments, page 7, lines 5-16, filed 1/28/2026, with respect to claims 9, 7 and 18, have been fully considered and are persuasive. The 35 U.S.C. § 112 rejections have been withdrawn.
Applicant’s arguments, see page 8, lines 1-6, filed 1/28/2026, with respect to claims 1, 3-6, 8, 9, 11-17, and 19-20, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. See the rejection on the merits below for a full rejection of the amended claims.
Applicant's arguments, see page 8, lines 7-25, filed 1/28/2026, with respect to claims 1, 3-6, 8, 9, 11-17, and 19-20, have been fully considered but they are not persuasive.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant argues that “merely compressing raw audio to a lower dimensional space, in a one dimensional space, generally as at best disclosed by Jukebox, does not teach or suggest ‘generating a sequence of discrete representations by encoding the raw audio representation’ ‘of the human-created sound’ ‘for generating multi-level music,’ as recited by independent claim 1.” The appended limitations were met by Zhou as stated in the non-final rejection dated 11/3/2025; the verbatim contiguous phrase as recited in claim 1 is, “generating a sequence of discrete representations by encoding the raw audio representation.”
The crux of Applicant’s argument here appears to hinge on the dimensionality of Jukebox in light of “for generating multi-level music” from the preamble of claim 1. (See MPEP § 2111.02 for the effect of the preamble.) “Multi-level music” is defined in instant ¶0013 as “multiple streams corresponding to different instruments/sound types.” Applicant’s argument appears to be targeting Jukebox’s output of a single stream to define a patentable distinction over the references of record.
In response, it is important to note that in instant claim 1, the phrase, “multi-level music” is recited with respect to the user selection and decoding limitations. “Multi-level music” is not recited in the encoding limitations. Furthermore, although “a raw audio representation of a human created sound” may comprise a composite waveform of multiple instruments, “a raw audio representation,” within its plain meaning, connotes a stream that is input to the encoder rather than a plurality of streams, and thus is inconsistent with Applicant’s apparent assertion that Applicant’s string of juxtaposed limitations recited in the arguments implies multi-level music encoding. (See MPEP § 2111.01 for plain meaning.) Applicant may be basing this argument on an alternative embodiment disclosed in the instant specification, e.g. “In some implementations, interfaces 342 can receive multiple raw audio representations as inputs.” (¶0029). However, this alternative embodiment and Applicant’s arguments are incommensurate with the scope of instant claim 1, which requires only that during the output stage must the stream bifurcate.
Therefore, it is unnecessary for the encoder to encode additional dimensionalities to accommodate a raw audio input as recited in claim 1 within BRI. Instead, only the decoder/production elements are limited to “producing multi-level music.” See MPEP § 2111 for BRI.
For this reason, the rejection is maintained against these arguments.
Applicant's arguments, see page 8, line 26 – page 9, line 21, filed 1/28/2026, with respect to claims 1, 3-6, 8, 9, 11-17, and 19-20, have been fully considered but they are not persuasive.
In the non-final rejection dated 11/3/2025, Zhou was cited to meet the limitation, “receiving user steerings specifying desired properties of the multi-level music.” Here, Applicant argues instead against Jukebox. The instant specification in ¶0013 defines “user steerings” as “genre, artist, style, etc.” In the non-final rejection dated 11/3/2025, the limitation “and applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the one or more user steerings” was rejected over a Jukebox quotation, which recites in part: “The Transformer takes the embeddings of the tokens z1:T −1 prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song.” (fig. 8a caption). Thus, it can be seen that within the BRI of the claims, Jukebox is basing its machine learning model on user steerings which conceptually flow from Zhou’s “music style… and/or other features from the user input” (¶0050) that determines the received user steerings.
As detailed in the non-final rejection dated 11/3/2025 and discussed above, it can be seen that Zhou teaches or reasonably suggests “receiving user steerings,” and Jukebox teaches or reasonably suggests, “applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the one or more user steerings.” Therefore, the rejection is maintained against these arguments.
Applicant's arguments, see page 9, line 22 – page 10, line 8, filed 1/28/2026, with respect to claims 2, 7, 10, and 18, have been fully considered but they are not persuasive.
Because the rejection of claims 1, 8, and 16 has been maintained, claims 2, 7, 10, and 18 are not placed in condition for allowance by virtue of their respective dependencies upon the aforementioned base claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-6, 8, 11-17, and 19-20 are rejected under 35 U.S.C. 103 as unpatentable over Zhou et al. (US 20220223125 A1, July 14, 2022), hereinafter Zhou, in view of Dhariwal et al. ("Jukebox: A Generative Model for Music," April 30, 2020, retrieved from the application file wrapper), hereinafter Jukebox, and further in view of Baevski et al. (VQ-WAV2VEC: Self-Supervised Learning of Discrete Speech Representations," February 16, 2020, retrieved October 24, 2025 from https://arxiv.org/pdf/1910.05453), hereinafter Baevski, and Alinoori ("Music-STAR: A Style Translation System for Audio-Based Re-Instrumentation," March 3, 2022, retrieved June 4, 2026 from https://yorkspace.library.yorku.ca/items/7ceff38c-642f-498c-a4bd-b3aa3bb947e7).
Regarding claim 1, Zhou teaches a method of generating multi-level music (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument.") from human- created sound (Zhou ¶0025: "the audio input may be a piece of music audio hummed or uploaded by a user"), the method comprising: receiving a raw audio representation of the human-created sound (Zhou ¶0025: "In a song generation system 100, a user input 110 may be received. Herein, the user input 110 may include text input as well as optional audio input… The audio input may include a piece of audio with a reference melody, which is used to generate a melody of a song, for example, the audio input may be a piece of music audio hummed or uploaded by a user."); receiving user steerings specifying desired properties of the multi-level music (Zhou ¶0050: "the instrument set is selected according to the determined music style, extracted emotions, and/or other features from the user input"); and generating instrument-specific code sequences corresponding to a plurality of instruments, based on the sequence of predicted embeddings (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument. With a given or selected instrument set, the multi-track arrangement process may arrange for respective parts or bars of the melody to generate an arrangement track, and align different arrangement tracks in time with respective bars of the melody. In some examples, during the multi-track arrangement process, the arrangement of the current bar of the melody on each track may be as follows: for the track is arranged within the current bar of the melody based on the current bar of the melody (for example, as the main melody of the current time) and a note sequence played by each instrument in all the instruments generated in the previous bar of the melody. In one implementation, the multi-track arrangement process may be implemented through a machine learning model, such as a long short-term memory (LSTM) sequence model.").
Zhou does not explicitly disclose generating a sequence of discrete representations by encoding the raw audio representation; converting the sequence of discrete representations to a sequence of embeddings in a same dimensionality, the sequence of embeddings being a vector of values; applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings; and producing the multi-level music by converting the instrument-specific code sequences into a plurality of instrument-specific audio waveforms, based on the user steerings and associated with the human-created sound of the raw audio representation.
However, Jukebox teaches or suggests generating a sequence of discrete representations by encoding the raw audio representation (Jukebox § 2.1: "However, we use the VQ-VAE… to compress raw audio to a lower-dimensional space. A one-dimensional VQ-VAE learns to encode an input sequence x = hxti T t=1 using a sequence of discrete tokens z = hzs ∈ [K]i S s=1, where K denotes the vocabulary size and we call the ratio T/S the hop length."); and applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings (Jukebox fig. 8a caption: "The structure of our prior models, performing next-token prediction at each level. The Transformer takes the embeddings of the tokens z1:T −1 prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song").
Furthermore, Baevski teaches or suggests converting the sequence of discrete representations to a sequence of embeddings in a same dimensionality, the sequence of embeddings being a vector of values (Baevski § 3: "The quantization module replaces the original representation z by ˆz = ei from a fixed size codebook e ∈ R V ×d which contains V representations of size d.").
Additionally, Alinoori teaches or suggests producing the multi-level music by converting the instrument-specific code sequences (Alinoori § 3.4.1.1: "The pre-trained WaveNet decoders are attached to the Music-STAR encoder during inference to translate the code into the target instruments, and the summa tion of their results will demonstrate the rearrangement.") into a plurality of instrument-specific audio waveforms (Alinoori § 4.1.4: "In the case of clarinet vibraphone to strings-piano translation, two autoencoders take in the input mixture, one outputs the piano track, while the other outputs the strings track."), based on the user steerings (Alinoori Introduction: "In this thesis, the term rearrangement refers to selecting musical instruments different from those in the original performance and automatically deriving an audio performance using the new set of instruments.") and associated with the human-created sound of the raw audio representation (Alinoori § 3.4.1: "Unsupervised Music-STAR performs a semi-separation task through the encoding process, where the encoder learns to capture the pitch content of one of the two instruments in the mixture.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of Zhou by adding the discrete representations and embeddings of Jukebox and Baevski to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1) and the output stage of Alinoori to assist musicians and composers in experimenting with their pieces’ instrumentation (Alinoori Introduction).
Regarding claim 3, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 2 as discussed above.
Baevski further suggests receiving a first portion of the raw audio representation of the human-created sound and a corresponding part of the predicted embeddings, of the sequence of predicted embeddings, is generated before a second part of the raw audio representation of the human- created sound is received (Baevski fig. 1 caption: "training requires future time step prediction"; Baevski § 3: "We first map 30ms segments of raw speech to a dense feature representation z at a stride of 10ms using the encoder network f."; Baevski § 2.1: "Given an aggregated representation ci , the model is trained to distinguish a sample zi+k that is k steps in the future from distractor samples ˜z drawn from a distribution pn, by minimizing the contrastive loss for steps k = 1, . . . , K").
Regarding claim 4, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 3 as discussed above.
Baevski further suggests producing a next predicted embedding of the sequence of predicted embeddings is produced by applying a neural network to raw audio received after a previous predicted embedding of the sequence of predicted embeddings (Baevski § 3: "We follow the same architectural choices as wav2vec (§2.1) with two convolutional networks f : X 7→ Z and g : Z 7→ C ˆ for feature extraction and aggregation, as well as a new quantization module q : Z 7→ Zˆ to build discrete representations (Figure 1a)." Baevski § 3: "We first map 30ms segments of raw speech to a dense feature representation z at a stride of 10ms using the encoder network f."; Baevski § 2.1: "Given an aggregated representation ci , the model is trained to distinguish a sample zi+k that is k steps in the future from distractor samples ˜z drawn from a distribution pn, by minimizing the contrastive loss for steps k = 1, . . . , K." Here, k = 1 represents a next predicted embedding.).
Regarding claim 5, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 1 as discussed above.
Zhou further teaches that the machine learning model comprises a Long Short-Term Memory (LSTM) network (Zhou ¶0049: "the multi-track arrangement process may be implemented through a machine learning model, such as a long short-term memory (LSTM) sequence model").
Regarding claim 6, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 1 as discussed above.
Jukebox further suggests converting the instrument-specific code sequences into instrument-specific audio waveforms via Mel spectrograms (Jukebox § 6: "the approaches above serve as sophisticated generative models for raw audio to be conditioned on a compact and controllable representation of audio such as Mel spectrograms").
Regarding claim 8, Zhou teaches or suggests: a non-transitory computer-readable storage medium (Zhou ¶0114: "Software can reside on computer readable medium.") facilitating generation of multi-level music (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument.") from human-created sound (Zhou ¶0025: "the audio input may be a piece of music audio hummed or uploaded by a user"), the computer-readable storage medium storing instructions that, when executed, cause: receiving a raw audio representation of the human-created sound (Zhou ¶0025: "In a song generation system 100, a user input 110 may be received. Herein, the user input 110 may include text input as well as optional audio input… The audio input may include a piece of audio with a reference melody, which is used to generate a melody of a song, for example, the audio input may be a piece of music audio hummed or uploaded by a user."); receiving user steerings specifying desired properties of the multi-level music (Zhou ¶0050: "the instrument set is selected according to the determined music style, extracted emotions, and/or other features from the user input"); and generating instrument-specific code sequences corresponding to a plurality of instruments, based on the sequence of predicted embeddings (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument. With a given or selected instrument set, the multi-track arrangement process may arrange for respective parts or bars of the melody to generate an arrangement track, and align different arrangement tracks in time with respective bars of the melody. In some examples, during the multi-track arrangement process, the arrangement of the current bar of the melody on each track may be as follows: for the track is arranged within the current bar of the melody based on the current bar of the melody (for example, as the main melody of the current time) and a note sequence played by each instrument in all the instruments generated in the previous bar of the melody. In one implementation, the multi-track arrangement process may be implemented through a machine learning model, such as a long short-term memory (LSTM) sequence model.").
Zhou does not explicitly disclose generating a sequence of discrete representations by encoding the raw audio representation; converting the sequence of discrete representations to a sequence of embeddings in a same dimensionality; applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings; and producing the multi-level music by converting the instrument-specific code sequences into a plurality of instrument-specific audio waveforms, based on the user steerings and associated with the human-created sound of the raw audio representation.
However, Jukebox teaches or suggests generating a sequence of discrete representations by encoding the raw audio representation (Jukebox § 2.1: "However, we use the VQ-VAE… to compress raw audio to a lower-dimensional space. A one-dimensional VQ-VAE learns to encode an input sequence x = hxti T t=1 using a sequence of discrete tokens z = hzs ∈ [K]i S s=1, where K denotes the vocabulary size and we call the ratio T/S the hop length."); and applying a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings (Jukebox fig. 8a caption: "The structure of our prior models, performing next-token prediction at each level. The Transformer takes the embeddings of the tokens z1:T −1 prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song").
Furthermore, Baevski teaches or suggests converting the sequence of discrete representations to a sequence of embeddings in a same dimensionality (Baevski § 3: "The quantization module replaces the original representation z by ˆz = ei from a fixed size codebook e ∈ R V ×d which contains V representations of size d.").
Additionally, Alinoori teaches or suggests producing the multi-level music by converting the instrument-specific code sequences (Alinoori § 3.4.1.1: "The pre-trained WaveNet decoders are attached to the Music-STAR encoder during inference to translate the code into the target instruments, and the summa tion of their results will demonstrate the rearrangement.") into a plurality of instrument-specific audio waveforms (Alinoori § 4.1.4: "In the case of clarinet vibraphone to strings-piano translation, two autoencoders take in the input mixture, one outputs the piano track, while the other outputs the strings track."), based on the user steerings (Alinoori Introduction: "In this thesis, the term rearrangement refers to selecting musical instruments different from those in the original performance and automatically deriving an audio performance using the new set of instruments.") and associated with the human-created sound of the raw audio representation (Alinoori § 3.4.1: "Unsupervised Music-STAR performs a semi-separation task through the encoding process, where the encoder learns to capture the pitch content of one of the two instruments in the mixture.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer-readable storage medium of Zhou by adding the discrete representations and embeddings of Jukebox and Baevski to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1) and the output stage of Alinoori to assist musicians and composers in experimenting with their pieces’ instrumentation (Alinoori Introduction).
Regarding claim 11, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Baevski further teaches or suggests that the instructions, when executed, further cause: receiving a first portion of the raw audio representation of the human-created sound and a corresponding part of the predicted embeddings, of the sequence of predicted embeddings, is generated before a second part of the raw audio representation of the human-created sound is received (Baevski fig. 1 caption: "training requires future time step prediction"; Baevski § 3: "We first map 30ms segments of raw speech to a dense feature representation z at a stride of 10ms using the encoder network f."; Baevski § 2.1: "Given an aggregated representation ci , the model is trained to distinguish a sample zi+k that is k steps in the future from distractor samples ˜z drawn from a distribution pn, by minimizing the contrastive loss for steps k = 1, . . . , K").
Regarding claim 12, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 11 as discussed above.
Baevski further teaches or suggests that the instructions, when executed, further cause: producing a next predicted embedding of the sequence of predicted embeddings by applying a neural network to raw audio received after a previous predicted embedding of the sequence of predicted embeddings (Schneider § 2.1: "The output of the encoder is a low frequency feature representation zi ∈ Z which encodes about 30 ms of 16 kHz of audio and the striding results in representations zi every 10ms. Next, we apply the context network g : Z 7→ C to the output of the encoder network to mix multiple latent representations zi . . . zi−v into a single contextualized tensor ci = g(zi . . . zi−v) for a receptive field size v… The layers in both the encoder and context networks consist of a causal convolution with 512 channels, a group normalization layer and a ReLU nonlinearity" Schneider § 2.2: "We train the model to distinguish a sample zi+k that is k steps in the future from distractor samples ˜z drawn from a proposal distribution pn, by minimizing the contrastive loss for each step k = 1, . . . , K.").
Regarding claim 13, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Zhou further teaches or suggests that the machine learning model comprises a Long Short-Term Memory (LSTM) network (Zhou ¶0049: "the multi-track arrangement process may be implemented through a machine learning model, such as a long short-term memory (LSTM) sequence model").
Regarding claim 14, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Jukebox further suggests that the instructions, when executed, further cause: converting the instrument-specific code sequences into instrument-specific audio waveforms via Mel spectrograms (Jukebox § 6: "the approaches above serve as sophisticated generative models for raw audio to be conditioned on a compact and controllable representation of audio such as Mel spectrograms").
Regarding claim 15, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Zhou further teaches or suggests that the sequence of embeddings comprises a vector of values (Zhou ¶0055: "In some examples, words in the text 310 and notes in each song 320 may be embedded in a dense vector space").
Regarding claim 16, Zhou teaches an apparatus facilitating generation of multi-level music from human-created sound (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument."), the apparatus comprising: one or more processors (Zhou ¶0109: "The apparatus 1500 may comprise one or more processors 1510, and a memory 1520 storing computer-executable instructions"); and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus: receive a raw audio representation of the human-created sound (Zhou ¶0025: "In a song generation system 100, a user input 110 may be received. Herein, the user input 110 may include text input as well as optional audio input… The audio input may include a piece of audio with a reference melody, which is used to generate a melody of a song, for example, the audio input may be a piece of music audio hummed or uploaded by a user."); receive user steerings specifying desired properties of the multi- level music (Zhou ¶0050: "the instrument set is selected according to the determined music style, extracted emotions, and/or other features from the user input); generate instrument-specific code sequences corresponding to a plurality of instruments, based on the sequence of predicted embeddings (Zhou ¶0049: "arranging the melody includes performing multi-track arrangement on the melody based on a given or selected instrument set, in which each track may correspond to a musical instrument. With a given or selected instrument set, the multi-track arrangement process may arrange for respective parts or bars of the melody to generate an arrangement track, and align different arrangement tracks in time with respective bars of the melody. In some examples, during the multi-track arrangement process, the arrangement of the current bar of the melody on each track may be as follows: for the track is arranged within the current bar of the melody based on the current bar of the melody (for example, as the main melody of the current time) and a note sequence played by each instrument in all the instruments generated in the previous bar of the melody. In one implementation, the multi-track arrangement process may be implemented through a machine learning model, such as a long short-term memory (LSTM) sequence model.").
Zhou does not explicitly disclose: convert the sequence of discrete representations to a sequence of embeddings in a same dimensionality; generate a sequence of discrete representations by encoding the raw audio representation; apply a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings; ; and produce the multi-level music by converting the instrument-specific code sequences into a plurality of instrument-specific audio waveforms, based on the user steerings and associated with the human-created sound of the raw audio representation.
However, Jukebox teaches or suggests generate a sequence of discrete representations by encoding the raw audio representation (Jukebox § 2.1: "However, we use the VQ-VAE… to compress raw audio to a lower-dimensional space. A one-dimensional VQ-VAE learns to encode an input sequence x = hxti T t=1 using a sequence of discrete tokens z = hzs ∈ [K]i S s=1, where K denotes the vocabulary size and we call the ratio T/S the hop length."); and apply a machine learning model to produce a sequence of predicted embeddings based on the sequence of embeddings and based on the user steerings (Jukebox fig. 8a caption: "The structure of our prior models, performing next-token prediction at each level. The Transformer takes the embeddings of the tokens z1:T −1 prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song").
Furthermore, Baevski teaches or suggests convert the sequence of discrete representations to a sequence of embeddings in a same dimensionality (Baevski § 3: "The quantization module replaces the original representation z by ˆz = ei from a fixed size codebook e ∈ R V ×d which contains V representations of size d.").
Additionally, Alinoori teaches or suggests producer the multi-level music by converting the instrument-specific code sequences (Alinoori § 3.4.1.1: "The pre-trained WaveNet decoders are attached to the Music-STAR encoder during inference to translate the code into the target instruments, and the summa tion of their results will demonstrate the rearrangement.") into a plurality of instrument-specific audio waveforms (Alinoori § 4.1.4: "In the case of clarinet vibraphone to strings-piano translation, two autoencoders take in the input mixture, one outputs the piano track, while the other outputs the strings track."), based on the user steerings (Alinoori Introduction: "In this thesis, the term rearrangement refers to selecting musical instruments different from those in the original performance and automatically deriving an audio performance using the new set of instruments.") and associated with the human-created sound of the raw audio representation (Alinoori § 3.4.1: "Unsupervised Music-STAR performs a semi-separation task through the encoding process, where the encoder learns to capture the pitch content of one of the two instruments in the mixture.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the apparatus of Zhou by adding the discrete representations and embeddings of Jukebox and Baevski to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1) and the output stage of Alinoori to assist musicians and composers in experimenting with their pieces’ instrumentation (Alinoori Introduction).
Regarding claim 17, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches an apparatus comprising the features of claim 16 as discussed above.
Zhou further teaches or suggests that the user steerings comprise one or more of genres, artists, styles, types of instruments, temperatures, or frequency penalties (Zhou ¶0050: "the instrument set is selected according to the determined music style, extracted emotions, and/or other features from the user input).
Regarding claim 19, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches an apparatus comprising the features of claim 16 as discussed above.
Baevski further teaches or suggests that when the one or more processors further execute the instructions, the apparatus is configured to: receive a first portion of the raw audio representation of the human-created sound and a corresponding part of the predicted embeddings, of the sequence of predicted embeddings, is generated before a second part of the raw audio representation of the human-created sound is received (Baevski fig. 1 caption: "training requires future time step prediction"; Baevski § 3: "We first map 30ms segments of raw speech to a dense feature representation z at a stride of 10ms using the encoder network f."; Baevski § 2.1: "Given an aggregated representation ci , the model is trained to distinguish a sample zi+k that is k steps in the future from distractor samples ˜z drawn from a distribution pn, by minimizing the contrastive loss for steps k = 1, . . . , K").
Regarding claim 20, Zhou (in view of Jukebox and further in view of Baevski and Alinoori ) teaches an apparatus comprising the features of claim 16 as discussed above.
Zhou further teaches or suggests that the sequence of embeddings is a vector of values (Zhou ¶0055: "In some examples, words in the text 310 and notes in each song 320 may be embedded in a dense vector space").
Claims 2, 10, and 18 are rejected under 35 U.S.C. 103 as unpatentable over Zhou in view of Jukebox and further in view of Baevski, Alinoori, and Hsu et al. ("HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units," June 14, 2021, retrieved October 24, 2025 from https://arxiv.org/pdf/2106.07447), hereinafter Hsu.
Regarding claim 2, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 1 as discussed above.
Alinoori further teaches that the instrument-specific audio waveforms comprises audio of different sounds types associated with different instruments (Alinoori § 4.1.4: "In the case of clarinet vibraphone to strings-piano translation, two autoencoders take in the input mixture, one outputs the piano track, while the other outputs the strings track.").
Zhou (in view of Jukebox and further in view of Baevski and Alinoori) does not explicitly disclose producing the sequence of predicted embeddings after the raw audio representation is received in full.
However, Hsu discloses producing the sequence of predicted embeddings after the raw audio representation is received in full (Hsu abstract: "h utilizes an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only, which forces the model to learn a combined acoustic and language model over the continuous inputs."; Hsu § II(B): "A masked prediction model f takes as input X˜ and predicts a distribution over the target indices at each timestep pf (· | X, t ˜ ).").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of Zhou (as modified by Jukebox, Baevski, and Alinoori) by adding the predicted embeddings of Hsu to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1).
Regarding claim 10, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Zhou (in view of Jukebox and further in view of Baevski and Alinoori) does not explicitly disclose that the instructions, when executed, further cause: producing the sequence of predicted embeddings is produced after the raw audio representation is received in full.
However, Hsu teaches or suggests that the instructions, when executed, further cause: producing the sequence of predicted embeddings is produced after the raw audio representation is received in full (Hsu abstract: "h utilizes an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only, which forces the model to learn a combined acoustic and language model over the continuous inputs."; Hsu § II(B): "A masked prediction model f takes as input X˜ and predicts a distribution over the target indices at each timestep pf (· | X, t ˜ ).").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer readable storage medium of Zhou (as modified by Jukebox, Baevski, and Alinoori) by adding the predicted embeddings of Hsu to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1).
Regarding claim 18, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches an apparatus comprising the features of claim 16 as discussed above.
Zhou (in view of Jukebox and further in view of Baevski) does not explicitly disclose that the sequence of predicted embeddings is produced after the raw audio representation is received in full.
However, Hsu teaches or suggests that when the one or more processors further execute the instructions, the apparatus is configured to: produce the sequence of predicted summed embeddings is produced after the raw audio representation is received in full (Hsu abstract: "h utilizes an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only, which forces the model to learn a combined acoustic and language model over the continuous inputs."; Hsu § II(B): "A masked prediction model f takes as input X˜ and predicts a distribution over the target indices at each timestep pf (· | X, t ˜ ).").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the apparatus of Zhou (as modified by Jukebox, Baevski, and Alinoori) by adding the predicted embeddings of Hsu to use generative models to generate diverse high-fidelity music in the raw audio domain (Jukebox § 1).
Claims 7 and 9 are rejected under 35 U.S.C. 103 as unpatentable over Zhou in view of Jukebox and further in view of Baevski, Alinoor, and Mancusi et al. ("Unsupervised Source Separation via Bayesian Inference in the Latent Domain," March 30, 2022, retrieved June 5, 2026 from https://arxiv.org/pdf/2110.05313), hereinafter Mancusi.
Regarding claim 7, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a method for generating multi-level music comprising the features of claim 1 as discussed above.
Zhou further teaches or suggests that the raw audio representation comprises a first raw audio representation (Zhou ¶0025: "The audio input may include a piece of audio with a reference melody, which is used to generate a melody of a song, for example, the audio input may be a piece of music audio hummed or uploaded by a user.").
Jukebox further teaches or suggests that the sequence of discrete representations comprises a first sequence of discrete representations (Jukebox § 2.1: "A one-dimensional VQ-VAE learns to encode an input sequence x = hxti T t=1 using a sequence of discrete tokens z = hzs ∈ [K]i S s=1"); the sequence of embeddings comprises a first sequence of embeddings (Jukebox § 2.1: "quantizes
h
s
→
e
z
s
by mapping each
h
s
to its nearest vector
e
z
s
from a codebook C =
{
e
k
}
k
=
1
K
"); and applying the machine learning model to produce the sequence of predicted embeddings based on the summed up sequence of embeddings (Jukebox fig. 8(a) caption: "The structure of our prior models, performing next-token prediction at each level. The Transformer takes the embeddings of the tokens
z
1
:
T
-
1
prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song").
Zhou (in view of Jukebox and further in view of Baevski and Alinoori) does not explicitly disclose receiving a second raw audio representation; generating a second sequence of discrete representations by encoding the second raw audio representation; converting the second sequence of discrete representations to a second sequence of embeddings in a same dimensionality; and summing the first sequence of embeddings and the second sequence of embeddings to produce a summed sequence of embeddings.
However, Mancusi teaches or suggests receiving a second raw audio representation (Mancusi fig. 2 caption: "Training scheme of the LQ-VAE: reconstructions ˆx1, ˆ x2 are obtained from input pairs x1, x2 as in the VQ-VAE, leading to the loss LVQ-VAE"); generating a second sequence of discrete representations by encoding the second raw audio representation (Mancusi § 4.1: "Our task is to separate a mixture signal m = 1/2 x1 + 1/2 x2 into x1 ~ p1data and x2 ~ p2data, where p1data and p2data represent the distributions of each instrument class in the time domain." Mancusi § 4.2: "QLt = BQ(1/2BQ (E (x1,t)) + 1/2BQ (E (x2,t))." Mancusi § 3.1: "BI : CS → [K]S is an indexer mapping the codes ek1 , … , ekS into the associated codebook indices z1 = k1,…, zS = kS"); converting the second sequence of discrete representations to a second sequence of embeddings in a same dimensionality (Mancusi § 3.1: "A convolutional encoder E : [-1, 1] T → R(S x D), with S << T, where S is the length of the latent sequence and D denotes the number of channels; A bottleneck block B = BI ○ BQ, where BQ : R(S x D) → CS ⊆ R(S x D) is a vector quantizer"); and summing the first sequence of embeddings and the second sequence of embeddings (Mancusi § 4.2: "At each step s, we compare a variable term mlatent,s with a constant matrix BQ ( 1/2 ez1 + 1/2 ez2)representing all possible (scaled) sums over all codes in C.") to produce a summed sequence of embeddings (Mancusi § 4.2: "Minimizing this loss pushes the quantized latent code representing a mixture of two arbitrary source signals (LQt term) to be equal to the sum of the quantized latent codes, corresponding to the single sources (QLt term), therefore enforcing the discrete codes to behave in an approximately linear way.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of Zhou (as modified by Jukebox, Baevski, and Alinoori) by leveraging the structure of Jukebox to add the second raw audio representation and summed embeddings of Mancusi (Mancusi § 1).
Regarding claim 9, Zhou (in view of Jukebox and further in view of Baevski and Alinoori) teaches a non-transitory computer-readable storage medium comprising the features of claim 8 as discussed above.
Zhou further teaches or suggests that the raw audio representation comprises a first raw audio representation (Zhou ¶0025: "The audio input may include a piece of audio with a reference melody, which is used to generate a melody of a song, for example, the audio input may be a piece of music audio hummed or uploaded by a user.").
Jukebox further teaches or suggests that the sequence of discrete representations comprises a first sequence of discrete representations (Jukebox § 2.1: "A one-dimensional VQ-VAE learns to encode an input sequence x = hxti T t=1 using a sequence of discrete tokens z = hzs ∈ [K]i S s=1"); the sequence of embeddings comprises a first sequence of embeddings (Jukebox § 2.1: "quantizes
h
s
→
e
z
s
by mapping each
h
s
to its nearest vector
e
z
s
from a codebook C =
{
e
k
}
k
=
1
K
"); and applying the machine learning model to produce the sequence of predicted embeddings based on the summed up sequence of embeddings (Jukebox fig. 8(a) caption: "The structure of our prior models, performing next-token prediction at each level. The Transformer takes the embeddings of the tokens
z
1
:
T
-
1
prepended by the sum of the artist and genre embeddings, in addition to the time embedding that encodes relative and absolute timing of the segments in the duration of the song").
Zhou (in view of Jukebox and further in view of Baevski and Alinoori) does not explicitly disclose that the instructions, when executed, further cause: receiving a second raw audio representation; generating a second sequence of discrete representations by encoding the second raw audio representation; converting the second sequence of discrete representations to a second sequence of embeddings in a same dimensionality; and summing the first sequence of embeddings and the second sequence of embeddings to produce a summed sequence of embeddings.
However, Mancusi teaches or suggests that the instructions, when executed, further cause: receiving a second raw audio representation (Mancusi fig. 2 caption: "Training scheme of the LQ-VAE: reconstructions ˆx1, ˆ x2 are obtained from input pairs x1, x2 as in the VQ-VAE, leading to the loss LVQ-VAE"); generating a second sequence of discrete representations by encoding the second raw audio representation (Mancusi § 4.1: "Our task is to separate a mixture signal m = 1/2 x1 + 1/2 x2 into x1 ~ p1data and x2 ~ p2data, where p1data and p2data represent the distributions of each instrument class in the time domain." Mancusi § 4.2: "QLt = BQ(1/2BQ (E (x1,t)) + 1/2BQ (E (x2,t))." Mancusi § 3.1: "BI : CS → [K]S is an indexer mapping the codes ek1 , … , ekS into the associated codebook indices z1 = k1,…, zS = kS"); converting the second sequence of discrete representations to a second sequence of embeddings in a same dimensionality (Mancusi § 3.1: "A convolutional encoder E : [-1, 1] T → R(S x D), with S << T, where S is the length of the latent sequence and D denotes the number of channels; A bottleneck block B = BI ○ BQ, where BQ : R(S x D) → CS ⊆ R(S x D) is a vector quantizer"); and summing the first sequence of embeddings and the second sequence of embeddings (Mancusi § 4.2: "At each step s, we compare a variable term mlatent,s with a constant matrix BQ ( 1/2 ez1 + 1/2 ez2)representing all possible (scaled) sums over all codes in C.") to produce a summed sequence of embeddings (Mancusi § 4.2: "Minimizing this loss pushes the quantized latent code representing a mixture of two arbitrary source signals (LQt term) to be equal to the sum of the quantized latent codes, corresponding to the single sources (QLt term), therefore enforcing the discrete codes to behave in an approximately linear way.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer-readable storage medium of Zhou (as modified by Jukebox, Baevski, and Alinoori) by leveraging the structure of Jukebox to add the second raw audio representation and summed embeddings of Mancusi (Mancusi § 1).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHILIP SCOLES whose telephone number is (703)756-1831. The examiner can normally be reached Monday-Friday 8:30-4:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei Hammond can be reached on 571-270-7938. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHILIP G SCOLES/
Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837
Prosecution Insights