DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 14, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over US10446126 (Kaye), hereinafter US’126, in view of US20210335333 (Miles), hereinafter US’333.
US10446126 (Kaye), hereinafter US’126, in view of US20210335333 (Miles), hereinafter US’333.
Regarding claim 1, US’126 discloses ‘A method, comprising: generating, by a computing system executing a music composition program, output music content (US’126, [Abstract]:”A musical audio composition is generated based on a content library”; the library contains “sequences and instruments” and a “succession of sequences are selected to create a series of segments in the chain”) that includes multiple overlapping musical expressions in time (US’126, a musical “Sequence” as “a partial musical composition” and a “Composite musical sequence” as “an integral whole musical composition comprised of distinct partial musical sequences”; final audio can be the “sum of all these layers” and defines a musical event as occurring at “specific moment in time”), wherein the generating includes selecting expressions and combining expressions (US’126, Fig. 1, “In order to generate each next segment 140, sequences are selected, transposed, and combined into composite sequence 295; Instruments are selected, transposed according to the musical characteristics of said composite sequence, and combined 296. Selected source audio samples are modulated and mixed into final output 297, resulting in the output audio data 311.”);
receiving, by the computing system, user feedback at a point in time while the output music content is being played (US’126, Fig. 4:”each segment 107 optionally has one or more feedback 104 enabling the implementation of machine learning in order to enhance the performance of the system based on feedback from listeners”. Each feedback includes “rating 185:, “credit 184” identifying the listener, and “detail 186” containing information concerning “this particular listener’s response to this segment”);
based on the user feedback and based on characteristics of the output music content associated with the point in time (US’126, feedback is associated with a particular segment, and each segment has an “offset 172” in chronological order, “start 174 specifying the number of seconds which the start of this segment is located relative to the start of the chain” and “finish 175” specifying the end time):
and generating, by the computing system, additional output music content based on the expression and composition embeddings (US’126, Fig. 1 col. 8, lines 46-52:”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295. Instruments are selected, transposed according to the musical characteristics of said composite sequence, and combined 296. Selected source audio samples are modulated and mixed into final output audio 297, resulting in the output audio data 311”; Fig. 4, “each segment 107 optionally has one or more feedback 104 enabling the implementation of machine learning in order to enhance the performance of the system based on feedback from listeners).
US’126 does not expressly disclose ‘determining, by the computing system … one or more expression embeddings generated based on expressions selected for inclusion in the output music content
and one or more composition embeddings generated based on combined expressions in the output music content.
However, US’333 discloses ‘determining, by the computing system … one or more expression embeddings generated based on expressions selected for inclusion in the output music content (US’333, ¶[0083]:”Some of the features that may be analyzed are done so on the piece of music level, while others may be performed on certain tracks”; hardware/software containing “features (e.g., first-order or piece of music level) 2050, “features (e.g. second-order or track level) 2060” and “features (e.g. third-order or preference level0 2070”);
and one or more composition embeddings generated based on combined expressions in the output music content (US’333, features determined at the entire piece-of-music and track level. It states :”The features for the entire piece of music may be determined as described above. The entire piece of music features may include key, duration and tempo”; a “single composite modeled preference score for each piece of music” while permitting the representation at individual-track level);
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’126 to analyze musical content at both the individual track and overall composition levels, as taught by US’333, to provide more granular listener-preference information for subsequent music generation. And thereby facilitating US’126’s stated use of listener feedback to improve subsequent musical selection and modulation.
Regarding claim 2, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 (in view of US’333) further discloses ‘wherein: at least one of the one or more expression embeddings is a vector (US’333, ¶[0080]:”For vectors, an L2 norm of the difference vector || Math || may be more appropriate…”) that represents a first set of features (US’333, ¶[0085]:”The features include, by non-limiting example, only, track-level features, such as timbre, harmony, rhythm, texture, dynamics, and melody…”) extracted from a musical expression (US’333, ¶[0060]:”the corpus musical information 113 of that piece is first automatically separated into five different sets of instrument separated information 140…”; ¶[0083]:”Some of the features that may be analyzed are done so on the piece of music level, while others may be performed on certain tracks”) included in the output music content (US’126, col. 8, lines 46-51:” sequences are selected, transposed, and combined into a composite sequence 295. Instruments are selected, transposed according to the musical characteristics of said composite sequence, and combined 296. Selected source audio samples are modulated and mixed into final output audio 297…”);
and at least one of the one or more composition embeddings is a vector (US’333, ¶[0080]:”For vectors, an L2 norm of the difference vector || Math || may be more appropriate…”) that represents a second set of features (US’333, ¶[0085]:”The features include, by non-limiting example, only, track-level features, such as timbre, harmony, rhythm, texture, dynamics, and melody…”) extracted from a combined set of expressions (US’333, ¶[0060] :after separating the music into instrument-specific information, it is “also retained as an intact piece of music for a separate analysis of the full musical information 113 of that piece…”) in the output music content (US’126, col. 8, lines 46-51:” sequences are selected, transposed, and combined into a composite sequence 295. Instruments are selected, transposed according to the musical characteristics of said composite sequence, and combined 296. Selected source audio samples are modulated and mixed into final output audio 297…”).
Regarding claim 3, US’126 (in view of US’333) discloses ‘The method of claim 2, as discussed above.
US’126 (in view of US’333) further discloses ‘wherein: the first set of features includes two or more of the following features: brightness, range, and complexity (US’333, ¶[0189]:“Spectral centroid is a feature that correlates with how ‘bright’ something sounds”; ¶¶[0178]-[0179]: determining a standard deviation of pitch values for each section “to quantify changes in range across the sections” and determining complexity at different resolutions and durations and comparing the complexity of successive sections).
Regarding claim 14, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 (in view of US’333) further discloses ‘further comprising: suggesting, by the computing system (US’333, ¶¶[0027]-[0028]: determination engine 101 as processor executable software and its technical effects include producing “improved media selections, preference information, predictions, and recommendations to a user”; ¶[0035]: “determination engine 101 provides improved selections, preference information, predictions, and recommendations to a user”) for a manual music composition tool (US’333, ¶[0028]:computing system 100 is digital media equipment that can “generate, record, edit, play, and store media”; ¶[0037]: input to determination engine 101 can be a “piece of music in progress of composition”), one or more musical expressions (US’333, ¶[0039]: ”new chord progressions are generated … to form verses and choruses”)
based on the expression and composition embeddings (US’333, ¶[0083]:features may be analyzed at the “piece of music level” while others are analyzed on “certain tracks”; ¶[0085]:”track-level features” including timbre, harmony, rhythm, texture, dynamics, and melody, and “piece of music-level features” including key, tempo, duration, and lyrics; ”learn representations of the input that capture the salient characteristics of the input distribution”).
Regarding claim 15, US’126 (in view of US’333) teaches limitations corresponding to the method of claim 1 as discussed above. US’333 discloses ‘A non-transitory computer-readable medium having instructions stored thereon that are executable by a computing device to perform operations comprising (US’333, ¶[0206]:”The various functional units illustrated in the figures and/or described herein may be implemented as … a program, software, or firmware, stored in a non-transitory computer readable medium … executable by a general purpose computer, a processor, or a processor core”):
Accordingly, the remaining limitations of claim 15 are taught by US’126 (in view of US’333) for the same reasons discussed above with respect to the corresponding limitations of claim 1.
generating, by executing a music composition program, output music content that includes multiple overlapping musical expressions in time, wherein the generating includes selecting expressions and combining expressions;
receiving user feedback at a point in time while the output music content is being played;
determining, based on the user feedback and based on characteristics of the output music content associated with the point in time: one or more expression embeddings generated based on expressions selected for inclusion in the output music content;
and one or more composition embeddings generated based on combined expressions in the output music content; and generating additional output music content based on the expression and composition embeddings.
Regarding claim 20, US’126 (in view of US’333) teaches the limitations corresponding to the method of claim 1 as discussed above. US’126 discloses ‘A system, comprising (US’126, col. 5, lines 60-62:” Configuration" or "config" is defined as the arrangement or set-up of the hardware and software that make up a computer system”):
US’333 further discloses ‘one or more processors (US’333, ¶[0029]:”According to an embodiment, the computing system 100 includes the one or more processors 102 (any computing hardware) and the memory 103 (any non-transitory tangible media)”); and one or more non-transitory computer-readable media having instructions stored thereon that are executable by the one or more processors to (US’333, ¶[0029]: ”the computing system 100 includes the one or more processors 102 (any computing hardware) and the memory 103 (any non-transitory tangible media)”; ¶[0207]: ”… include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media”):
Accordingly, the remaining limitations of claim 20 are taught by US’126 (in view of US’333) for the same reasons discussed above with respect to the corresponding limitations of claim 1.
generate, based on execution of a music composition program, output music content that includes multiple overlapping musical expressions in time,
wherein the generating includes selecting expressions and combining expressions;
receive user feedback at a point in time while the output music content is being played;
determine, based on the user feedback and based on characteristics of the output music content associated with the point in time:
one or more expression embeddings generated based on expressions selected for inclusion in the output music content;
and one or more composition embeddings generated based on combined expressions in the output music content;
and generate additional output music content based on the expression and composition embeddings.
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US20220208217 (Liu), hereinafter US’217.
Regarding claim 4, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above. as discussed above.
US’126 (in view of US’333) further discloses ‘wherein at least one of the one or more expression embeddings, at least one of the one or more composition embeddings (US’333, ¶¶[0083]-[0085]:”track-level features, such as timbre, harmony, rhythm, texture, dynamics, and melody, piece of music-level features, such as key, tempo, duration, and lyrics”), or both is:
US’126 (in view of US’333) does not expressly disclose ‘a multi-dimensional embedding generated by a contrastive machine learning model, wherein the contrastive model is trained to provide outputs for different input musical content, wherein a distance between the outputs in a multi-dimensional space corresponds to musical differences between the different input musical content.
However, US’217 discloses ‘a multi-dimensional embedding (US’217, ¶[0037]:”The ML logic 315 is configured/trained to learn a characteristic vector of fixed length, or embedding 420, from a musical work”; ¶[0039]:”The final output of the ML logic 315 is a vector of length 64, which corresponds to the embedding 420”) generated by a contrastive machine learning model (US’217, ¶[0037]:”An example of the ML logic 315 is trained by using a Siamese framework with a contrastive loss function”),
wherein the contrastive model is trained to provide outputs for different input musical content (US’217, ¶[0037]:”during training, the ML logic 315 takes a pair of works as input, either a positive pair if the works are covering the same song or a negative pair if they are from different songs, and learns two embeddings”),
wherein a distance between the outputs in a multi-dimensional space (US’217, ¶[0037]: the model “learns two embeddings such that the Euclidean distance between two positive embeddings is small while the distance between two negative embeddings is large”; ¶[0040]:”ƒ is the embedding function of the proposed system, ∥.Math.∥ is the Euclidean distance”, between outputs) corresponds to musical differences between the different input musical content (US’217, ¶[0040]:the two inputs “represent the feature inputs of two musical works, y.sub.i indicates if the two works are covering the same song or not”; ¶[0041]:”the Euclidean distance between the embeddings of two works covering the same song is also minimized, while their distance is maximized by a margin of m if they are from different songs”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the method of US’126 (in view of US’333) to generate the musical embeddings using the contrastive machine-learning technique taught by US’217, in order to provide embedding representations in which musical similarity or differences between musical content are reflected by distances between the respective embeddings.
Regarding claim 16, US’126 (in view of US’333) discloses ‘The non-transitory computer-readable medium of claim 15, as discussed above.
wherein at least one of the one or more expression embeddings, at least one of the one or more composition embeddings, or both is: a multi-dimensional embedding generated by a contrastive machine learning model,
wherein the contrastive model is trained to provide outputs for different input musical content,
wherein a distance between the outputs in a multi-dimensional space corresponds to musical differences between the different input musical content. (Claim 16 corresponds to claim 4)
Claims 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, in further view of US’217 and “Self-Supervised Contrastive Learning for Singing Voices”, IEEE/ACM, (Hiromu Yakura), hereinafter Yakura.
Regarding claim 5, US’126 (in view of US’333 & US’217) discloses ‘The method of claim 4, as discussed above.
US’126 (in view of US’333 & US’217) further discloses ‘further comprising: training, by the computing system, the contrastive model by (US’217, ¶[0037]: ”The ML logic 315 is configured/trained to learn a characteristic vector of fixed length, or embedding 420, from a musical work. An example of the ML logic 315 is trained by using a Siamese framework with a contrastive loss function”):
providing the first and second music content to the contrastive model (US’217, ¶[0037]: ”during training, the ML logic 315 takes a pair of works as input, either a positive pair if the works are covering the same song or a negative pair if they are from different songs…”);
and comparing differences in outputs generated by the contrastive model (US’217, ¶[0037]: the model “learns two embeddings such that the Euclidean distance between two positive embeddings is small while the distance between two negative embeddings is large).
US’217 does not expressly disclose ‘adjusting first music content to generate second music content
[and]
‘to known differences corresponding to the adjusting.
However, Yakura discloses ‘adjusting first music content to generate second music content (Yakura, p.1617-1618, III-C: raising or lowering pitch by three semitones and speeding up or slowing down the musical content by respective factors of x1.70 and x.065, with one of the transformations applied to a singing-voice sample to generate a negative anchor);
[and]
‘…to known differences corresponding to the adjusting (Yakura, pp. 1616-1617, III-A: “Consequently, we can acquire feature representations sensitive to changes caused by the transformations”)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the contrastive training of US’217 to generate one of the paired musical inputs by applying a musical transformation to the other input, as taught by Yakura, to train the model to produce representations sensitive to musical differences caused by the applied transformation.
Regarding claim 17, US’126 (in view of US’333 & US’217) discloses ‘The non-transitory computer-readable medium of claim 16, as discussed above.
wherein the operations further comprise: training the contrastive model by:
adjusting first music content to generate second music content;
providing the first and second music content to the contrastive model;
and comparing differences in outputs generated by the contrastive model to known differences corresponding to the adjusting. (Claim 17 corresponds to claim 5)
Claims 6-8, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US20220205798 (Rivella), hereinafter US’798.
Regarding claim 6, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
US’126 does not expressly disclose ‘wherein the generating uses different emphasis for different embeddings.
However, US’798 discloses ‘wherein the generating uses (US’798, ¶[0094]: after obtaining a weighted user embedding, “the recommendation system creates the recommendations of a list of songs to be played”, with songs selected/sorted according to similarity between their item embeddings and the user embedding) different emphasis (US’798, ¶[0093]:”The descriptor elements of the music tracks do not have the same weights in the average. In fact, newer music tracks have a heavier weight while previous tracks have lower weights.”) for different embeddings (US’798, ¶[0092]:user embedding as the centroid of the embeddings of the descriptor elements of the music tracks (“item embeddings”), with those music tracks receiving unequal weights in the average).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to further modify the method of US’126 (in view of US’333) to apply different weights to different musical embeddings, as taught by US’798, to account for differences in the user’s musical preferences when determining subsequent music content.
Regarding claim 7, US’126 (in view of US’333 & US’798) discloses ‘The method of claim 6, as discussed above.
US’126 (in view of US’333 & US’798) further discloses ‘wherein a first emphasis for a first embedding is greater than a second emphasis for a second embedding based on user feedback corresponding to the first embedding being received later in time than user feedback corresponding to the second embedding (US’798, ¶[0093]:user representation is generated from music track item embeddings, wherein “newer music tracks have a heavier weight while previous tracks have lower weights”;” a music track at time t−1 (the last music track played) is α, the weight of the music track at time t−2 is α.sup.2, and so on. The weight decreases exponentially in the past”).
Regarding claim 8, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
US’126 does not expressly disclose ‘wherein the generating is based on an aggregation of multiple embeddings of the expression embeddings.
However, US’798 discloses ‘wherein the generating is based on (US’798, ¶[0094]:”After obtaining a user embedding, the recommendation system creates the recommendations of a list of songs to be played with the most interesting music tracks, sorted according to the cosine similarity between their item embeddings and the user embedding”) an aggregation of multiple embeddings (US’798, ¶[0092]:”all the music tracks are summarized in the history in a single vector representation; this is obtained by calculating a weighted average of all the item embeddings of the music tracks included in the history”) of the expression embeddings (US’798, ¶[0092]:”The result is an embedding of the user and in the vector space it represents the centroid of all the embeddings of the descriptor elements of the music tracks (item embeddings)”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to further modify the method of US’126 (in view of US’333) to aggregate multiple musical-expression embeddings as taught by US’798, to provide a combined representation of the user’s musical preferences for determining subsequent music content.
Regarding claim 13, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [additional music output] (US’126, ; col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
US’126 does not expressly disclose ‘… is further based on embedding information shared by a first user with a second user.
However, US’798 discloses ‘…is further based on embedding information (US’798, ¶[0095]:”the system independently calculates embeddings for all occupants on board the vehicle”; ”multiple user embeddings are generated”; music is determined using the relationship between “the item embedding and the n embeddings of users”) shared by a first user with a second user (US’798, ¶[0095]:”the recommendation system of the invention adopts the method described above also for multiple occupants of a vehicle who are on board the vehicle in the same time interval”; “the recommended music track must be close to all users”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’126 (in view of US’333) to further base the generation of music of embedding information shared among multiple users, as taught by US’798, in order to generate musical content corresponding to the preferences of the multiple users.
Regarding claim 18, US’126 (in view of US’333) discloses ‘The non-transitory computer-readable medium of claim 15, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
wherein: the generating uses different emphasis for different embeddings;
and a first emphasis for a first embedding is greater than a second emphasis for a second embedding based on user feedback corresponding to the first embedding being received later in time than user feedback corresponding to the second embedding.
However, US’798 discloses ‘wherein: the generating uses (US’798, ¶[0094]: after obtaining a weighted user embedding, “the recommendation system creates the recommendations of a list of songs to be played”, with songs selected/sorted according to similarity between their item embeddings and the user embedding) different emphasis (US’798, ¶[0093]:”The descriptor elements of the music tracks do not have the same weights in the average. In fact, newer music tracks have a heavier weight while previous tracks have lower weights.”) for different embeddings (US’798, ¶[0092]:user embedding as the centroid of the embeddings of the descriptor elements of the music tracks (“item embeddings”), with those music tracks receiving unequal weights in the average);
and a first emphasis for a first embedding is greater than a second emphasis for a second embedding based on user feedback corresponding to the first embedding being received later in time than user feedback corresponding to the second embedding (US’798, ¶[0093]:user representation is generated from music track item embeddings, wherein “newer music tracks have a heavier weight while previous tracks have lower weights”;” a music track at time t−1 (the last music track played) is α, the weight of the music track at time t−2 is α.sup.2, and so on. The weight decreases exponentially in the past”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to further modify the method of US’126 (in view of US’333) to apply different weights to different musical embeddings, as taught by US’798, to account for differences in the user’s musical preferences when determining subsequent music content.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US20220391432 (Nazari), hereinafter US’432.
Regarding claim 9, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
US’126 does not expressly disclose ‘wherein: the determining further includes determining one or more third embeddings based on genre of music content interacted with by the user; and the generating is further based on the one or more third embeddings.
However, US’432 discloses ‘wherein: the determining further includes determining one or more third embeddings (US’432, ¶[0073]:”the content domain metadata includes information corresponding to artist/artist affinity 406, genre 408, meta-genre 410, micro-genre 412, etc. that can be utilized to generate respective embeddings”) based on genre of music content (US’432, ¶[0073]:” artist/artist affinity 406, genre 408, meta-genre 410, micro-genre 412 metadata ...”) interacted with by the user (US’432, ¶[0073]:these embeddings are generated “for client device 102-1 based on consumed media in music history 110-1”; music history includes “user created playlists, user consumed, favorited content items” and other user information);
and the generating is further based on the one or more third embeddings (US’432, ¶[0073]: “artist/artist affinity 406, genre 408, meta-genre 410, and micro-genre 412 metadata for the music domain are utilized to generate respective artist/artist affinity, genre, meta-genre, and micro-genre embeddings for a user (e.g., embeddings generated for client device 102-1 based on consumed media in music history 110-1”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’126 to further base the selection or modulation of musical sequences or musical instrument audio on the genre embeddings of US’432, which are generated for a user based on consumed media in the user’s music history, thereby using the user’s consumed history in order to generate musical content that more closely corresponds to the user’s music-consumption preferences.
Regarding claim 10, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, ; col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
US’126 does not expressly disclose ‘wherein: the determining further includes determining one or more fourth embeddings based on one or more artists of music content interacted with by the user; and the generating is further based on the one or more fourth embeddings.
However, US’432 discloses ‘wherein: the determining further includes determining one or more fourth embeddings based on one or more artists of music content interacted with by the user (US’432, ¶[0073]:”content domain metadata includes information corresponding to artist/artist affinity 406, genre 408, meta-genre 410, micro-genre 412, etc. that can be utilized to generate respective embeddings. For example, artist/artist affinity 406, genre 408, meta-genre 410, and micro-genre 412 metadata for the music domain are utilized to generate respective artist/artist affinity, genre, meta-genre, and micro-genre embeddings for a user (e.g., embeddings generated for client device 102-1 based on consumed media in music history 110-1”);
and the generating is further based on the one or more fourth embeddings (US’432, ¶[0073]: “artist/artist affinity 406, genre 408, meta-genre 410, and micro-genre 412 metadata for the music domain are utilized to generate respective artist/artist affinity, genre, meta-genre, and micro-genre embeddings for a user (e.g., embeddings generated for client device 102-1 based on consumed media in music history 110-1”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’126 to further base the selection or modulation of musical sequences or musical instrument audio on the genre embeddings of US’432, which are generated for a user based on consumed media in the user’s music history, thereby using the user’s consumed history in order to generate musical content that more closely corresponds to the user’s music-consumption preferences.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US20230129350 (Bryan), hereinafter US’350.
Regarding claim 11, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 further discloses ‘and the generating [each next segment] (US’126, ; col. 8, lines 46-50 :”In order to generate each next segment 140, sequences are selected, transposed, and combined into a composite sequence 295 … and combined 296”).
However, US’126 does not expressly disclose ‘wherein: the determining further includes determining one or more fifth embeddings (US’350, ¶[0043]: ”audio model 111 to first compute short length audio embeddings”; ¶[0045]:” a catalog audio sequence 400 averages short length audio embeddings within each musically motivated section (e.g., sections 402A-402C) to generate audio embeddings 404A-404”) based on musical sections (US’350, ¶[0043]:”automatically divides audio sequences into musically motivated sections (e.g., intro, verse, chorus, etc.)”; ¶[0043]:averages the embeddings corresponding to each identified section) being played in the output music content (US’350, ¶[0045]:” The candidate audio sequence id, start time of the section, and end time of the section are associated with each of audio embeddings 404A-404C”; ¶[0047]:”a playhead for each of the candidate audio sequences 502A-C can be set to play at the start of a corresponding matching audio sections”);
and the generating is further based on the one or more fifth embeddings (US’350, ¶[0045]:” … (e.g., sections 402A-402C) to generate audio embeddings 404A-404”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’126 to further base the generation of each next segment on the section-level audio embeddings of US’350, which represent respective musically motivated sections, to account for the characteristics of individual musical sections when generating subsequent musical content.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US11443346 (Sankar), hereinafter US’346.
Regarding claim 12, US’126 (in view of US’333) discloses ‘The method of claim 1, as discussed above.
US’126 (in view of US’333) does not expressly disclose ‘further comprising: combining, by the computing system, embeddings determined based on feedback from multiple different users to generate at least one of the expression embeddings and the composition embeddings.
US’346 discloses ‘further comprising: combining, by the computing system, embeddings (US’346, col. 6, lines 54-56: “neural preference aggregator… to aggregate the user preference embeddings 32 into group preference embeddings 34”) determined based on feedback (US’346, col. 8, lines 1-2: the neural group recommender considers “implicit feedback of user visits to restaurants, rather than explicit ratings”) from multiple different users (US’346, col. 8, lines 59-61:“aggregate(s) the user preference embeddings … into group preference embeddings”) to generate at least one of the expression embeddings and the composition embeddings (US’346, Fig. 3B, col. 6, lines 53-56: “The base group recommender 28 is also provided with neural preferences aggregator 30B to aggregate the user preference embeddings 32 into group preference embeddings 34…“).
It would have been obvious to modify US’126 (in view of US’333) to aggregate feedback-derived embeddings from multiple different users, as taught by US’346, when determining the expression and/or composition representation, in order to incorporate shared preferences of multiple users when determining subsequent musical content.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over US’126, in view of US’333, and in further view of US’432 & US’350.
Regarding claim 19, US’126 (in view of US’333) discloses ‘The non-transitory computer-readable medium of claim 15, as discussed above.
US’126 (in view of US’333) does not expressly disclose ‘wherein the determining further includes determining one or more embeddings based on: an artist being played; a genre being played; and a musical section being played.
However, US’432 discloses ‘wherein the determining further includes determining one or more embeddings based on: an artist being played US’432, ¶[0073]:”content domain metadata includes information corresponding to artist/artist affinity 406, genre 408, meta-genre 410, micro-genre 412, etc. that can be utilized to generate respective embeddings. For example, artist/artist affinity 406, genre 408, meta-genre 410, and micro-genre 412 metadata for the music domain are utilized to generate respective artist/artist affinity, genre, meta-genre, and micro-genre embeddings for a user (e.g., embeddings generated for client device 102-1 based on consumed media in music history 110-1”);
a genre being played (US’432, ¶[0073]:” artist/artist affinity 406, genre 408, meta-genre 410, micro-genre 412 metadata ...”);
US’350 discloses ‘and a musical section (US’350, ¶[0043]:”automatically divides audio sequences into musically motivated sections (e.g., intro, verse, chorus, etc.)”;) being played (US’350, ¶[0045]:” The candidate audio sequence id, start time of the section, and end time of the section are associated with each of audio embeddings 404A-404C”; ¶[0047]:”a playhead for each of the candidate audio sequences 502A-C can be set to play at the start of a corresponding matching audio sections”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the artist- and genre-based embeddings of US’432 and the musical section embeddings of US’350 into the system of US’126(in view of US’333) to provide a more comprehensive representation of a user’s musical preferences for generating subsequent musical content.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US20220059063 (“Music Generator Generation of Continuous Personalized Music”) teaches selecting and combining audio files to generate output music content and receiving feedback during playback.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICOLE K GILLESPIE whose telephone number is (571)482-4187. The examiner can normally be reached Monday-Friday 7:30-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei K Hammond can be reached at (571)270-3819. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICOLE K GILLESPIE/Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837
Prosecution Insights