DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 8, lines 7-17, filed 8/26/2026, with respect to the 35 U.S.C. 112(b) rejection of claim 17 and the objection to claim 18, have been fully considered and are persuasive. The 35 U.S.C. 112(b) rejection of claim 17 and the objection to claim 18 have been withdrawn in repose to Applicant’s amendments.
Applicant's arguments filed, see page 9, lines 1-22, filed 8/26/2026, with respect to claims 1-2, 5-12, 15, and 17-18, have been fully considered but they are not persuasive.
Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Applicant has offered no substantive argument against the rejection of record. Instead, Applicant has argued only conclusively that “[amended] independent claims 1 and 15 provide features that… [a]re not taught or suggested in combination by the cited references.”
Applicant should submit an argument under the heading “Remarks” pointing out disagreements with the examiner's contentions. Applicant must also discuss the references applied against the claims, explaining how the claims avoid the references or distinguish from them.
In response to Applicant's argument that rejecting amended claim 1 would require an excessive number of references, reliance on a large number of references in a rejection does not, without more, weigh against the obviousness of the claimed invention. See In re Gorman, 933 F.2d 982, 18 USPQ2d 1885 (Fed. Cir. 1991).
Applicant’s arguments, see page 9, lines 23-26 with respect to claims 21-24 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 8, 10-12, 15, and 18 are rejected under 35 U.S.C. 103 as unpatentable over Pobloth et al. (US 20190034428 A1, January 31, 2019), hereinafter Pobloth, in view of Silverstein (US 20170263227 A1, September 14, 2017), hereinafter Silverstein, and further in view of Chen (US 20150110294 A1, April 23, 2015), Becherer et al. (US 20190138262 A1, May 9, 2019), hereinafter Becherer, and Groeschel et al. (US 20130170672 A1, July 4, 2013), hereinafter Groeschel.
Regarding claim 1, Pobloth teaches a computer-implemented method of generating a musical composition to convey a sequence of moods, the method comprising: segmenting, by a computer-based musical composition system, a movie into a sequence of time intervals based on visual characteristics and audio characteristics of each frame, each time interval delimited by a respective start time and a respective stop time (Pobloth ¶¶0076-0077: "S104: The mood detector 200 determines the length of the segment 150 a, 150 b, 150 c in accordance with which type (e.g., text based, audio based, or video based) of mood indicator 140 a, 140 b, 140 c, 140 d has been detected. According to the illustrative example of FIG. 1 the segment 150 a has a length corresponding to a duration in time of τ1+τ2 and occurs between t=TA−τ1 and t=TA+τ2, the segment 150 b has a length corresponding to a duration in time of τ3+τ4 and occurs between t=TB−τ3 and t=TB+τ4, and the segment 150 c has a length corresponding to a duration in time of τ5+τ6 and occurs between t=(TC+TD)/2−τ5 and t=(TC+TD)/2+τ6."), wherein the computer-based musical composition system stores a set of mood labels (Pobloth ¶0002: "One category of metadata is “mood” which can describe a scene of audio-visual media content (such as a movie scene) and/or the persons in that scene with mood attributes such as 'happy', 'sad', 'humorous', 'feel good', 'sentimental', 'controversial', 'cynical', etc."); and assigning, by the computer-based musical composition system, a respective mood label to each time interval (Pobloth ¶0052: "According to an embodiment the metadata y comprises the mood descriptive value and information of a time segment 150 a, 150 b, 150 c in the multimedia file 100, x over which time segment 150 a, 150 b, 150 c the mood descriptive value holds. The output from the mood classification module 220 can thus be a sequence of mood-classifications, where each mood-classification contains a mood descriptive value and the time segment 150 a, 150 b, 150 c over which it holds.").
Pobloth does not explicitly disclose that the computer-based musical composition system stores a set of mappings between mood labels and musical characteristics; and for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics; and generating, by the computer-based musical composition system, a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval; varying a volume of the musical composition over the time intervals to anti- correlate with a volume of the movie over the time intervals, wherein varying the volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals comprises: partitioning the movie into a sequence of consecutive time windows; determining a mean sound volume for each time window; scaling the mean sound volume of each time window to fit in a range; determining an anti-sound volume for each time window based on the mean sound volume of each time window; adjusting a volume of the musical composition over the time windows based on the anti-sound volume of each time window; and combining the volume-adjusted musical composition with audio for the movie.
However, Silverstein teaches that the computer-based musical composition system stores a set of mappings between mood labels and musical characteristics (Silverstein ¶0930: "The above principles and considerations will be used by the system designer(s) when defining or creating transformational mappings between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters stored in system operating parameter tables that are loaded into subsystem B39 and used during the automated music composition and generation system of the present invention."); and for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶¶0501-0502: "As shown in FIG. 28D, for each emotion-type musical experience descriptor supported by the system and selected by the system user, a probability measure is provided for each key supported by the system, and this probability-based parameter table is used during the automated music composition and generation process of the present invention. The primary function of the key generation table is to provide a framework to determine the key(s) of a musical piece, section, phrase, or other structure."); and generating, by the computer-based musical composition system, a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval (Silverstein ¶0038: "Another object of the present invention is to provide an Automated Music Composition and Generation Process supporting automated virtual-instrument music synthesis driven by linguistic and/or graphical icon based musical experience descriptors supplied by the system user, wherein linguistic-based musical experience descriptors, and a video, audio-recording, image, or event marker, supplied as input through the system user interface, and are used by the Automated Music Composition and Generation Engine of the present invention to generate musically-scored media (e.g. video, podcast, image, slideshow etc.) or event marker using virtual-instrument music synthesis, which is then supplied back to the system user via the system user interface.").
Furthermore, Chen teaches or suggests varying a volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals (Chen ¶¶0005-0006: "The method attenuates the volume of the slave tracks in order to provide at least the minimum loudness separation between the slave tracks and the master tracks. The amount of attenuation for a slave is determined based on the loudness levels of the slave and of a master. Some embodiments perform ducking on a window-by-window basis."), wherein varying the volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals comprises: determining an anti-sound volume for each time window based on the mean sound volume of each time window (Chen ¶0064: "The worst case differential of each window is the smallest difference (or the most negative difference) between the master's loudness and the slave's loudness of each window. It occurs at a point in the window in which the slave is the loudest relative to the master. In some embodiments, the worst case differential of a window is indicative of the amount of attenuation that is needed for the slave in that window."); adjusting a volume of the musical composition over the time windows based on the anti-sound volume of each time window (Chen ¶0066: "The ducking operation, however, does attenuate slave audio for windows 3, 4, and 6. In each of these windows, the ducking operation attenuates the entire window by an amount that is based on the worst case differential between the slave's loudness and the master's loudness for that window."); and combining the volume-adjusted musical composition with audio for the movie (Chen ¶0140: "The audio mixer 1560 performs attenuation or amplification on individual tracks of the audio content before mixing the attenuated or amplified audio signals for output to the audio module 1585.").
Additionally, Becherer teaches or suggests partitioning the movie into a sequence of consecutive time windows (Becherer ¶0075: "In the illustrative examples of FIGS. 2-6, the observation window is a time window that is fifty milliseconds longs and, thus, contains five audio slices. It starts at time 'tn' and ends at time 'tn+50 ms,' where 'tn' corresponds to the start time of the audio slice."); and determining a mean sound volume for each time window (Becherer ¶0022: "Thereafter, a fifty milliseconds observation window is used to compute an average RMS level across five audio slices. The average RMS level is computed from the summed-area table instead of the five individual RMS levels of the five audio slices.").
Additionally, Groeschel teaches or suggests scaling the mean sound volume of each time window to fit in a range (Groeschel ¶0069: "Then, in the optional step 305, a scale factor for the non-dominant signal 207 may be determined directly from the mixing metadata 202 a and the mixing balance input 203. In step 306, the non-dominant signal 207 is scaled. In the case where a scale factor for the non-dominant signal 207 was determined in step 305 based on the mixing metadata 202 a and the mixing balance input 203, step 306 includes the scaling of the non-dominant signal 207 according to the scale factor determined in step 305. Finally, in step 307 the scaled non-dominant signal 207 s is combined with the dominant signal 206 as the mixed signal 208 of the end step 308.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of generating a musical composition to convey a sequence of moods of Pobloth by adding the corresponding musical characteristics and music generation of Silverstein to enable composing and generating music pieces for use in scoring diverse kinds of media products without requiring musical knowledge, theory, or expertise (Silverstein ¶0028), as well as the anti-correlation of Chen, the mean sound volume of Becherer, and the scaling Groeschel to provide content-aware loudness separation between a slave track and a master track (Chen abstract).
Regarding claim 2, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Pobloth further teaches that segmenting the movie into a sequence of time intervals includes identifying sequences of times of the movie that each delimit a respective mood (Pobloth ¶0052: "The output from the mood classification module 220 can thus be a sequence of mood-classifications, where each mood-classification contains a mood descriptive value and the time segment 150 a, 150 b, 150 c over which it holds.").
Regarding claim 8, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Pobloth further teaches that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes, for a movie, assigning a respective mood label to each scene based on audio characteristics of each scene (Pobloth ¶¶0089-0090: "S106 c: The mood detector 200 detects at least one audio feature in the segment 150 a, 15013, 150 c of the multimedia file 100, x by the mood classification module 220 performing audio analysis of the audio part 120 of the segment 150 a, 15013, 150 c of the multimedia file 100, x. S106 d: The mood detector 200 associates the segment 150 a, 15013, 150 c with the at least one audio feature.").
Regarding claim 10, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 8 as discussed above.
Pobloth further teaches that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes assigning a respective mood label to each scene based on semantic properties of each scene (Pobloth ¶¶0085-0087: "S106 aa: The mood detector 200 performs speech to text recognition on the audio part 120 of the segment 150 a, 150 b, 150 c of the multimedia file 100, x to generate the text transcript. S106 ab: The mood detector 200 performs natural language processing of the text transcript to identify a combination of keywords. S106 ac: The mood detector 200 determines the mood descriptive value according to the combination of keywords.").
Regarding claim 11, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Silverstein further teaches that for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶0869: "The above principles and considerations will be used by the system designer(s) when defining or creating “transformational mappings” (i.e. statistical or theoretical relationships) between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters (i.e. values) stored in system operating parameter (SOP) tables that are loaded into subsystem B5 and used during the automated music composition and generation system of the present invention.") includes assigning at least one respective chord progression to the time interval (Silverstein ¶0673: "FIGS. 27V1, 27V2 and 27V3 show the Sub-Phrase Chord Progression Generation Subsystem (B19) used in the Automated Music Composition and Generation Engine of the present invention. Chord, or the sounding of two or more notes (usually at least three) simultaneously, is a fundamental building block of any musical piece. The Sub-Phrase Chord Progression Generation Subsystem B19 determines what the chord progression will be for each sub-phrase of the musical piece. This information is based on either user inputs (if given), computationally-determined value(s), or a combination of both."), the at least one chord progression selected from a set of chord progressions that correspond to the mood label assigned to the time label per the mapping between mood labels and musical characteristics (Silverstein ¶0675: "The Parameter Transformation Engine Subsystem B51 generates the probability-weighted set of sub-phrase chord progression parameter tables for the various musical experience descriptors selected by the system user and supplied to the input subsystem B0. The probability-based parameter tables (i.e. chord root table, chord function root modifier table, and beat root modifier table) employed in the subsystem is set up for the exemplary “emotion-type” musical experience descriptor—HAPPY—and used during the automated music composition and generation process of the present invention.").
Regarding claim 12, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 11 as discussed above.
Silverstein further teaches aligning chord progressions (Silverstein ¶0685: "During operation, Phrase Chord Progression Generation Subsystem B18 receives the output from Initial Chord Generation Subsystem B17 and modifies, changes, adds, and deletes chords from each sub-phrase to generate the chords of each phrase. For example, if a phrase consists of two sub-phrases that each contain an identical chord progression, there might be a one half probability that the first chord in the second sub-phrase is altered to create a more musical chord progression (following a data set or parameter table created and loaded by subsystem B51) for the phrase and a one half probability that the sub-phrase chord progressions remain unchanged.") in abutting segments of the movie or song (Silverstein ¶0904: "Additionally, an upcoming chord's position in the piece of music, phrase, sub-phrase, and measure affects the default landscape of what chord roots might be selected in the future. For example a chord previous to a downbeat at the end of a phrase might ensure that the subsequent chord be a I chord or other chord that accurately resolves the chord progression.").
Regarding claim 15, Pobloth teaches a computer program product comprising a non-transitory processor-readable storage medium storing data and/or processor-executable instructions (Pobloth ¶0001: "Embodiments presented herein relate to a method, a mood detector, a computer program, and a computer program product") that, when executed by at least one processor of a computer-based musical composition system, cause the computer-based musical composition system to: segment a movie or song into a sequence of time intervals based on visual characteristics and audio characteristics of each frame, each time interval delimited by a respective start time and a respective stop time (Pobloth ¶¶0076-0077: "S104: The mood detector 200 determines the length of the segment 150 a, 150 b, 150 c in accordance with which type (e.g., text based, audio based, or video based) of mood indicator 140 a, 140 b, 140 c, 140 d has been detected. According to the illustrative example of FIG. 1 the segment 150 a has a length corresponding to a duration in time of τ1+τ2 and occurs between t=TA−τ1 and t=TA+τ2, the segment 150 b has a length corresponding to a duration in time of τ3+τ4 and occurs between t=TB−τ3 and t=TB+τ4, and the segment 150 c has a length corresponding to a duration in time of τ5+τ6 and occurs between t=(TC+TD)/2−τ5 and t=(TC+TD)/2+τ6."), wherein the computer-based musical composition system stores a set of mood labels (Pobloth ¶0002: "One category of metadata is “mood” which can describe a scene of audio-visual media content (such as a movie scene) and/or the persons in that scene with mood attributes such as 'happy', 'sad', 'humorous', 'feel good', 'sentimental', 'controversial', 'cynical', etc."); assign a respective mood label to each time interval; for each time interval (Pobloth ¶0052: "According to an embodiment the metadata y comprises the mood descriptive value and information of a time segment 150 a, 150 b, 150 c in the multimedia file 100, x over which time segment 150 a, 150 b, 150 c the mood descriptive value holds. The output from the mood classification module 220 can thus be a sequence of mood-classifications, where each mood-classification contains a mood descriptive value and the time segment 150 a, 150 b, 150 c over which it holds.").
Pobloth does not explicitly disclose that the computer program product stores a set of mood labels and a set of mappings between mood labels and musical characteristics; assign a respective mood label to each time interval; for each time interval, assign at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics; and generate a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval; and vary a volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals, wherein the data and/or processor-executable instructions that, when executed by at least one processor of the computer-based musical composition system, cause the computer-based musical composition system to, for a movie, vary a volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals, cause the computer-based musical composition system to: partition the movie into a sequence of consecutive time windows; determine a mean sound volume for each time window; scale the mean sound volume of each time window to fit in a range; determine an anti-sound volume for each time window based on the mean sound volume of each time window; adjust a volume of the musical composition over the time windows based on the anti-sound volume of each time window; and combine the volume-adjusted musical composition with audio for the movie.
However, Silverstein teaches that the computer-based musical composition system stores a set of mappings between mood labels and musical characteristics (Silverstein ¶0930: "The above principles and considerations will be used by the system designer(s) when defining or creating transformational mappings between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters stored in system operating parameter tables that are loaded into subsystem B39 and used during the automated music composition and generation system of the present invention."); assign at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶¶0501-0502: "As shown in FIG. 28D, for each emotion-type musical experience descriptor supported by the system and selected by the system user, a probability measure is provided for each key supported by the system, and this probability-based parameter table is used during the automated music composition and generation process of the present invention. The primary function of the key generation table is to provide a framework to determine the key(s) of a musical piece, section, phrase, or other structure."); and generate a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval (Silverstein ¶0038: "Another object of the present invention is to provide an Automated Music Composition and Generation Process supporting automated virtual-instrument music synthesis driven by linguistic and/or graphical icon based musical experience descriptors supplied by the system user, wherein linguistic-based musical experience descriptors, and a video, audio-recording, image, or event marker, supplied as input through the system user interface, and are used by the Automated Music Composition and Generation Engine of the present invention to generate musically-scored media (e.g. video, podcast, image, slideshow etc.) or event marker using virtual-instrument music synthesis, which is then supplied back to the system user via the system user interface.").
Furthermore, Chen teaches or suggests vary a volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals (Chen ¶¶0005-0006: "The method attenuates the volume of the slave tracks in order to provide at least the minimum loudness separation between the slave tracks and the master tracks. The amount of attenuation for a slave is determined based on the loudness levels of the slave and of a master. Some embodiments perform ducking on a window-by-window basis."), wherein the data and/or processor-executable instructions that, when executed by at least one processor of the computer-based musical composition system, cause the computer-based musical composition system to, for a movie, vary a volume of the musical composition over the time intervals to anti-correlate with a volume of the movie over the time intervals (Chen ¶0064: "The worst case differential of each window is the smallest difference (or the most negative difference) between the master's loudness and the slave's loudness of each window. It occurs at a point in the window in which the slave is the loudest relative to the master. In some embodiments, the worst case differential of a window is indicative of the amount of attenuation that is needed for the slave in that window."), cause the computer-based musical composition system to: adjust a volume of the musical composition over the time windows based on the anti-sound volume of each time window (Chen ¶0066: "The ducking operation, however, does attenuate slave audio for windows 3, 4, and 6. In each of these windows, the ducking operation attenuates the entire window by an amount that is based on the worst case differential between the slave's loudness and the master's loudness for that window."); and combine the volume-adjusted musical composition with audio for the movie (Chen ¶0140: "The audio mixer 1560 performs attenuation or amplification on individual tracks of the audio content before mixing the attenuated or amplified audio signals for output to the audio module 1585.").
Additionally, Becherer teaches or suggests partition the movie into a sequence of consecutive time windows (Becherer ¶0075: "In the illustrative examples of FIGS. 2-6, the observation window is a time window that is fifty milliseconds longs and, thus, contains five audio slices. It starts at time 'tn' and ends at time 'tn+50 ms,' where 'tn' corresponds to the start time of the audio slice."); and determine a mean sound volume for each time window (Becherer ¶0022: "Thereafter, a fifty milliseconds observation window is used to compute an average RMS level across five audio slices. The average RMS level is computed from the summed-area table instead of the five individual RMS levels of the five audio slices.").
Additionally, Groeschel teaches or suggests scale the mean sound volume of each time window to fit in a range (Groeschel ¶0069: "Then, in the optional step 305, a scale factor for the non-dominant signal 207 may be determined directly from the mixing metadata 202 a and the mixing balance input 203. In step 306, the non-dominant signal 207 is scaled. In the case where a scale factor for the non-dominant signal 207 was determined in step 305 based on the mixing metadata 202 a and the mixing balance input 203, step 306 includes the scaling of the non-dominant signal 207 according to the scale factor determined in step 305. Finally, in step 307 the scaled non-dominant signal 207 s is combined with the dominant signal 206 as the mixed signal 208 of the end step 308.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the computer program product of Pobloth by adding the corresponding musical characteristics and music generation of Silverstein to enable composing and generating music pieces for use in scoring diverse kinds of media products without requiring musical knowledge, theory, or expertise (Silverstein ¶0028), as well as the anti-correlation of Chen, the mean sound volume of Becherer, and the scaling Groeschel to provide content-aware loudness separation between a slave track and a master track (Chen abstract).
Regarding claim 18, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a computer program product comprising the features of claim 15 as discussed above.
Silverstein further teaches the data and/or processor-executable instructions that, when executed by at least one processor of the computer-based musical composition system, cause the computer-based musical composition system to, for each time interval, assign at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶0869: "The above principles and considerations will be used by the system designer(s) when defining or creating “transformational mappings” (i.e. statistical or theoretical relationships) between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters (i.e. values) stored in system operating parameter (SOP) tables that are loaded into subsystem B5 and used during the automated music composition and generation system of the present invention."), cause the computer-based musical composition system to, for each time interval, assign at least one respective chord progression to the time interval (Silverstein ¶0673: "FIGS. 27V1, 27V2 and 27V3 show the Sub-Phrase Chord Progression Generation Subsystem (B19) used in the Automated Music Composition and Generation Engine of the present invention. Chord, or the sounding of two or more notes (usually at least three) simultaneously, is a fundamental building block of any musical piece. The Sub-Phrase Chord Progression Generation Subsystem B19 determines what the chord progression will be for each sub-phrase of the musical piece. This information is based on either user inputs (if given), computationally-determined value(s), or a combination of both."), the at least one chord progression selected from a set of chord progressions that correspond to the mood interval assigned to the time label per the mapping between mood labels and musical characteristics (Silverstein ¶0675: "The Parameter Transformation Engine Subsystem B51 generates the probability-weighted set of sub-phrase chord progression parameter tables for the various musical experience descriptors selected by the system user and supplied to the input subsystem B0. The probability-based parameter tables (i.e. chord root table, chord function root modifier table, and beat root modifier table) employed in the subsystem is set up for the exemplary “emotion-type” musical experience descriptor—HAPPY—and used during the automated music composition and generation process of the present invention.").
Claims 5-7, 9, and 17 are rejected under 35 U.S.C. 103 as unpatentable over Pobloth in view of Silverstein, and further in view of Chen, Becherer, Groeschel, and Li (US 20210020149 A1, January 21, 2021), hereinafter Li.
Regarding claim 5, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) does not explicitly disclose that segmenting the movie into a sequence of time intervals includes segmenting the movie into scenes based on dynamic characteristics of each scene.
However, Li teaches that segmenting the movie into a sequence of time intervals include segmenting the movie into scenes based on dynamic characteristics of each scene (Li ¶0012: "The video analysis module 10 makes an analysis according to color tone, storyboard pace, video dialogue (such as a plot, a word of turn in speech and the like), length and category, director's special requirement and characteristic, actors expression, movement, weather, scene, buildings, spatial and temporal factors, things, creature, character, character personality; video content analysis of the video analysis module 10 includes: a color analysis, a content analysis and a character expression analysis.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of generating a musical composition to convey a sequence of moods of Pobloth (as modified by Silverstein, Chen, Becherer, and Groeschel) by adding the segmenting the movie into scenes based on dynamic characteristics of each scene of Li to perform adequate matching between video and musical characteristics (Li ¶0004).
Regarding claim 6, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) does not explicitly disclose that segmenting the movie into a sequence of time intervals includes segmenting the movie into scenes based on semantic interpretation of dialog within each scene.
However, Li teaches that segmenting the movie into a sequence of time intervals includes segmenting the movie into scenes based on semantic interpretation of dialog within each scene (Li ¶0012: "the video dialogue is processed to look for a storyline or delete a word of turn in speech, so as to clearly present keywords and arrange the same according to dependency (or influence), and proportionally locate a corresponding emotional parameter").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of generating a musical composition to convey a sequence of moods of Pobloth (as modified by Silverstein, Chen, Becherer, and Groeschel) by adding the segmenting the movie into scenes based on semantic interpretation of dialog within each scene of Li to perform adequate matching between video and musical characteristics (Li ¶0004).
Regarding claim 7, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 1 as discussed above.
Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) does not explicitly disclose that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes assigning a respective mood label to each scene based on a distribution of colors within each scene.
However, Li teaches that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes assigning a respective mood label to each scene based on a distribution of colors within each scene (Li ¶0012: "The video analysis module 10 makes an analysis according to color tone, storyboard pace, video dialogue (such as a plot, a word of turn in speech and the like), length and category, director's special requirement and characteristic, actors expression, movement, weather, scene, buildings, spatial and temporal factors, things, creature, character, character personality; video content analysis of the video analysis module 10 includes: a color analysis, a content analysis and a character expression analysis. Referring to FIG. 2, which shows a structure of color analysis categories for analyses of color function, color value; a content analysis distinguishes who, how, when, where and what (such as a year, a location, a time, a plot and the like) based on a scene, a person, an item and lighting in a video; a character expression analysis determines an emotion of a person, a plot, a likely conversation and the like in a video according to an expression; by combining the described video content analysis, vector values of various videos can be obtained respectively.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of generating a musical composition to convey a sequence of moods of Pobloth (as modified by Silverstein, Chen, Becherer, and Groeschel) by adding the assigning a respective mood label to each scene based on a distribution of colors within each scene of Li to perform adequate matching between video and musical characteristics (Li ¶0004).
Regarding claim 9, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a method comprising the features of claim 8 as discussed above.
Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) does not explicitly disclose that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes, for a movie, assigning a respective mood label to each scene based on dynamic characteristics of each scene.
However, Li teaches that assigning, by the computer-based musical composition system, a respective mood label to each time interval includes assigning a respective mood label to each scene based on dynamic characteristics of each scene (Li ¶0014: "the present invention classifies and induces a final result between a video and music according to a classification function commonly used in audio matching, wherein a related video type is determined and set according to a story property, and is mainly decided according to a part to be emphasized in audio matching; for example, a character (including a character personality and inner feelings), a plot, a scene (including a location or a city), a time, a point of action and the like").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the method of generating a musical composition to convey a sequence of moods of Pobloth (as modified by Silverstein, Chen, Becherer, and Groeschel) by adding the assigning a respective mood label to each scene based on dynamic characteristics of each scene of Li to perform adequate matching between video and musical characteristics (Li ¶0004).
Regarding claim 17, Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) teaches a computer program product comprising the features of claim 15 as discussed above.
Pobloth (in view of Silverstein and further in view of Chen, Becherer, and Groeschel) does not explicitly disclose the data and/or processor-executable instructions that, when executed by at least one processor of the computer-based musical composition system, cause the computer-based musical composition system to assign a respective mood label to each time interval segment the movie into a sequence of time intervals, cause the computer-based musical composition system to assign a respective mood label to each scene based on a distribution of colors within each scene.
However, Li teaches the data and/or processor-executable instructions that, when executed by at least one processor of the computer-based musical composition system, cause the computer-based musical composition system to assign a respective mood label to each time interval segment the movie into a sequence of time intervals, cause the computer-based musical composition system to assign a respective mood label to each scene based on a distribution of colors within each scene (Li ¶0012: "The video analysis module 10 makes an analysis according to color tone, storyboard pace, video dialogue (such as a plot, a word of turn in speech and the like), length and category, director's special requirement and characteristic, actors expression, movement, weather, scene, buildings, spatial and temporal factors, things, creature, character, character personality; video content analysis of the video analysis module 10 includes: a color analysis, a content analysis and a character expression analysis. Referring to FIG. 2, which shows a structure of color analysis categories for analyses of color function, color value; a content analysis distinguishes who, how, when, where and what (such as a year, a location, a time, a plot and the like) based on a scene, a person, an item and lighting in a video; a character expression analysis determines an emotion of a person, a plot, a likely conversation and the like in a video according to an expression; by combining the described video content analysis, vector values of various videos can be obtained respectively.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the computer program product of Pobloth (as modified by Silverstein, Chen, Becherer, and Groeschel) by adding the segmentation based on scenes based on visual characters of each frame of Li to perform adequate matching between video and musical characteristics (Li ¶0004).
Claims 21-24 are rejected under 35 U.S.C. 103 as unpatentable over Sun et al. (CN 112614511 A, April 6, 2021), hereinafter Sun, in view of Silverstein.
Regarding claim 21, Sun teaches a computer-implemented method of generating a musical composition (Sun ¶0022: "WaveNet is a one-dimensional convolutional neural network used for speech modeling.") to convey a sequence of moods (Sun ¶0003: "Moreover, the emotions conveyed in a song can change as the music plays."), the method comprising: segmenting, by a computer-based musical composition system, a song (Sun ¶0008: "S2: Divide the music, singing voice, and lyrics into m segments according to the breakpoints of the singing voice, based on the original song melody") into a sequence of time intervals each delimited by a respective start time and a respective stop time (Sun ¶0033: "Specifically, in one implementation, the singing voice is segmented according to the duration of the pause at the break point being greater than 5 seconds."), wherein the computer-based musical composition system stores a set of mood labels (Sun ¶0025: "Thayer's Emotional Model: Thayer believed that emotions are mainly influenced by two factors: energy and stress. He created a two-dimensional Thayer emotional model to characterize emotions and divided emotions into four categories: 'frustration', 'satisfaction', 'energy', and 'anxiety/mania'."); assigning, by the computer-based musical composition system, a respective mood label to each time interval (Sun ¶0012: "Preferably, machine learning classification methods or deep learning are used to predict the Thayer sentiment ring sentiment label for each segment of singing voice.").
Sun does not explicitly disclose that the computer-based musical composition system stores a set of mappings between mood labels and musical characteristics; and for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics; and generating, by the computer-based musical composition system, a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval.
However, Silverstein teaches that the computer-based musical composition system stores a set of mappings between mood labels and musical characteristics (Silverstein ¶0930: "The above principles and considerations will be used by the system designer(s) when defining or creating transformational mappings between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters stored in system operating parameter tables that are loaded into subsystem B39 and used during the automated music composition and generation system of the present invention."); and for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶¶0501-0502: "As shown in FIG. 28D, for each emotion-type musical experience descriptor supported by the system and selected by the system user, a probability measure is provided for each key supported by the system, and this probability-based parameter table is used during the automated music composition and generation process of the present invention. The primary function of the key generation table is to provide a framework to determine the key(s) of a musical piece, section, phrase, or other structure."); and generating, by the computer-based musical composition system, a musical composition that includes the sequence of time intervals and each assigned musical characteristic corresponding to each time interval (Silverstein ¶0038: "Another object of the present invention is to provide an Automated Music Composition and Generation Process supporting automated virtual-instrument music synthesis driven by linguistic and/or graphical icon based musical experience descriptors supplied by the system user, wherein linguistic-based musical experience descriptors, and a video, audio-recording, image, or event marker, supplied as input through the system user interface, and are used by the Automated Music Composition and Generation Engine of the present invention to generate musically-scored media (e.g. video, podcast, image, slideshow etc.) or event marker using virtual-instrument music synthesis, which is then supplied back to the system user via the system user interface.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the mood labels and time intervals of Sun by adding the corresponding musical characteristics and music generation of Silverstein to enable composing and generating music pieces for use in scoring diverse kinds of media products without requiring musical knowledge, theory, or expertise (Silverstein ¶0028).
Regarding claim 22, Sun (in view of Silverstein) teaches a computer-implemented method of generating a musical composition comprising the features of claim 21 as discussed above.
Sun further teaches that segmenting the song into a sequence of time intervals includes identifying sequences of times of the song (Sun ¶0008: "S2: Divide the music, singing voice, and lyrics into m segments according to the breakpoints of the singing voice, based on the original song melody") that each delimit a respective mood (Sun ¶0009: "S3: For each segment of singing voice, predict the emotional label and represent the prediction results as a matrix A1 based on the number of times the emotional label of each segment falls in different quadrants.").
Regarding claim 23, Sun (in view of Silverstein) teaches a computer-implemented method of generating a musical composition comprising the features of claim 21 as discussed above.
Silverstein further teaches that for each time interval, assigning, by the computer-based musical composition system, at least one respective musical characteristic to the time interval based at least in part on the mood label assigned to the time interval and a stored mapping between mood labels and musical characteristics (Silverstein ¶0869: "The above principles and considerations will be used by the system designer(s) when defining or creating “transformational mappings” (i.e. statistical or theoretical relationships) between (i) certain allowable combinations of emotion, style and timing/spatial parameters supplied by the system user(s) to the input output subsystem B0 of the system, and (ii) certain music-theoretic parameters (i.e. values) stored in system operating parameter (SOP) tables that are loaded into subsystem B5 and used during the automated music composition and generation system of the present invention.") includes assigning at least one respective chord progression to the time interval (Silverstein ¶0673: "FIGS. 27V1, 27V2 and 27V3 show the Sub-Phrase Chord Progression Generation Subsystem (B19) used in the Automated Music Composition and Generation Engine of the present invention. Chord, or the sounding of two or more notes (usually at least three) simultaneously, is a fundamental building block of any musical piece. The Sub-Phrase Chord Progression Generation Subsystem B19 determines what the chord progression will be for each sub-phrase of the musical piece. This information is based on either user inputs (if given), computationally-determined value(s), or a combination of both."), the at least one chord progression selected from a set of chord progressions that correspond to the mood label assigned to the time label per the mapping between mood labels and musical characteristics (Silverstein ¶0675: "The Parameter Transformation Engine Subsystem B51 generates the probability-weighted set of sub-phrase chord progression parameter tables for the various musical experience descriptors selected by the system user and supplied to the input subsystem B0. The probability-based parameter tables (i.e. chord root table, chord function root modifier table, and beat root modifier table) employed in the subsystem is set up for the exemplary “emotion-type” musical experience descriptor—HAPPY—and used during the automated music composition and generation process of the present invention.").
Regarding claim 24, Sun (in view of Silverstein) teaches a computer-implemented method of generating a musical composition comprising the features of claim 23 as discussed above.
Silverstein further teaches aligning chord progressions (Silverstein ¶0685: "During operation, Phrase Chord Progression Generation Subsystem B18 receives the output from Initial Chord Generation Subsystem B17 and modifies, changes, adds, and deletes chords from each sub-phrase to generate the chords of each phrase. For example, if a phrase consists of two sub-phrases that each contain an identical chord progression, there might be a one half probability that the first chord in the second sub-phrase is altered to create a more musical chord progression (following a data set or parameter table created and loaded by subsystem B51) for the phrase and a one half probability that the sub-phrase chord progressions remain unchanged.") in abutting segments of the movie or song (Silverstein ¶0904: "Additionally, an upcoming chord's position in the piece of music, phrase, sub-phrase, and measure affects the default landscape of what chord roots might be selected in the future. For example a chord previous to a downbeat at the end of a phrase might ensure that the subsequent chord be a I chord or other chord that accurately resolves the chord progression.").
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHILIP SCOLES whose telephone number is (703)756-1831. The examiner can normally be reached Monday-Friday 8:30-4:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei Hammond can be reached on 571-270-7938. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHILIP G SCOLES/
Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837