DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement(s) (IDS(s)) submitted on 8/21/2023, 12/12/2024, 7/3/2025, 4/29/2026, and 6/1/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered by the examiner.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6, and 11 are rejected under 35 U.S.C. 102 as anticipated by Sumita (US 20080034948 A1, February 14, 2008).
Regarding claim 1, Sumita discloses an audio analysis method realized by a computer system (Sumita ¶0002: "The present invention relates to a tempo detection apparatus and a tempo-detection computer program."), the audio analysis method comprising: estimating a plurality of beat points of a musical piece by analyzing an audio signal representing a performance sound of the musical piece (Sumita ¶0052: "In the figure, the tempo detection apparatus includes an input section 100 for receiving an acoustic signal; the scale-note-power detection section 101 for applying a fast Fourier transform (FFT) to the received acoustic signal at predetermined time intervals (frames) and for obtaining the power of each note in a scale at each frame interval from the obtained power spectrum; the tempo-candidate detection section 102 for summing up, for all the notes in the scale, an incremental value of the power of each note in the scale at each frame interval to obtain the total of the incremental values of the powers, indicating the degree of change of all the notes at each frame interval, and for detecting an average beat interval and the position of each beat, from the total of the incremental values of the powers"); receiving an instruction from a user to change a location of at least one beat point of the plurality of beat points (Sumita ¶0052: "A beat-detection position is corrected by pressing a “correct beat position” button. When this button is pressed, a crosshairs cursor appears on the screen. If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again."); and updating a plurality of locations of the plurality of beat points in response to the instruction from the user (Sumita ¶0052: "the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.").
Regarding claim 6, Sumita discloses an audio analysis system (Sumita ¶0002: "The present invention relates to a tempo detection apparatus and a tempo-detection computer program.") comprising: an electronic controller including at least one processor (Sumita ¶0013: "The computer may be not only a general-purpose computer having a central processing unit but also a special-purpose computer. The computer needs to have a central processing unit but there is no other special limitations."), the electronic controller being configured to execute an analysis processing unit configured to estimate a plurality of beat points of a musical piece by analyzing an audio signal representing a performance sound of the musical piece (Sumita ¶0052: "In the figure, the tempo detection apparatus includes an input section 100 for receiving an acoustic signal; the scale-note-power detection section 101 for applying a fast Fourier transform (FFT) to the received acoustic signal at predetermined time intervals (frames) and for obtaining the power of each note in a scale at each frame interval from the obtained power spectrum; the tempo-candidate detection section 102 for summing up, for all the notes in the scale, an incremental value of the power of each note in the scale at each frame interval to obtain the total of the incremental values of the powers, indicating the degree of change of all the notes at each frame interval, and for detecting an average beat interval and the position of each beat, from the total of the incremental values of the powers"), an instruction receiving unit configured to receive an instruction from a user to change a location of at least one beat point of the plurality of beat points (Sumita ¶0052: "A beat-detection position is corrected by pressing a “correct beat position” button. When this button is pressed, a crosshairs cursor appears on the screen. If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again."), and a beat point updating unit configured to update a plurality of locations of the plurality of beat points in response to the instruction from the user (Sumita ¶0052: "the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.").
Regarding claim 11, Sumita discloses a non-transitory computer-readable medium storing a program that causes a computer system to execute a process (Sumita ¶0002: "The present invention relates to a tempo detection apparatus and a tempo-detection computer program."), the process comprising: estimating a plurality of beat points of a musical piece by analyzing an audio signal representing a performance sound of the musical piece (Sumita ¶0052: "In the figure, the tempo detection apparatus includes an input section 100 for receiving an acoustic signal; the scale-note-power detection section 101 for applying a fast Fourier transform (FFT) to the received acoustic signal at predetermined time intervals (frames) and for obtaining the power of each note in a scale at each frame interval from the obtained power spectrum; the tempo-candidate detection section 102 for summing up, for all the notes in the scale, an incremental value of the power of each note in the scale at each frame interval to obtain the total of the incremental values of the powers, indicating the degree of change of all the notes at each frame interval, and for detecting an average beat interval and the position of each beat, from the total of the incremental values of the powers"); receiving an instruction from a user to change a location of at least one beat point of the plurality of beat points (Sumita ¶0052: "A beat-detection position is corrected by pressing a “correct beat position” button. When this button is pressed, a crosshairs cursor appears on the screen. If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again."); and updating a plurality of locations of the plurality of beat points in response to the instruction from the user (Sumita ¶0052: "the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.").
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 7, and 12 are rejected under 35 U.S.C. 103 as unpatentable over Sumita in view of Bock et al. (Joint Beat and Downbeat Tracking with Recurrent Neural Networks," August 11, 2016. Retrieved June 6, 2026 from https://archives.ismir.net/ismir2016/paper/000186.pdf), hereinafter Bock.
Regarding claim 2, Sumita discloses an audio analysis method comprising the features of claim 1 as discussed above.
Sumita does not explicitly disclose that the estimating includes performing a feature extraction process in which feature data including a feature value of the audio signal is generated for each of a plurality of analysis time points on a time axis, a probability calculation process in which by inputting the feature data generated for each of the analysis time points to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points, output data representing probability that each of the analysis time points corresponds to a beat point are generated, and a beat point estimation process in which the plurality of beat points are estimated from the output data generated by the probability calculation process.
However, Bock teaches or suggests that the estimating (Bock § 2: "The proposed method consists of a recurrent neural network (RNN) similar to the ones proposed in [2,3], and is trained to jointly detect the beats and downbeats of an audio signal in a supervised classification task.") includes performing a feature extraction process in which feature data including a feature value of the audio signal is generated for each of a plurality of analysis time points on a time axis (Bock § 2.1: "The audio signal is split into overlapping frames and weighted with a Hann window of same length before being transferred to a time-frequency representation with the Short-time Fourier Transform (STFT). Two adjacent frames are located 10ms apart, which corresponds to a rate of 100fps (frames per second)."), a probability calculation process (Bock § 2.2.1: "A softmax classification layer with three units is used to model the beat, downbeat, and non-beat classes.") in which by inputting the feature data generated for each of the analysis time points (Bock § 2.1: "To aid the network during training, we add the first order differences of the spectrograms to our input features. Hence, the final input dimension of the neural network is 314.") to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points (Bock § 2.2.2: "We train the network on the datasets described in Section 3.1 — except the ones marked with an asterisk (*) which are used for testing only — with 8-fold cross validation based on a random splits. We initialise the network weights and biases with a uniform random distribution with range [-0.1, 0.1] and train it with stochastic gradient decent minimising the cross entropy error with a learning rate of 10-5 and 0.9 momentum."), output data representing probability that each of the analysis time points corresponds to a beat point are generated (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and a beat point estimation process in which the plurality of beat points are estimated from the output data generated by the probability calculation process (Bock § 2.3.6: "The sequence of beat B and downbeat times D are determined by the set of time frames k which were assigned to a beat or downbeat state").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis method of Sumita by adding the estimating of Bock to improve Sumita's beat estimation by learning the relevant features directly from the audio (Bock § 4).
Regarding claim 7, Sumita discloses an audio analysis system comprising the features of claim 6 as discussed above.
Sumita does not explicitly disclose that the analysis processing unit includes a feature extraction unit configured to generate feature data including a feature value of the audio signal for each of a plurality of analysis time points on a time axis, a probability calculation unit configured to, by inputting the feature data generated for each of the analysis time points to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points, generate output data representing probability that each of the analysis time points corresponds to a beat point, and a beat point estimation unit configured to estimate the plurality of beat points from the output data generated by the probability calculation unit.
However Bock teaches or suggests that the analysis processing unit (Bock § 2: "The proposed method consists of a recurrent neural network (RNN) similar to the ones proposed in [2,3], and is trained to jointly detect the beats and downbeats of an audio signal in a supervised classification task.") includes a feature extraction unit configured to generate feature data including a feature value of the audio signal for each of a plurality of analysis time points on a time axis (Bock § 2.1: "The audio signal is split into overlapping frames and weighted with a Hann window of same length before being transferred to a time-frequency representation with the Short-time Fourier Transform (STFT). Two adjacent frames are located 10ms apart, which corresponds to a rate of 100fps (frames per second)."), a probability calculation unit (Bock § 2.2.1: "A softmax classification layer with three units is used to model the beat, downbeat, and non-beat classes.") configured to, by inputting the feature data generated for each of the analysis time points to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points (Bock § 2.2.2: "We train the network on the datasets described in Section 3.1 — except the ones marked with an asterisk (*) which are used for testing only — with 8-fold cross validation based on a random splits. We initialise the network weights and biases with a uniform random distribution with range [-0.1, 0.1] and train it with stochastic gradient decent minimising the cross entropy error with a learning rate of 10-5 and 0.9 momentum."), generate output data representing probability that each of the analysis time points corresponds to a beat point (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and a beat point estimation unit configured to estimate the plurality of beat points from the output data generated by the probability calculation unit (Bock § 2.3.6: "The sequence of beat B and downbeat times D are determined by the set of time frames k which were assigned to a beat or downbeat state").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis system of Sumita by adding the estimating of Bock to improve Sumita's beat estimation by learning the relevant features directly from the audio (Bock § 4).
Regarding claim 12, Sumita discloses a non-transitory computer-readable medium comprising the features of claim 11 as discussed above.
Sumita does not explicitly disclose that the estimating includes performing a feature extraction process in which feature data including a feature value of the audio signal is generated for each of a plurality of analysis time points on a time axis, a probability calculation process in which by inputting the feature data generated for each of the analysis time points to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points, output data representing probability that each of the analysis time points corresponds to a beat point are generated, and a beat point estimation process in which the plurality of beat points are estimated from the output data generated by the probability calculation process.
However, Bock teaches or suggests that the estimating (Bock § 2: "The proposed method consists of a recurrent neural network (RNN) similar to the ones proposed in [2,3], and is trained to jointly detect the beats and downbeats of an audio signal in a supervised classification task.") includes performing a feature extraction process in which feature data including a feature value of the audio signal is generated for each of a plurality of analysis time points on a time axis (Bock § 2.1: "The audio signal is split into overlapping frames and weighted with a Hann window of same length before being transferred to a time-frequency representation with the Short-time Fourier Transform (STFT). Two adjacent frames are located 10ms apart, which corresponds to a rate of 100fps (frames per second)."), a probability calculation process (Bock § 2.2.1: "A softmax classification layer with three units is used to model the beat, downbeat, and non-beat classes.") in which by inputting the feature data generated for each of the analysis time points (Bock § 2.1: "To aid the network during training, we add the first order differences of the spectrograms to our input features. Hence, the final input dimension of the neural network is 314.") to an estimation model that has learned a relationship between training feature data corresponding to time points on a time axis and training output data representing probability that the time points correspond to beat points (Bock § 2.2.2: "We train the network on the datasets described in Section 3.1 — except the ones marked with an asterisk (*) which are used for testing only — with 8-fold cross validation based on a random splits. We initialise the network weights and biases with a uniform random distribution with range [-0.1, 0.1] and train it with stochastic gradient decent minimising the cross entropy error with a learning rate of 10-5 and 0.9 momentum."), output data representing probability that each of the analysis time points corresponds to a beat point are generated (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and a beat point estimation process in which the plurality of beat points are estimated from the output data generated by the probability calculation process (Bock § 2.3.6: "The sequence of beat B and downbeat times D are determined by the set of time frames k which were assigned to a beat or downbeat state").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer-readable medium of Sumita by adding the estimating of Bock to improve Sumita's beat estimation by learning the relevant features directly from the audio (Bock § 4).
Claims 3-4, 8-9, and 13-14 are rejected under 35 U.S.C. 103 as unpatentable over Sumita in view of Bock, and further in view of Sandler et al. (US 20200104706 A1, April 2, 2020), hereinafter Sandler.
Regarding claim 3, Sumita (in view of Bock) teaches an audio analysis method comprising the features of claim 2 as discussed above.
Sumita further teaches or suggests that in the updating of the plurality of locations (Sumita ¶0123: "If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again."), applying the location of the at least one beat point or a changed location to which the location of the at least one beat point has been changed in accordance with the instruction from the user (Sumita ¶0123: "the position where the mouse was clicked is set as a tentative beat position"), and a plurality of updated beat points after the updating of the estimation model are estimated (Sumita ¶0123: "the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.").
Bock further teaches or suggests that beat points are estimated by performing the probability calculation process that uses the estimation model that has been updated (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and performing the beat point estimation process that uses output data generated by the probability calculation process that uses the estimation model that has been updated (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.").
Sumita (in view of Bock) does not explicitly disclose that additional training is executed by applying, to the estimation model, in a state in which an adaptation block is added between a first part on an input side of the estimation model and a second part on an output side of the estimation model, to perform updating of the estimation model.
However, Sandler teaches or suggests that additional training is executed by applying, to the estimation model (Sandler ¶0046: "After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch. In particular, in some implementations, the new values for the patch parameters can be learned while keeping the remainder of the model parameters fixed."), in a state in which an adaptation block is added between a first part on an input side of the estimation model and a second part on an output side of the estimation model, to perform updating of the estimation model (Sandler ¶¶0055-0056: "As yet another example, the machine-learned model can include a plurality of layers and the model patch can include an additional layer that is added to the model. For example, the additional layer can be an additional intermediate layer that is structurally positioned between at least two of the plurality of layers of the model. Alternatively or additionally, single neurons and/or edges can be added to the network as part or all of the patch. After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis method of Sumita (as modified by Bock) by adding the additional training of Sandler to personalize the user model (Sandler ¶ 0060).
Regarding claim 4, Sumita (in view of Bock and further in view of Sandler) teaches an audio analysis method comprising the features of claim 3 as discussed above.
Bock further teaches or suggests that the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is performed by using a state transition model (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.") including a plurality of states corresponding to any of a plurality of tempos (Bock § 2.3.1: "We divide the states pace into discrete states s to make inference feasible. These states s(φ, ˙ φ, r) lie in a three dimensional space indexed by the bar position state φ∈ {1..Φ}, the tempo state ˙ φ∈{1.. ˙Φ}, and the time signature state r(e.g.r∈{3/4,4/4}). States that fall on a downbeat position (φ=1) constitute the set of downbeat states D, all states that fall on a beat position define the set of beat states B.).
Regarding claim 8, Sumita (in view of Bock) teaches an audio analysis system comprising the features of claim 7 as discussed above.
Sumita further teaches or suggests that the beat point updating unit includes an estimation model updating unit configured to execute, the location of the at least one beat point or a changed location to which the location of the at least one beat point has been changed in accordance with the instruction from the user (Sumita ¶0123: "If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.").
Bock further teaches or suggests that the probability calculation unit configured to generate output data using the updated estimation model (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and the beat point estimation unit configured to estimate a plurality of updated beat points after the updating of the estimation model, by using the output data generated by using the updated estimation model (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.").
Sumita (in view of Bock) does not explicitly disclose an estimation model updating unit configured to execute additional training by applying, to the estimation model, in a state in which an adaptation block is added between a first part on an input side and a second part on an output side of the estimation model, to perform updating of the estimation model.
However, Sandler teaches or suggests an estimation model updating unit configured to execute additional training by applying, to the estimation model (Sandler ¶0046: "After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch. In particular, in some implementations, the new values for the patch parameters can be learned while keeping the remainder of the model parameters fixed."), in a state in which an adaptation block is added between a first part on an input side and a second part on an output side of the estimation model, to perform updating of the estimation model (Sandler ¶¶0055-0056: "As yet another example, the machine-learned model can include a plurality of layers and the model patch can include an additional layer that is added to the model. For example, the additional layer can be an additional intermediate layer that is structurally positioned between at least two of the plurality of layers of the model. Alternatively or additionally, single neurons and/or edges can be added to the network as part or all of the patch. After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis system of Sumita (as modified by Bock) by adding the additional training of Sandler to personalize the user model (Sandler ¶ 0060).
Regarding claim 9, Sumita (in view of Bock and further in view of Sandler) teaches an audio analysis system comprising the features of claim 8 as discussed above.
Bock further teaches or suggests that the beat point estimation unit is configured to estimate the plurality of updated beat points using a state transition model (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.") including a plurality of states corresponding to any of a plurality of tempos (Bock § 2.3.1: "We divide the states pace into discrete states s to make inference feasible. These states s(φ, ˙ φ, r) lie in a three dimensional space indexed by the bar position state φ∈ {1..Φ}, the tempo state ˙ φ∈{1.. ˙Φ}, and the time signature state r(e.g.r∈{3/4,4/4}). States that fall on a downbeat position (φ=1) constitute the set of downbeat states D, all states that fall on a beat position define the set of beat states B.).
Regarding claim 13, Sumita (in view of Bock) teaches a non-transitory computer-readable medium comprising the features of claim 12 as discussed above.
Sumita further teaches or suggests in the updating of the plurality of locations (Sumita ¶0123: "If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again."), the location of the at least one beat point or a changed location to which the location of the at least one beat point has been changed in accordance with the instruction from the user (Sumita ¶0123: "the position where the mouse was clicked is set as a tentative beat position").
Bock further teaches or suggests that a plurality of updated beat points after the updating of the estimation model are estimated by performing the probability calculation process that uses the estimation model that has been updated (Bock § 2.2.1: "The output of the neural network are three activation functions bk, dk, and nok, which represents the probability of a frame k being a beat but no down beat, downbeat or non-beat position."), and performing the beat point estimation process that uses output data generated by the probability calculation process that uses the estimation model that has been updated (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.").
Sumita (in view of Bock) does not explicitly disclose that additional training is executed by applying, to the estimation model, in a state in which an adaptation block is added between a first part on an input side of the estimation model and a second part on an output side of the estimation model, to perform updating of the estimation model.
However, Sandler teaches or suggests that additional training is executed by applying, to the estimation model (Sandler ¶0046: "After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch. In particular, in some implementations, the new values for the patch parameters can be learned while keeping the remainder of the model parameters fixed."), in a state in which an adaptation block is added between a first part on an input side of the estimation model and a second part on an output side of the estimation model, to perform updating of the estimation model (Sandler ¶¶0055-0056: "As yet another example, the machine-learned model can include a plurality of layers and the model patch can include an additional layer that is added to the model. For example, the additional layer can be an additional intermediate layer that is structurally positioned between at least two of the plurality of layers of the model. Alternatively or additionally, single neurons and/or edges can be added to the network as part or all of the patch. After modifying the model to include the model patch, the model can be re-trained by learning new values for the parameters of the model patch.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer-readable medium of Sumita (as modified by Bock) by adding the additional training of Sandler to personalize the user model (Sandler ¶ 0060).
Regarding claim 14, Sumita (in view of Bock and further in view of Sandler) teaches a non-transitory computer-readable medium comprising the features of claim 13 as discussed above.
Bock further teaches or suggests that the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is performed by using a state transition model (Bock § 2.3: "We use the output of the neural network as observations of a dynamic Bayesian network (DBN) which jointly infers the meter, tempo, and phase of a (down-)beat sequence.") including a plurality of states corresponding to any of a plurality of tempos (Bock § 2.3.1: "We divide the states pace into discrete states s to make inference feasible. These states s(φ, ˙ φ, r) lie in a three dimensional space indexed by the bar position state φ∈ {1..Φ}, the tempo state ˙ φ∈{1.. ˙Φ}, and the time signature state r(e.g.r∈{3/4,4/4}). States that fall on a downbeat position (φ=1) constitute the set of downbeat states D, all states that fall on a beat position define the set of beat states B.).
Claims 5, 10, and 15 are rejected under 35 U.S.C. 103 as unpatentable over Sumita in view of Bock, and further in view of Sandler and Maezawa (US 20140260912 A1, September 18, 2014).
Regarding claim 5, Sumita (in view of Bock and further in view of Sandler) teaches an audio analysis method comprising the features of claim 4 as discussed above.
Sumita (in view of Bock and further in view of Sandler) does not explicitly disclose that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval, in the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated, a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point, and in the updating of the locations, the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is executed under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user, to estimate the plurality of updated beat points.
However, Maezawa teaches or suggests that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval (Maezawa ¶0049: "From among probability models (Hidden Markov Models) described as sequences of states qb, n classified according to combination of a value of beat period b (value proportional to reciprocal of tempo) in a frame ti and a value of the number of frames n between the next beat, a probability model having the most likely sequence of observation likelihoods representative of probability of concurrent observation of the onset feature value XO and BPM feature value XB as observed values is selected (see FIG. 2)."), in the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated, a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point (Maezawa ¶0076: "In the sequence Q, furthermore, it is estimated that a beat exists in frames t1, t5, and t8 corresponding to states qmax (t1), qmax (t5) and qmax (t8) where the value of the number n of frames is '0'." Maezawa teaches that a beat is identified where the maximum-likelihood state of n=0, which is the beat-interval endpoint.), and in the updating of the locations, the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is executed under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user (Maezawa ¶0082: "For instance, furthermore, in a case where the user has corrected the “BPM-ness” of frame ti such that the probability that the value of the beat period b is “βe” is raised, the CPU 12 a sets the likelihoods P (XB (t)|Zb≠βe,n (ti)) of states where the value of the beat period b is not “βe” at a value which is sufficiently small."), to estimate the plurality of updated beat points (Maezawa ¶0086: "In accordance with user's input correction information, furthermore, the sound signal analysis apparatus 10 corrects log observation likelihoods L, and re-estimates beat positions and changes in tempo in a musical piece in accordance with the corrected log observation likelihoods L. Therefore, the sound signal analysis apparatus 10 re-calculates (re-selects) states qmax of the maximum likelihoods of one or more frames situated in front of and behind the corrected frame. Consequently, the sound signal analysis apparatus 10 can obtain estimation results which bring about smooth changes in beat intervals and tempo from the corrected frame to the one or more frames situated in front of and behind the corrected frame.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis method of Sumita (as modified by Bock and Sandler) by adding the beat and time interval calculations of Maezawa to bring about smooth changes in beat intervals and tempo (Maezawa ¶0083).
Regarding claim 10, Sumita (in view of Bock and further in view of Sandler) teaches an audio analysis system comprising the features of claim 9 as discussed above.
Sumita (in view of Bock and further in view of Sandler) does not explicitly disclose that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval, the beat point estimation unit is configured to execute a beat point estimation process in which a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point, and the beat point estimation unit is configured to execute the beat point estimation process under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user, to estimate the plurality of updated beat points.
However, Maezawa teaches or suggests that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval (Maezawa ¶0049: "From among probability models (Hidden Markov Models) described as sequences of states qb, n classified according to combination of a value of beat period b (value proportional to reciprocal of tempo) in a frame ti and a value of the number of frames n between the next beat, a probability model having the most likely sequence of observation likelihoods representative of probability of concurrent observation of the onset feature value XO and BPM feature value XB as observed values is selected (see FIG. 2)."), the beat point estimation unit is configured to execute a beat point estimation process in which a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point (Maezawa ¶0076: "In the sequence Q, furthermore, it is estimated that a beat exists in frames t1, t5, and t8 corresponding to states qmax (t1), qmax (t5) and qmax (t8) where the value of the number n of frames is '0'." Maezawa teaches that a beat is identified where the maximum-likelihood state of n=0, which is the beat-interval endpoint.), and the beat point estimation unit is configured to execute the beat point estimation process under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user (Maezawa ¶0082: "For instance, furthermore, in a case where the user has corrected the “BPM-ness” of frame ti such that the probability that the value of the beat period b is “βe” is raised, the CPU 12 a sets the likelihoods P (XB (t)|Zb≠βe,n (ti)) of states where the value of the beat period b is not “βe” at a value which is sufficiently small."), to estimate the plurality of updated beat points (Maezawa ¶0086: "In accordance with user's input correction information, furthermore, the sound signal analysis apparatus 10 corrects log observation likelihoods L, and re-estimates beat positions and changes in tempo in a musical piece in accordance with the corrected log observation likelihoods L. Therefore, the sound signal analysis apparatus 10 re-calculates (re-selects) states qmax of the maximum likelihoods of one or more frames situated in front of and behind the corrected frame. Consequently, the sound signal analysis apparatus 10 can obtain estimation results which bring about smooth changes in beat intervals and tempo from the corrected frame to the one or more frames situated in front of and behind the corrected frame.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the audio analysis method of Sumita (as modified by Bock and Sandler) by adding the beat and time interval calculations of Maezawa to bring about smooth changes in beat intervals and tempo (Maezawa ¶0083).
Regarding claim 15, Sumita (in view of Bock and further in view of Sandler) teaches a non-transitory computer-readable medium comprising the features of claim 14 as discussed above.
Sumita (in view of Bock and further in view of Sandler) does not explicitly disclose that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval, in the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated, a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point, and in the updating of the locations, the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is executed under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user, to estimate the plurality of updated beat points.
However, Maezawa teaches or suggests that the plurality of states of the state transition model correspond to different combinations of each of the plurality of tempos and each of a plurality of transition points within a beat interval (Maezawa ¶0049: "From among probability models (Hidden Markov Models) described as sequences of states qb, n classified according to combination of a value of beat period b (value proportional to reciprocal of tempo) in a frame ti and a value of the number of frames n between the next beat, a probability model having the most likely sequence of observation likelihoods representative of probability of concurrent observation of the onset feature value XO and BPM feature value XB as observed values is selected (see FIG. 2)."), in the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated, a time point at which a state corresponding to an end point of the beat interval, from among the plurality of transition points, is observed is estimated as a beat point (Maezawa ¶0076: "In the sequence Q, furthermore, it is estimated that a beat exists in frames t1, t5, and t8 corresponding to states qmax (t1), qmax (t5) and qmax (t8) where the value of the number n of frames is '0'." Maezawa teaches that a beat is identified where the maximum-likelihood state of n=0, which is the beat-interval endpoint.), and in the updating of the locations, the beat point estimation process that uses the output data generated by the probability calculation process that uses the estimation model that has been updated is executed under a constraint condition that the state corresponding to the end point of the beat interval is observed at a time point corresponding to the changed location changed in accordance with the instruction from the user (Maezawa ¶0082: "For instance, furthermore, in a case where the user has corrected the “BPM-ness” of frame ti such that the probability that the value of the beat period b is “βe” is raised, the CPU 12 a sets the likelihoods P (XB (t)|Zb≠βe,n (ti)) of states where the value of the beat period b is not “βe” at a value which is sufficiently small."), to estimate the plurality of updated beat points (Maezawa ¶0086: "In accordance with user's input correction information, furthermore, the sound signal analysis apparatus 10 corrects log observation likelihoods L, and re-estimates beat positions and changes in tempo in a musical piece in accordance with the corrected log observation likelihoods L. Therefore, the sound signal analysis apparatus 10 re-calculates (re-selects) states qmax of the maximum likelihoods of one or more frames situated in front of and behind the corrected frame. Consequently, the sound signal analysis apparatus 10 can obtain estimation results which bring about smooth changes in beat intervals and tempo from the corrected frame to the one or more frames situated in front of and behind the corrected frame.").
It would have been prima facie obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified the non-transitory computer-readable medium of Sumita (as modified by Bock and Sandler) by adding the beat and time interval calculations of Maezawa to bring about smooth changes in beat intervals and tempo (Maezawa ¶0083).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHILIP SCOLES whose telephone number is (703)756-1831. The examiner can normally be reached Monday-Friday 8:30-4:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei Hammond can be reached on 571-270-7938. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHILIP G SCOLES/
Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837
Prosecution Insights