DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims filed: 10-04-24
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10-04-24 was filed after the mailing date of the 10-04-24. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
The claims recite extracting the beat information by deriving the hop size and sample rate from the audio frame, predict the next beat locations and interval based on the proxy audio frames in real time, and identify the stability of the tempo. See MPEP 2106. Specifically, claims 1,11 are directed to signal processing generally and the mathematical concepts and steps necessary thereto. Also, the claims may be considered equivalent to the human performance of mental steps necessary to implement the recited mathematical concepts such as an algorithm
This judicial exception is not integrated into a practical application because rather than limit the abstract idea in a manner that comprise of improvement to an existing technology and demonstration of concept, the claims merely recite mathematical calculation for obtaining the probability of the beat information based on the audio frame and its proxies.
The claim(s) do not include additional elements that are sufficient to amount to significantly more than the judicial exception the claims merely state conventional computing elements within an apparatus which do no more than implement the recited and well-known calculations, steps, and methods. The computing element amount to no more than a general-purpose machine that demonstrates no improvement than what is already established. Claims 2-10, 12-19 do not remedy and are similarly rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 are rejected under 35 U.S.C. 103 as being unpatentable over Jochelson (US Patent) 7518053 B1 in view of Jehan (US Publication) 20070291958 A1.
Regarding claim 1, Jochelson (US Pat. 7518053 B1) teaches A method for analyzing music rhythm in real time, which is operated in a system, the method comprising: receiving and decoding an audio to acquire frame information of the audio (In Fig 1a, the input music (audio) is not excessively modified when being matched to another reference music or beat sound track. Modification is either time-scale or sampling rate conversions. See [Col 1 L61-65]), wherein the frame information includes a sampling rate (In Fig 1a, beat matching is applied on a “frame by frame” basis using a variable sampling rate converter to modify both the audio input stream to match the reference stream, see [Col 3 L65-67, Col 4 L1]); obtaining a hop size according to a frame size and an overlapped frame size (Fig 5a, the frame overlap is the hop size, see [Col 14 L50-54]. Mentions the frame overlap with relation with hop window, see [Col 5 L52-65]), and calculating a frame rate according to the sampling rate and the hop size (Determine the hop # [or beats in a hop window] for the analysis frames. The overlap is calculated from duration of the both the hop and frame analysis, see [Col 5 L42-50]. Both the hop window of the first analysis and reference frames contain samples, [Col 5 L66-67, Col 6 L1-5]. Apply a sampling rate converter [ASRC] to adjust the timescale, advance current sample location, then extract the analysis frame both input and reference streams, see [Col 6 L6-54]); calculating an initial value of a beat period according to the sampling rate and an initial BPM (beats per minute) value (The initial sampling rate is 44.1 kHz, or 441K samples from a 10 sec interval, and a tempo of 120 BPM for about 20 locations in a frame, see [Col 4 L2-7]), so as to obtain a beat location based on a quantity of sampling points in a beat (the beat location generator can provide beat sample locations with simple increments by the product of the sampling rate, see [Col 4 L35-38]); and speculating a next beat location by a recursive algorithm, wherein the frame rate is referred to for calculating a quantity of audio frames in a past period of time (The overlap is calculated from duration of the both the hop and frame analysis, see [Col 5 L42-50]), and a new beat period is calculated according to beats per minute (initial BPM is roughly 120bpm, see [Col 5 L58-60]), so as to speculate the next beat location according to the new beat period (The beat source replaces the reference stream, since the next virtual reference analysis frame is a product of the sampling rate multiplied by time from one beat to the next beat, which will be the next beat location, see [Col 6 L21-31]);
wherein the system is configured to re-calculate the new beat period according to the quantity of the audio frames in the past period of time at intervals, so as to re-speculate the next beat location (Extract the analysis frame for both streams, compute the conversion ratio for the analysis frames based on the last two beat locations, covert to hop, apply sampling converter, apply current samples based on hop window and samples consumed. Repeat steps. See [Col 6 L32-48]).
Jochelson does not explicitly teach by a recursive algorithm (lacks an explicit determination of the sampling rate of the incoming streams and the analysis and adjustment of detected beat in real time)
Jehan discloses a recursive algorithm (A recursive algorithm operative to determine properties of incoming audio including the sample rate, see para 48).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of a recursive algorithm as taught by Jehan in Jochelson’s invention. The motivation would have been reducing signal complexity without perceptual loss. In para 47, in Jehan
Claim 2, 5 - 12 are rejected under 35 U.S.C. 103 as being unpatentable over Jochelson (US Patent) 7518053 B1 in view of Jehan (US Publication) 20070291958 A1, in further view of Sumita (US Publication) 20080034948 A1
Regarding claim 2, Jochelson does not explicitly teach wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm.
Sumita discloses wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations (To get the beat positions from the periodical peaks, the tempo-candidate detection apparatus (Fig 2 [102]) takes the average beat interval between peaks and then applies (Fig 5 [S102], the auto-correlation of the total increment values. See para 75-76. In Fig 10, the fluctuation calculation section, accounts for each of the N most recent adjacent beat intervals, see para 103-110), and the system determines whether or not the audio has a stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if the percentage does not exceed the permissible value P, the tapping position where it is stable is determine to be the starting beat position. See para 108-110).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm as taught by Sumita in Jochelson’s invention. The motivation would have been to provide a tempo detection apparatus and a tempo-detection computer program capable of detecting an tempo and beat positions without an error. In para 8, in Sumita
Regarding claim 5, Jochelson does not explicitly teach wherein the frame rate indicates a quantity of proxy audio frames in one second, and the proxy audio frame is obtained based on the frame size of an original audio frame and the overlapped frame size set by the system.
Sumita disclose wherein the frame rate indicates a quantity of proxy audio frames in one second (In Fig 4 [101b], the given frame size [which also the original: 512] is your proxy audio frame (see para 64), the sample rate is 3,675 Hz (see para 58), by deriving the sample rate by the hope size [original frame – overlap]; the frame rate is about 115 frame per second. Note: the reference explicitly state calculation but a person with knowledge in the art could replicate these calculations. Also, since the time resolution is 8.7 milliseconds (see para 64), derive one sec [1000 ms] by 8.7 ms provides the same frames per second, and the proxy audio frame is obtained based on the frame size of an original audio frame (the number of FFT is set to 512, which is the given frame size, see para 64) and the overlapped frame size set by the system (the window shift (hop size) is set to 32 samples and window overlap is (15/16), so the overlapped frame size is 480 samples. Though not show within the art, the original audio frame subtract from the overlap frame equals the hop, see para 64).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the frame rate indicates a quantity of proxy audio frames in one second, and the proxy audio frame is obtained based on the frame size of an original audio frame and the overlapped frame size set by the system as taught by Sumita in Jochelson’s invention. The motivation would have been, it is better to increase the time resolution with the frequency resolution suppressed. In para 63, in Sumita
Regarding claim 6, Jochelson does not explicitly teach wherein, in the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern according to a correlation of the frame information of the audio at different time points, and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function.
Sumita disclose wherein, in the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern (In Fig 5 [S102], the average beat interval can be obtained from the autocorrelation of the total of the incremental values in power of the notes in the scales. See para 76) according to a correlation of the frame information of the audio at different time points (See Fig 7, the time delay {tau} is an integer multiple of the period of peaks: L(t), phi(tau) becomes a large value. See para 78), and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function (the maximum value of phi(tau) is obtained in a prescribed range of {tau}, the temp of the musical piece is obtained, see para 78).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern according to a correlation of the frame information of the audio at different time points, and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function as taught by Sumita in Jochelson’s invention. The motivation would have been detecting an average beat interval (or tempo) and beat positions without an error. In para 8, in Sumita
Regarding claim 7, Jochelson does not explicitly teach wherein the auto-correlation function introduces an onset value and the onset value of each one of the proxy audio frames in a time period is calculated; wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period.
Sumita disclose wherein the auto-correlation function introduces an onset value (Fig 2 [310]) and the onset value of each one of the proxy audio frames in a time period is calculated (In Fig 5 [S100-S102], the L(t) is calculated at every frame time (t) across the entire beat period of the input waveform, specifically total value L(t) indicates the degree of change in all notes in each frame interval, where the value suddenly becomes large when notes start sounding. See para 71-73); wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period (The autocorrelation [phi(tau)] is calculated from L(t), when time delay is an integer multiple of the period becomes large and the maximum value of the autocorrelation in a range of tau, the tempo is obtained. This shows max values calculated from the onset values in the time period is determined for speculating the new beat period. See para 77-78).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of wherein the auto-correlation function introduces an onset value and the onset value of each one of the proxy audio frames in a time period is calculated; wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period as taught by Sumita in Jochelson’s invention. The motivation would have been detecting an average beat interval (tempo) and beat positions without an error. In para 8, in Sumita
Regarding claim 8, Jochelson does not explicitly teach wherein the system introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period, and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame; wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period, and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period.
Sumita disclose wherein the system (Fig 1, shows the structure of a personal computer, comprising: CPU, RAM, interface, and a series of other components, para 42) introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period (In Fig 4 [101a], the power of each note on the scale of the power spectrum, which is the maximum power correspond to the frequencies falling in the range of 50 cents above and below the fundamental frequency of each note in the scale is set to the power of the note. See para 66), and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame (beat detection value is the total incremental values of power across all notes at frame time, L(t), see para 72);
wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period (After obtaining the average beat interval, the degrees of changes of all notes at frames separated by beat intervals are added up with the starting frame being shifted by one frame, and the starting frame which maximizes the total value is regarded as the starting beat position [see para 6]. Following the tempo-candidate detection section [Fig 2 (102), specifically Fig 5 [S102], the average beat interval is obtained by applying autocorrelation, see para 76), and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period (In Fig 11, the next beat location is determined to be a position where cross-correlation b/w L(t) and M(t) becomes maximum in beat position, see para 111. Also, a third beat position is found the same way, see para 114).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of wherein the system introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period, and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame; wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period, and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period as taught by Sumita in Jochelson’s invention. The motivation would have been to determine beat positions as off-beats. In para 7, in Sumita
Regarding claim 9, Jochelson does not explicitly teach wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm.
Sumita discloses wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations (To get the beat positions from the periodical peaks, the tempo-candidate detection apparatus (Fig 2 [102]) takes the average beat interval between peaks and then applies (Fig 5 [S102], the auto-correlation of the total increment values. See para 75-76. In Fig 10, the fluctuation calculation section, accounts for each of the N most recent adjacent beat intervals, see para 103-110), and the system determines whether or not the audio has a stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if the percentage does not exceed the permissible value P, the tapping position where it is stable is determine to be the starting beat position. See para 108-110).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm as taught by Sumita in Jochelson’s invention. The motivation would have been to provide a tempo detection apparatus and a tempo-detection computer program capable of detecting a tempo and beat positions without an error. In para 8, in Sumita
Regarding claim 10, Jochelson does not explicitly teach wherein, when the system determines that the audio has the stable rhythm, a beat-prompting note is added at each beat location.
Sumita discloses wherein, when the system (Fig 1, shows the structure of a personal computer, comprising: CPU, RAM, interface, and a series of other components, para 42) determines that the audio has the stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if it does not exceed the permissible value P, the tapping position where it is stable. See para 108-110), a beat-prompting note (Fig 14, the sound, like a metronome, is played back at timing of the beat position, see para 122) is added at each beat location (Fig 14, the play-position pointer can be used to check for errors in any beat position while listening to music, para 122).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of when the system determines that the audio has the stable rhythm, a beat-prompting note is added at each beat location as taught by Sumita in Jochelson’s invention. The motivation would have been determination of detection errors. In para 122, in Sumita
Regarding claim 11, Jochelson teaches receiving and decoding an audio to acquire frame information of the audio (In Fig 1a, the input music (audio) is not excessively modified when being matched to another reference music or beat sound track. Modification is either time-scale or sampling rate conversions. See [Col 1 L61-65]), wherein the frame information includes a sampling rate (In Fig 1a, beat matching is applied on a “frame by frame” basis using a variable sampling rate converter to modify both the audio input stream to match the reference stream, see [Col 3 L65-67, Col 4 L1]);
obtaining a hop size according to a frame size and an overlapped frame size (Fig 5a, the frame overlap is the hop size, see [Col 14 L50-54]. Mentions the frame overlap with relation with hop window, see [Col 5 L52-65]), and calculating a frame rate according to the sampling rate and the hop size (Determine the hop # [or beats in a hop window] for the analysis frames. The overlap is calculated from duration of the both the hop and frame analysis, see [Col 5 L42-50]. Both the hop window of the first analysis and reference frames contain samples, [Col 5 L66-67, Col 6 L1-5]. Apply a sampling rate converter [ASRC] to adjust the timescale, advance current sample location, then extract the analysis frame both input and reference streams, see [Col 6 L6-54]); calculating an initial value of a beat period according to the sampling rate and an initial BPM (beats per minute) value (The initial sampling rate is 44.1 kHz, or 441K samples from a 10 sec interval, and a tempo of 120 BPM for about 20 locations in a frame, see [Col 4 L2-7]), so as to obtain a beat location based on a quantity of sampling points in a beat (the beat location generator can provide beat sample locations with simple increments by the product of the sampling rate, see [Col 4 L35-38]);
and speculating a next beat location by a recursive algorithm, wherein the frame rate is referred to for calculating a quantity of audio frames in a past period of time (The overlap is calculated from duration of the both the hop and frame analysis, see [Col 5 L42-50]), and a new beat period is calculated according to beats per minute (initial BPM is roughly 120bpm, see [Col 5 L58-60]), so as to speculate the next beat location according to the new beat period (The beat source replaces the reference stream, since the next virtual reference analysis frame is a product of the sampling rate multiplied by time from one beat to the next beat, which will be the next beat location, see [Col 6 L21-31]);
wherein the system is configured to re-calculate the new beat period according to the quantity of the audio frames in the past period of time at intervals, so as to re-speculate the next beat location (Extract the analysis frame for both streams, compute the conversion ratio for the analysis frames based on the last two beat locations, covert to hop, apply sampling converter, apply current samples based on hop window and samples consumed. Repeat steps. See [Col 6 L32-48]).
Jochelson does not explicitly teach by a recursive algorithm (lacks an explicit determination of the sampling rate of the incoming streams and the analysis and adjustment of detected beat in real time).
Jehan discloses a recursive algorithm (A recursive algorithm operative to determine properties of incoming audio including the sample rate, see para 48).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of by a recursive algorithm as taught by Jehan in Jochelson’s invention. The motivation would have been reducing signal complexity without perceptual loss. In para 47, in Jehan.
Jochelson does not explicitly teach which is installed in a device, the system comprising: an audio analysis module, wherein the audio analysis module is connected with an audio-processing circuit of the device for performing a method
Sumita discloses which is installed in a device (Fig 2 [100], input section, though not mentioned in detail, it is receiving component for a series of other devices, like microphone, CD-ROM, and etc. See para 54 ), the system (Fig 1, shows the structure of a personal computer, comprising: CPU, RAM, interface, and a series of other components, para 42) comprising: an audio analysis module (Fig 2, tempo detection apparatus, para 51), wherein the audio analysis module is connected with an audio-processing circuit (Fig 1 [11], CPU, para 42) of the device for performing a method (Fig 5, the procedure of processing performed by tempo-candidate detection section [102], para 69-70).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of installed in a device, the system comprising: an audio analysis module, wherein the audio analysis module is connected with an audio-processing circuit of the device for performing a method as taught by Sumita in Jochelson’s invention. The motivation would have been to provide a tempo detection apparatus and a tempo-detection computer program capable of detecting an average beat interval and beat positions without an error. In para 8, in Sumita.
Regarding claim 12, Jochelson does not explicitly teach wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm.
Sumita discloses wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations (To get the beat positions from the periodical peaks, the tempo-candidate detection apparatus (Fig 2 [102]) takes the average beat interval between peaks and then applies (Fig 5 [S102], the auto-correlation of the total increment values. See para 75-76. In Fig 10, the fluctuation calculation section, accounts for each of the N most recent adjacent beat intervals, see para 103-110), and the system determines whether or not the audio has a stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if the percentage does not exceed the permissible value P, the tapping position where it is stable is determine to be the starting beat position. See para 108-110).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm as taught by Sumita in Jochelson’s invention. The motivation would have been to provide a tempo detection apparatus and a tempo-detection computer program capable of detecting an tempo and beat positions without an error. In para 8, in Sumita
Claim 3 – 4, 13 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Jochelson (US Patent) 7518053 B1 in view of Jehan (US Publication) 20070291958 A1, in further view of Sumita (US Publication) 20080034948 A1, in further view of Yamashita (US Patent) 7534951 B2.
Regarding claim 3, Jochelson does not explicitly teach wherein, when the audio is a multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time for forming a mono-channel audio that is provided for real-time music rhythm analysis.
Jochelson does not explicitly teach multi-channel audio, an averaging operation
Sumita disclose multi-channel audio, an averaging operation (a digital signal received via a stereo signal, converted to monaural signal using a simplify the processing, see para 54.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of multi-channel audio, an averaging operation as taught by Sumita in Jochelson’s invention. The motivation would have been to simplify the processing. In para 54, in Sumita
Yamashita discloses wherein, when the audio is a multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time (Following figure 6, the power spectrum is commuted with respect to the input audio data by shifting the window by 1/8, so that an amount of 2 W/8 overlaps, see [Col 9 L37-41]. In determining the playback of the music content has been completed, in the power spectrum computation section [Fig 4 (211), also Fig 5 (B)], the window shifted by the amount of one division interval (W/8). The process then returns to [Fig 6 (step S1)], where it performs from step S1 to step S7 repletely, see [Col 10 L11-20]. The beat extraction is performed, an output of the waveform is shown in Fig 4 [21] is obtained in synchronization with the audio data, see [Col 10 L26-31]), for forming a mono-channel audio that is provided for real-time music rhythm analysis (By only playing back a CD, possible to recognize rhythm of music being played back in real time, see [Col 19 L60-67]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the audio is a pulse-code-modulation multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time for forming a mono-channel audio that is provided for real-time music rhythm analysis as taught by Yamashita in Jochelson’s invention. The motivation would have been to automatically recognize the rhythm of the piece of music being played back and possible to display lyrics in real time in time with the piece of music as in karaoke of the related art. In Col 19 L60-67, in Yamashita
Regarding claim 4, Jochelson does not explicitly teach wherein each of the audio frames in each channel of the multi-channel audio is the digital signals in a form of pulse-code modulation.
Yamashita disclose wherein each of the audio frames in each channel of the multi-channel audio (each musical performer is mixed down in the form of two channels of stereo, see Col L54-57) is the digital signals (In Fig 2 [111], I/O port, a digital audio signal that is transferred via system bus [100], see Col 5 L51-54) in a form of pulse-code modulation (A compact disc [CD], a simple audio waveform of PCM and is delivered to a user, see Col 1 L60-63).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of each of the audio frames in each channel of the multi-channel audio is the digital signals in a form of pulse-code modulation as taught by Yamashita in Jochelson’s invention. The motivation would have been to automatically recognize the rhythm of the piece of music being played back and possible to display lyrics in real time in time with the piece of music as in karaoke of the related art. In see Col 19 L64-67, in Yamashita
Regarding claim 13, Jochelson does not explicitly teach wherein, when the audio is a pulse-code-modulation multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time for forming a mono-channel audio that is provided for real-time music rhythm analysis.
Jochelson does not explicitly teach multi-channel audio, an averaging operation
Sumita disclose multi-channel audio, an averaging operation (a digital signal received via a stereo signal, converted to monaural signal using a simplify the processing, see para 54.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of multi-channel audio, an averaging operation as taught by Sumita in Jochelson’s invention. The motivation would have been to simplify the processing. In para 54, in Sumita.
Yamashita discloses wherein, when the audio is a pulse-code-modulation (Compact Disc, or CDs, use audio waveform of PCM, which is standard. See [Col 1 L54-63]) multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time (Following figure 6, the power spectrum is commuted with respect to the input audio data by shifting the window by 1/8, so that an amount of 2 W/8 overlaps, see [Col 9 L37-41]. In determining the playback of the music content has been completed, in the power spectrum computation section [Fig 4 (211), also Fig 5 (B)], the window shifted by the amount of one division interval (W/8). The process then returns to [Fig 6 (step S1)], where it performs from step S1 to step S7 repletely, see [Col 10 L11-20]. The beat extraction is performed, an output of the waveform is shown in Fig 4 [C] is obtained in synchronization with the audio data, see [Col 10 L26-31]) for forming a mono-channel audio that is provided for real-time music rhythm analysis (By only playing back a CD, possible to recognize rhythm of music being played back in real time, see [Col 19 L60-67]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the audio is a pulse-code-modulation multi-channel audio, an averaging operation is performed on digital signals of multiple audio frames at a same time for forming a mono-channel audio that is provided for real-time music rhythm analysis as taught by Yamashita in Jochelson’s invention. The motivation would have been to automatically recognize the rhythm of the piece of music being played back and possible to display lyrics in real time in time with the piece of music as in karaoke of the related art. In Col 19 L60-67, in Yamashita.
Regarding claim 14, Jochelson does not explicitly teach wherein the frame rate indicates a quantity of proxy audio frames in one second, and the proxy audio frame is obtained based on the frame size of an original audio frame and the overlapped frame size set by the system.
Sumita disclose wherein the frame rate indicates a quantity of proxy audio frames in one second (In Fig 4 [101b], the given frame size [which also the original: 512] is your proxy audio frame (see para 64), the sample rate is 3,675 Hz (see para 58), by deriving the sample rate by the hope size [original frame – overlap]; the frame rate is about 115 frame per second. Note: the reference explicitly state calculation but a personal knowable of the art do the calculations. Also, since the time resolution is 8.7 milliseconds (see para 64), derive one sec [1000 ms] by 8.7 ms provides the same frames per second, and the proxy audio frame is obtained based on the frame size of an original audio frame (the number of FFT is set to 512, which is the given frame size, see para 64) and the overlapped frame size set by the system (the window shift (hop size) is set to 32 samples and window overlap is (15/16), so the overlapped frame size is 480 samples. Though not show within the art, the original audio frame subtract from the overlap frame equals the hop, see para 64).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the frame rate indicates a quantity of proxy audio frames in one second, and the proxy audio frame is obtained based on the frame size of an original audio frame and the overlapped frame size set by the system as taught by Sumita in Jochelson’s invention. The motivation would have been it is better to increase the time resolution with the frequency resolution suppressed. In para 63, in Sumita
Regarding claim 15, Jochelson does not explicitly teach wherein, in the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern according to a correlation of the frame information of the audio at different time points, and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function.
Sumita disclose wherein, in the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern (In Fig 5 [S102], the average beat interval can be obtained from the autocorrelation of the total of the incremental values in power of the notes in the scales. See para 76) according to a correlation of the frame information of the audio at different time points (See Fig 7, the time delay {tau} is an integer multiple of the period of peaks: L(t), phi(tau) becomes a large value. See para 78), and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function (the maximum value of phi(tau) is obtained in a prescribed range of {tau}, the temp of the musical piece is obtained, see para 78).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the process of calculating the new beat period, an auto-correlation function is adopted to find out a repeating pattern according to a correlation of the frame information of the audio at different time points, and the new beat period is re-calculated based on a period having a maximum of the auto-correlation function as taught by Sumita in Jochelson’s invention. The motivation would have been detecting an average beat interval (or tempo) and beat positions without an error. In para 8, in Sumita
Regarding claim 16, Jochelson does not explicitly teach wherein the auto-correlation function introduces an onset value and the onset value of each one of the proxy audio frames in a time period is calculated; wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period.
Sumita disclose wherein the auto-correlation function introduces an onset value (Fig 2 [310]) and the onset value of each one of the proxy audio frames in a time period is calculated (In Fig 5 [S100-S102], the L(t) is calculated at every frame time (t) across the entire beat period of the input waveform, specifically total value L(t) indicates the degree of change in all notes in each frame interval, where the value suddenly becomes large when notes start sounding. See para 71-73); wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period (The autocorrelation [phi(tau)] is calculated from L(t), when time delay is an integer multiple of the period becomes large and the maximum value of the autocorrelation in a range of tau, the tempo is obtained. This shows max values calculated from the onset values in the time period is determined for speculating the new beat period. See para 77-78).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of wherein the auto-correlation function introduces an onset value and the onset value of each one of the proxy audio frames in a time period is calculated; wherein a maximum of the onset values or values calculated from the onset values in the time period is determined for speculating the new beat period as taught by Sumita in Jochelson’s invention. The motivation would have been detecting an average beat interval (so-called tempo) and beat positions without an error. In para 8, in Sumita
Regarding claim 17, Jochelson does not explicitly teach wherein the system introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period, and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame; wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period, and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period.
Sumita disclose wherein the system (Fig 1, shows the structure of a personal computer, comprising: CPU, RAM, interface, and a series of other components, para 42) introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period (In Fig 4 [101a], the power of each note on the scale of the power spectrum, which is the maximum power correspond to the frequencies falling in the range of 50 cents above and below the fundamental frequency of each note in the scale is set to the power of the note. See para 66), and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame (beat detection value is the total incremental values of power across all notes at frame time, L(t), see para 72);
wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period (In Fig 6b, following Fig 5a, using harmonics and DFT magnitudes for each BPM is calculated, these estimates are used to selected the next beat period), and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period (In Fig 11, the next beat location is determined to be a position where cross-correlation b/w L(t) and M(t) becomes maximum in beat position, see para 111. Also, a third beat position is found the same way, see para 114).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of wherein the system introduces a beat detection value for calculating a maximum beat characteristic value of the sampling points around the onset values of the proxy audio frames in the time period, and the maximum beat characteristic value acts as the beat detection value of the proxy audio frame; wherein the beat detection values of all of the beats in a past beat period are further referred to for speculating beat detection values in a next beat period, and a maximum of the beat detection values in the next beat period is regarded as a beat location of the next beat period as taught by Sumita in Jochelson’s invention. The motivation would have been to determine beat positions as off-beats. In para 7, in Sumita
Regarding claim 18, Jochelson does not explicitly teach wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm.
Sumita discloses wherein, after the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations (To get the beat positions from the periodical peaks, the tempo-candidate detection apparatus (Fig 2 [102]) takes the average beat interval between peaks and then applies (Fig 5 [S102], the auto-correlation of the total increment values. See para 75-76. In Fig 10, the fluctuation calculation section, accounts for each of the N most recent adjacent beat intervals, see para 103-110), and the system determines whether or not the audio has a stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if the percentage does not exceed the permissible value P, the tapping position where it is stable is determine to be the starting beat position. See para 108-110).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the next beat location of the new beat period is obtained, multiple time differences of multiple adjacent beat locations are calculated by retracing multiple beat locations, and the system determines whether or not the audio has a stable rhythm as taught by Sumita in Jochelson’s invention. The motivation would have been to provide a tempo detection apparatus and a tempo-detection computer program capable of detecting an tempo and beat positions without an error. In para 8, in Sumita.
Regarding claim 19, Jochelson does not explicitly teach wherein, when the system determines that the audio has the stable rhythm, a beat-prompting note is added at each beat location.
Sumita discloses wherein, when the system (Fig 1, shows the structure of a personal computer, comprising: CPU, RAM, interface, and a series of other components, para 42) determines that the audio has the stable rhythm (Fig 10 [S406], the section calculates, for each beat interval, a beat fluctuation percentage and if it does not exceed the permissible value P, the tapping position where it is stable. See para 108-110), a beat-prompting note (Fig 14, the sound, like a metronome, is played back at timing of the beat position, see para 122) is added at each beat location (Fig 14, the play-position pointer can be used to check for errors in any beat position while listening to music, para 122).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of when the system determines that the audio has the stable rhythm, a beat-prompting note is added at each beat location as taught by Sumita in Jochelson’s invention. The motivation would have been determination of detection errors. In para 122, in Sumita
Regarding claim 20, Jochelson does not explicitly teach wherein, when the audio-processing circuit outputs the processed audio via an output interface, the audio is synthesized with the beat-prompting note that is added at each beat location.
Sumita discloses wherein, when the audio-processing circuit (Fig 1 [11], CPU) outputs the processed audio via an output interface (Fig 1 [15], I/O interphase), the audio is synthesized with the beat-prompting note (Fig 14, the sound, like a metronome, is played back at timing of the beat position, see para 122) that is added at each beat location.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of the audio-processing circuit outputs the processed audio via an output interface, the audio is synthesized with the beat-prompting note that is added at each beat location as taught by Sumita in Jochelson’s invention. The motivation would have been determination of detection errors. In para 122, in Sumita
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
Katekar (US Publication) 20250259655 A1 – interface and beat tracking module with synchronization
Maezawa (US Publication) 20140260911 A1- Tempo stability judgment, not specifically audio frames
Sasaki (US Publication) 20070169614 A1: prompting sound for a beat, similar to a metronome.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCUS A BARBOZA whose telephone number is (571)272-9626. The examiner can normally be reached Monday-Friday 7:30 am to 5 pm, Alternate Fridays: off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carolyn R. Edwards can be reached at 571-270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARCUS A BARBOZA/ Examiner, Art Unit 2692
/CAROLYN R EDWARDS/Supervisory Patent Examiner, Art Unit 2692
Prosecution Insights