DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Comments
Based upon an incorrectly cited prior art reference to Hodgson, the non final rejection filed 3-9-2026 has been vacated and a new nonfinal rejection has been submitted.
Double Patenting
The terminal disclaimer filed has been approved and the double patenting rejection has been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) ) 1,2,7,8,10,11,12,13,15-18,20,21,24,25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bogdanov (US 20060149533 A1), and further in view of Bakar et al (US 20140282695 A1).
As per claim 1, Bogdanov discloses a system comprising: memory; and a processor operatively coupled to the memory and configured to:
define a plurality of trigger points at different time locations within an audio signal (fig. 1, and para. 30, where the trigger points are the points from within each audio file from which each fingerprint is obtained and the audio signal is the collection of audio files as read by stage 120, additionally, the multiple trigger points each associated with a fingerprint are defined in multiple times at different time points for a given song per para 46, additionally the multiple fingerprints per song which each have different times/triggerpoints based on the shifting in para 46, );
where trigger points from the plurality of trigger points are associated with different timing offsets from a same time point of the audio signal (the processor has defined multiple trigger points for a fingerprint which are the time shifted fingerprints which are offset in time from each other and also from any other common reference point per para 46)
generate, based on the audio signal, a fingerprint for each trigger point of the plurality of trigger points, wherein each of the fingerprints characterizes a different frame of the audio signal (each fingerprint is for a different audio file which each comprises a different frame of the audio signal, also each time shifted); and
associate a different event of the plurality of trigger points (an event defined by the respective metadata for each respective audio file and associated with each respective fingerprint at the respective trigger point per para. 34, notably the supplemental content in para 21: a retrieval engine may be utilized to match supplemental content to the fingerprints, which is a set of respective different events). However, Bogdanov does not specify:
wherein at least one event of the events includes play-back of other audio at a time corresponding to a time location of the trigger point corresponding to the at least one event, simultaneously with and in synchronization with play-back of the audio signal and
Bakar teaches that watermarks can be used to trigger playback of advertisements ((the ad audio is the other audio)) (abstract). Bakar further discloses that the watermark can be detected during playback of the audio per (para 34 the received audio is presented, para 44 the watermark can be a sound, para. 35 the ad is caused to be presented, para 26, the ad can be added to the presented media content ) It would have been obvious to one skilled in the art to trigger advertisements based off the detection of the watermark of Bogdanov for the purpose of implementing advertising.
As per claim 2, the system of claim 1, wherein each trigger point is associated with a timing offset from a start of the audio signal (the predetermined time per para. 38 and also the time shifted fingerprints per para 45).
As per claim 7, the system of claim 1, wherein a plurality of trigger points are defined (the trigger points defined by the boundaries of each of the 155 frames cited in para. 43, and the another 155 frames cited in para. 50) , a plurality of fingerprints corresponding to the plurality of trigger points are generated (the fingerprint cited in para. 43-46 and the another fingerprint cited in para.50), and each of the fingerprints characterizes a different frame of the audio signal (either a frame within the first 155 frames, or the entire collection of 155 frames read as a ‘frame’), each frame having a fixed duration of a same width (frames by definition have a fixed duration/width).
As per claim 8, The system of claim 1, wherein the audio signal comprises a mono uncompressed audio (para. 40, the summed L and R signals are a mono signal, and the audio is not compressed).
As per claim 10, The system of claim 1, wherein each fingerprint comprises a fingerprint width A recognized width is required to form a finger in a fingerprint), fingerprint identification number (para. 30 the identifier associated with the audio from which the fingerprint was derived), a number of peaks in the fingerprint (a fingerprint is defined by a number of peaks which are part of defining said finger), and coordinates of each peak in the fingerprint (the peaks as determined by the digital processor implementing the fingerprint creation module are determined in distinct memory areas/coordinates for each finger and its associated peak because a fingerprint is defined by a relative set of fingers that each must be stored in respective registers in order to be recognized and processed as per the claim 1 rejection.
As per claim 11, The system of claim 1, wherein each fingerprint comprises a number of peaks in the fingerprint and coordinates of each peak in the fingerprint (as required in order for a digital processor to represent and use a fingerprint as cited in the claim 1 rejection).
As per claim 12, the system of claim 1, further comparing: a first device comprising the memory and the processor; wherein the processor of the first device is further configured to transmit, to the second device, fingerprint information corresponding to the generated fingerprint (client pc 700, para. 60),
and a second device (recognition server 750 para. 62), wherein the second device is configured to:
identify a trigger point within an audio signal corresponding a fingerprint detected in the audio signal based on the fingerprint information (the point in time where the fingerprint is recognized by the recognition server); and
trigger an event associated with the fingerprint (para. 64), a timing of the event synchronized to the identified trigger point within the audio signal (the recognition server is in sync with/based on the client pc identifying the trigger point).
As per claim 13, the system of claim 1, further comparing:
a first device comprising the memory and the processor; and a second device, wherein the processor of the first device is further configured to transmit, to the second device, fingerprint information corresponding to the generated fingerprint and audio signal (client pc 700, para. 62),
wherein the second device is configured to:
receive a second audio signal (the fingerprint based on/corresponding to the first audio signal from the first device, recognition server 750 para. 62) from the first device;
detect a trigger point within the audio signal (per claim 12 rejection), wherein detecting the trigger point within the audio signal includes:
processing the audio signal to provide a plurality of frames (the 155 frames per the claim 7 rejection),
and identifying, in the plurality of frames, a frame including a fingerprint associated with the trigger point based on the fingerprint information (the detection of the fingerprint is based on frames including a frame); and trigger an event associated with the fingerprint in the first frame, a timing of the event synchronized to the detected trigger point within the audio signal (per the claim 12 rejection).
As per claim 15, the claim 1 rejection discloses a method performed by a device comprising a processor and memory, the method comprising: defining one or more trigger points at one or more different time locations within an audio signal; where trigger points from the plurality of trigger points are associated with different timing offsets from a same time point of the audio signal ; generating, based on the audio signal, a fingerprint for each trigger point of the plurality of trigger points, wherein each of the fingerprints characterizes a different frame of the audio signal; and associating a different event of the plurality of trigger points.;(per the claim 1 rejection);
However Bogdanov does not specify
wherein at least one event of the events includes play-back of other audio at a time corresponding to a time location of the trigger point corresponding to the at least one event simultaneously with and in synchronization with play-back of the audio signal.
Bakar teaches that watermarks can be used to trigger playback of advertisements ((the ad audio is the other audio)) (abstract). Bakar further discloses that the watermark can be detected during playback of the audio per (para 34 the received audio is presented, para 44 the watermark can be a sound, para. 35 the ad is caused to be presented, para 26, the ad can be added to the presented media content ) It would have been obvious to one skilled in the art to trigger advertisements based off the detection of the watermark of Bogdanov for the purpose of implementing advertising.
As per claim 16, the method of claim 15, wherein the fingerprint is generated from a plurality of peaks identified within a Fast Fourier Transform (FFT) of the audio track at the time location of the fingerprint (per the claim 3 rejection).
As per claim 17, the method of claim 16, wherein each of the plurality of peaks is a local magnitude maxima of a 2D coordinate space created by each FFT block (per claim 4 rejection).
As per claim 18, The method of claim 16, wherein each of the plurality of peaks in the fingerprint is separated from other peaks in the fingerprint by a minimum spacing (per claim 5 rejection).
As per claim 20, the method of claim 15, wherein each fingerprint comprises a fingerprint width, fingerprint identification number, a number of peaks in the fingerprint, and coordinates of each peak in the fingerprint (per claim 10 rejection).
As per claim 21, A non-transitory computer readable medium having stored therein computer- readable instructions that, when executed by one or more processors, cause an apparatus connected to the one or more processors to: define one or more trigger points at one or more time locations within an audio signal; where trigger points from the plurality of trigger points are associated with different timing offsets from a same time point of the audio signal generate, based on the audio signal, a fingerprint for each trigger point of the plurality of trigger points, wherein each of the fingerprints characterizes a different frame of the audio signal; and associating a different event of the plurality of trigger points.;(per the claim 1 rejection).
; and associate an event with each trigger point (the processor based system of the claim 1 rejection requires memory and software to implement the cited functions);
However Bogdanov does not specify
wherein at least one event of the events includes play-back of other audio at a time corresponding to a time location of the trigger point corresponding to the at least one event simultaneously with and in synchronization with play-back of the audio signal and.
Bakar teaches that watermarks can be used to trigger playback of advertisements ((the ad audio is the other audio)) (abstract) Bakar further discloses that the watermark can be detected during playback of the audio per (para 34 the received audio is presented, para 44 the watermark can be a sound, para. 35 the ad is caused to be presented, para 26, the ad can be added to the presented media content ) . It would have been obvious to one skilled in the art to trigger advertisements based off the detection of the watermark of Bogdanov for the purpose of implementing advertising.
As per claim 24, the claim 1 rejection discloses the system of claim 1, but the cited prior art does not specify:
wherein at least one of the events includes controlling of electronic game-play coordinated to the audio signal;
The examiner takes official notice it would have been obvious to one skilled in the art at the time of filing that advertisements can be used within video games for the purpose of providing advertising.
As per claim 25, the system of claim 1, wherein the different event include generation of mechanical movements in synchronization with time locations of plurality of trigger points while controlling speakers/apparatuses (the speakers required to playback the audio comprise mechanical apparatus that playback the ad, via the processor implementing the watermark detection and triggering).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-6,14,22,23,26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bogdanov (US 20060149533 A1) and further in view of Bakar et al (US 20140282695 A1) as applied to claim 1 above, and further in view of Gurijala et al (US 10236031 B1).
As per claim 22, Bogdanov discloses an apparatus comprising: memory; and a processor operatively coupled to the memory and configured to control the apparatus to:
define a plurality of trigger points within an audio signal, each of the trigger points corresponding to a different time location within the audio signal (the trigger points defined by the boundaries of each of the 155 frames cited in para. 43, and the another 155 frames cited in para. 50) ,);
where trigger points from the plurality of trigger points are associated with different timing offsets from a same time point of the audio signal (per the claim 1 rejection)
generate, a fingerprint for each trigger point, wherein each fingerprint comprises a plurality of peaks identified in the audio signal at the time location corresponding to the trigger point (per claim 1 rejection); and
associate an event with each trigger point. (as per the claim 1 rejection);
However Bogdanov does not specify
wherein at least one event of the events includes play-back of other audio at a time corresponding to a time location of the trigger point corresponding to the at least one event simultaneously with and in synchronization with play-back of the audio signal.
Bakar teaches that watermarks can be used to trigger playback of advertisements ((the ad audio is the other audio)) (abstract) Bakar further discloses that the watermark can be detected during playback of the audio per (para 34 the received audio is presented, para 44 the watermark can be a sound, para. 35 the ad is caused to be presented, para 26, the ad can be added to the presented media content ). It would have been obvious to one skilled in the art to trigger advertisements based off the detection of the watermark of Bogdanov for the purpose of implementing advertising.
However Bogdanov does not specify.
Generating the fingerprint based on peaks identified within a Fast Fourier Transform (FFT) of the audio signal.
Gurijala teaches that audio recognition platforms can use FFT an FFT to identify a fingerprint, which is a plurality of identified peaks (para 447: fingerprints for audio-video signal segments include frequency domain peak locations, or fingerprints formed from the amplitude change in Fourier magnitude values in bands of sub-segments within an audio or video signal segment). It would have been obvious for one skilled in the art at the time of filing to use an FFT for the purpose of using a well known process to determine the fingerprint in the system of Bogdanov.
As per claim 23, the apparatus of claim 22, wherein each fingerprint comprises a fingerprint width, fingerprint identification number, a number of peaks in the fingerprint, and coordinates of each peak in the fingerprint (per the claim 10 rejection).
As per claim 3, Bogdanov discloses the system of claim 1, but does not specify wherein the fingerprint is generated from a plurality of peaks identified within a Fast Fourier Transform (FFT) of the audio signal at the time location of the fingerprint.
Gurijala teaches that audio recognition platforms can use FFT an FFT to identify a fingerprint, which is a plurality of identified peaks (para. 447). It would have been obvious for one skilled in the art at the time of filing to use an FFT for the purpose of using a well known process to determine the fingerprint in the system of Bogdanov.
As per claim 26, the system of claim 1, wherein each fingerprint comprises a plurality of peaks per the same obviousness analysis per the claim 3 rejection, and generating the fingerprint comprises:
reading the audio signal in Fast Fourier Transform (FFT) sized buffers for performing FFT on the audio signal (required in order to implement the cited FFT processing);
deriving a magnitude spectrum from the FFT of the audio signal (the output of a fft is by definition a derived magnitude spectrum); and
classifying peaks in the magnitude spectrum as peaks of the fingerprint (the identifying step taught by Gurijala as per the claim 3 rejection).
As per claim 4, The system of claim 3, wherein each of the plurality of peaks is a local magnitude maxima of a 2D coordinate space created by each FFT block (the spectrum created by the fft is a 2d coordinate space and digital fingerprints are by definition a collection of local magnitude maxima).
As per claim 5, The system of claim 3, wherein each of the plurality of peaks in the fingerprint is separated from other peaks in the fingerprint by a minimum spacing (The cited fingerprints in the claim 1-3 rejections are by definition defined by a minimum spacing from other fingers because a finger is by definition a known structure which is delineated by time and amplitude from other fingers).
As per claim 6, The system of claim 5, wherein the minimum spacing includes a minimum width and a minimum height (The cited fingerprints in the claim 1-3 rejections are by definition defined by a minimum spacing from other fingers because a finger is by definition a known structure which is delineated by time (minimum width) and amplitude (minimum height) from other fingers).
As per claim 14, Bogdanov discloses the system of claim 13, but does not disclose wherein the second device comprises a microphone and the second device receives the audio signal via the microphone.
The examiner takes official notice it is well known in the art that audio signals can be transferred/received via microphones for the purpose of capturing acoustic audio, additionally it is well known to use microphones/audio as a means of communication between devices.
Response to Arguments
The submitted arguments have been considered but are moot in view of the new grounds of rejection.
As per applicant’s argument’s that Bakar does not teach playing back the audio with the retrieved/triggered audio playback, Bakar para 26 teaches to add to or replace a portion of the media content, where adding to would be simultaneous playback.
Allowable Subject Matter
Claims 9,19 would be objected to as being dependent upon a rejected base claim, and would be allowable over the prior art of record if rewritten in independent form including all of the limitations of the base claim and any intervening claims, assuming the double patenting is overcome with a terminal disclaimer.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER KRZYSTAN whose telephone number is 571-272-7498, and whose email address is alexander.krzystan@uspto.gov
The examiner can usually be reached on m-f 7:30-4:00 est.
If attempts to reach the examiner by telephone or email are unsuccessful, the examiner’s supervisor, Fan Tsang can be reached on (571) 272-7547.
The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications.
/ALEXANDER KRZYSTAN/Primary Examiner, Art Unit 2653
Examiner Alexander Krzystan
August 18, 2026
Prosecution Insights