Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claim 14 objected to because of the following informalities: the claim comprises a solitary instance of enumeration “the time-series data to 1.) make a hard decision…”. The recitation thereof appears superfluous to the claim and the absence of further enumeration conveys the appearance of incompleteness on the remainder of the claim. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 5, 10, 20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Claim 5 recites “the detected sound event,” the chain of antecedent of the recitation is ambiguous and requires correction. Claims 10 and 20 recite “the entire time segment,” and “the segment,” respectively neither resolve a clear antecedent.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1, 14 rejected under 35 U.S.C. 102a1 as being anticipated by Wang: 10418957.
Regarding claim 1
Wang teaches:
A computer-implemented method for event detection in time-series data, wherein the method uses a processor coupled with a memory configured to store instructions implementing the method, wherein the instructions, when executed by the processor carry out steps of the method, comprising:
processing the time-series data (Wang: Col 3:63-4:14; Fig 1B: system receives audio data, samples and subsamples same into data with a coarser and/or finer time scale generates scores based thereon) to make a hard decision on a time span of an event indicative of continuous activity of the event within the time-series data (Wang: Col 3:29-3:42; 11:35-11:60; Fig 1A, 1B, 6: region proposal network (RPN) processes the time series data to output begin and end times of a determined interval of an audio event, the hard decision is considered an initial decision)
and make a soft decision on a presence of the event for the entire time span (Wang: Col 3:32-3:62; 5:20-5:38: generation of a likelihood score which “indicates how likely an audio event is detected during that interval of time,” applied to each of a plurality of frames, times therein, the soft decision is considered a further processed decision);
applying an event-level threshold to the soft decision on the presence of the event for the entire time span to produce a result of the event detection (Wang: Col 3:32-3:62; 5:20-5:38, 11:35-12:8: threshold applied to each frame, each of likelihood scores therein to determine and output a result); and
outputting the result of the event detection (Wang: Col 3:32-3:62; 5:20-5:38: threshold applied to each frame, each of likelihood scores therein to determine and output a result).
Regarding claim 14—the claim is considered to recite substantially similar subject matter to claim 1 under 35 USC 102 as discussed supra and is similarly rejected
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-11, 14-20 rejected under 35 U.S.C. 103 as being unpatentable over Wang: 10418957 further in view of Cakir: Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection (copy provided by Examiner; copyright 2017; and hereinafter Cak).
Regarding claim 1
Wang teaches:
A computer-implemented method for event detection in time-series data, wherein the method uses a processor coupled with a memory configured to store instructions implementing the method, wherein the instructions, when executed by the processor carry out steps of the method, comprising:
processing the time-series data (Wang: Col 3:63-4:14; Fig 1B: system receives audio data, samples and subsamples same into data with a coarser and/or finer time scale generates scores based thereon) to make a hard decision on a time span of an event indicative of continuous activity of the event within the time-series data (Wang: Col 3:29-3:42; 11:35-11:60; Fig 1A, 1B, 6: region proposal network (RPN) processes the time series data to output begin and end times of a determined interval of an audio event)
and make a soft decision on a presence of the event for the entire time span (Wang: Col 3:32-3:62; 5:20-5:38: generation of a likelihood score which “indicates how likely an audio event is detected during that interval of time,” applied to each of a plurality of frames, times therein);
applying a threshold to the soft decision on the presence of the event for the entire time span to produce a result of the event detection (Wang: Col 3:32-3:62; 5:20-5:38, 11:35-12:8: threshold applied to each frame, each of likelihood scores therein to determine and output a result); and
outputting the result of the event detection (Wang: Col 3:32-3:62; 5:20-5:38: threshold applied to each frame, each of likelihood scores therein to determine and output a result).
It may be argued that Wang strongly suggests but does not explicitly teach a broadly reasonable interpretation of the recited applying an event-level threshold such as by thresholding candidates against an entire time span with respect to one or more events.
In a related field of endeavor Cak teaches a system and method for polyphonic sound event detection (Cak: Abstract) comprising receiving and processing time series data (Cak: Fig 1: timewise segments of input audio analyzed to detect particular events, classes thereof therein); determining a decision on a time span of an event indicative of continuous activity of the event within the time-series data by utilizing an event-level threshold based on a presence of the event for the entire time span such as by applying the threshold to the decision on the presence of the event for the entire time span to produce a result of the event class detection (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: system determines event activity probabilities for particular event classes which, when present, binarize a frame to 1 by thresholding the detected class such that consecutive frames exceeding a threshold with respect to a particular class form an activity region for the particular class which is detected as present therein based on onset/offset times thereof such as comprised within a final event class activity prediction). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the event class thresholding taught or suggested by Cak to thereby reify the soft decision of Wang for at least the purpose of determining the presence of an event and/or event class in each and all of the consecutive frames which represent the presence of the sound event and for at least the purpose of improving the decisioning of Wang by the inclusion of polyphone detection; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 2
Wang in view of Cak teaches or suggests:
The method of claim 1, wherein the soft decision is an event bounding box that comprises the time span of the event (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size and stride and/or to segment window of particular lengths into a determined number of segments); (Cak: § II, A; Fig 1: such as by resolving or determining of onset/offset times with respect to particular sound event classes); a type of the event (Wang: Col 13:20-13:27; Fig 6: such as by detecting a particular event of a particular type); (Cak: § II, A; Fig 1: such as particular sound event classes), and a confidence score for the presence of the event (Wang: Abstract, etc.: such as a likelihood score that individual, pluralities, etc. of frames correspond to a sound event); (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by determining sound class activity prediction scores, thresholding same). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 3
Wang in view of Cak teaches or suggests:
The method of claim 2, wherein the time-series data comprises an audio stream, wherein the event is a sound event (Wang: Abstract); (Cak: Abstract), and wherein the soft decision is a sound event bounding box that comprises a sound class of the sound event as the type of the event (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size, and/or stride; and/or to segment windows of determined lengths into a determined number of sub-segments); (Cak: § II, A; Fig 1: such as by resolving or determining of onset/offset times with respect to particular sound event classes), the time span as an extent of the sound event in the audio stream (Wang: Col 13:20-13:27; Fig 6: such as by detecting a particular event of a particular type); (Cak: § II, A; Fig 1: such as particular sound event classes), and an overall confidence score indicating the probability of presence of the sound class in the sound event (Wang: Abstract, etc.: such as a likelihood score that individual, pluralities, etc. of frames correspond to a sound event); (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by determining sound class activity prediction scores, thresholding same). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 4
Wang in view of Cak teaches or suggests:
The method of claim 3, further comprising controlling a machine based on the result of the event detection (Wang: Col 5:32-5:39; Fig 1A, 1B, etc.: such as causing an action to be performed based thereon). While Wang in view of Cak does not explicitly discusses “controlling a machine,” merely causing an action, turning on data logging, or causing a command to be executed this is considered substantially similar as output commands for causing an action such as the change of state of a switch may additionally cause a processor, device, or machine to awaken, sleep, change power modes, change state, etc. Examiner takes official notice that controlling a machine as a result of an output command was well-kwon in the art before the effective filing date of the instant application and would have comprised an obvious inclusion; one of ordinary skill in the art would have expected only predictable results therefrom. Thus the claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 5
Wang in view of Cak teaches or suggests:
The method of claim 3, further comprising identifying a source of a sound associated with the detected sound event, based on the result of the event detection (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size and stride and/or to segment window of particular lengths into a determined number of segments); (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by resolving or determining of onset/offset times class-wise prediction values with respect to particular sound event classes and subsequently binarizing same using a threshold thereby presence of a particular sound class based on activity prediction scores, thresholding, etc.). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 6
Wang in view of Cak teaches or suggests:
The method of claim 3, wherein making the hard decision on the time span comprises identifying frames in the audio stream that belong to the sound class (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size and stride and/or to segment window of particular lengths into a determined number of segments); (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by resolving or determining of onset/offset times class-wise prediction values with respect to particular sound event classes and subsequently binarizing same using a threshold thereby presence of a particular sound class based on activity prediction scores, thresholding, etc.). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 7
Wang in view of Cak teaches or suggests:
The method of claim 6, wherein identifying the frames that belong to the sound class comprises:
computing a probability of presence of the sound class in each frame of the audio stream; and applying a class threshold to the probability of presence of the sound class in each frame to filter a plurality of frames whose probability of presence exceeds the class threshold (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size and stride and/or to segment window of particular lengths into a determined number of segments); (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by resolving or determining of onset/offset times class-wise prediction values with respect to particular sound event classes and subsequently binarizing same using a threshold thereby presence of a particular sound class based on activity prediction scores, thresholding, etc.). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 8
Wang in view of Cak teaches or suggests:
The method of claim 7, wherein making the hard decision on the time span further comprises determining a start time instance and an end time instance of the time span based on the plurality of filtered frames (Wang: Col 12:11-12:25, 12:55-12:67: filter manages overlapping front and/or back ends of time windows in such a way as to adjust start and end time instances such that the decisions are conducted on the best fit time windows); (Cak: § III. B, D, neural network configurations: a filter shape varied to determent particular sequence lengths with respect to variations upon time axes, frames, corresponding seconds). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 9
Wang in view of Cak teaches or suggests:
The method of claim 8, wherein a time of occurrence of a sequentially first frame of the plurality of filtered frames is selected as the start time instance of the time span and a time of occurrence of a sequentially last frame of the plurality of filtered frames is selected as the end time instance of the time span (Wang: Col 12:11-12:25, 12:55-12:67: filter manages overlapping front and/or back ends of time windows in such a way as to adjust start and end time instances such that the decisions are conducted on the best fit time windows). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 10
Wang in view of Cak teaches or suggests:
The method of claim 1, wherein processing the time-series data to make the soft decision on the presence of the event for the entire time segment comprises
computing a composite confidence score based on the probability of presence of the sound class in each frame of the entire time span (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by determining sound class activity prediction scores, thresholding same to arrive at a binary confidence score, that is a 1 indicating presence of a particular class over a particular span). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 11
Wang in view of Cak teaches or suggests:
The method of claim 10, wherein the computing includes one or more of obtaining the average, obtaining the maximum, obtaining the minimum, or obtaining the median of the probability of presence of the sound class in each frame of the entire time span (Cak: § II, A; § II, B, 4_binarization; Figs 1, 2(4), 3: such as by determining sound class activity prediction scores, thresholding same to arrive at a binary confidence score, wherein a maximum of the prediction score exceeding a threshold is binarized such that a 1 indicates presence of a particular class of a span). While Wang in view of Cak does not explicitly discuss selections among machine learning processes of averaging, maximizing, minimizing, taking a median, etc. Examiner takes official notice that such processes were well-known in the art before the effective filing date of the instant application and would have comprised an obvious inclusion such as for applying user desired machine learning processes to parameters pre or post processed based thereon. Thus the claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 14—the claim is considered to recite substantially similar subject matter to claim 1 under 35 USC 103 as discussed supra and is similarly rejected
Regarding claim 15
Wang in view of Cak teaches or suggests:
The method of claim 14, wherein the time-series data comprises an audio stream (Wang: Abstract); (Cak: Abstract),, and wherein the soft decision comprises a sound class of a detected sound event in the audio stream (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event a bounding box, such as different feature matrices with particular feature size and stride and/or to segment window of particular lengths into a determined number of segments); (Cak: § II, A; Fig 1: such as by resolving or determining of onset/offset times with respect to particular sound event classes) and the time span as an extent of the detected sound event in the audio stream (Wang: Col 13:20-13:27; Fig 6: such as by detecting a particular event of a particular type); (Cak: § II, A; Fig 1: such as particular sound event classes). The claim is considered obvious over Wang as modified by Cak as addressed in the base claim as it would have been obvious to apply the further teaching of Wang and/or Cak to the modified device of Wang and Cak; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 16—the claim is considered to recite substantially similar subject matter to claim 4 under 35 USC 103 as discussed supra and is similarly rejected
Regarding claim 17—the claim is considered to recite substantially similar subject matter to claim 5 under 35 USC 103 as discussed supra and is similarly rejected
Regarding claim 18—the claim is considered to recite substantially similar subject matter to claim 6 under 35 USC 103 as discussed supra and is similarly rejected
Regarding claim 19—the claim is considered to recite substantially similar subject matter to claim 7 under 35 USC 103 as discussed supra and is similarly rejected
Regarding claim 20—the claim is considered to recite substantially similar subject matter to claim 8 under 35 USC 103 as discussed supra and is similarly rejected
Claims 12 rejected under 35 U.S.C. 103 as being unpatentable over Wang: 10418957 further in view of Cakir: Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection (copy provided by Examiner; copyright 2017 and hereinafter Cak) as applied to claims 1-11, 14-20 supra and further in view of Ema: 20210405909.
Regarding claim 12
Wang in view of Cak teaches or suggests:
The method of claim 3, further comprising:
computing for each frame of the audio stream, a class presence confidence score as a probability of presence of the sound class in each frame of the audio stream (Wang: Col 13:20-13:27; Fig 6: such as by detecting a particular event of a particular type; such as by a classifier); (Cak: § II. A; § II. B, 4_binarization § III. A-D; Figs 1, 2(4), 3: such as by determining sound class activity prediction scores, thresholding same to arrive at a binary confidence score, that is a 1 indicating presence of a particular class over a particular span)
determining tentative onset times and tentative offset times of tentative events; and processing the tentative onset times and tentative offset times of tentative events to obtain time spans of sound events (Wang: Col 10:28-10:50, 10:58-11:15, 11:28-11:63; Fig 8, 9: such as by determining with respect to particular start and end times of an event different feature matrices with particular feature size, and/or stride; and/or to segment windows of determined lengths into a determined number of sub-segments); (Cak: § II. A; § II. B, 4_binarization § III. A-D; Figs 1, 2(4), 3: detection of events is predicted and binarized to determine active frames, binarize same and reify event boundaries thereby).
.
Wang in view of Cak does not explicitly teach filtering the class presence confidence scores with an ideal step filter in continuous time to determine a delta score for each frame as a difference between the average of class presence confidence scores in a predefined time period after a respective frame and the average of class presence confidence scores in the same- length time period before the respective frame; utilizing delta scores to process decision outputs.
In a related field of endeavor Ema teaches a system and method for management of data by the generation of differential data comprising filtering data using an ideal step filter (Ema: ¶ 66: a step value of a moving average or boxcar filter is optimized); in continuous time (Ema: ¶ 37, etc.: data continuously acquired in a time series from sensors; such as utilizing an embodiment processed over a time series); to determine a delta or differential score(s) for each frame (Ema; ¶ 66-69: thereby generating differential data acquired from maximum and minimum values by subtracting the filtered average).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the delta scores taught or suggested by Ema to better optimize change point detection in by taking a difference among averages of class-wise confidence scores in the manner taught or suggested by Wang in view of Cak for at least the point of utilizing change point detection of hard decision onset and offset times to manage, process, and improve the detected time spans of sound classes; one of ordinary skill in the art would have expected only predictable results therefrom.
Allowable Subject Matter
Claim 13 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL C MCCORD whose telephone number is (571)270-3701. The examiner can normally be reached 730-630 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CAROLYN EDWARDS can be reached at (571) 270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL C MCCORD/Primary Examiner, Art Unit 2692