Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 9-14, and 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Saki et al (US 20220165292) in view of Heintz et al (EP 2988105).
Consider Claim 1. Saki teaches receiving audio data corresponding to a sound detected at a premises ([0092] “or example, the particular SEC model 114 can be designated for use at home”, e.g. designated for use at home interpreted as premises, Saki); providing the audio data to an active instance of a sound classification model ([0005] “audio data samples as input to a sound event classification model.”, Saki); generating, by the active instance of the sound classification model, classification data that indicates the sound is unrecognized ([0043] “If the SEC model 112 (or another of the available SEC models 114) recognized the sound class of the audio data samples 110, the SEC engine 120 generates an output 124 that indicates the sound class 122 of the audio data samples”, e.g. sound class of audio data interpreted as classification data, e.g. If the SEC model recognizes the sound class is interpreted as indicating the sound is unrecognized (SEC model may or may not recognize the sound class), Saki); providing the audio data to a user device associated with a user ([0026] “… updating the sound event classification model for the user device…”, e.g. updating SEC model for user device interpreted as providing audio data to user device associated with user, Saki); receiving, from the user device, updated classification data that indicates an identity of the sound and one or more environments in which the sound is expected or unexpected ([0026] “Because the drift may correspond to sounds that were encountered by a user device but that were unrecognized by the user device, updating the sound event classification model for the user device to accommodate these encountered variations of sound classes enables the user device to more accurately identify specific variations of sound classes that are commonly encountered by that particular user device.”, e.g. updating sound event classification model for user device interpreted as updated classification data, e.g. accommodate these encountered variations of sound classes interpreted as identity of the sound and whether sound is expected or unexpected, Saki); storing the audio data and the [updated classification data] in a data bucket corresponding to the identity of the sound ([0048] “
the drift data 144 includes the audio data samples 110 and a label, where the label identifies the SEC model 112, indicates a sound class associated with the audio data samples 110, or both”, e.g. SEC stores drift data which is interpreted as storing audio data, e.g. labels are updated to the drift data, Saki); determining the data bucket contains a threshold quantity of user-classified audio data ([0027] “and the signal is stored in the recording database in the format of an audio file”, e.g. recording database involve groups (data buckets) and these groups are interpreted to contain threshold quantity of user-classified audio data, Saki); generating a new instance of the sound classification model ([0053] “end the unknown data 146 and the user-specified sound class to another device for use to generate a new or updated SEC model”, Saki); retraining the new instance of the sound classification model ([0051] “The model updater 152 uses the drift data 144 as labeled training data to update training of the SEC model 112 using backpropagation “, e.g. using drift data as labeled training data to update training of SEC interpreted as retraining the new instance of sound classification model, Saki) using the user-classified audio data ([0051] “the model updater 152 provides audio data samples from the drift data 144 of the model update data “, e.g. audio data sample interpreted as user-classified audio data, Saki); and replacing the active instance of the classification model with the new instance of the sound classification model ([0056] “In the example illustrated in FIG. 2, training data 202 is used to update the SEC model 208.”, e.g. using training data to update SEC model interpreted as replacing active instance with new instance, Saki).
Saki does not teach providing the audio data to a user device associated with a user.
Heintz teaches providing the audio data to a user device associated with a user ([0027] “In order to be able to process the recordings, they must be imported and played back in order to listen to a recording.” e.g. The act of processing recordings, importing, listening, followed by a playback to something (i.e. user or device) is interpreted as providing audio data to user device associated with a user, Heintz)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Heintz into the teachings of Saki, to yield the result of a sound classification mode that can determine and then store audio scenes associated with sound classes.
Consider Claims 2, 9, and 16. Saki teaches determining the identity of the sound is new ([0030] “If the user assigns a new sound class label to the unknown data”, e.g. assigning a new sound class involves determining if an identity of a sound is new or familiar, Saki).
Saki does not teach generating a new data bucket corresponding to the identity of the sound; and storing the audio data and the updated classification data in the new data bucket.
Heintz teaches generating a new data bucket ([0025] “After the start of the module "Sensor Manager/Menu" "SensorManager", a new project can be added,” e.g. new project interpreted as generating a new data bucket, Heintz) corresponding to the identity of the sound; and storing the audio data and the updated classification data in the new data bucket ([0010] “in that the recording database of the acoustic sensor system is connected to the recording module and the signal is stored in the recording database in the format of an audio file,” e.g. signals being recorded that are then stored in recording database interpreted as storing audio data and updating classification data in the new data bucket, Heintz).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Heintz into the teachings of Saki, to yield the result of a sound classification mode that can determine and then store audio scenes associated with sound classes.
Consider Claim 3, 10, and 17. Saki teaches providing test audio data comprising the sound to the new instance of the sound classification model ([0057] “Additionally, in some implementations, other audio data samples (e.g., audio data samples previously used to train the SEC model 208) may also be provided to the SEC model “, e.g. using previous audio data samples interpreted as providing test audio data for new instance of the sound classification model, Saki); and determining whether the new instance of the sound classification model correctly identifies the sound ([0056] “ The label 204 corresponding to an entry of the drift data 144 identifies a sound class to which the sound is expected to belong”, e.g. identifying a sound class to which the sound is expected to belong interpreted as the new sound classification model correctly identifying the sound, Saki) wherein the active instance of the sound classification models replaced with the new instance of the sound classification model in response to the new instance of the sound classification model correctly identifying the sound (0058] “When the training termination condition is satisfied, the model updater 152 stores the updated SEC model 214, where the updated SEC model 214 corresponds to the SEC model 208 with link weights based on the updated link weights 212 applied during training.”, e.g. updated SEC model updating link weights from training interpreted as new instance of sound classification model correctly identifying the sound, Saki).
Consider Claims 4, 11, and 18. Saki teaches the test audio data comprises a plurality of sound clips of the sound ([0097] “For example, the device 100 provides the output 124 of FIG. 1 to the display 620 indicating a sound class of a sound 102 (such as a car horn)”, e.g. car horn is one of the many examples of a plurality of sound clips of the sound, Saki); and determining whether the new instance of the sound classification model correctly identifies the sound ([0056] “ The label 204 corresponding to an entry of the drift data 144 identifies a sound class to which the sound is expected to belong”, e.g. identifying a sound class to which the sound is expected to belong interpreted as the new sound classification model correctly identifying the sound, Saki) is based on the new instance correctly identifying a threshold number of the plurality of sound clips ([0049] “ For example, if the SEC engine 120 uses a detection threshold of 0.95, the drift threshold may have a value of 0.80, of 0.75, or some other value less than 0.95”, e.g. detection threshold interpreted as threshold number, Saki).
Consider Claims 5, 12, and 19. Saki teaches wherein the audio data corresponds to the sound ([0003] “To illustrate, the labeled training data may include audio data samples for many different doorbells”, e.g. audio data sample for many different doorbells interpreted as audio data corresponding to the sound, Saki) and another sound that is recognized by the active instance of the sound classification model (See Fig. 16 below, e.g. Box 1604 interpreted as another sound recognized by active instance of sound classification model, Saki) the method further comprising generating second classification data that indicates an identity of the other sound ([0102] “For example, the SEC engine 120 uses one or more of the SEC models 114 to generate the sound event classification data which may be provided to the display 620 to indicate that a recognized sound event, such as a car horn, “, e.g. SEC model generating sound classification data interpreted as generating sound classification data indicating an identity of other sound (i.e. the car horn), Saki).
PNG
media_image1.png
778
571
media_image1.png
Greyscale
Consider Claims 6, 13 and 20. Saki teaches determining, based on environment data that identifies expected sounds of an environment ([0027] “is used to determine scene data indicating a sound environment (or audio scene) associated with the audio data samples. “, e.g. scene data indicating sound environment interpreted as environment data, e.g. association of environment data with audio data samples interpreted as the expected sounds of environment, Saki), whether the second classification data indicates the other sound is expected for an environment associated with the premises ([0026] “To illustrate, the sound event classification model may be trained to recognize the sound of a bass drum played in a concert hall but may not recognize the bass drum if it is played by a marching band in an outdoor parade”, e.g. sound event classification model trained to recognize another sound (i.e. bass drum) involves second classification data, e.g. training sound event classification model to recognize sound of bass drum played in concert hall but not trained to recognize sound of bass drum played by marching band is interpreted as indicates other sound is expected for environment associated with the premises, Saki); and initiating a premises alert in response to the environment data indicating the sound is unexpected for the environment associated with the premises ([0026] “ The transfer learning techniques disclosed herein facilitate updating a sound event classification model to account for such drift, which enables the sound event classification model to detect a broader range of sounds within a sound class. “, e.g. drift is referred to as the variation within a sound class, thus updating the SEC model to account for drift is interpreted as a premises alert in response to environment data indicating sound is unexpected for environment, Saki).
Consider Claims 7 and 14. Saki teaches generating and transmitting a notification to the user that indicates the sound is unrecognized ([0043] “For example, the output 124 may be sent to a display to notify a user of detection of the sound class 122 associated with the sound 102 or may be sent to another device…”, e.g. notifying user of detection of sound class may involve detection of a sound that is either recognized or unrecognized thus it is interpreted as generating/transmitting notification to user when sound is unrecognized, Saki).
Consider Claim 15. Saki teaches a listening device that detects a sound and generates audio data based on the sound ([0038] “The device 100 of FIG. 1 includes one or more microphones 104 configured to generate audio signals 106 based on sound 102 detected within an acoustic environment”, Saki); a communication device that communicates the audio data from the listening device ([0095] “ a mobile communication device, a smart phone, a cellular phone, a laptop computer,”, e.g. examples of devices interpreted as a communication device, Saki); and one or more control devices that: receive the audio data ([0101] “The headset 802 includes the sensors 134, such as the microphone(s) “, e.g. headset receives audio data via microphone, e.g. See Fig 8 below,
PNG
media_image2.png
841
688
media_image2.png
Greyscale
);
provide the audio data to an active instance of a sound classification model ([0005] “audio data samples as input to a sound event classification model.”, Saki); generate, by the active instance of the sound classification model, classification data that indicates the sound is unrecognized ([0043] “If the SEC model 112 (or another of the available SEC models 114) recognized the sound class of the audio data samples 110, the SEC engine 120 generates an output 124 that indicates the sound class 122 of the audio data samples”, e.g. sound class of audio data interpreted as classification data, e.g. If the SEC model recognizes the sound class is interpreted as indicating the sound is unrecognized (SEC model may or may not recognize the sound class), Saki); receive, from the user device, updated classification data that indicates an identity of the sound and one or more environments in which the sound is expected or unexpected ([0026] “Because the drift may correspond to sounds that were encountered by a user device but that were unrecognized by the user device, updating the sound event classification model for the user device to accommodate these encountered variations of sound classes enables the user device to more accurately identify specific variations of sound classes that are commonly encountered by that particular user device.”, e.g. updating sound event classification model for user device interpreted as updated classification data, e.g. accommodate these encountered variations of sound classes interpreted as identity of the sound and whether sound is expected or unexpected, Saki); store the audio data and the updated classification data in a data bucket corresponding to the identity of the sound ([0048] “a the drift data 144 includes the audio data samples 110 and a label, where the label identifies the SEC model 112, indicates a sound class associated with the audio data samples 110, or both”, e.g. SEC stores drift data which is interpreted as storing audio data, e.g. labels are updated to the drift data, Saki); determine the data bucket contains a threshold quantity of user-classified audio data ([0027] “and the signal is stored in the recording database in the format of an audio file”, e.g. recording database involve groups (data buckets) and these groups are interpreted to contain threshold quantity of user-classified audio data, Saki); generate a new instance of the classification model ([0053] “end the unknown data 146 and the user-specified sound class to another device for use to generate a new or updated SEC model”, Saki); retrain the new instance of the classification model ([0051] “The model updater 152 uses the drift data 144 as labeled training data to update training of the SEC model 112 using backpropagation “, e.g. using drift data as labeled training data to update training of SEC interpreted as retraining the new instance of sound classification model, Saki) using the user-classified audio data ([0051] “the model updater 152 provides audio data samples from the drift data 144 of the model update data “, e.g. audio data sample interpreted as user-classified audio data, Saki); and replace the active instance of the classification model with the new instance of the classification model ([0056] “In the example illustrated in FIG. 2, training data 202 is used to update the SEC model 208.”, e.g. using training data to update SEC model interpreted as replacing active instance with new instance, Saki).
Saki does not teach provide the audio data to a user device associated with a user.
Heintz teaches provide the audio data to a user device associated with a user ([0027] “In order to be able to process the recordings, they must be imported and played back in order to listen to a recording.” e.g. The act of processing recordings, importing, listening, followed by a playback to something (i.e. user or device) is interpreted as providing audio data to user device associated with a user, Heintz)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Heintz into the teachings of Saki, to yield the result of a sound classification mode that can determine and then store audio scenes associated with sound classes.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Saki et al (US 20220165292 ) in view of Eswara et al (US 11765501).
Consider Claim 8. Saki teaches receiving first audio data corresponding to a first instance of a sound, the first instance of the sound detected at a premises ([0092] “For example, the particular SEC model 114 can be designated for use at home”, e.g. designated for use at home interpreted as premises, Saki); providing the first audio data to an active instance of a sound classification model ([0005] “audio data samples as input to a sound event classification model.”, Saki); generating, by the active instance of the sound classification model, first classification data that indicates the sound is unrecognized ([0043] “If the SEC model 112 (or another of the available SEC models 114) recognized the sound class of the audio data samples 110, the SEC engine 120 generates an output 124 that indicates the sound class 122 of the audio data samples”, e.g. sound class of audio data interpreted as classification data, e.g. If the SEC model recognizes the sound class is interpreted as indicating the sound is unrecognized (SEC model may or may not recognize the sound class), Saki); providing the first audio data to a user device associated with a user ([0026] “… updating the sound event classification model for the user device…”, e.g. updating SEC model for user device interpreted as providing audio data to user device associated with user, Saki); receiving, from the user device, updated classification data that indicates an identity of the sound ([0026] “Because the drift may correspond to sounds that were encountered by a user device but that were unrecognized by the user device, updating the sound event classification model for the user device to accommodate these encountered variations of sound classes enables the user device to more accurately identify specific variations of sound classes that are commonly encountered by that particular user device.”, e.g. updating sound event classification model for user device interpreted as updated classification data, e.g. accommodate these encountered variations of sound classes interpreted as identity of the sound and whether sound is expected or unexpected, Saki); storing the first audio data ([0048] “the drift detector 128 stores that the audio data samples 110 as drift data 144 associated with the particular sound class”, e.g. audio data sample interpreted as first audio data, Saki) and the updated classification data in a data bucket corresponding to the identity of the sound ([0048] “ the drift data 144 includes the audio data samples 110 and a label, where the label identifies the SEC model 112, indicates a sound class associated with the audio data samples”, e.g. label associated with audio data sample interpreted as updated classification data, e.g. sound class interpreted as a data bucket, Saki); determining the data bucket contains a threshold quantity of user-classified audio data for the sound ([0027] “and the signal is stored in the recording database in the format of an audio file”, e.g. recording database involve groups (data buckets) and these groups are interpreted to contain threshold quantity of user-classified audio data, Saki); generating a new instance of the sound classification model ([0053] “end the unknown data 146 and the user-specified sound class to another device for use to generate a new or updated SEC model”, Saki); retraining the new instance of the sound classification model ([0051] “The model updater 152 uses the drift data 144 as labeled training data to update training of the SEC model 112 using backpropagation “, e.g. using drift data as labeled training data to update training of SEC interpreted as retraining the new instance of sound classification model, Saki) using the user-classified audio data ([0051] “the model updater 152 provides audio data samples from the drift data 144 of the model update data “, e.g. audio data sample interpreted as user-classified audio data, Saki); replacing the active instance of the classification model with the new instance of the sound classification model ([0056] “In the example illustrated in FIG. 2, training data 202 is used to update the SEC model 208.”, e.g. using training data to update SEC model interpreted as replacing active instance with new instance, Saki; receiving second audio data corresponding to a second instance of the sound ([0054] “ make the updated SEC model 154 available to evaluate audio data samples 110 received after the updated SEC model 154 is generated”, Saki); providing the second audio data to the new instance of the sound classification model ([0054] “ make the updated SEC model 154 available to evaluate audio data samples 110 received after the updated SEC model 154 is generated”, Saki); generating, by the new instance of the sound classification model, second classification data that indicates the identity of the sound ([0043] “the SEC engine 120 generates an output 124 that indicates the sound class 122 of the audio data samples 110”, Saki).
Saki does not teach determining, based on environment data that identifies expected sounds of an environment associated with the premises, whether the sound is expected for the environment; and
initiating a premises alert in response to the environment data indicating the sound is unexpected for the environment.
Eswara teaches determining, based on environment data that identifies expected sounds of an environment associated with the premises, whether the sound is expected for the environment (Col. 17 line 66 – Col. 18 line 11 “The controller 102 may identify similarities between the incoming split audio stream features and the audio classification models. The controller 102 may further use this comparison to determine if the audio stream includes normal sounds or noises 258. For example, if the incoming split audio stream features are similar to the audio features of a normal sound audio model, the controller 102 may determine the sounds are normal or expected and take no further action 260. If the incoming split audio stream features are similar to the audio features of an abnormal sound audio model, the controller 102 may determine the sounds are abnormal or unexpected,”, and Col. 1 lines 49-55 “using at least part of the normal audio stream as a baseline... to determine whether the incoming audio stream... includes an abnormal audio event for the particular environment”, e.g. abnormal audio event for particular environment interpreted as a environment data, e.g. The controller identifying similarities between incoming split audio stream features and determining which sounds are normal/expected is interpreted as identifying expected sounds of an environment associated with premises, Eswara); and initiating a premises alert in response to the environment data indicating the sound is unexpected for the environment (Col. 18 lines 41-43 “If is further contemplated that after a predetermined number of abnormal sound events, an alert may be generated 262 and transmitted to an operator.”, e.g. alert being generated and transmitted to operator interpreted as initiating a premises alert, e.g. the predetermined number of abnormal sound events is interpreted as related to the environment data e.g. generating the alert is interpreted as indicating a sound is unexpected, Eswara).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Eswara into the teachings of Saki, to yield the result of detecting whether a sound is expected or unexpected in an environment and when the sound is unexpected, alerting the premise.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIET NGUYEN whose telephone number is (571)270-0174. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at (571) 272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VIET NGUYEN/Patent Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
