DETAILED ACTION
Introduction
1. This office action is in response to Applicant's submission filed on 01/21/2025. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 21-40 are currently pending and examined below, while claims 1-20 were previously cancelled.
Drawings
2. The drawings filed on 01/21/2025 have been accepted and considered by the Examiner.
Priority
3. The Applicants priority to U.S. Non-Provisional Patent Application # 16822744, filed March 18, 2020, has been accepted and considered in this office action.
Double Patenting
4. The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper time-wise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Omum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed e-terminal disclaimer (e-TD) in compliance with 37 CFR 1.321 (c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a non-statutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. Effective January 1, 1994, a registered attorney or agent of record may sign an e-terminal disclaimer. An e-terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 21-40 of the instant Application are rejected on the ground of non-statutory obviousness-type double patenting as being unpatentable over claims 1-19 of U.S. Patent # 11551685. Although the conflicting claims are not identical, they are not patentably distinct from each other because the claims of the present application are broader in scope than those of U.S. Patent # 11551685 and hence the claims of U.S. Patent # 11551685 can anticipate those of the present invention. That is, the claims of U.S. U.S. Patent # 11551685 contain every limitation of the claims of the present application or the claims of the present application are obvious variants thereof. It should be noted that this is in fact a non-provisional non-statutory obviousness-type double patenting rejection because the conflicting claims have in fact been patented.
As an example; claim 21 of the instant application and claim 1 of U.S. Patent # U.S. Patent # 11551685 both teach a computer-implemented method, comprising receiving first audio data representing first speech; processing the first audio data to determine acoustic feature data; performing automatic speech recognition (ASR) processing based on the first audio data to determine ASR data; inputting the acoustic feature data and the ASR data to at least one machine learning component, the at least one machine learning component being configured to classify input data as corresponding to a device-directed speech event; determining, using the at least one machine learning component, that the first audio data corresponds to a first device-directed speech event; and based at least in part on the first audio data corresponding to the first device-directed speech event, causing natural language processing to be completed based on the ASR data. One of ordinary skill in the art would recognize that it would have been obvious at the time of the invention to drop narrower limitations in order to have a patent with wider applicability and freedom to operate. In other words, the narrower claim 1 of U.S. Patent # 11551685 anticipates the broader claim 1 of the instant application. Also, removal of the additional steps is obvious: In re Karlson, 136 USPQ 184 (1963): "Omission of an element and its function is an obvious expedient if the remaining elements perform the same functions as before".
Claims 21-40 of the instant Application are also rejected on the ground of non-statutory obviousness-type double patenting as being unpatentable over claims 1-21 of U.S. Patent # 12236950. Although the conflicting claims are not identical, they are not patentably distinct from each other because the claims of the present application are broader in scope than those of U.S. Patent # 12236950 and hence the claims of U.S. Patent # 12236950 can anticipate those of the present invention. That is, the claims of U.S. U.S. Patent # 12236950 contain every limitation of the claims of the present application or the claims of the present application are obvious variants thereof. It should be noted that this is in fact a non-provisional non-statutory obviousness-type double patenting rejection because the conflicting claims have in fact been patented.
As an example; claim 21 of the instant application and claim 1 of U.S. Patent # U.S. Patent # 12236950 both teach a computer-implemented method, comprising receiving first audio data representing first speech; processing the first audio data to determine acoustic feature data; performing automatic speech recognition (ASR) processing based on the first audio data to determine ASR data; inputting the acoustic feature data and the ASR data to at least one machine learning component, the at least one machine learning component being configured to classify input data as corresponding to a device-directed speech event; determining, using the at least one machine learning component, that the first audio data corresponds to a first device-directed speech event; and based at least in part on the first audio data corresponding to the first device-directed speech event, causing natural language processing to be completed based on the ASR data. One of ordinary skill in the art would recognize that it would have been obvious at the time of the invention to drop narrower limitations in order to have a patent with wider applicability and freedom to operate. In other words, the narrower claim 1 of U.S. Patent # 12236950 anticipates the broader claim 1 of the instant application. Also, removal of the additional steps is obvious: In re Karlson, 136 USPQ 184 (1963): "Omission of an element and its function is an obvious expedient if the remaining elements perform the same functions as before".
Information Disclosure Statement
5. The Information Statements (IDSs) filed on 10/22/2025, 08/25/2025 have been accepted/considered and are in compliance with the provisions of 37 CFR 1.97.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
6. Claims 21, 23-24, 28-31, 33-34 and 38-40 are rejected under 35 U.S.C. 102 (a) (2) as being anticipated by Sharifi (U.S. Patent Application Publication # 2022/0093104 A1).
With regards to claim 21, Sharifi teaches a computer-implemented method, comprising receiving first audio data representing first speech (Para 26, audio data associated with a speech input);
processing the first audio data to determine acoustic feature data (Para 37, teaches processing acoustic features related to pronunciation);
performing automatic speech recognition (ASR) processing based on the first audio data to determine ASR data (Para 27, teaches ASR based transcription generation);
inputting the acoustic feature data and the ASR data to at least one machine learning component, the at least one machine learning component being configured to classify input data as corresponding to a device-directed speech event (Para 37, further teaches a speaker classifier that compares audio data e.g., acoustic features related to pronunciation, timing, etc. of the speech input to acoustic features for a speaker profile associated with one or more users of the user device. The speaker profile may be learned and/or generated during a speaker enrollment process by one or more users of a household that are authorized to use a user device, such as a smart speaker. If the audio data of the speech input matches the acoustic features of the speaker profile for one or more users associated with user device, the speaker classifier outputs a high speaker-identification score indicating that the speech input was likely spoken by a user associated with the user device);
determining, using the at least one machine learning component, that the first audio data corresponds to a first device-directed speech event (Para 37, further teaches that the signal generator could use the high speaker-identification score to provide content metadata indicating a high likelihood that the corresponding ASR request is genuine. On the other hand, the speaker classifier may provide a low speaker-identification score when the audio data of the speech input does not match acoustic features of a speaker profile for a user associated with the user device. Accordingly, the speaker-identification score may correspond to a confidence value or probability of the audio data matching a known speaker profile);
and based at least in part on the first audio data corresponding to the first device-directed speech event, causing natural language processing to be completed based on the ASR data (Para 38, teaches a broadcast audio classifier that analyzes the audio data of the speech input to provide the broadcasted-speech score for the speech input indicating the likelihood that the speech input corresponds to broadcasted or synthesized speech output from a non-human source , such as a television, a radio, a computer, or any other audio output device capable of outputting broadcasted and/or synthesized speech. As used herein, broadcasted speech refers to speech spoken by a human e.g., newscaster, actor, radio personality, etc. but that corresponds to audio content emanating/broadcasting from a non-human source during a media event, such as a commercial, radio program, television show, and/or movie. Synthesized speech, on the other hand, refers to non-human speech generated by, for example, a text-to-speech system. The broadcast audio classifier is capable of detecting watermarks or other features that may be appended to audio content emanating/broadcasting from a non-human source and/or may be self-learning to differentiate between speech output from real humans in proximity to the user device and speech output from non-human sources that is synthesized speech or being broadcasted during a media event).
With regards to claim 23, Sharifi teaches the computer-implemented method of claim 21, further comprising determining, using a wake-word detection component, an indicator that the first audio data includes a representation of a wake-word (Para 40, teaches a hot-word detector that calculates the hot-word confidence score for the speech input and compares the hot-word confidence score to a hot-word confidence score threshold. The hot-word confidence score threshold represents a hot-word confidence score that, when detected by the hot-word detector, triggers the user device to wake-up from a sleep-state to capture the remaining portion of the speech input that corresponds to the voice query and generate the ASR request to be sent to the query processing stack);
and inputting the indicator to the at least one machine learning component in addition to the acoustic feature data and the ASR data (Para 40, further teaches that the input to the processing stack also includes providing content metadata that includes the hot-word confidence score of the speech input i.e., an initial portion of the speech input that may indicate that although the hot-word confidence score threshold was satisfied to trigger the user device to wake-up, the hot-word confidence score may be low enough to indicate that the speaker was far away and/or spoke some other phrase that sounds similar to the hot-word and therefore did not intend to invoke the user device. Thus, the hot-word confidence score can contribute to content metadata indicating whether or not the ASR request is likely genuine).
With regards to claim 24, Sharifi teaches the computer-implemented method of claim 21, further comprising processing, by an ASR component, a first portion of the first audio data to determine the ASR data, wherein the ASR data corresponds to the first portion of the first audio data (Para 4, teaches that the ASR component first processes the wake-word or the hot-word, e.g. when a user of a voice enabled device utters the following speech: “Hey Google, what restaurants are still open right now?”, the voice enabled device may wake-up in response to detecting a hot-word “Hey Google”, and provide the terms following the hot-word that correspond to a voice query “what nearby restaurants are still open right now?” to the server-based processing stack for processing).
With regards to claim 28, Sharifi teaches the computer-implemented method of claim 21, further comprising processing, by a wake-word detection component, the first audio data (Para 40, teaches a hot-word detector that calculates the hot-word confidence score for the speech input and compares the hot-word confidence score to a hot-word confidence score threshold. The hot-word confidence score threshold represents a hot-word confidence score that, when detected by the hot-word detector, triggers the user device to wake-up from a sleep-state to capture the remaining portion of the speech input that corresponds to the voice query and generate the ASR request to be sent to the query processing stack);
and failing to detect, by the wake-word detection component, a representation of a wake-word in the first audio data (Para 40, further teaches that in instances where a user speaks a designated hot-word “Ok Google” clearly and is near the user device, the hot-word confidence score may be high e.g., >0.9. In some instances, a user near the user device may speak a phrase such as “Ok poodle” that sounds similar to the designated hot-word “Ok Google”, thereby resulting in a lower confidence score e.g., 0.7 but still satisfying the hot-word confidence score threshold e.g., 0.68. Moreover, hot-word confidence scores may decrease if the speaker is farther from the user device or speaks less clearly. Accordingly, providing content metadata that includes the hot-word confidence score of the speech input i.e., an initial portion of the speech input may indicate that although the hot-word confidence score threshold was satisfied to trigger the user device to wake-up, the hot-word confidence score may be low enough to indicate that the speaker was far away and/or spoke some other phrase that sounds similar to the hot-word and therefore did not intend to invoke the user device. Thus, the hot-word confidence score can contribute to content metadata indicating whether or not the ASR request is likely genuine).
With regards to claim 29, Sharifi teaches the computer-implemented method of claim 21, further comprising after determination that the first speech corresponds to a first device-directed speech event, presenting, by a device, an output corresponding to an indication that natural language processing is occurring (Para 34, teaches that a query processing stack on the remote system that receives each ASR request that has not been dropped or timed out, including the audio data and content metadata associated with the speech input, from the QoS manager in descending order of ranking. The query processing stack includes at least the ASR module, the interpreter module, or the TTS module. The ASR module performs a variety of operations on the ASR request, such as processing, noise modeling, acoustic modeling, language model, annotation, etc., to generate a speech recognition result e.g., transcription for the speech input. The ASR module sends this speech recognition result to the interpreter to determine an intent of the ASR request and generate a response. An ASR request requesting the current time would be satisfied by the query processing stack determining and generating a response of the current time in the time zone of the user. The TTS module may convert this response from text to speech and output the response in audio form to the user device, which is then output as synthesized speech to the user via speakers of the user device or alternatively, the response may be outputted to the user device in text form, which is then transmitted to the user via a screen of the user device. The user device may also receive a response in the form of text or other data from the query processing stack and convert the response to speech using an on-device TTS module).
With regards to claim 30, Sharifi teaches the computer-implemented method of claim 21, further comprising after determination that the first speech corresponds to a first device-directed speech event, discontinuing generating output audio using a loudspeaker of a device proximate to the first speech (Paragraphs 46-47, teach that a situation wherein the environmental condition signal may indicate that there are several user devices in proximity of the user device, conditions of the network the user device is connected to is overloaded, GPS coordinates of the user device, whether the user device is outside, presently moving, approaching an area of poor cellular or data reception, etc. In this scenario, the query processing stack is unable to immediately process the ASR request and may simply drop the ASR request and optionally inform the user that the request cannot be completed at the moment. To this end, paragraphs 34 and 43, teach the use of an output speaker and proximity sensors).
With regards to claims 31, 33-34 and 38-40, these are system claims for the corresponding method claims 21, 23-24 and 28-30. These two sets of claims are related as method and apparatus of using the same, with each claimed system element's function corresponding to the claimed method step. Accordingly, claims 31, 33-34 and 38-40 are similarly rejected under the same rationale as applied above with respect to method claims 21, 23-24 and 28-30.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 22, 25-27 and 32-37 are rejected under 35 U.S.C. 103 as being unpatentable over Sharifi in view of Chang (U.S. Patent Application Publication # 2020/0335091 A1).
With regards to claim 26, Sharifi may not explicitly detail the limitation wherein the ASR data comprises lattice data. This is taught by Chang (Para 38, teaches a speech recognizer that includes a feature extraction module configured to receive audio data encoding the utterance and generate audio features indicating acoustic characteristics of the audio data. The audio features are input to the joint ASR and endpointing model or “joint model” as well as to the EOQ endpointer. Output of the joint model is evaluated using a beam search process or another process. The recognizer may execute the beam search process on a speech lattice obtained from outputs of the joint model to produce a transcription for the utterance encoded in the audio data).
Sharifi and Chang can be considered as analogous art as they belong to a similar field of endeavor in speech processing. It would thus have been obvious to one having ordinary skill in the art to advantageously combine the teachings of Chang (Use of speech lattice to produce transcription) with those of Sharifi (Use of a speech recognition based personal assistant) so as to quickly and accurately determining that an utterance has ended (Chang, para 30).
With regards to claim 27, Sharifi may not explicitly detail the limitation wherein the at least one machine learning component comprises at least one recurrent neural network (RNN). This is taught by Chang (Para 14, teaches that the streaming speech recognition model may include a Recurrent Neural Network-Transducer).
Sharifi and Chang can be considered as analogous art as they belong to a similar field of endeavor in speech processing. It would thus have been obvious to one having ordinary skill in the art to advantageously combine the teachings of Chang (Use of RNN to recognize speech) with those of Sharifi (Use of a speech recognition based personal assistant) so as to quickly and accurately determining that an utterance has ended (Chang, para 30).
With regards to claim 22, Sharifi may not explicitly detail the limitation further comprising detecting an endpoint of the first speech represented in the first audio data, wherein determining that the first speech corresponds to a first device-directed speech event occurs after detection of the endpoint. However, Chang teaches this limitation (Para 38, teaches a speech recognizer that includes a feature extraction module configured to receive audio data encoding the utterance and generate audio features indicating acoustic characteristics of the audio data. The audio features are input to the joint ASR and endpointing model or “joint model” as well as to the EOQ endpointer. Output of the joint model is evaluated using a beam search process or another process. The recognizer may execute the beam search process on a speech lattice obtained from outputs of the joint model to produce a transcription for the utterance encoded in the audio data. The speech recognizer may trigger endpoint detection responsive to receiving an endpoint signal from either one of the joint model or the EOQ endpointer, whichever occurs first. The endpoint signal corresponds to an endpoint indication output by the joint model or the EOQ endpointer that indicates an end of an utterance. The endpoint indication e.g., endpoint signal may include an endpoint token in the transcription selected by the beam search. Once the speech recognizer triggers endpoint detection, the user device can end detection of the utterance, e.g., by stopping the processing of further audio using the joint model or in some implementations disabling input from an array of microphones to at least some portions of the system. The speech recognizer may provide an instruction to deactivate the microphones, also known as a microphone closing event. The endpoint signal can also trigger the device to perform another action, such as to initiate a response to the utterance by requesting or providing search results, carrying out a command, and so on).
Sharifi and Chang can be considered as analogous art as they belong to a similar field of endeavor in speech processing. It would thus have been obvious to one having ordinary skill in the art to advantageously combine the teachings of Chang (Use of RNN to recognize speech) with those of Sharifi (Use of a speech recognition based personal assistant) so as to quickly and accurately determining that an utterance has ended (Chang, para 30).
With regards to claim 25, please see the rejection of claim 22 above.
With regards to claims 32, 25-37, these are system claims for the corresponding method claims 22, 25-27. These two sets of claims are related as method and apparatus of using the same, with each claimed system element's function corresponding to the claimed method step. Accordingly, claims 32, 35-37 are similarly rejected under the same rationale as applied above with respect to method claims 22, 25-27.
Conclusion
8. The following prior art, made of record but not relied upon, is considered pertinent to applicant's disclosure: Gadd (U.S. Patent Application Publication # 2005/0033582 A1), Cooper (U.S. Patent Application Publication # 2004/0225650 A1). These references are also included in the PTO-892 form attached with this office action.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. If you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). In case you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEERAJ SHARMA whose contact information is given below. The examiner can normally be reached on Monday to Friday 8 am to 5 pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis-Desir can be reached on 571-272-7799 (Direct Phone). The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
/NEERAJ SHARMA/
Primary Examiner, Art Unit 2659
571-270-5487 (Direct Phone)
571-270-6487 (Direct Fax)
neeraj.sharma@uspto.gov (Direct Email)
Prosecution Insights