DETAILED ACTION
This office action is in response to Applicant’s submission filed on 3/20/2025. Claims 1-20 are pending in the application of which Claims 1 and 11 are independent and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365 is acknowledged. The prior-filed application (Provisional application No. 63/570015 Filed on 3/26/2024) is acknowledged.
Information Disclosure Statement
The information disclosure statement(s)(IDS) submitted on 6/23/2025 has been considered by the examiner.
Duplicate Claims, Warning
Applicant is advised that should claims 9, and 19 be found allowable, claim 10, and 20 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 5, 7-15, and 17-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter without significantly more. The claims as whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea.
The independent claim 1 and 11 recites: “… receiving an audio input; converting the audio input to a textual message; analyzing the textual message with a natural language processing technique to identify an intent; generating a structured command based on the identified intent; and transmitting the structured command to a user device. “
The aforementioned limitations, under its broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting “user device”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “user device”, “a “receiving an audio input”, “converting” to textual, “analyzing” to identify an intent; transmitting command to a user device.
In the context of this claim encompasses individual hears an audio message, subsequently write down the message he heard, figuring out the intent of the message and what command or action it carries, and write the command or action on a piece of paper and pass it to the intended person.
All of these steps can be performed in the mind and/or using a pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. There is no additional element with the exception of server, and client device which is recited at a high level of generality. Such generic computer components amount to no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
The dependent claims of 2-5, 7-10, 12-15, and 17-20 do not add limitations that could help the Claim as a whole to amount to significantly more than the Abstract idea identified for the Independent Claim:
Similarly, dependent claims -5, 7-10, 12-15, and 17-20 are also not patent eligible as they include additional steps that are directed towards an abstract idea as they can be practically performed in the mind without being integrated into a practical application or including any additional elements sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 7-14, and 17-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Björn Hoffmeister (US10388274B1)(herein " Hoffmeister ").
Regarding claims 1 and 11, Hoffmeister teaches [A computing system comprising: a processor; and a non-transitory computer-readable storage device storing computer-executable instructions, the instructions operable to cause the processor to perform operations comprising: - claim 1], [A computer-implemented method, performed by at least one processor, comprising: - claim 11] (Hoffmeister, Col. 30, ll. 20-24:” … may include one or more controllers/processors (1804/1904), that may each include a central processing unit (CPU) for processing data and computer-readable instructions, and a memory (1806/1906) for storing data and instructions of the respective device.”, and Col. 37, ll. 16-21:” … a computer method or as an article of manufacture such as a memory device or non-transitory computer readable storage medium. The computer readable storage medium may be readable by a computer and may comprise instructions for causing a computer or other device to perform processes described in the present disclosure.”)
receiving an audio input; (Hoffmeister, Col. 4, ll. 9-12:” … a device 110 receives an audio input 11 corresponding to a spoken utterance from the user 10. The device 110 may convert the audio 11 into audio data 111 and send the audio data to the server(s) 120.”)
converting the audio input to a textual message; (Hoffmeister, Col. 6, ll. 34-40:” Upon receipt by the server(s) 120, an ASR module 250 may convert the audio data 111 into text. The ASR transcribes audio data into text data representing the words of the speech contained in the audio data. The text data may then be used by other components for various purposes, such as executing system commands, inputting data, etc.”)
analyzing the textual message with a natural language processing technique to identify an intent; (Hoffmeister, Col. 8, ll. 35-47:” The NLU process takes textual input (such as processed from ASR 250 based on the utterance 11) and attempts to make a semantic interpretation of the text. That is, the NLU process determines the meaning behind the text based on the individual words and then implements that meaning. NLU processing 260 interprets a text string to derive an intent or a desired action from the user as well as the pertinent pieces of information in the text that allow a device (e.g., device 110) to complete that action. For example, if a spoken utterance is processed using ASR 250 and outputs the text “call mom” the NLU process may determine that the user intended to activate a telephone in his/her device and to initiate a call with a contact matching the entity “mom.” “)
generating a structured command based on the identified intent; and (Hoffmeister, Col. 8, ll. 43-58:” For example, if a spoken utterance is processed using ASR 250 and outputs the text "call mom" the NLU process may determine that the user intended to activate a telephone in his/her device and to initiate a call with a contact matching the entity "mom." The NLU may process several textual inputs related to the same utterance. For example, if the ASR 250 outputs N text segments (as part of an N-best list), the NLU may process all N outputs to obtain NLU results. The NLU process may be configured to parsed and tagged to annotate text as part of NLU processing. For example, for the text "call mom," "call" may be tagged as a command (to execute a phone call) and "mom" may be tagged as a specific entity and target of the command (and the telephone number for the entity corresponding to "mom" stored in a contact list may be included in the annotated result).")
transmitting the structured command to a user device. (Hoffmeister, Col 8, ll. 7-19 Following ASR processing, the ASR results may be sent by the speech recognition engine 258 to other processing components, which may be local to the device performing ASR and/or distributed across the network(s) 199. For example, ASR results in the form of a single textual representation of the speech, an N-best list including multiple hypotheses and respective scores, lattice, etc. may be sent to a server, such as server 120, for natural language understanding (NLU) processing, such as conversion of the text into commands for execution, either by the device 110, by the server 120, or by another device (such as a server running a specific application like a search engine, etc.).”) Note: device 110 is mapped to the user device.
Regarding claims 2 and 12, Hoffmeister teaches the computer system and the method of claims 1, and 11 respectively.
Hoffmeister, further teaches wherein the operations comprise, prior to receiving the audio input: receiving an initial audio input; and (Hoffmeister, Col. 5, ll. 59-60:” The wakeword detection module 220 may compare audio data to stored models or data to detect a wakeword.”, and Col. 6, ll. 21-23:”Once the wakeword is detected, the local device 110 may “wake” and begin transmitting audio data 111 corresponding to input audio 11 to the server(s) 120 for speech processing.”). Note: wakeword reads on an initial audio input.
detecting a predefined keyword in the initial audio input. (Hoffmeister, Col. 5, ll. 59-64:” The wakeword detection module 220 may compare audio data [initial audio input] to stored models or data to detect a wakeword [predefined keyword]. One approach for wakeword detection applies general large vocabulary continuous speech recognition (LVCSR) systems to decode the audio signals, with wakeword searching conducted in the resulting lattices or confusion networks.”)
Regarding claims 3 and 13, Hoffmeister teaches the computer system and the method of claims 2 and 12 respectively.
Hoffmeister, further teaches wherein detecting a predefined keyword in the initial audio input comprises processing the audio input to infer a probability of a keyword. (Hoffmeister, Col. 6, ll. 49-67 "The different ways a spoken utterance may be interpreted (i.e., the different hypotheses) may each be assigned a probability or a confidence score representing the likelihood that a particular set of words matches those spoken in the utterance. The confidence score may be based on a number of factors including, for example, the similarity of the sound in the utterance to models for language sounds (e.g., an acoustic model 253 stored in an ASR Models Storage 252), and the likelihood that a particular word which matches the sounds would be included in the sentence at the specific location (e.g., using a language or grammar model). Thus, each potential textual interpretation of the spoken utterance (hypothesis) is associated with a confidence score. Based on the considered factors and the assigned confidence score, the ASR process 250 outputs the most likely text recognized in the audio data. The ASR process may also output multiple hypotheses in the form of a lattice or an N-best list with each hypothesis corresponding to a confidence score or other score (such as probability scores, etc.).")
Regarding claims 4 and 14, Hoffmeister teaches the computer system and the method of claims 3 and 13 respectively.
Hoffmeister, further teaches wherein detecting a predefined keyword in the initial audio input comprises identifying an inferred word with a probability greater than a predefined threshold. (Hoffmeister, Col. 21, ll. 21-24:” If the first stage detects a potential keyword (or keyphrase), then the classifier 1202 may output a confidence score. If the confidence score exceeds a given threshold, then the keyword (or keyphrase) is accepted.”)
Regarding claims 7 and 17, Hoffmeister teaches the computer system and the method of claims 1 and 11 respectively.
Hoffmeister, further teaches wherein analyzing the textual message with the natural language processing technique to identify the intent comprises parsing text within the textual message to determine the intent. (Hoffmeister, Col. 8, ln 35-58 "The NLU process takes textual input (such as processed from ASR 250 based on the utterance 11) and attempts to make a semantic interpretation of the text. That is, the NLU process determines the meaning behind the text based on the individual words and then implements that meaning. NLU processing 260 interprets a text string to derive an intent or a desired action from the user as well as the pertinent pieces of information in the text that allow a device (e.g., device 110) to complete that action. … For example, if the ASR 250 outputs N text segments (as part of an N-best list), the NLU may process all N outputs to obtain NLU results. The NLU process may be configured to parsed and tagged to annotate text as part of NLU processing. For example, for the text "call mom," "call" may be tagged as a command (to execute a phone call) and "mom" may be tagged as a specific entity and target of the command (and the telephone number for the entity corresponding to "mom" stored in a contact list may be included in the annotated result).")
Regarding claims 8 and 18, Hoffmeister teaches the computer system and the method of claims 1 and 11 respectively.
Hoffmeister, further teaches wherein analyzing the textual message with the natural language processing technique to identify the intent comprises applying a predefined schema comprising a command syntax to extract the intent. (Hoffmeister, Col. 8, ll. 52-58: "The NLU may process several textual inputs related to the same utterance. For example, if the ASR 250 outputs N text segments (as part of an N-best list), the NLU may process all N outputs to obtain NLU results. The NLU process may be configured to parsed and tagged to annotate text as part of NLU processing. For example, for the text "call mom," "call" may be tagged as a command (to execute a phone call) and "mom" may be tagged as a specific entity and target of the command (and the telephone number for the entity corresponding to "mom" stored in a contact list may be included in the annotated result)."; Col. 9, line 65-Col. 10, line 33: "The intents identified by the IC module 264 are linked to domain-specific grammar frameworks (included in 276) with "slots" or "fields" to be filled. For example, if "play music" is an identified intent, a grammar (276) framework or frameworks may correspond to sentence structures such as "Play {Artist Name}," "Play {Album Name}," "Play {Song name}," "Play {Song name} by {Artist Name}," etc. However, to make recognition more flexible, these frameworks would ordinarily not be structured as sentences, but rather based on associating slots with grammatical tags. For example, the NER module 260 may parse the query to identify words as subject, object, verb, preposition, etc., based on grammar rules and models, prior to recognizing named entities. The identified verb may be used by the IC module 264 to identify intent, which is then used by the NER module 262 to identify frameworks. A framework for an intent of "play" may specify a list of slots/fields applicable to play the identified "object" and any object modifier (e.g., a prepositional phrase), such as {Artist Name}, {Album Name}, {Song name}, etc. The NER module 260 then searches the corresponding fields in the domain-specific and personalized lexicon(s), attempting to match words and phrases in the query tagged as a grammatical object or object modifier with those identified in the database(s). This process includes semantic tagging, which is the labeling of a word or combination of words according to their type/semantic meaning. Parsing may be performed using heuristic grammar rules, or an NER model may be constructed using techniques such as hidden Markov models, maximum entropy models, log linear models, conditional random fields (CRF), and the like. For instance, a query of "play mother's little helper by the rolling stones" might be parsed and tagged as {Verb}: "Play," {Object}: "mother's little helper," {Object Preposition}: "by," and {Object Modifier}: "the rolling stones."")
Regarding claims 9 and 19, Hoffmeister teaches the computer system and the method of claims 1, and 11 respectively.
Hoffmeister, further teaches wherein analyzing the textual message with the natural language processing technique to identify the intent comprises applying a phrase mapping file to the textual message. (Hoffmeister, Col. 25, line 57- Col. 26, line 5: "For example, if query text includes "where was Obama born," the system may determine that the relation is [born in] and that rephrased queries may include "Obama was born where," "Obama's birthplace is what," or the like. The system may also be capable of performing NER for any subjects/objects named in the query text and may use different versions of the entity name in the rephrased query (for example "where was President Obama born"). The system may then take the text of the individual rephrasings and perform (1374) a web search for each rephrased query or from the original first query text. That is, the text of each individual rephrasing is run through a search engine to find results (for example from data source(s) 180) for each query and the system attempts to identify the missing subject or object from the unstructured text of the web searches").
Regarding claims 10, and 20, Hoffmeister teaches the computer system and the method of claims 1, and 11 respectively.
Hoffmeister, further teaches wherein analyzing the textual message with the natural language processing technique to identify the intent comprises applying a phrase mapping file to the textual message. (Hoffmeister, Col. 25, line 57- Col. 26, line 5: "For example, if query text includes "where was Obama born," the system may determine that the relation is [born in] and that rephrased queries may include "Obama was born where," "Obama's birthplace is what," or the like. The system may also be capable of performing NER for any subjects/objects named in the query text and may use different versions of the entity name in the rephrased query (for example "where was President Obama born"). The system may then take the text of the individual rephrasings and perform (1374) a web search for each rephrased query or from the original first query text. That is, the text of each individual rephrasing is run through a search engine to find results (for example from data source(s) 180) for each query and the system attempts to identify the missing subject or object from the unstructured text of the web searches").
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Hoffmeister, and in further view of Trausti Thor Kristjansson (US20170092268A1)(herein " Kristjansson ").
Regarding claims 5 and 15, Hoffmeister teaches the computer system and the method of claims 1, and 11 respectively.
Hoffmeister, does not teach, however Kristjansson teaches wherein converting the audio input to the textual message comprises analyzing the audio input with an acoustic model comprising a neural network trained on high-noise audio samples. (Kristjansson, Par. 0034: "…the computing system 120 inputs the feature vectors 122, the noise vector 124 and the additional data 126 to the neural network 140. The neural network 140 has been trained to act as an acoustic model.”, and Par. 0045:” The neural network 270 has been trained to estimate likelihoods that a combination of feature vectors and a noise-vector and non-speech vectors that represent particular phonetic units. For example, during training, input to the neural network 270 may be a combination of acoustic feature vectors and a noise-vector corresponding to the utterance from which the acoustic feature vectors were derived. Many inputs combining acoustic feature vectors and a noise-vector can be used to train the neural network 270, and the various training data sets can include acoustic feature vectors and noise-vectors derived from utterances from multiple speakers.”, and Par. 0047:” At the output layer 273, the neural network 270 indicates likelihoods that the speech in the window 220 under analysis (e.g., the window w7 corresponding to acoustic feature vector v7) corresponds to specific phonetic units.”)
Kristjansson is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hoffmeister further in view of Kristjansson to analyze the audio input with an acoustic model comprising a neural network trained on high-noise audio samples. Motivation to do so would improve noise robustness for speech recognition and speech enhancement (Kristjansson, Par. 0015).
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Hoffmeister, and Kristjansson, and in further view of Zuluaga-Gomez et al. (“A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers”)(herein “Zuluaga-Gomez”).
Regarding claims 6 and 16, Hoffmeister teaches the computer system and the method of claims 5 and 15 respectively.
Hoffmeister, as modified above, does not teach, however Zuluaga-Gomez teaches wherein the neural network is fine-tuned with a plurality of keywords associated with aircraft. (ABS:” … The overall pipeline is composed of the following submodules: (i) an automatic speech recognition (ASR) system that transforms audio into a sequence of words; (ii) a high-level air traffic control (ATC)-related entity parser that understands the transcribed voice communication; and … Our system employs state-of-the-art AI-based tools such as Wav2Vec 2.0, Conformer, BERT and Tacotron models. … Our ASR system can reach as low as 5.5% and 15.9% absolute word error rates (WER) on high- and low-quality ATC audio. We also demonstrate that adding surveillance data into the ASR can yield a callsign detection accuracy of more than 96%.”, and Page 3:” Our pipeline is built entirely on open-source and state-of-the-art pre-trained AI models that have been fine-tuned on the ATC domain. The Wav2Vec 2.0 and XLSR [26,27] models are used for ASR, BERT [28] is employed for natural language understanding (NLU), …”, and Page 6:” Here, the authors follow the practice of fine-tuning a Wav2Vec 2.0 model [26] with public and private ATC databases. This system reaches on-par performances with hybrid-based ASR models, demonstrating that this new paradigm for ASR development also performs well in the ATC domain. In E2E-ASR, the system encodes directly an acoustic and language model, and it produces transcripts in an E2E manner. … A high-level entity parser system (such as ours) can be implemented by fine-tuning a pre-trained LM for the NER task. Currently, state-of-the-art NER models utilize pre-trained LMs such as BERT [28], RoBERTa [67] or DeBERTa [68]. For the proposed virtual simulation-pilot, we use a finetuned BERT on ATC text data.”, and Page 9:” … One of the challenges for VAD in ATC communications is the presence of a high level of background noise. The noise comes from various sources, e.g., the engines of aircraft, wind or even other ATCos.”)
Zuluaga-Gomez is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hoffmeister, as modified above, further in view of Zuluaga-Gomez to wherein the neural network is fine-tuned with a plurality of keywords associated with aircraft. Motivation to do so would improve the system’s performance in recognizing and responding to ATC communications (Zuluaga-Gomez, Page 19).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Tur et al. (“Intent determination and spoken utterance classification”) teaches in Page 93:” … one key spoken language understanding (SLU) task is interpreting users’ intentions from their speech utterances. This task is closely related to dialogue based natural language understanding, where the user does not need to know a predefined set of command words in order to use the system successfully. …In a call-routing system, a critical part of understanding a speech utterance is classification into pre-defined types of intents or call types. … the spoken utterance classification techniques and illustrate their successful applications to intent determination systems, as emerged partly from commercial call-routing applications. … the tasks ranging from simple keyword detection based approaches to more sophisticated predicate/argument style semantic classification schemes using context that integrates with template filling.”
Examiner's Note: Examiner has cited particular columns and line numbers and/or paragraph numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DARIOUSH AGAHI whose telephone number is (408)918-7689. The examiner can normally be reached Monday - Thursday and alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DARIOUSH AGAHI, P.E.
Primary Examiner
/DARIOUSH AGAHI/Primary Examiner, Art Unit 2656
