Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 8/6/2025 are being considered by the examiner.
Response To Arguments
In light of amendments, the arguments raised in reference to 101 are moot as new art is used to reject the claims and new 101 arguments are presented.
In light of amendments, the arguments raised in reference to 102/103 are moot as new art is used to reject the claims and new art rejections are presented.
In light of the above, the examiner maintains the 101, 102 and 103 rejections.
In light of amendments the examiner is entering Claim Objections.
Claim Objections
Listed claims are objected to for the informalities shown and maybe addressed with the suggested amendments:
Claim 1, 7, 13, 19 and 25 : …identify and end of the one or more speech segments
Claim 26 : …the one or more neural networks
Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-30 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
The independent claims 1, 7, 13, 19, 25 recite One or more processors A processor, comprising: circuitry to use one or more neural networks to: receive text representations of one or more digital audio signals: determine, for each time step within a window of time for one or more speech segments of the one or more digital audio signals, a ratio of speech characters and non-speech characters within the window of time for the one or more speech segments comprising text representations: and identify an end of one or more speech segments based, at least in part, on the ratio of speech and non-speech within the one or more speech segments of, wherein the non-speech comprises one or more non-speech sounds. The limitation receive, determine, identify as drafted covers performance of the limitation in the mind. This amounts to a human listening to recorded speech that has a transcription, and writing ratio of speech to non-speech characters to determine end of speech.
This judicial exception is not integrated into a practical application. In particular claim 1 recites additional element of circuits which are a form of generic computer equipment, digital audio signals is an additional element which is nothing more than instructions to implement the abstract idea on a computer. As evidenced in the as-filed specs, there is no special processing occurring just because the signals are digitized:“¶[0046] In at least one embodiment, audio capture device 106 can capture one or more audio signals 110 using microphone 108, which can be passed to an audio analyzer 112. In at least one embodiment, audio analyzer 112 may be on a same device as audio capture device 106, or may be on a separate device, such as a remote device available over a wireless connection or a remote server available over a network connection. In at least one embodiment, audio analyzer 112 can include a speech detection pipeline 114 to analyze captured audio signals and generate one or more transcripts 116, or generated text for these audio signals, as discussed in more detail later herein.” neural networks are generic high-level computer models running on generic hardware and model activities that can be done in the human mind with a pen and a paper, and as such not a technology that is improved by the applicant. In the as-filed Specifications “¶[0059] In at least one embodiment, inference and/or training logic 615 may include, without limitation, code and/or data storage 601 to store forward and/or output weight and/or input/output data, and/or other parameters to configure neurons or layers of a neural network trained and/or used for inferencing in aspects of one or more embodiments. In at least one embodiment, training logic 615 may include, or be coupled to code and/or data storage 601 to store graph code or other software to control timing and/or order, in which weight and/or other parameter information is to be loaded to configure, logic, including integer and/or floating point units (collectively, arithmetic logic units (ALUs).”
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using a computer is noted as a general computer. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
Claim 2, 8, 14, 20 and 26 recite wherein the circuitry is further to use a connectionist temporal classification (CTC) function with one or more neural networks to generate probabilities of each of one or more characters representing speech and non-speech based on features extracted from one or more audio signals containing the one or more speech segments. This amounts to a person A listening to B speak. In a given 1 min interval, A writes down for every 5 sec if there was any speech in that time, and for that 5-sec he writes down ‘speech’ as a label for that segment and if there is no speech, he writes down ‘no-speech’, and he then adds the number of ‘speech’ and ‘no-speech’ labels for the 1 min segment and if there is a preponderance of ‘no-speech’ in that segment, he marks that segment as ‘no-speech’. connectionist temporal classification (CTC) function with one or more neural networks due to lack of specificity is mapped to general purpose elements and as such not a technology that is improved by the invention. No other limitations are present.
Claims 3, 9, 15, 21 and 27 recite wherein the circuitry is further to analyze the probabilities of each of the one or more characters using a greedy decoder to generate a string of characters of individual time steps. This amounts to a person A listening to B speak and mentally deciding that probability of letters is H: .9, E: .9, L: .9, L:0.9, O: 0.9. greedy decoder due to lack of specificity is mapped to general purpose elements and as such not a technology that is improved by the invention. No other limitations are present.
Claims 4, 10, 16, 22 and 28 recite wherein the circuitry is further to analyze the string of characters using a sliding window of a specified length, wherein the end of the one or more speech segments is determined in response to a percentage of blank characters contained within the sliding window being determined to satisfy an end of speech threshold. This amounts to a person A listening to B for 11 seconds and writing letters at each second “H E L L O_ _ _ _ _ _” after listening to B and mentally calculating that ratio of blank characters is 7/11 and is greater than 6/11 indicating end of speech. No other limitations are present.
Claims 5, 11, 17, 23 and 29 recite wherein the probabilities of each of the one or more characters are decoded up to the end of the one or more speech segments to generate one or more text transcripts of the one or more speech segments . This amounts to a person A listening to B for 11 seconds and writing letters at each second “H E L L O_ _ _ _ _ _” after listening to B and mentally calculating that ratio of blank characters is 7/11 and is greater than 6/11 indicating end of speech. No other limitations are present.
Claims 6, 12, 18, 24, 30 recite wherein transcripts of the one or more speech segments are to be provided as input to one or more voice-controllable devices. This amounts to a person A writing “H E L L O” The voice-controllable device is an extra-solution element used for the purpose of displaying data, and does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing of the abstract idea. No other limitations are present.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3,7, 8, 9, 13, 14, 15, 19, 20, 21, 25, 26, 27 are rejected under 35 U.S.C. 103 as being unpatentable over Wein, in further view of Zhou (US 20200005765 A1).
With respect to claim 1, 7, 13, 19 and 25 Wein teaches
(claim 1) One or more processors comprising: ([0030] The processing system 206 can comprise a microprocessor and other circuitry that retrieves and executes software)
(claim 7) A system comprising:
(claim 13) A method comprising:
(claim 19) A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
(claim 25) A voice transcription system, comprising: one or more processors to use one or more neural networks
[[use one or more neural networks to]]:
[[receive text representations of one or more digital audio signals]] (Abrash ¶ [0020] In step 108, the user begins speaking, at time t=S. The user command to commence speech recognition, received at time t=T.sub.S, and the actual start of the user speech, at time t=S, are only approximately synchronized; the user may begin speaking before or after the command to commence speech recognition [text representation of digital signal] received in step 106.;
determine, for each time step within a window of time for one or more speech segments of the one or more digital audio signals , a ratio of speech characters and non-speech characters within the window of time for the one or more speech segments [[comprising text representations ]](Abrash¶ [0041] In one embodiment, the method 300 may be adapted to detect the speech stopping frame as well as the speech starting frame (e.g., in accordance with step 212 of the method 200). However, in step 304, the method 300 would count the number, F.sub., of frames of the received audio signal in which the most likely word is silence in the last N.sub. preceding frames [window size is N.sub. frame; each frame consists of several time steps]. Then, when that number, F.sub.2, meets a second predefined threshold, T.sub.2, speech recognition processing is terminated (e.g., effectively identifying the frame at which recognition processing is terminated as the speech endpoint). In either case, the method 300 is robust to noise and produces accurate speech recognition results with minimal computational complexity [most likely word silence compared to on-silence=ratio of speech and non-speech characters determines end of speech], ¶[0056] Thus, the present invention represents a significant advancement in the field speech recognition. One or more Hidden Markov Models are implemented to endpoint (potentially augmented) audio signals for speech recognition processing, resulting in an endpointing method that is more efficient, more robust to noise and more reliable than existing endpointing methods. The method is more accurate and less computationally complex than conventional methods, making it especially useful for speech recognition applications in which input audio signals may contain background noise and/or other non-speech sounds); and
identify an end of one or more speech segments based, at least in part, on the ratio of speech and non-speech within the one or more speech segments, wherein the non-speech comprises one or more non-speech sounds (Abrash¶ [0041] In one embodiment, the method 300 may be adapted to detect the speech stopping frame as well as the speech starting frame (e.g., in accordance with step 212 of the method 200). However, in step 304, the method 300 would count the number, F.sub., of frames of the received audio signal in which the most likely word is silence in the last N.sub. preceding frames [window size is N.sub. frame; each frame consists of several time steps]) .
Abrash does not explicitly disclose however Zhou teaches circuitry to use one or more circuits to use one or more neural networks (Zhou ¶[0047] Connectionist temporal classification (CTC) 172 utilizes an objective function that allows RNN 352 to be trained for sequence transcription tasks without requiring any prior alignment between the input and target sequences. The output layer contains a single unit for each of the transcription labels, such as characters or phonemes plus an extra unit referred to as the "blank" which corresponds to a null emission. Given a length T input sequence X, the output vectors yt are normalized with the softmax function, then interpreted as the probability of emitting the label or blank with index k at time t).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify end of speech determination of Abrash with the neural network of Zhou in order to optimizes a performance metric, such as CER or WER, defined over output transcriptions ([0032], Zhou);
Neither Abrash nor Zhou explicitly disclosed however Basye teaches receive text representations of one or more digital audio signals ([0039] FIG. 4 is a flow diagram illustrating a speech isolation process for speech recognition according to one aspect of the present disclosure. An ASR device receives a plurality of audio signals from a microphone array, as illustrated in block 402. The ASR device processes the received plurality of audio signals to generate one or more beamformed signals, as illustrated in block 404. The ASR device identifies speech within the one or more beamformed signals, as illustrated in block 406. The ASR device selects a beamformed signal comprising speech, as shown in block 408. The ASR device then identifies an utterance in the selected beamformed audio signal, as shown in block 410. The ASR device then performs speech recognition on the utterance, as shown in block 412. The ASR device may also determine an end of the utterance, as shown in block 414)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify end of speech determination of Abrash in view of the neural network of Zhou to include text representation of Basye in order to perform postprocessing efficiently (¶¶ [0031], [0039, Basye);
With respect to claim 2, 8, 14, 20 and 26 Zhou further teaches wherein the one or more circuits are further to use a connectionist temporal classification (CTC) function with one or more neural networks to generate probabilities of each of one or more characters representing speech and non-speech based on features extracted from one or more audio signals containing the one or more speech segments ([0047] Connectionist temporal classification (CTC) 172 utilizes an objective function that allows RNN 352 to be trained for sequence transcription tasks without requiring any prior alignment between the input and target sequences. The output layer contains a single unit for each of the transcription labels, such as characters or phonemes plus an extra unit referred to as the "blank" which corresponds to a null emission. Given a length T input sequence X, the output vectors yt are normalized with the softmax function, then interpreted as the probability of emitting the label or blank with index k at time t).
With respect to claims 3, 9, 15, 21, 27 Zhou further teaches wherein the one or more circuits are further to analyze the probabilities of each of the one or more characters using a greedy decoder to generate a string of characters of individual time steps ([0047] For the connectionist temporal classification (CTC), consider an entire neural network to be simply a function that takes in some input sequence of length T and outputs some output sequence y also of length T [generated string of characters], and [0048] Connectionist temporal classification (CTC) 172 utilizes an objective function that allows RNN 352 to be trained for sequence transcription tasks without requiring any prior alignment between the input and target sequences. The output layer contains a single unit for each of the transcription labels, such as characters or phonemes plus an extra unit referred to as the "blank" which corresponds to a null emission. Given a length T input sequence X, the output vectors yt are normalized with the softmax function, then interpreted as the probability of emitting the label or blank with index k at time t, and [0058] Training with the defined objective is efficient, since both sampling and greedy decoding are cheap).
With respect to claims 4, 10, 16, 22 Abrash teaches further to analyze the string of characters [[using a sliding window of a specified length]], wherein the end of the one or more speech segments is determined in response to a percentage of blank characters contained [[within the sliding window]] being determined to satisfy an end of speech threshold (Claim 55. The computer readable medium of claim 51, wherein said step of locating a second speech endpoint comprises: counting a number of frames of said audio signal for which a most likely word in a pre-defined quantity of preceding frames is silence [non-speech character]; determining whether said number of frames exceeds a second pre-defined threshold; [counting silence words in frames that exceed a threshold equates to proportion of non-speech characters])
Zhou further teaches using a sliding window of specified length ([0043] FIG. 2 shows a block diagram for preprocessor 148 which includes spectrogram generator 225 which takes as input, sampled speech audio wave 252 and computes, for each speech input, a spectrogram with a sliding 20 ms window and 10 ms step size.)
With respect to claim 5, 11, 17, 23, 29 Zhou further teaches to analyze the string wherein the probabilities of each of the one or more characters are decoded up to the end of the one or more speech segments in order to generate one or more text transcripts of the one or more speech segments ([0053] FIG. 4 shows an example whole transcription sampled [decoded] by e sampling module 125 from softmax probabilities generated by the RNN 352 after processing a speech sample annotated with a “HALO” transcription. The illustrated example would use CER as the evaluation metric. Another example could include words instead of characters, and calculate WER. In FIG. 4, the x axis shows the letters predicted for each 20 ms window, and the y axis lists the twenty-six letters of the alphabet and blank 472 and space 482. The bright red entries correspond to letters sampled by the sampling module 125. The sampled whole transcription is “HHHEE_LL_LLLOOO”. In some implementations, a collapsing module (not show) enforces CTC collapsing rules and removes repeated letters and blanks to produce a final whole transcription “HELLO”.).
Claims 6, 12, 18, 24, 30 are rejected under 35 U.S.C. 103 as being unpatentable over Abrash, Zhou and Basye in further view of Gorny (US 20210020181 A1).
With respect to claim 6, 12, 18, 24 and 30 none of Abrash and Zhou explicitly disclose, however, Gorny teaches wherein transcripts of the one or more speech segments are to be provided as input to one or more voice- controllable devices ([0042] According to embodiments, transcription module 206 accesses local device audio data 214 and transcribes the audio data stored in local device audio data 214 into a local device text transcript and [0006] In embodiments of the disclosed subject matter, the computer merges the audio transcription data from each of the two or more communication devices into a master audio transcript. The computer transmits the master audio transcript to each of the two or more communication devices.).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify end of speech determination of Abrash in view of the neural network of Zhou in view of text representation of Basye to include the transcripts of Gorny in order for devices to view master transcript in real time ([0018], Gorny);
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ATHAR N PASHA whose telephone number is (408)918-7675. The examiner can normally be reached Monday-Thursday Alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached on (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ATHAR N PASHA/Examiner, Art Unit 2657