Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is sent in response to Applicant’s communication received on 03/24/2025 for the application number 19087830. The office hereby acknowledges receipt of the following placed of record in the file: Specification, Abstract, Oath/Declaration and claims.
Status of the claims
Claims 11-25 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/24/2025 was filed before the mailing date of the first office action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 11-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as explained below.
Claim 11 recites a method for supporting the hearing comprehension of a user of a hearing instrument, the method comprising:
using the hearing instrument to capture speech-containing ambient sound from surroundings of the user;
automatically converting speech contained in the captured ambient sound into text data;
outputting the text data to the user as at least one of: a graphical representation of text on a screen of the hearing instrument or of a peripheral device connected to the hearing instrument for data transmission purposes, or synthesized speech formed as a sound signal;
automatically determining at least one of a direction of origin or at least one speaker trait for the speech contained in the ambient sound in a manner resolved with respect to time;
And varying the graphical representation of the text data and the synthesized speech based on at least one of the identified direction of origin or the at least one speaker trait in a manner resolved with respect to time.
Step (a) comprises a mental process. This step can be performed by a human as a person can listen to and hear speech containing ambient sound from the person’s surroundings.
Step (b) comprises a mental process. This step can be performed by a human as a person can listen to spoken words and mentally recognize and convert the spoken words into corresponding text or words.
Step (c) comprises a mental process. This step can be performed by a human as a person can communicate the recognized speech to another person in written form or by repeating the recognized speech aloud.
Step (d) comprises a mental process. This step can be performed by a human as a person can listen to speech and determine where the speech originates or recognize a characteristic of the person speaking, such as the identity, gender, or characteristics of the speaker’s voice.
Step (e) comprises a mental process. This step can be performed by a human as a person can alter the manner in which recognizes speech is written or spoken based on the location or characteristics of the speaker and can change the written or spoken representation according to changes in time.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any
statutory category. See MPEP 2106.03. The claim recites at least method. Thus, the claim is a
process, which is one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the
claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim
“recites” a judicial exception when the judicial exception is “set forth” or “described” in
the claim. As discussed above, the broadest reasonable interpretation of steps (a)-(e)
recites a mental process.
Specifically, step (a) can be performed by a human as a person can listen to and hear speech containing ambient sound from the person’s surroundings.
Step (b can be performed by a human as a person can listen to spoken words and mentally recognize and convert the spoken words into corresponding text or words.
Step (c) can be performed by a human as a person can communicate the recognized speech to another person in written form or by repeating the recognized speech aloud.
Step (d) can be performed by a human as a person can listen to speech and determine where the speech originates or recognize a characteristic of the person speaking, such as the identity, gender, or characteristics of the speaker’s voice.
Step (e) can be performed by a human as a person can alter the manner in which recognizes speech is written or spoken based on the location or characteristics of the speaker and can change the written or spoken representation according to changes in time.
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a
whole integrates the recited judicial exception into a practical application of the exception or
whether the claim is “directed to” the judicial exception. This evaluation is performed by (1)
identifying whether there are any additional elements recited in the claim beyond the judicial
exception, and (2) evaluating those additional elements individually and in combination to
determine whether the claim as a whole integrates the exception into a practical application. See
MPEP 2106.04(d).
The claim recites additional elements including a hearing instrument, a screen of a hearing instrument, or a peripheral device connected to the hearing instrument. However, these elements are recited at a high level of generality and perform generic computer functions, such as executing instructions and storing data. The use of these elements to capture ambient sound, convert speech into text data, determine a direction of origin or speaker traits, and output or vary text or synthesized speech constitutes a generic computational technique that merely automates the mental processes described above. Such processing does not impose any meaningful limit on the judicial exception. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. (Step 2A, Prong Two: NO), and the claim is directed to the judicial exception. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole
amounts to significantly more than the recited exception i.e., whether any additional element, or
combination of additional elements, adds an inventive concept to the claim. As explained with
respect to Step 2A, Prong Two, a hearing instrument, a screen of a hearing instrument, or a peripheral device connected to the hearing instrument comprise additional elements that do not contribute to the patentability of the claim as a whole. They constitute at best mere instructions to “apply” the abstract ideas, which cannot provide an inventive concept. See MPEP 2106.05(f). At Step 2B, the evaluation of the insignificant extra solutional activity consideration takes into account whether or not the extra-solutional activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). As known in the art these elements are well routine and conventional. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solutional activity which do not provide an inventive concept. The claim is not patent eligible.
Claim 12 recites a mental process as a human can communicate recognized speech in written and spoken form as the speech is received.
Claim 13 recites a mental process as a human can change the location of written information or use a directional indicator based on the perceived location of a speaker.
Claim 14 recites a mental process as a human can associate written speech with an image of the person or sound source from which the speech originated.
Claim 15 recites a mental process as a human can visually distinguish written speech by changing its color, size, font, or background.
Claim 16 adds the generation of stereo audio corresponding to the perceived direction of a sound source, which constitutes an additional function performed using conventional components.
Claim 17 recites a mental process as a human can alter the manner in which speech is reproduced, such as its timbre or pitch, based on a recognized characteristic of the speaker.
Claims 18-23 are analogous to claims 11, 13-17 in that they recite substantially the same limitations. They are therefore rejected for the same reasons.
Claims 24 and 25 recite substantially the same mental processes as claim 11, implemented using a hearing instrument and generic signal-processing components. These additional elements merely automate the mental processes described above and do not limit the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 11, 12, 15, 17, 18, 21, 23-25 are rejected under 35 U.S.C. 103 as being unpatentable over Menezes et al. (US 20170243582 A1) in view of Skuratovsky et al. (US 20070118378 A1).
Regarding claim 11, Menezes teaches a method for supporting the hearing comprehension of a user of a hearing instrument (Fig. 2 teaches the system as a whole), the method comprising: using the hearing instrument to capture speech-containing ambient sound from surroundings of the user (Para 0028, “The assistive hearing device 200 includes a microphone (or a microphone array) 204 that captures audio 206 containing speech as well as background noise or sounds.”); automatically converting speech contained in the captured ambient sound into text data (Para 0029, “A speech recognition module 224 on the assistive hearing device 200 converts the received audio 206 to text 228.”); outputting the text data to the user as at least one of: a graphical representation of text on a screen of the hearing instrument or of a peripheral device connected to the hearing instrument for data transmission purposes, or synthesized speech formed as a sound signal (Para 0029, “The speech recognition module 224 can output the transcript 228 to a display 234.”); automatically determining at least one of a direction of origin or at least one speaker trait for the speech contained in the ambient sound in a manner resolved with respect to time (Para 0029, “the speech recognition module can determine which person is speaking to the hearing impaired user 208 by using the speech models to distinguish which person is speaking”).
Menezes does not teach varying the graphical representation of the text data and the synthesized speech based on at least one of the identified direction of origin or the at least one speaker trait in a manner resolved with respect to time.
However, Skuratovsky teaches varying the graphical representation of the text data and the synthesized speech based on at least one of the identified direction of origin or the at least one speaker trait in a manner resolved with respect to time (Para 0024, “A next phase can include automatically identifying a gender for the spoken passages.” And para 0027, “The TTS system 130 can dynamically apply different voice configurations to different portions of text within the text source 105 according to the attributes determined for each respective portion of text” and para 0035, “In step 250, any attributes corresponding to the portion of text also can be displayed.” And finally, para 0041, “This information can be used by a TTS system when producing an audible rendition of the text to dynamically select an appropriate voice configuration on a word-by-word, phrase-by-phrase, etc. basis according to the attributes determined for the particular portion of text being rendered at any given time.” Wherein the system varies both the graphical output and the synthesized speech according to one speaker trait (gender)).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Skuratovsky in order to better distinguish between speakers by providing graphical and synthesized speech outputs corresponding to characteristics of each speaker (Para 0027).
Regarding claim 12, Menezes teaches outputting the text data and the synthesized speech to the user in real time (Para 0048, “This process can occur in real-time so that the user sees a transcript of the speech on a display at the same time that the speech is spoken” and para 0047, “This process 500 can occur in real-time so that the user can hear the enhanced speech at essentially the same time that the speech is being spoken”).
Regarding claim 15, Menezes does not teach varying at least one of a text color, a text size, a font or a background color of an associated text box for the graphical representation of the text data in a manner resolved with respect to time.
However, Skuratovsky teaches varying at least one of a text color, a text size, a font or a background color of an associated text box for the graphical representation of the text data in a manner resolved with respect to time (Para 0035, “The attributes can be displayed…whether through color coding, by placing the attribute proximate… placing tags or other markers in-line with the text” and para 0037, “male gender, female gender, or a particular speaker identity, can be highlighted within the text as it is displayed and/or audibly rendered”).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Skuratovsky in order to visually distinguish text associated with different speakers, thus making the transcriptions easier to interpret (Para 0035).
Regarding claim 17, Menezes does not teach varying at least one of a voice timbre or a tonal pitch of the synthesized speech based on the at least one speaker trait in a manner resolved with respect to time.
However, Skuratovsky teaches varying at least one of a voice timbre or a tonal pitch of the synthesized speech based on the at least one speaker trait in a manner resolved with respect to time (Para 0027, “The TTS system 130 can dynamically apply different voice configurations to different portions of text within the text source 105 according to the attributes determined for each respective portion of text” and para 0033, “Other attributes that may be used in determining a voice configuration are gender, tone, prosody, and pitch.”).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Skuratovsky in order to reproduce speech using specific characteristics and thus make the speech more representative of the speaker (Para 0033).
Claim 18 is analogous to claim 11 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 21 is analogous to claim 15 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 23 is analogous to claim 17 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claims 24 & 25 are analogous to claims 11 & 18 in that they recites substantially the same limitations in system form, with the claimed functions being performed by corresponding hearing components. Menezes teaches the claimed hearing instrument, input transducer, signal processing output, speech recognition, direction/speaker analysis, and processing components. Claims 24 & 25 are therefore rejected for similar reasons as above
Claims 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over
Menezes (US 20170243582 A1) in view of Skuratovsky (US 20070118378 A1) as above and further in view of Teshima et al. (US 20170243600 A1).
Regarding claim 13, Menezes does not teach varying at least one of a display location for the graphical representation of the text data or a direction-indicating symbol associated with the text based on the identified direction of origin in a manner resolved with respect to time.
However, Teshima teaches varying at least one of a display location for the graphical representation of the text data or a direction-indicating symbol associated with the text based on the identified direction of origin in a manner resolved with respect to time (Para 0108, “the sound source location identification section 32 instructs the output section 28 to display the captioned speech content acquired by the processing of step S40 in the direction of emitted audio identified by the processing of step S30.”).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Teshima in order to allow the user to more readily identify the source of the speech (Para 0108)
Claim 19 is analogous to claim 13 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claims 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over
Menezes (US 20170243582 A1) in view of Skuratovsky (US 20070118378 A1) as above and further in view of Cowburn (US 10074381 B1)
Regarding claim 14, Menezes does not teach producing the graphical representation of the text data in a form of a text insertion into a real image sequence of the surroundings of the user captured during capture of the ambient sound, and locally associating the text insertion with a depiction of a related sound source in the real image sequence.
However, Cowburn teaches producing the graphical representation of the text data in a form of a text insertion into a real image sequence of the surroundings of the user captured during capture of the ambient sound (Col. 13, Ln. 39-44, “the presentation module 602 generates and causes display of a presentation of a space at a client device … the client device 102 may have an integrated camera, and may capture and display an image of a space. The presentation of the space may include a depiction of a first person”), and locally associating the text insertion with a depiction of a related sound source in the real image sequence (Col. 15, Ln. 19-30, “the presentation module 602 generates a speech bubble that includes a presentation of the text string, based on the speech properties, a location of the source of the speech in the presentation, and the text string” and “a tail portion to indicate a source of the speech”).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Cowburn in order to allow the user to more readily identify which depicted person produced the speech (Col. 13).
Claim 20 is analogous to claim 14 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claims 16 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over
Menezes (US 20170243582 A1) in view of Skuratovsky (US 20070118378 A1) as above and further in view of Teshima et al. (US 20170243600 A1).
Regarding claim 16, Menezes does not teach generating the synthesized speech based on the direction of origin as a stereo signal with a stereophonic sound characteristic corresponding exactly or approximately to a stereophonic sound characteristic of the ambient sound.
However, Cutler teaches generating the synthesized speech based on the direction of origin as a stereo signal with a stereophonic sound characteristic corresponding exactly or approximately to a stereophonic sound characteristic of the ambient sound (Para 0061, “One particular way of doing this in the case of two or more speakers is to use stereo or surround sound in order to place the synthesized speech at a distinct spatial position from the perspective of the receiving user” and “a head related transfer function (HRTF) may be used to place the synthesized speech at a distinct position, including giving the sensation of forward or backward and/or up or down directions.”).
It would have been obvious to one of ordinary skill in the art to modify Menezes before the effective filing date to incorporate the teachings of Cutler in order to preserve directional details of the original ambient speech (Para 0061).
Claim 22 is analogous to claim 16 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL ALAN FOSTER JR. whose telephone number is (571)272-8874. The examiner can normally be reached M - F 8:00am - 5:00pm, Alternate Fridays Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL A FOSTER JR/ Examiner, Art Unit 2654
/HAI PHAN/ Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights