DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 8, 9 and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-7 and 11-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Riemer (US 2022/0103952), Blumenau (US 2008/0240475) or Wexler et al (US 2023/0045237).
Claims 1 and 14. Riemer, via Fig. 1, teaches a method and a device comprising:
receiving audio data comprising one or more voices of one or more users present in an environment, wherein a user wearing an earpiece is present in the environment and is distinct from the one or more users; (The speech analyzer may comprise a neural network trained to identify individual sentences in an audio stream, e.g. in an audio stream comprising noisy speech. The neural network may be trained in an off-line procedure based on a large database of known sentences spoken by different persons in different noise environments. Optimized parameters (e.g. weights) of the trained neural network may be stored in the speech analyzer. Parameters of the neural network may be updated during use of the hearing aid in a particular learning mode, e.g. in cooperation with the user interface, e.g. implemented in an auxiliary device, [0033-0034]. The hearing aid may comprise an earpiece configured to be worn at or in an ear of the user, or to be partially implanted in the head at an ear of the user, [0035-0045]).
storing the received audio data in a memory; (configured to store audio data representing one or more time segments of an electric input signal, Abstract and The memory unit (MEM) may form part of or include the input buffer (IBUF) and comprise stored audio data according to the present disclosure, [0194]);
based on receiving an input from the user wearing the earpiece, selecting a portion of the audio data to replay, wherein the input is received while the user wearing the earpiece is present in the environment; (The present application relates to the field of hearing aids, in particular to storage, analysis and replay of recently spoken words or sentences by a person in the hearing aid wearers acoustic environment, [0124-0132]); and
causing, by control circuitry, the selected portion of the audio data to be replayed by the earpiece of the user wearing the earpiece. (see the previous step).
Claims 1 and 14. Blumenau teaches a method and a device comprising:
receiving audio data comprising one or more voices of one or more users present in an environment, wherein a user wearing an earpiece is present in the environment and is distinct from the one or more users; (Hearing assistive apparatus according to the invention assists a user of the hearing assistive apparatus in hearing sound that occurs in the vicinity of the user. A user of hearing assistive apparatus according to the invention can be any sentient being capable of hearing. It is anticipated that, typically, a user of hearing assistive apparatus according to the invention will be a person; however, a user of hearing assistive apparatus according to the invention can also be an animal. In many embodiments of the invention, hearing assistive apparatus according to the invention is adapted to be "worn" (the user of such hearing assistive apparatus according to the invention is sometimes referred to herein as a "wearer" of the hearing assistive apparatus), [0036]);
storing the received audio data in a memory; (When the digital data storage device 802 stores an amount of sound data that corresponds to a specified replay time, at the time of inputting new sound data into the digital data storage device, the control logic 804 causes the sound data that has been stored longest in the digital data storage device 802 to be output from the digital data storage device 802 and input to the output buffer 803. [0052] and Fig. 8);
based on receiving an input from the user wearing the earpiece, selecting a portion of the audio data to replay, wherein the input is received while the user wearing the earpiece is present in the environment; (Hearing assistive apparatus according to the invention can be implemented to enable operation in a replay mode (in which the hearing assistive apparatus replays sound that occurred prior to the current time, e.g., sound that occurred a specified duration of time, such as 5, 10 or 30 seconds, prior to the current time), [0009-0020, 0034-0036] and Figs. 3-11B); and
causing, by control circuitry, the selected portion of the audio data to be replayed by the earpiece of the user wearing the earpiece. (See previous step).
Claims 1 and 14, Wexler, via Fig. 1A, teaches a method and a device comprising:
receiving audio data comprising one or more voices of one or more users present in an environment, wherein a user wearing an earpiece is present in the environment and is distinct from the one or more users; (The hearing aid system includes a wearable camera configured to capture a plurality of images from an environment of a user and at least one microphone configured to capture sounds from the environment of the user., [0006-0029]);
storing the received audio data in a memory; (The voice of the user may be recognized based on a voiceprint associated with the user, which may be stored in memory 550, database 2050, or other suitable locations, [0241]);
based on receiving an input from the user wearing the earpiece, selecting a portion of the audio data to replay, wherein the input is received while the user wearing the earpiece is present in the environment; (The periods missed by replaying previous periods of sound may then be provided to user 100 at an increased rate, for example by reducing the silence periods between words, or in any other suitable manner. In some embodiments, automatically replaying the sound may be performed based on a gesture performed by the user rather than a spoken indication by the user, [0338, 0557, 0584, 0587]); and
causing, by control circuitry, the selected portion of the audio data to be replayed by the earpiece of the user wearing the earpiece. (See previous step).
Claim 2. (Original) The method of claim 1, wherein the earpiece comprises the control circuitry, the earpiece corresponding to headphones or a hearing aid, and the input is received via an interface of the earpiece. (See the independent claims).
Claim 3 . (Original) The method of claim 1, wherein a mobile device comprises the control circuitry, and the input is received via a user interface of the mobile device. (Riemer: a wireless communication device, e.g. a mobile phone (such as a smartphone), [0110]; Wexler: Fig. 38).
Claims 4 and 17. (Original) The method of claim 1, wherein the memory in which the audio data is stored is a temporary memory having a particular memory capacity, the method further comprising: identifying at least one of a number of the one or more users or an ambient noise level of the environment; and based on the particular memory capacity and at least one of the number of the one or more users or the ambient noise level, adjusting one or more parameters of the audio data being stored in the temporary memory. (Wexler: the plurality of audio samples may include audio samples 4412 and 4414, as described above with respect to FIG. 44. The buffer may be any storage location where audio samples are stored, at least temporarily, for analysis and/or processing. In some embodiments, the buffer may correspond to memory 550, [0513]).
Claims 5 and 18. (Original) The method of claim 1, wherein the selecting the portion of the audio data to replay comprises: determining a first timepoint within the audio data that corresponds to when the input was received; (Riemer: The top graph of FIG. 3 schematically illustrates a time period reflecting the contents of a sound input to the hearing aid (represented by the electric input signal, see e.g. IN in FIG. 1, 2). Between time t.sub.2 and t.sub.3 (time range Δt.sub.23), the user's voice is detected to be present (indication ‘User's voice#1’) as detected by an own voice detector, see middle graph (denoted ‘OVC (Own voice control signal)’ vs. ‘Time’, the OVC taking on values between 0 and 1, here shown as binary). Between time t.sub.1 and t.sub.2 and between time t.sub.3 and t.sub.4 (i.e. in time ranges Δt.sub.12 and Δt.sub.34, respectively), the user's voice is detected to be present (indication ‘User's voice#1’) (e.g. as concluded by the own voice detector indicating that the user's voice is NOT detected in these time periods), [0184]) and selecting, as the selected portion of the audio data, a portion of the audio data corresponding to a last portion of the audio data that was detected prior to the first timepoint. (Riemer: the top graph of FIG. 4 comprising time segments denoted ‘Other voice’ (time segments Δt.sub.11′, Δt.sub.1″2, Δt.sub.3′4), ‘Noise’ (time segments Δt.sub.1′1″, Δt.sub.33′), and ‘User's voice’ (time segment Δt.sub.23). The more detailed classification of the acoustic environment is reflected in the values of the VAC and OVC signals in the two middle graphs. The bottom graph schematically illustrates the contents of the input buffer in case the input controller depends on the own voice control signal (OVC) as well as the (general) voice activity control signal (VAC). The stored time segments are only segments ‘Other voice (Δt.sub.11′, Δt.sub.1″2, and Δt.sub.3′4) i.e. excluding the time segment containing the user's own voice and noise (no voice), [0185]).
Claims 6 and 19. (Original) The device of claim 18, wherein the control circuitry is further configured to: identify a particular voice of the one or more voices that was a last voice detected prior to the first timepoint; (Wexler: Voice recognition component 2041 may analyze the audio signal representative of sound 2020 to determine whether voice 2012 matches a voiceprint of an individual in database 2050. Accordingly, database 2050 may contain voiceprint data associated with a number of individuals, similar to the stored facial identification data described above. After determining a match, individual 2010 may be determined to be a recognized individual of user 100, [0217]) determine, from the audio data, a second timepoint, occurring prior to the first timepoint, when a voice segment corresponding to the particular voice during the last portion of the audio data began; and select, as the selected portion of the audio data, a portion of the audio data from the second timepoint to the first timepoint. (Riemer: the top graph of FIG. 4 comprising time segments denoted ‘Other voice’ (time segments Δt.sub.11′, Δt.sub.1″2, Δt.sub.3′4), ‘Noise’ (time segments Δt.sub.1′1″, Δt.sub.33′), and ‘User's voice’ (time segment Δt.sub.23). The more detailed classification of the acoustic environment is reflected in the values of the VAC and OVC signals in the two middle graphs. The bottom graph schematically illustrates the contents of the input buffer in case the input controller depends on the own voice control signal (OVC) as well as the (general) voice activity control signal (VAC). The stored time segments are only segments ‘Other voice (Δt.sub.11′, Δt.sub.1″2, and Δt.sub.3′4) i.e. excluding the time segment containing the user's own voice and noise (no voice), [0185]).
Claims 7 and 20. (Original) The method of claim 1, wherein the selecting the portion of the audio data to replay comprises: determining a first timepoint within the audio data that corresponds to when the input was received; and selecting, as the selected portion of the audio data, a portion of the audio data from a second timepoint occurring at a predetermined period of time prior to the first timepoint. (Riemer: the top graph of FIG. 4 comprising time segments denoted ‘Other voice’ (time segments Δt.sub.11′, Δt.sub.1″2, Δt.sub.3′4), ‘Noise’ (time segments Δt.sub.1′1″, Δt.sub.33′), and ‘User's voice’ (time segment Δt.sub.23). The more detailed classification of the acoustic environment is reflected in the values of the VAC and OVC signals in the two middle graphs. The bottom graph schematically illustrates the contents of the input buffer in case the input controller depends on the own voice control signal (OVC) as well as the (general) voice activity control signal (VAC). The stored time segments are only segments ‘Other voice (Δt.sub.11′, Δt.sub.1″2, and Δt.sub.3′4) i.e. excluding the time segment containing the user's own voice and noise (no voice), [0185]).
Claim 11. (Original) The method of claim 1, wherein causing the portion of the audio data to be replayed further comprises altering the portion of the audio data to cause one or more of removing background noise, translating the portion of the audio data into another language, or changing a speed of the replay of the portion of the audio data. (Wexler: the conditioning may include amplification of the audio signal relative to other audio signals received from outside of the region associated with the look direction of the user. Amplification may be performed by various means, such as operation of a directional microphone configured to focus on audio sounds emanating from the region, or varying one or more parameters associated with the microphone to cause the microphone to focus on audio sounds emanating from the region. The amplification may include attenuating or suppressing one or more audio signals received by the microphone from directions outside the region associated with the look direction of user 110, [0203]).
Claim 12. (Original) The method of claim 1, wherein the selecting the portion of the audio data to replay comprises: determining a relevance of each portion of the audio data by: determining one or more entities within each portion of the audio data using natural language processing; and comparing the one or more entities within each portion of the audio data to information stored in a user profile of the user wearing the earpiece. (Wexler: in some embodiments, the at least one voice trait may include a list of the individual and the selective conditioning includes removing the lisp. In this case, processor 210 may replace “th” sounds with an “s” sound in the first audio signal. However, because the “th” sound may be properly used in some words by a speaker having a lisp, processor 210 may include a natural language processing algorithm to determine if a word identified in the first audio signal should be pronounced with a “th” rather than an “s”, and, as a result, refrain from replacing the “th” with an “s” sound, [0594]; Processor 210 may store a rule that when the friend's voice signature matches the first audio signal (indicating that the user is speaking with the friend), some segments of the first audio signal should be replaced with an audio signal properly saying the “l” sound. Processor 210 may further employ natural language processing methods to determine if the original sound was correct and refrain from inserting the “l” sound, for instance. Processor 210 may also selectively condition the voice to a higher pitch and remove overtones responsible for the gravelly quality of the friend's voice, [0601]).
Claim 13. (Original) The method of claim 2, wherein the user interface of the mobile device (Riemer: The hearing aid may comprise and/or be connectable to a user interface. The hearing aid may comprise the user interface, [0023-0028]) comprises a timeline indicating the portions of the audio data spoken by each of the one or more users and detected pauses indicating multiple portions spoken by a same user of the one or more users, and wherein the input comprises selecting a timepoint on the timeline. (Riemer: The hearing aid may comprise a speech analyzer connected to the input buffer and configured to analyze at least one, such as all, of the one or more time segments of the electric input signal, [0020-0023] wherein The hearing aid may comprise a replay controller for controlling the presenting of said one or more time segments or selected sub-sections thereof to the user via said output transducer. The replay controller may be configured to control a weighting of the stored signal from the input buffer and the current signal originating from the input transducer. The replay controller may be controlled vi the user interface, e.g. via the output control signal, [0035]. Also see Figs. 4 and 5 for indicating sub-sections of the stored time segments containing speech).
Claim 15. (Original) The device of claim 14, wherein the device comprises the earpiece, the earpiece corresponding to headphones or a hearing aid. (See the independent claims).
16. (Original) The device of claim 14, wherein the device comprises a mobile device. (Riemer: a wireless communication device, e.g. a mobile phone (such as a smartphone), [0110]; Wexler: Fig. 38).
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUNG-HOANG J. NGUYEN whose telephone number is (571)270-1949. The examiner can normally be reached Reg. Sched. 6:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUNG-HOANG J NGUYEN/Primary Examiner, Art Unit 2691
Prosecution Insights