DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending and claims 1, 9 and 17 are independent claims.
Response to Arguments
Applicant's arguments filed on 02/09/2026, pages 7-11, have been fully considered but they are not persuasive.
The Applicant argues that the combination of Benattar, Glaser, and Pardo fail to teach or suggest any extraction of audio signals from noisy audio signals for incorporation in outputs to users (e.g., during run-time). For instance, the Applicant argues that "extracting from the first noisy audio signal a first audio signal associated with the first person" is not addressed by Banattar. More specifically, the Applicant argues that the description of Benattar does not relate to an extraction of an audio signal. The Applicant argues that Banattar does not deal with extracting speech from noise (Arguments, pages 8-9).
The Examiner respectfully disagrees. Contrary to the argument of the Applicant, Benattar teaches creating profiles for a noise for isolating/extracting and analyzing audio and speech. Banattar teaches active (i.e., during run-time) noise control systems utilizing various active filtration techniques and also teaches ambient sound to source audio at the ear of a listener. The active noise control devices taught in Banattar are capable of addressing a wide variation of ambient sound and dominant noise. Benattar also teaches systems that may be implemented in one or more digital signal processors and/or adaptive filters operating on ambient, directional or directionless, source and noise audio in order to enhance delivery of desirable audio and damp delivery of undesirable audio. Benattar also teaches multi-channel digital signal processing to divide/separate ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech. An embodiment of the invention may dynamically adjust attenuation rates across channels and frequency ranges, may have a feature that enables a user to apply adaptive filters to each channel either independently or across all channels simultaneously. Benattar also teaches identifying modulation characteristics within a mixed signal as well. The teaching even has the advantageous features of a system that may facilitate adjustment of filtration on the basis of direction of sound sources; signal detection methodology of acoustic measurement among modulation rates, synchronous energy (opening and closing of vocal folds) or signal to noise ratios depending on both the environment and the nature of the sound which is desirable (i.e. speech or other ambient sounds). Benattar also teaches creating profiles for a noise, such as a jackhammer or a person the user does not want to hear, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller and communications may occur over a network or may occur when people are talking within “earshot” of each other. Thus, Banattar deals with extracting speech or any desired audio from noise and works during run-time as well (Banattar, para 0010-0011; Banattar, para 0042-0047, Benattar, para 0097-0101).
The Applicant argues that the combination of Benattar, Glaser, and Pardo fail to teach or suggest at least converting the noisy audio signal into a frequency domain and extracting the first audio signal of the first person from the converted noisy audio signal in claim 7. The Applicant further argues that the cited paragraphs do not teach the claim limitations (Arguments, pages 9-10).
The Examiner respectfully disagrees. Applying the audio signature or audio fingerprinting or the acoustic fingerprinting and other audio processing technologies in Benattar sound spectrographing can be used to recognize sounds with unique characteristics which distinguish them, even with very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them. Thus, Benattar teaches converting the noisy audio signal into a frequency domain and extracting the first audio signal of the first person from the converted noisy audio signal. Benattar also teaches sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies that may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them, but also effectively read existing spectrographs which may exist in a personal ambient sound library of a user, or which may otherwise reside in a database of available ambient sound spectrographs and decode such spectrographs to inform the digital signal processing and active filtration systems of those patterns which should be treated as desired ambient sounds and thus included in the customized listening environment of a user when they are present in the ambient environment. Benattar’s claimed invention provides a system for managing a sound library and audio profiles. Thus, Banattar teaches extracting the first audio signal of the first person from the converted noisy audio signal. Thus, the sound spectrographing technology and acoustic fingerprinting of Benattar enables converting the first noisy audio signal into a frequency domain, thereby obtaining a converted first noisy audio signal (Benattar, para 0032-0033, para 0060).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-14, and 16-20 are rejected under 35 U.S.C. under 35 U.S.C. 103 as being unpatentable over Benattar Pat App No. US 20160162254 A1 (Benattar) in view of Glaser et al. Pat US 20190354343 A1 (Glaser).
Regarding Claim 1, Benattar discloses a method performed in an environment of a user (Benattar, para 0003, The invention relates to audio processing systems), the environment comprising a plurality of people participating in at least one conversation, the plurality of people comprise at least a first person and a second person (para 0031, a listener may be able to hear important information or hold a conversation with another person), the user having at least one hearable device used for providing audio output to the user (Benattar, para 0099, The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone), the method comprising:
obtaining a first activation selection of the first person (Benattar, para 0105, The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber);
capturing a first noisy audio signal from the environment (Benattar, para 0097, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array. The adaptive audio controller may also be connected to a set of active audio profiles 403. The active audio profiles 403 may be selected from profiles stored in the sound library 404. The sound library 404 may contain audio profiles created by sampling audio information detected by the ambient microphone. If a user wants to establish a profile for certain characteristic audio, the audio may be sampled and characterized in order to create a profile. The sample audio may be used to create an audio profile such as a specific voice, machinery, or other noise. Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401);
extracting from the first noisy audio signal a first audio signal associated with the first person, said extracting comprises applying speech separation (Benattar, para 0042-0047, The system may be implemented in one or more digital signal processors and/or adaptive filters operating on ambient, directional or directionless, source and noise audio in order to enhance delivery of desirable audio and damp delivery of undesirable audio… multi-channel digital signal processing to divide ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech…An embodiment of the invention may dynamically adjust attenuation rates across channels and frequency ranges, may have a feature that enables a user to apply adaptive filters to each channel either independently or across all channels simultaneously… Advantageous features of a system may facilitate adjustment of filtration on the basis of direction of sound sources; signal detection methodology of acoustic measurement among modulation rates, synchronous energy (opening and closing of vocal folds) or signal to noise ratios depending on both the environment and the nature of the sound which is desirable (i.e. speech or other ambient sounds));
outputting to the user via the at least one hearable device (Benattar, para 0082, One or more active profiles 103 may be used by the audio customization engine 101 to customize audio signals provided to an audio output device 104, for example, headphones), a first output signal comprising at least the first audio signal, thereby enhancing an audibility of a voice of the first person for a first duration and reducing an audibility of a voice of the second person for the first duration (para 0037-0038, There are many uses for such an enhancement to an active noise control and customized audio system. This may be used to enhance perception of an individual speaker in a lecture environment, for example, a university professor in a lecture hall. The system may also be used by friends in a noisy environment such as in a school hallway, a bar/club or at a concert. This could eliminate the need for yelling to be heard or straining to hear a friend. At the same time the user can keep the headphone on the user's ears and continue to listen to source and/or ambient audio at a normal or customized level. A user may select which sounds are to be heard from both the ambient environment and the source signal, and to apply a variety of adjustments/mixing controls to that combined sound environment to ensure the appropriate blending of the sounds, such adjustments to include, but are not limited to, relative volume, timing delays, distance compensation between microphones or both microphones and source signals and a wide variety of other adjustments);
extracting from the second noisy audio signal a second audio signal associated with the second person, said extracting comprises applying speech separation (Benattar, para 0098-101, The audio processor may have an auto mode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry. The audio divider may be connected to an audio controller implemented by the DSP which may dictate the manner in which the divided audio input signals are handled. The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other; [i.e., “people (more than one person) are talking within hearing (“earshot”) range” as “second audio signal associated with the second person”]);
outputting to the user, via the at least one hearable device, a second output signal comprising at least the second audio signal, thereby enhancing the audibility of the voice of the second person for a second duration, the second duration is subsequent to the first duration (Benattar, para 0097-0101, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array…Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401, to either enhance or attenuate audio corresponding to the characteristics of the sample…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other’ [i.e., ““people (more than one person) are talking within hearing (“earshot”) range” as “…the audibility of the voice of the second person…” ]).
Benattar does not specifically disclose the user having a mobile device used for obtaining user input, obtaining a second activation selection of the second person, and capturing a second noisy audio signal from the environment of the user.
However, Glaser, in the same field of endeavor discloses:
the user having a mobile device used for obtaining user input (Glaser, para 0034, In another implementation, the set of microphones includes a distributed set of personal computing devices capable of recording audio. The distributed recording device variation can preferably enable ad-hoc implementation of the system in different environments. In one exemplary implementation, multiple smart phones of different users could act collectively as a microphone array as shown in FIG. 3…the microphones integrated into hearing aids and worn in or on a user's ears may serve to collect audio input; ALTERNATIVE: Glaser, para 0021, The audio control inputs can specify a location/region and types of audio modifications such as how to adjust the volume and/or equalize the audio originating from that location. The set of positional audio control inputs are then used to generate/augment an audio signal that drives a personal audio system. For example, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her).
obtaining a second activation selection of the second person (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech);
capturing a second noisy audio signal from the environment of the user (Glaser, para 0041, An audio source input preferably includes a location property and one or more audio processing operator property. The location property can be a distinct point, but may alternatively be a region. In some variations, it may additionally be mapped to a particular source in addition to location. For example, some implementations may use CV-based monitoring system to determine that a machine and a person are in the same general location, and two audio control inputs may be assigned to that same general location but individually assigned to audio generated by the machine (e.g., noise) and audio generated by the person (e.g., speech)); and
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Glaser in the method of Benattar because this would enable a listener to combine and customize one or more sources of sound, both ambient and otherwise, to personalize and enhance a listening experience (Glaser, para 0022).
Regarding Claim 2, Benattar in view of Glaser discloses the method of claim 1.
Furthermore, Glaser discloses:
wherein at least one of the first and second activation selections is performed by the user via the mobile device (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech)
Regarding Claim 3, Benattar in view of Glaser discloses the method of Claim 1, wherein at least one of the first and second activation selections is performed automatically based on an estimated intention of the user (Benattar, para 0104-0105, The request authorization operation initiates an authorization request to another subscriber. Once that subscriber receives the request, it may be accepted, rejected, or ignored. According to an embodiment, once the request is accepted, the subscriber record of the accepting subscriber is updated to reflect permission granted to the request of the subscriber for use of the audio profile. The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber; [ “the request of the subscriber for use of the audio profile” as “based on an … intention of the user”]).
Regarding Claim 4, Benattar in view of Glaser discloses the method of Claim 1, wherein the second output signal comprises the second audio signal and an audio signal associated with the first person, thereby enhancing the audibility of the voices of the first and second persons for the second duration (Benattar, para 0033-0036, The user can select one or more profiles from a library for enhancement of the perception of audio. The system may operate by caching profiles and allowing users to download selected profiles. This can be done by having a repository of sound profiles organized by participants in the system. When a user wants to enhance perception of audio matching another participant's voice, the other participant's voice profile can be obtained from the repository and associated with the requesting user…The voice profile library and/or the active voice profiles may be saved locally on a user device. Audio processing and profile storage may be on a user client device or a server device depending on computational and communication resources available; [“The user can select one or more profiles from a library for enhancement of the perception of audio… the active voice profiles…on a user device. Audio processing and profile storage may be on a user client device or a server device” as “enhancing the audibility of the voices of the first and second persons for the second duration”]).
Regarding Claim 5, Benattar in view of Glaser discloses the method of Claim 1.
Furthermore, Glaser discloses:
wherein, in response to a muting selection of the first person, the second output signal is generated to not include audio associated with the first person (Glaser, para 0054, the significance of the content may determine whether that audio source is amplified or deamplified/muted. In another example, the content of multiple speakers may be analyzed to determine other content that is part of a shared conversation so that a listener can follow a conversation while reducing the interference of a competing conversation; [i.e., “audio source is … deamplified/muted” as “a muting selection of the first person”; “a listener can follow a conversation while reducing the interference of a competing conversation” as “the second output signal is generated to not include audio associated with the first person”] ).
Regarding Claim 6, Benattar in view of Glaser discloses the method of Claim 1,
Furthermore, Glaser discloses:
wherein the second activation selection is performed via a map view displayed on the mobile device, the map view indicating locations of at least a portion of the plurality of people relative to a location of the user (Glaser, para 0036, The spatial audio analyzer 120 functions generate an audio map from multiple audio inputs. The spatial audio analyzer 120 preferably uses one or more forms of acoustic source localization. The spatial audio analyzer 120 preferably includes configuration to analyze the relative delay in arrival time of audio signals detected from the collected audio inputs, and uses such time delay phase shift detection to produce a map of audio sources in space).
Regarding Claim 9, Benattar discloses a computer program product comprising a non-transitory computer readable storage medium retaining program instructions, which program instructions when read by a processor, cause the processor to perform a method in an environment of a user (Benattar, para 0060, The system may include an article of manufacture, a method, a system, and an apparatus for an audio customization system. The article of manufacture of the invention may include a computer-readable medium comprising software for a system for generating an audio signature or audio fingerprints. The invention may be embodied in hardware and/or software and may be implemented in one or more of a general purpose computer), the environment comprising a plurality of people participating in at least one conversation, the plurality of people comprise at least a first person and a second person (para 0031, a listener may be able to hear important information or hold a conversation with another person), the user having at least one hearable device used for providing audio output to the user (Benattar, para 0099, The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone), the method comprising:
obtaining a first activation selection of the first person (Benattar, para 0105, The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber);
capturing a first noisy audio signal from the environment (Benattar, para 0097, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array. The adaptive audio controller may also be connected to a set of active audio profiles 403. The active audio profiles 403 may be selected from profiles stored in the sound library 404. The sound library 404 may contain audio profiles created by sampling audio information detected by the ambient microphone. If a user wants to establish a profile for certain characteristic audio, the audio may be sampled and characterized in order to create a profile. The sample audio may be used to create an audio profile such as a specific voice, machinery, or other noise. Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401);
extracting from the first noisy audio signal a first audio signal associated with the first person, said extracting comprises applying speech separation (Benattar, para 0045, multi-channel digital signal processing to divide ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech);
outputting to the user via the at least one hearable device (Benattar, para 0082, One or more active profiles 103 may be used by the audio customization engine 101 to customize audio signals provided to an audio output device 104, for example, headphones), a first output signal comprising at least the first audio signal, thereby enhancing an audibility of a voice of the first person for a first duration and reducing an audibility of a voice of the second person for the first duration (para 0037-0038, There are many uses for such an enhancement to an active noise control and customized audio system. This may be used to enhance perception of an individual speaker in a lecture environment, for example, a university professor in a lecture hall. The system may also be used by friends in a noisy environment such as in a school hallway, a bar/club or at a concert. This could eliminate the need for yelling to be heard or straining to hear a friend. At the same time the user can keep the headphone on the user's ears and continue to listen to source and/or ambient audio at a normal or customized level. A user may select which sounds are to be heard from both the ambient environment and the source signal, and to apply a variety of adjustments/mixing controls to that combined sound environment to ensure the appropriate blending of the sounds, such adjustments to include, but are not limited to, relative volume, timing delays, distance compensation between microphones or both microphones and source signals and a wide variety of other adjustments);
extracting from the second noisy audio signal a second audio signal associated with the second person, said extracting comprises applying speech separation (Benattar, para 0098-101, The audio processor may have an auto mode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry. The audio divider may be connected to an audio controller implemented by the DSP which may dictate the manner in which the divided audio input signals are handled. The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other; [i.e., “people (more than one person) are talking within hearing (“earshot”) range” as “second audio signal associated with the second person”]);
outputting to the user, via the at least one hearable device, a second output signal comprising at least the second audio signal, thereby enhancing the audibility of the voice of the second person for a second duration, the second duration is subsequent to the first duration (Benattar, para 0097-0101, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array…Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401, to either enhance or attenuate audio corresponding to the characteristics of the sample…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other’ [i.e., ““people (more than one person) are talking within hearing (“earshot”) range” as “…the audibility of the voice of the second person…” ]). Benattar does not specifically disclose the user having a mobile device used for obtaining user input, obtaining a second activation selection of the second person, and capturing a second noisy audio signal from the environment of the user.
However, Glaser, in the same field of endeavor discloses:
the user having a mobile device used for obtaining user input (Glaser, para 0034, In another implementation, the set of microphones includes a distributed set of personal computing devices capable of recording audio. The distributed recording device variation can preferably enable ad-hoc implementation of the system in different environments. In one exemplary implementation, multiple smart phones of different users could act collectively as a microphone array as shown in FIG. 3…the microphones integrated into hearing aids and worn in or on a user's ears may serve to collect audio input; ALTERNATIVE: Glaser, para 0021, The audio control inputs can specify a location/region and types of audio modifications such as how to adjust the volume and/or equalize the audio originating from that location. The set of positional audio control inputs are then used to generate/augment an audio signal that drives a personal audio system. For example, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her).
obtaining a second activation selection of the second person (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech);
capturing a second noisy audio signal from the environment of the user (Glaser, para 0041, An audio source input preferably includes a location property and one or more audio processing operator property. The location property can be a distinct point, but may alternatively be a region. In some variations, it may additionally be mapped to a particular source in addition to location. For example, some implementations may use CV-based monitoring system to determine that a machine and a person are in the same general location, and two audio control inputs may be assigned to that same general location but individually assigned to audio generated by the machine (e.g., noise) and audio generated by the person (e.g., speech));
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Glaser in the method of Benattar because this would enable a listener to combine and customize one or more sources of sound, both ambient and otherwise, to personalize and enhance a listening experience (Glaser, para 0022).
Regarding Claim 10, Benattar in view of Glaser discloses the computer program product of Claim 9.
Furthermore, Glaser discloses:
wherein at least one of the first and second activation selections is performed by the user via the mobile device (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech).
Regarding Claim 11, Benattar in view of Glaser discloses the computer program product of Claim 9, wherein at least one of the first and second activation selections is performed automatically based on an estimated intention of the user (Benattar, para 0104-0105, The request authorization operation initiates an authorization request to another subscriber. Once that subscriber receives the request, it may be accepted, rejected, or ignored. According to an embodiment, once the request is accepted, the subscriber record of the accepting subscriber is updated to reflect permission granted to the request of the subscriber for use of the audio profile. The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber; [ “the request of the subscriber for use of the audio profile” as “based on an … intention of the user”]).
Regarding Claim 12, Benattar in view of Glaser discloses the computer program product of Claim 9, wherein the second output signal comprises the second audio signal and an audio signal associated with the first person, thereby enhancing the audibility of the voices of the first and second persons for the second duration (Benattar, para 0033-0036, The user can select one or more profiles from a library for enhancement of the perception of audio. The system may operate by caching profiles and allowing users to download selected profiles. This can be done by having a repository of sound profiles organized by participants in the system. When a user wants to enhance perception of audio matching another participant's voice, the other participant's voice profile can be obtained from the repository and associated with the requesting user…The voice profile library and/or the active voice profiles may be saved locally on a user device. Audio processing and profile storage may be on a user client device or a server device depending on computational and communication resources available; [“The user can select one or more profiles from a library for enhancement of the perception of audio… the active voice profiles…on a user device. Audio processing and profile storage may be on a user client device or a server device” as “enhancing the audibility of the voices of the first and second persons for the second duration”]).
Regarding Claim 13, Benattar in view of Glaser discloses the computer program product of Claim 9.
Furthermore, Glaser discloses:
wherein, in response to a muting selection of the first person, the second output signal is generated to not include audio associated with the first person (Glaser, para 0054, the significance of the content may determine whether that audio source is amplified or deamplified/muted. In another example, the content of multiple speakers may be analyzed to determine other content that is part of a shared conversation so that a listener can follow a conversation while reducing the interference of a competing conversation; [i.e., “audio source is … deamplified/muted” as “a muting selection of the first person”; “a listener can follow a conversation while reducing the interference of a competing conversation” as “the second output signal is generated to not include audio associated with the first person”] ).
Regarding Claim 14, Benattar in view of Glaser discloses the computer program product of Claim 9.
Furthermore, Glaser discloses:
wherein the second activation selection is performed via a map view displayed on the mobile device, the map view indicating locations of at least a portion of the plurality of people relative to a location of the user (Glaser, para 0036, The spatial audio analyzer 120 functions generate an audio map from multiple audio inputs. The spatial audio analyzer 120 preferably uses one or more forms of acoustic source localization. The spatial audio analyzer 120 preferably includes configuration to analyze the relative delay in arrival time of audio signals detected from the collected audio inputs, and uses such time delay phase shift detection to produce a map of audio sources in space).
Regarding Claim 17, An apparatus comprising a processor and coupled memory, the processor being adapted to perform a method in an environment of a user (Benattar, para 0060, The system may include an article of manufacture, a method, a system, and an apparatus for an audio customization system. The article of manufacture of the invention may include a computer-readable medium comprising software for a system for generating an audio signature or audio fingerprints. The invention may be embodied in hardware and/or software and may be implemented in one or more of a general purpose computer; [a general purpose computer would have a processor(s) and coupled memory]), the environment comprising a plurality of people participating in at least one conversation, the plurality of people comprise at least a first person and a second person (para 0031, a listener may be able to hear important information or hold a conversation with another person), t the user having at least one hearable device used for providing audio output to the user (Benattar, para 0099, The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone), the method comprising:
obtaining a first activation selection of the first person (Benattar, para 0105, The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber);
capturing a first noisy audio signal from the environment (Benattar, para 0097, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array. The adaptive audio controller may also be connected to a set of active audio profiles 403. The active audio profiles 403 may be selected from profiles stored in the sound library 404. The sound library 404 may contain audio profiles created by sampling audio information detected by the ambient microphone. If a user wants to establish a profile for certain characteristic audio, the audio may be sampled and characterized in order to create a profile. The sample audio may be used to create an audio profile such as a specific voice, machinery, or other noise. Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401);
extracting from the first noisy audio signal a first audio signal associated with the first person, said extracting comprises applying speech separation (Benattar, para 0045, multi-channel digital signal processing to divide ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech);
outputting to the user via the at least one hearable device (Benattar, para 0082, One or more active profiles 103 may be used by the audio customization engine 101 to customize audio signals provided to an audio output device 104, for example, headphones), a first output signal comprising at least the first audio signal, thereby enhancing an audibility of a voice of the first person for a first duration and reducing an audibility of a voice of the second person for the first duration (para 0037-0038, There are many uses for such an enhancement to an active noise control and customized audio system. This may be used to enhance perception of an individual speaker in a lecture environment, for example, a university professor in a lecture hall. The system may also be used by friends in a noisy environment such as in a school hallway, a bar/club or at a concert. This could eliminate the need for yelling to be heard or straining to hear a friend. At the same time the user can keep the headphone on the user's ears and continue to listen to source and/or ambient audio at a normal or customized level. A user may select which sounds are to be heard from both the ambient environment and the source signal, and to apply a variety of adjustments/mixing controls to that combined sound environment to ensure the appropriate blending of the sounds, such adjustments to include, but are not limited to, relative volume, timing delays, distance compensation between microphones or both microphones and source signals and a wide variety of other adjustments);
extracting from the second noisy audio signal a second audio signal associated with the second person, said extracting comprises applying speech separation (Benattar, para 0098-101, The audio processor may have an auto mode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry. The audio divider may be connected to an audio controller implemented by the DSP which may dictate the manner in which the divided audio input signals are handled. The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other; [i.e., “people (more than one person) are talking within hearing (“earshot”) range” as “second audio signal associated with the second person”]);
outputting to the user, via the at least one hearable device, a second output signal comprising at least the second audio signal, thereby enhancing the audibility of the voice of the second person for a second duration, the second duration is subsequent to the first duration (Benattar, para 0097-0101, FIG. 4 shows a system layout. An adaptive audio controller 401 may be provided. The adaptive audio controller 401 may be connected to an audio source(s) 402 which may be one or more microphones or other audio sources including an ambient microphone array…Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 401, to either enhance or attenuate audio corresponding to the characteristics of the sample…The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other’ [i.e., ““people (more than one person) are talking within hearing (“earshot”) range” as “…the audibility of the voice of the second person…” ]).
Benattar does not specifically disclose the user having a mobile device used for obtaining user input, obtaining a second activation selection of the second person, and capturing a second noisy audio signal from the environment of the user.
However, Glaser, in the same field of endeavor discloses:
the user having a mobile device used for obtaining user input (Glaser, para 0034, In another implementation, the set of microphones includes a distributed set of personal computing devices capable of recording audio. The distributed recording device variation can preferably enable ad-hoc implementation of the system in different environments. In one exemplary implementation, multiple smart phones of different users could act collectively as a microphone array as shown in FIG. 3…the microphones integrated into hearing aids and worn in or on a user's ears may serve to collect audio input; ALTERNATIVE: Glaser, para 0021, The audio control inputs can specify a location/region and types of audio modifications such as how to adjust the volume and/or equalize the audio originating from that location. The set of positional audio control inputs are then used to generate/augment an audio signal that drives a personal audio system. For example, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her).
obtaining a second activation selection of the second person (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech);
capturing a second noisy audio signal from the environment of the user (Glaser, para 0041, An audio source input preferably includes a location property and one or more audio processing operator property. The location property can be a distinct point, but may alternatively be a region. In some variations, it may additionally be mapped to a particular source in addition to location. For example, some implementations may use CV-based monitoring system to determine that a machine and a person are in the same general location, and two audio control inputs may be assigned to that same general location but individually assigned to audio generated by the machine (e.g., noise) and audio generated by the person (e.g., speech)); and
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Glaser in the method of Benattar because this would enable a listener to combine and customize one or more sources of sound, both ambient and otherwise, to personalize and enhance a listening experience (Glaser, para 0022).
Regarding Claim 18, Benattar in view of Glaser discloses the apparatus of Claim 17.,
wherein at least one of the first and second activation selections is performed by the user via the mobile device (Glaser, para 0021, a user may use his smart phone to selectively mark different locations to amplify her friends sitting around her while reducing the volume of the background music and others sitting nearby, but who are not of interest. As another example, a user may choose to reduce the low frequency signals from a human speaker or selectively amplify just the frequencies used in human speech).
Regarding Claim 19, Benattar in view of Glaser discloses the apparatus of Claim 17, wherein at least one of the first and second activation selections is performed automatically based on an estimated intention of the user (Benattar, para 0104-0105, The request authorization operation initiates an authorization request to another subscriber. Once that subscriber receives the request, it may be accepted, rejected, or ignored. According to an embodiment, once the request is accepted, the subscriber record of the accepting subscriber is updated to reflect permission granted to the request of the subscriber for use of the audio profile. The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber; [ “the request of the subscriber for use of the audio profile” as “based on an … intention of the user”]).
Regarding Claim 20, Benattar in view of Glaser discloses the apparatus of Claim 17, wherein the second output signal comprises the second audio signal and an audio signal associated with the first person, thereby enhancing the audibility of the voices of the first and second persons for the second duration (Benattar, para 0033-0036, The user can select one or more profiles from a library for enhancement of the perception of audio. The system may operate by caching profiles and allowing users to download selected profiles. This can be done by having a repository of sound profiles organized by participants in the system. When a user wants to enhance perception of audio matching another participant's voice, the other participant's voice profile can be obtained from the repository and associated with the requesting user…The voice profile library and/or the active voice profiles may be saved locally on a user device. Audio processing and profile storage may be on a user client device or a server device depending on computational and communication resources available; [“The user can select one or more profiles from a library for enhancement of the perception of audio… the active voice profiles…on a user device. Audio processing and profile storage may be on a user client device or a server device” as “enhancing the audibility of the voices of the first and second persons for the second duration”]).
Claims 7-8 and 15-16 are rejected under 35 U.S.C. under 35 U.S.C. 103 as being unpatentable over Benattar in view of Glaser, and further in view of Pardo et al. Pat No. US 9093056 B2 (Pardo).
Regarding Claim 7, Benattar in view of Glaser discloses the method of Claim 1, wherein said extracting from the first noisy audio signal (Benattar, para 0045, multi-channel digital signal processing to divide ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech) comprises:
converting the first noisy audio signal into a frequency domain, thereby obtaining a converted first noisy audio signal (Benattar, para 0032-0033, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them, but also effectively read existing spectrographs which may exist in a personal ambient sound library of a user, or which may otherwise reside in a database of available ambient sound spectrographs and decode such spectrographs to inform the digital signal processing and active filtration systems of those patterns which should be treated as desired ambient sounds and thus included in the customized listening environment of a user when they are present in the ambient environment. It is an object of the invention to provide a system for managing a sound library and audio profiles);
extracting the first audio signal from the converted first noisy audio signal using an acoustic fingerprint of the first person (Benattar, para 0060, a system for generating an audio signature or audio fingerprints; ALTERNATIVE: Benattar, para 0032, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them); and
Benattar in view of Glaser do not specifically disclose multiplying the first audio signal with a filtration mask having a value of zero or one, the value of the filtration mask is determined based on an estimation of whether or not the first audio signal comprises the voice of the first person, wherein the estimation is performed based on the converted first noisy audio signal.
However, Pardo, in the same field of endeavor, discloses multiplying the first audio signal with a filtration mask having a value of zero or one (Pardo, col 23, ln 40-47, the mask is a binary mask having values of zero or one at time-frequency bins of the mask. The value of zero represents the time-frequency bins that represent the non-repeating structure in the audio signal. The value of one represents the time-frequency bins that represent the first repeating structure in the audio signal. Extracting the first repeating structure can include multiplying the binary mask to a spectrogram of the audio signal), the value of the filtration mask is determined based on an estimation of whether or not the first audio signal comprises the voice of the first person, wherein the estimation is performed based on the converted first noisy audio signal Pardo, col 20, ln 35-58, the mask M may be obtained using Equation 5, described above. The time-frequency mask M is then symmetrized and applied to the STFT X of the mixture signal…The system 1700 includes an input 1702, such as a device or module that receives input data, such as an audio recording including two or more sounds that may be separated from each other into separate components or audio signals, such as … voice signals) ).
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Pardo in the method of Benattar in view of Glaser because this introduces systems and methods that are previously trained to separate components of an audio recording, such as by learning statistical models of sound sources (e.g., a model of a person's voice) from a training database (Pardo, col 2, ln 39-42).
Regarding Claim 8, Benattar in view of Glaser and Padro discloses the method of Claim 7, wherein the estimation is performed based on at least one of: an acoustic signature of the first person (Benattar, para 0032, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them ).
Furthermore, Glaser discloses:
a direction of arrival of the voice of the first person (Glaser, para 0032, The microphones 110 can be omnidirectional microphones but may alternatively be directional microphones. Orientation of directional microphones may be used to facilitate locating audio source position; [“locating audio source position” as “direction of … the voice of the first person”]).
Regarding Claim 15, Benattar in view of Glaser discloses the computer program product of Claim 9, wherein said extracting from the first noisy audio signal (Benattar, para 0045, multi-channel digital signal processing to divide ambient sound environment into multiple channels based on frequency ranges, directionality, or audio characteristics, including but not limited to modulation rates that correspond to a wide variety of ambient sounds, including speech) comprises:
extracting the first audio signal from the converted first noisy audio signal using an acoustic fingerprint of the first person (Benattar, para 0060, a system for generating an audio signature or audio fingerprints; ALTERNATIVE: Benattar, para 0032, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them); and
converting the first noisy audio signal into a frequency domain, thereby obtaining a converted first noisy audio signal (Benattar, para 0032-0033, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them, but also effectively read existing spectrographs which may exist in a personal ambient sound library of a user, or which may otherwise reside in a database of available ambient sound spectrographs and decode such spectrographs to inform the digital signal processing and active filtration systems of those patterns which should be treated as desired ambient sounds and thus included in the customized listening environment of a user when they are present in the ambient environment. It is an object of the invention to provide a system for managing a sound library and audio profiles);
Benattar in view of Glaser do not specifically disclose multiplying the first audio signal with a filtration mask having a value of zero or one, the value of the filtration mask is determined based on an estimation of whether or not the first audio signal comprises the voice of the first person, wherein the estimation is performed based on the converted first noisy audio signal.
However, Pardo, in the same field of endeavor, discloses multiplying the first audio signal with a filtration mask having a value of zero or one (Pardo, col 23, ln 40-47, the mask is a binary mask having values of zero or one at time-frequency bins of the mask. The value of zero represents the time-frequency bins that represent the non-repeating structure in the audio signal. The value of one represents the time-frequency bins that represent the first repeating structure in the audio signal. Extracting the first repeating structure can include multiplying the binary mask to a spectrogram of the audio signal), the value of the filtration mask is determined based on an estimation of whether or not the first audio signal comprises the voice of the first person, wherein the estimation is performed based on the converted first noisy audio signal (Pardo, col 20, ln 35-58, the mask M may be obtained using Equation 5, described above. The time-frequency mask M is then symmetrized and applied to the STFT X of the mixture signal…The system 1700 includes an input 1702, such as a device or module that receives input data, such as an audio recording including two or more sounds that may be separated from each other into separate components or audio signals, such as … voice signals)
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Pardo in the method of Benattar in view of Glaser because this this introduces systems and methods that are previously trained to separate components of an audio recording, such as by learning statistical models of sound sources (e.g., a model of a person's voice) from a training database (Pardo, col 2, ln 39-42).
Regarding Claim 16, Benattar in view of Glaser discloses the computer program product of Claim 15, wherein the estimation is performed based on at least one of: and an acoustic signature of the first person (Benattar, para 0032, Sound spectrographing technology, acoustic fingerprinting, and other audio processing technologies may be used to recognize sounds with unique characteristics which distinguish them in minute ways from other, even very similar sounds, can both record the frequency and time patterns of sounds to identify and classify them ).
Furthermore, Glaser discloses:
a direction of arrival of the voice of the first person (Glaser, para 0032, The microphones 110 can be omnidirectional microphones but may alternatively be directional microphones. Orientation of directional microphones may be used to facilitate locating audio source position; [“locating audio source position” as “direction of … the voice of the first person”]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MULUGETA T. DUGDA whose telephone number is (703)756-1106. The examiner can normally be reached Mon - Fri, 4:30am - 7:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D. Shah can be reached at 571-270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MULUGETA TUJI DUGDA/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
05/30/2026
Prosecution Insights