DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 10-12, 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Popovich U.S. PAP 2022/036895 A1 in view of Bou U.S. PAP 2023/0274753.
Regarding claim 1 Popovich teaches a method for voice activity detection (VAD) (audio microphone system simultaneously provides in-room amplification and remote communication during a collaboration session with one or more local participants in a shared space and one or more remote participants, see abstract) comprising:
defining, by a targeting processor, a plurality of virtual microphones in a shared 3D space, wherein each virtual microphone is assigned a location coordinate in the shared 3D space (one or more arrays of microphones and one or more speakers each located in the shared space, at least one microphone system for amplification, an audio processor connected to the microphone system for amplification, the one or more arrays of microphones and the one or more speakers, and a computer connected to the audio processor, see par. [0015]; “virtual microphone” in this specification may include, but is not limited to, a microphone system focus point in 2D (x, y) and/or 3D (x, y, z) space, see par. [0042]);
determining, for each virtual microphone, a processed acoustic energy level continuously in real-time based on a microphone signal from a microphone array in the shared 3D space (The signal components 1107 from the microphone's element processor 1101 are summed at node 1110 to provide the combined microphone array 703 signal for each of the, e.g., 8192, virtual microphone bubbles. Each bubble signal is preferably converted into a power signal at node 1111 by squaring the signal samples, see par. [0100]);
selecting at least one virtual microphone from the plurality of virtual microphones having a highest processed acoustic energy level as a targeted virtual microphone to focus the microphone array (target processor/bubble processor system 912 preferably simultaneously focuses the microphone array 703 on the, e.g., 8192, virtual microphone points 602 in 3D space, see par. [0102]);
determining, for the targeted virtual microphone by a VAD engine, whether an acoustic type associated with the highest processed acoustic energy level is a speech sound or a non-speech sound (To prevent this accidental cancellation, a threshold or voice activity detection or a combination of the two is applied such that the signal is only passed to the combining processor A S1003 and to the combining processor B S1014 if there is adequate signal from someone speaking directly into the additional microphone for amplification 914, see par. [0086]).
and transmitting the VAD flag and the location coordinate of the targeted virtual microphone to at least one external process (Combining processor B S1014 combines the signals from the additional microphone for amplification 914 path and the signals from room microphone array 703 path so that they can be sent to the computer or appliance S1015 and or to the combining block S1008. At the combining step a positive or a negative gain may be applied to one or both signals to achieve a desired level for both signals, see par. [0093]).
However Popovich does not teach setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound.
In the same field of endeavor Bou teaches Methods, systems, and computer-readable media are provided for detecting voice activity, see abstract. Bou teaches echo and noise cancelation, reduction, and/or suppression subsystems may exhibit better performance when certain functions, e.g., adaptive functions, are frozen or paused during periods that the occupant is actively speaking. Accordingly, in various applications it may be desirable to accurately detect when a user is actively speaking, see par. [0002]. Bou teaches setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound (VAD output is a binary flag, see par. [0029].
It would have been obvious to one of ordinary skill in the art to combine the Popovich invention with the teachings of Bou for the benefit of accurately detecting when a speaker is actively speaking, see par. [0002].
Regarding claim 2 Popovich teaches The method according to claim 1, wherein the location coordinate of the targeted virtual microphone is set as a current position of a talker within the shared 3D space (he exact coordinates 603 of the sound source 804 can be measured in an (X, Y, Z) coordinate grid 603. This allows for precise location determination to a high degree of accuracy, which is limited by virtual microphone bubble 602 size. See par, [0060]).
Regarding claim 3 Popovich teaches the method according to claim 1, wherein the location coordinate of each virtual microphone is a distinct point in the shared 3D space comprising an X-coordinate, a Y-coordinate, and a Z-coordinate (he virtual microphone bubbles 602 which are arranged in a 3D grid with (X, Y, Z) vectors 603 to an individual (X, Y, Z) location 604 in the room 101, see par. [0060]).
Regarding claim 10 Bou teaches the method according to claim 1, further comprising: continuously monitoring the targeted virtual microphone for speech or non-speech sound; and setting the VAD flag to negative if non-speech sound is detected at the targeted virtual microphone (binary flag detector, see par. [0029]).
Regarding claim 11 Bou teaches the method according to claim 1, further comprising: selecting a second virtual microphone from the plurality of virtual microphones having a second highest processed acoustic energy level as a second targeted virtual microphone(second directional microphone, see par. [0032]);
determining, for the second targeted virtual microphone, whether the acoustic type is a speech sound or a non-speech sound by the VAD engine (directional microphone may be aimed away from the user and may be expected to pick up surrounding acoustic signals in the environment and pick up very little user speech, see par. [0032]);
setting a second VAD flag for the second targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound (In various examples, a double-talk detector may compare an energy in the signal from the first directional microphone to an energy in the signal from the second directional microphone. For instance, a double-talk detector may take a ratio of signal energies from the two microphones and if the ratio exceeds a threshold such may indicate that the user is talking, see par. [0032]);
and transmitting the second VAD flag and the location coordinate of the second targeted virtual microphone to the at least one external process ( first combination of microphone signals may form an array having an increased acoustic response in the direction of the user (or the expected location of the user's mouth) and the double-talk detector may compare a signal (or a signal energy) of a portion (e.g., frequency limited, time limited, or both) of the first combination to that of a second combination of the microphone signals that form an array having a reduced acoustic response in the direction of the user (or the user's mouth), see par. [0034]).
Regarding claim 12 Popovich teaches the method according to claim 1, wherein the microphone array comprises at least two microphones positioned in the 3D shared space (he system includes one or more arrays of microphones and one or more speakers each located in the shared space, see par. [0015]).
Regarding claim 16 Popovich teaches a non-transitory computer readable medium having stored thereon a program for a computer (non-transitory computer readable media, see par. [0048]), the program executing steps of:
defining, by a targeting processor, a plurality of virtual microphones in a shared 3D space, wherein each virtual microphone is assigned a location coordinate in the shared 3D space (one or more arrays of microphones and one or more speakers each located in the shared space, at least one microphone system for amplification, an audio processor connected to the microphone system for amplification, the one or more arrays of microphones and the one or more speakers, and a computer connected to the audio processor, see par. [0015]; “virtual microphone” in this specification may include, but is not limited to, a microphone system focus point in 2D (x, y) and/or 3D (x, y, z) space, see par. [0042]);
determining, for each virtual microphone, a processed acoustic energy level continuously in real-time based on a microphone signal from a microphone array in the shared 3D space (The signal components 1107 from the microphone's element processor 1101 are summed at node 1110 to provide the combined microphone array 703 signal for each of the, e.g., 8192, virtual microphone bubbles. Each bubble signal is preferably converted into a power signal at node 1111 by squaring the signal samples, see par. [0100]);
selecting at least one virtual microphone from the plurality of virtual microphones having a highest processed acoustic energy level as a targeted virtual microphone to focus the microphone array (target processor/bubble processor system 912 preferably simultaneously focuses the microphone array 703 on the, e.g., 8192, virtual microphone points 602 in 3D space, see par. [0102]);
determining, for the targeted virtual microphone by a VAD engine, whether an acoustic type associated with the highest processed acoustic energy level is a speech sound or a non-speech sound (To prevent this accidental cancellation, a threshold or voice activity detection or a combination of the two is applied such that the signal is only passed to the combining processor A S1003 and to the combining processor B S1014 if there is adequate signal from someone speaking directly into the additional microphone for amplification 914, see par. [0086]).
and transmitting the VAD flag and the location coordinate of the targeted virtual microphone to at least one external process (Combining processor B S1014 combines the signals from the additional microphone for amplification 914 path and the signals from room microphone array 703 path so that they can be sent to the computer or appliance S1015 and or to the combining block S1008. At the combining step a positive or a negative gain may be applied to one or both signals to achieve a desired level for both signals, see par. [0093]).
However Popovich does not teach setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound.
In the same field of endeavor Bou teaches Methods, systems, and computer-readable media are provided for detecting voice activity, see abstract. Bou teaches echo and noise cancelation, reduction, and/or suppression subsystems may exhibit better performance when certain functions, e.g., adaptive functions, are frozen or paused during periods that the occupant is actively speaking. Accordingly, in various applications it may be desirable to accurately detect when a user is actively speaking, see par. [0002]. Bou teaches setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound (VAD output is a binary flag, see par. [0029].
It would have been obvious to one of ordinary skill in the art to combine the Popovich invention with the teachings of Bou for the benefit of accurately detecting when a speaker is actively speaking, see par. [0002].
Regarding claim 17 Popovich teaches an apparatus for performing voice activity detection (VAD) comprising:
a targeting processor defining a plurality of virtual microphones in a shared 3D space, wherein each virtual microphone is assigned a location coordinate in the shared 3D space (one or more arrays of microphones and one or more speakers each located in the shared space, at least one microphone system for amplification, an audio processor connected to the microphone system for amplification, the one or more arrays of microphones and the one or more speakers, and a computer connected to the audio processor, see par. [0015]; “virtual microphone” in this specification may include, but is not limited to, a microphone system focus point in 2D (x, y) and/or 3D (x, y, z) space, see par. [0042]);
wherein the targeting processor: determines, for each virtual microphone, a processed acoustic energy level continuously in real-time based on a microphone signal from a microphone array in the shared 3D space (The signal components 1107 from the microphone's element processor 1101 are summed at node 1110 to provide the combined microphone array 703 signal for each of the, e.g., 8192, virtual microphone bubbles. Each bubble signal is preferably converted into a power signal at node 1111 by squaring the signal samples, see par. [0100]);
and selects at least one virtual microphone from the plurality of virtual microphones having a highest processed acoustic energy level as a targeted virtual microphone to focus a microphone array (target processor/bubble processor system 912 preferably simultaneously focuses the microphone array 703 on the, e.g., 8192, virtual microphone points 602 in 3D space, see par. [0102]);
and a VAD engine, wherein the VAD engine: determines an acoustic type associated with the highest processed acoustic energy level for the targeted virtual microphone (To prevent this accidental cancellation, a threshold or voice activity detection or a combination of the two is applied such that the signal is only passed to the combining processor A S1003 and to the combining processor B S1014 if there is adequate signal from someone speaking directly into the additional microphone for amplification 914, see par. [0086]).
and transmitting the VAD flag and the location coordinate of the targeted virtual microphone to a camera director system, wherein the camera director system controls a video stream of at least one video camera in the shared 3D space based on the VAD flag and the location coordinate of the targeted virtual microphone (Combining processor B S1014 combines the signals from the additional microphone for amplification 914 path and the signals from room microphone array 703 path so that they can be sent to the computer or appliance S1015 and or to the combining block S1008. At the combining step a positive or a negative gain may be applied to one or both signals to achieve a desired level for both signals, see par. [0093]).
However Popovich does not teach setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound.
In the same field of endeavor Bou teaches Methods, systems, and computer-readable media are provided for detecting voice activity, see abstract. Bou teaches echo and noise cancelation, reduction, and/or suppression subsystems may exhibit better performance when certain functions, e.g., adaptive functions, are frozen or paused during periods that the occupant is actively speaking. Accordingly, in various applications it may be desirable to accurately detect when a user is actively speaking, see par. [0002]. Bou teaches setting a VAD flag for the targeted virtual microphone negative if the acoustic type is non-speech sound and positive if the acoustic type is speech sound (VAD output is a binary flag, see par. [0029].
It would have been obvious to one of ordinary skill in the art to combine the Popovich invention with the teachings of Bou for the benefit of accurately detecting when a speaker is actively speaking, see par. [0002].
Claim(s) 4-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Popovich U.S. PAP 2022/036895 A1 in view of Bou U.S. PAP 2023/0274753 further in view of Wexler U.S. PAP 2021/0390957 A1.
Regarding claim 4 Popovich does not teach the method according to claim 1, wherein the VAD engine is a neural engine trained using labeled speech sounds, labeled non-speech sounds, and labeled silence-background noise.
In the same field of endeavor Wexler teaches to separate the speaker's voice from additional speakers or background noise in a noisy audio, a second pre-trained neural network may receive the noisy audio and the speaker's signature and output an audio (which may also be represented as attributes) of the voice of the speaker as extracted from the noisy audio, separated from the other speech or background noise. It will be appreciated that the same or additional neural networks may be used to separate the voices of multiple speakers. Alternatively, a neural network may receive voice signatures of two or more speakers and output the voice of each of the speakers separately. Accordingly, the system may generate two or more different audio outputs, each comprising the speech of the respective speaker. In some embodiments, if separation is impossible, the input voice may only be cleaned from background noise, see par. [0182].
It would be obvious to one of ordinary skill in the art to combine the Popovich in view of Bou with the teachings of Wexler for the benefit of separating a speakers voice from background noise in noisy audio, see par. [0182].
Regarding claim 5 Wexler teaches the method according to claim 1, wherein the at least one external process is a camera director system (FIG. 17A is a schematic illustration of an example of a user wearing an apparatus for a camera-based hearing aid device according to a disclosed embodiment, see par. [0038]).
Regarding claim 6 Wexler teaches the method according to claim 5, wherein the camera director system comprises: a plurality of video cameras in the shared 3D space, wherein each of the plurality of video cameras is associated with at least one predefined area within the shared 3D space, wherein the camera director ( Apparatus 110 may have one or more cameras 1730, which may correspond to image sensor 220. Camera 1730 may be configured to capture images of the surrounding environment of user 100, see par. [0144]) system: automatically selects a targeted video camera from the plurality of video cameras if the location coordinate of the targeted virtual microphone is within the predefined area of the targeted video camera (, camera 1730 may be configured to detect a representation of a chin of the user in the same images capturing the surrounding environment of the user, which may be used for other functions described in this disclosure, see par. [0144]);
and transmits a video stream from the selected targeted video camera to a unified communication client for broadcast (processor 210 may be configured to analyze images or videos containing representations of individual 2010 to determine when individual 2010 is speaking, for example, based on detected movement of the recognized individual's lips., see par. [0173]).
Regarding claim 7 Wexler teaches the method according to claim 6, wherein the at least one predefined area of each of the plurality of video cameras do not overlap in the shared 3D space ( processor 210 may perform further analysis based on captured images or videos to determine how to selectively condition audio signals associated with a recognized individual. In some embodiments, processor 210 may analyze the captured images to selectively condition audio associated with one individual relative to others., see par. [0177]).
Regarding claim 8 Wexler teaches the method according to claim 6, further comprising: adjusting a zoom level, a field of view, or orientation of the selected target video camera based on the location coordinate of the targeted virtual microphone (Processor 210 may recognize certain situations from the analyzed image data and adjust the aiming direction of image sensor 220 to capture relevant image data. For example, in one embodiment, processor 210 may detect an interaction with another individual and sense that the individual is not fully in view, because image sensor 220 is tilted down. Responsive thereto, processor 210 may adjust the aiming direction of image sensor 220 to capture image data of the individual., see par. [0090]).
Regarding claim 9 Wexler teaches the method according to claim 6, wherein the camera director system comprises: at least one video camera in the shared 3D space, wherein the at least one video camera is associated with at least one predefined area in the shared 3D space, wherein the camera director system transmits a video stream from the at least one video camera to a unified communication client for broadcast if the location coordinate of the targeted virtual microphone is within the at least one predefined area (Orientation adjustment unit 705 may be configured to permit the adjustment of image sensor 220. As illustrated in FIG. 7, orientation adjustment unit 705 may include an eye-ball type adjustment mechanism. In alternative embodiments, orientation adjustment unit 705 may include gimbals, adjustable stalks, pivotable mounts, and any other suitable unit for adjusting an orientation of image sensor 220, see par. [0110]; The electrical signals may be used to form an image or a video stream (i.e. image data) based on the detected signal, see par. [0066]).
Claim(s) 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Popovich U.S. PAP 2022/036895 A1 in view of Bou U.S. PAP 2023/0274753 further in view of Tran U.S. PAP 2026/0283481 A1.
Regarding claim 13 Povovich in view of Bou does not teach he method according to claim 1, wherein the microphone signal for VAD training or inference is divided into a plurality of overlapping frames, wherein a frame length of each overlapping frame is set to a value between 5 msec to 50 msec.
IN the same field of endeavor Tran teaches Acoustic event identification can be performed by a cascaded pipeline combining energy- and event-detection heuristics with one or more trained classifiers or neural networks. For example, voice activity detection (VAD) or a speech-non-speech classifier operating on the time-frequency representation segments speech regions, see par. [0329]. Features are computed over fixed-length, optionally overlapping windows, assigned precise timestamps, standardized by robust scaling, and consolidated into a unified feature vector per window for subsequent processing in S202, see par. [0259]. Segmentation into overlapping windows of a specified duration. For each window and channel, a power spectral density is estimated using a windowed periodogram, see par. [0298].
It would have been obvious to one of ordinary skill in the art to combine the Povovich in view of Bou invention with the teachings of Tran for the benefit of the technology can amount to an inventive distribution of functionality across a network for improving data aggregation, data processing, and/or user experience, such as through distributing data collection and automatic transmission functionality across a plurality of wireless weight sensor subsystems , see par. [0023].
Regarding claim 14 Tran teaches the method according to claim 13, wherein the overlap between successive frames is set to a value between 25% to 75% (windows with 0-50% overlap, see par. [0321]).
Regarding claim 15 Tran teaches the method according to claim 14, wherein the frame length is 20 msec, and wherein the overlap is 50% (50% overlap see par. [0321]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pertinent prior art available on form 892.
Wexler ‘362 teaches a system may comprise a wearable camera configured to capture images from an environment of a user; a microphone configured to capture sounds from the environment of the user; and a processor. The processor may be configured to receive at least one image of the plurality of images, the at least one image comprising a plurality of image portions associated with corresponding image portion timestamps; receive at least one audio signal representative of the sounds captured by the at least one microphone, identify an audio timestamp associated with a portion of the audio signal; identify an image portion from among the plurality of image portions, the image portion having an image portion timestamp associated with the audio timestamp; and analyze the image portion to identify a voice originating from an object represented in the image, see abstract.
Ferguson ‘201 teaches Automated conference room camera switching is a technique that utilizes multiple video cameras in a conference room to visually cover multiple areas of interest by automatically switching camera views as triggered by events happening in the room. A common trigger is the identification of a current talker and switching to a camera that includes a view of that person. See par. [0002].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Ortiz-Sanchez whose telephone number is (571)270-3711. The examiner can normally be reached Monday- Friday 9AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ORTIZ-SANCHEZ/Primary Examiner, Art Unit 2656
