Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
2. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 15 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 15 contains the trademark/trade name HDMI. Where a trademark or trade name is used in a claim as a limitation to identify or describe a particular material or product, the claim does not comply with the requirements of 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 1982). The claim scope is uncertain since the trademark or trade name cannot be used properly to identify any particular material or product. A trademark or trade name is used to identify a source of goods, and not the goods themselves. Thus, a trademark or trade name does not identify or describe the goods associated with the trademark or trade name. In the present case, the trademark/trade name is used to identify/describe a means for transferring an audio signal between two devices, accordingly, the identification/description is indefinite.
For purposes of examination the examiner will interpret the claim HDMI audio return channel as a digital audio communication path conforming to the HDMI ARC standard that is conventionally used in televisions, computers and set-top boxes for transmitting audio signals from a playback device to another device.
Claim Rejections - 35 USC § 102
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
4. Claims 1-14 and 16-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nesta (US 2016/0029120).
Regarding Claim 1:
Nesta discloses a method of capturing speech using a television system (Nesta: p[0003] discloses systems such as televisions (tv) Fig. 1 discloses a system that could operate on a television system) the television system comprising a microphone device and a playback device (Nesta: p[0016] discloses audio signals are generated from two or more microphones), the method comprising:
using a first, playback device of the television system to play a tv audio signal to generate acoustic tv audio representing the tv audio signal (Nesta: p[0016] discloses multiple audio signals (sub ban audio signal for a plurality of frequency bands k) are received from an audio source, one of these frequency bands could be noise from a television as described in the background of invention p[0003] playing a movie or any other content with noise)
using a microphone in a second, microphone device of the television system to capture a speech audio signal comprising a mixture of a first acoustic audio source comprising speech from a user and a second acoustic audio source comprising the acoustic tv audio (Nesta: p[0016] discloses there is at least wo microphones within this system, the sub band analysis filter receives a multichannel time domain signals from an audio source and generates a subband audio signal with a plurality of frequency bands, p[0022] describes this multichannel audio contains speech that is intended to be separated from the rest of the signal (i.e., background noise from a device such as a TV));
obtaining a representation of the tv audio signal, wherein the representation of the tv audio signal lacks a well-defined synchronization with the tv audio signal (Nesta: p[0007]-[0008] Figs. 1-3, disclose that echo cancellation may be performed in systems with loosely paired devices where playback and microphone signals are not tightly synchronized, further in p[0054] it is disclosed that “even where there is not an exact linearity, e.g., when the sampling rates between playback and recordings do not match well or when the playback signal is preprocessed before to be played through the loudspeakers” and that the echo cancellation remains effective “even in conditions where the is a large delay between the playback and recorded echo signals” Examiner interpretation {Nesta teaches obtaining a representation of the playback/TV audio signal that lacks a well-defined synchronization with the playback audio signal as claimed});
and processing the speech audio signal using a semi-blind source separation process to separate the speech from the user from the acoustic tv audio and determine at least one audio signal component comprising the speech from the user and in which the acoustic tv audio is suppressed (Nesta: p[0054] discloses using semi-blind multichannel separation in order to suppress non-linear residual echoes out of the intended signal which includes speech),
wherein the semi-blind source separation process uses the representation of the tv audio signal to guide the separation of the speech from the user from the acoustic tv audio, and wherein the at least one audio signal component comprises the captured speech (Nesta: Fig. 1, p[0018] discloses through the flow chart that the time domain reference 116 (the representation of the TV audio signal) is given to a single/double talk estimator 106which receives the input of the original sub bands and the conditioned outputs signals of the subband audio channel audio echo cancellation 104 which is then use used to guide or condition the subband semi-bss 108 in order to lead to a separated signal).
Regarding Claim 2:
Nesta further discloses the method of claim 1 wherein the semi-blind source separation process comprises a process that separates the speech from the user from the acoustic tv audio dependent upon an information content of the speech and the acoustic tv audio (Nesta: p[0018]-[0019] discloses single/double talk estimator generates a probability whether echo dominates near-end speech, based on subband features like ERLE (echo return loss enhancement); Examiner interpretation {the probability is seen as information content measure, and is used as weights to steer the demixing filters} p[0026]-[0028] discloses ICA (independent component analysis) update separates independent components near-end speech vs echo, based on non-Gaussianity and independence assumptions; Examiner interpretation {these are information measures based on theoretical applications of the signal content}).
Regarding Claim 3:
Nesta further discloses the method of claim 1 wherein processing the speech audio signal using the semi-blind source separation process determines a plurality of the audio signal components,
further including an audio signal component comprising the acoustic tv audio and in which the speech from the user is suppressed (Nesta: p[0019-[0020] discloses the semi-blind separation process decomposes the input mixture into multiple signal components, including a component comprising the residual echo corresponding to the playback/TV audio. In this component, the user’s near-end speech is suppressed, while a complementary component preserves the near-end speech with the echo suppressed);
and wherein using the representation of the tv audio signal to guide the separation of the speech from the user from the acoustic tv audio comprises resolving a source ambiguity by identifying one or more of the audio signal components which are similar to the acoustic tv audio (Nesta: p[0018-[0019] discloses echo dominance probability is computed from ERLE his is essentially a similarity/correlation measure between the reference TV signal and the microphone sub band signals, this probability is then used to weight the demixing algorithm).
Regarding Claim 4:
Nesta further discloses the method of claim 1 wherein the microphone comprises a microphone array, the method comprising using the microphone array to capture a multichannel speech audio signal, each channel of the multichannel speech audio signal comprising a mixture of speech from a user and the acoustic tv audio, and processing the channels of the speech audio signal using the semi-blind source separation process to separate the speech from the user from the acoustic tv audio (Nesta: p[0016], p[0026] p[0027] explicitly states two or more microphone and an unknown number of noise sources, each channel contains a mixture of near-end speech and playback echo, ICA is used to separate speech and noise/echo).
Regarding Claim 5:
Nesta further discloses the method of any preceding claim 1wherein the semi-blind source separation process comprises:
converting the speech audio signal to time-frequency domain data comprising a succession of time-frequency data frames for a succession of time windows, each having a plurality of frequency bands (Nesta: p[0016] discloses subband decomposition each representing a range of frequencies, maps directly to this limitations);
and determining a set of de-mixing matrices, one for each of the frequency bands, to apply to each time-frequency data frame to separate the speech from the user and the acoustic tv audio, the set of de-mixing matrices defining a vector of separated outputs (Nesta: p[0025]-p[0030] and p[0038]-p[0043] ICA update per subband, demixing matrices per frequency bin, defining separated outputs).
Regarding Claim 6:
Nesta further discloses the method of claim 5
wherein each row of one of the de-mixing matrices corresponds to one of the sources (Nesta: p[0043]-[0046] discloses demixing matrices in which each row corresponds to one of the separated sources. Specifically, the matrix multiplication shown in these paragraphs produces outputs ysub1-ysubN, with the weighting scheme ensuring that on output each row consistently corresponds to the near-end speech while other rows correspond to residual echo. Thus, each row corresponds to a separated source as claimed);
and permuting the rows of the de-mixing matrices using the representation of the tv audio signal such that for the succession of time windows each row corresponds to the same source (Nesta: p[0043]-[0046] discloses echo reference/weights guide the assignment to consistent sources, i.e., resolving permutation with TV representation).
Regarding Claim 7:
Nesta further discloses the method of claim 5 comprising converting the set of de-mixing matrices from the time-frequency domain to the time domain to determine a time domain demixing filter (Nesta: p[0045]-[0047] discloses);
and applying the time domain demixing filter to the speech audio signal in the time-domain to determine the least one audio signal component comprising the speech from the user and in which the acoustic tv audio is suppressed (Nesta: p[0045]-[0046] discloses applying filters to input to recover near-end source which will be separated from the echo).
Regarding Claim 8:
Nesta further discloses the method of claim 5 comprising
constructing, from the demixing matrices, a multichannel filter to operate on the time- frequency data frames (Nesta: p[0053]-[0054] discloses a deconvolution matrix this is analogous to a multichannel filter as it is built from individual filter coefficients which themselves are derived from the demixing matrix);
applying the multichannel filter to said time-frequency domain data to determine de-mixed time-frequency data (Nesta: p[0055] the subband separator and spectral filter bank apply the filter matrices in the subband frequency domain to yield de-mixed signals);
and converting the de-mixed time- frequency data to the time domain to recover de-mixed time domain data for the least one audio signal component comprising the speech from the user and in which the acoustic tv audio is suppressed (Nesta: p[0055] discloses a subband synthesis filter which combines the subband components back into a time-domain audio signal as described in previous p[0021] and p[0037]).
Regarding Claim 9:
Nesta further discloses the method of any preceding claim 1, wherein the processing the speech audio signal using semi-blind source separation further comprises:
reducing a time misalignment between the speech audio signal and the representation of the tv audio signal, prior to guiding the separation of the speech from the acoustic tv audio (Nesta: p[0054] discloses direct support for handling misalignment between the reference signal and the mic signal, it explicitly mentions sampling rate mismatch, preprocessing which causes offset and large playback vs recording signal delays).
Regarding Claim 10:
Nesta further discloses the method of claim 9, further comprising:
converting each of the speech audio signal and the representation of the tv audio signal into time-frequency domain data, each comprising a succession of time- frequency data frames (Nesta: p[0032] discloses converting both the microphone-captured speech mixture and the playback/TV reference signal into time-frequency (subband) data frames);
determining a pair of frames that are a closest match, comprising determining that a data frame of the converted representation is a closest match to a component of a data frame of the converted speech audio signal, wherein the component corresponds to the second acoustic audio source comprising the acoustic tv audio (Nesta: p[0018]-[0019] the single double talk estimator generates a probability from the subband echo to return a loss enhancement ration (ERLE), computed from the input and output signals for subband audio channel AEC. ERLE is a similarity/correlation measure between the reference signal and the microphone signal at a given frame/subband);
and indicating that the pair of frames relate to the same or nearby point in time (Nesta: p[0054] the disclosure teaches that after finding the matching components (via ERLE/correlation) the system can align them in time despite drift/delay indicating that the pair of frames relate to the same or nearby point in time).
Regarding Claim 11:
Nesta further discloses the method of any preceding claim 1 wherein obtaining a representation of the tv audio signal comprises obtaining a reduced bandwidth representation of the tv audio signal (Nesta: p[0054] discloses “There might be applications where the data throughput is sensible and the full playback reference signals cannot be transferred to the processing system” in other words, this states that instead of the full reference, a reduced version (less bandwidth, fewer samples, or partial information) may be provided)).
Regarding Claim 12:
Nesta further discloses the method of claim 11 wherein processing the speech audio signal comprises processing a digitized sample of the speech audio signal, and wherein the reduced bandwidth representation comprises a representation of an energy in the tv audio signal in a set of different frequencies at time intervals longer than an interval between the digitized samples of the speech audio signal (Nesta: p[0020] p[0054] discloses representing the playback/TV audio by reduced resolution information rather than full digitized samples, in particular describing spectral domain processing based on energy profiles in different frequency bands and further discloses operation where the full reference signals cannot be transferred, therefore relying on reduced-bandwidth, lower sampling rate (longer time intervals) representations).
Regarding Claim 13:
Nesta further discloses the method of any one of claim 1 wherein obtaining the representation of the tv audio signal comprises sending representation of the tv audio signal from the first, playback device to the second, microphone device (Nesta: p[0054]-[0055] discloses that the playback device (loudspeaker 632) produces the audio signal, a representation of this playback signal can be sent/transferred to the processing system that houses the microphones and echo cancellation module).
Regarding Claim 14:
Nesta further discloses the method of claim 13 wherein the second, microphone device comprises a set top box (Nesta: p[0003] discloses “Multimedia devices such as TVs, set-top boxes…”).
Regarding Claim 16:
Nesta further discloses the method of any one of claim 1 comprising obtaining the representation of the tv audio signal at the second, microphone device, and sending the tv audio signal from the second, microphone device to the first, playback device (Nesta: p[0022] discloses “system 100 performs selective source pickup…separating a near-end speech signal from the residual echo component at the output of the linear AEC” this shows that at the mic side, the system explicitly isolates a residual echo component that its received, then p[0055] discloses “device 600 includes microphones 630, loudspeaker 632 and the processing module 602” the process provides output back into the playback chain and p[0054] discloses that it allows cancellation of the echo even in conditions where there is a large delay between the playback and recorded echo signals).
Regarding Claim 17:
Nesta further discloses the method of any preceding claim 1 comprising using the television system as a video phone or video conferencing system, wherein the first, playback device provides an audio and video output for the video phone or video conferencing system, wherein the second, microphone device captures speech of a user of the video phone or video conferencing system, wherein the at least one audio signal component comprising the captured speech is provided for onward transmission to a remote video phone or video conferencing system station (Nesto: p[0003] discloses that the invention is for conferencing systems where playback and microphones are both in use).
Regarding Claim 18:
Nesta discloses a method of processing an audio signal (Nesta: p[0003] and Fig. 1discloses systems for processing audio signals the method comprising:
playing a first audio signal to generate first acoustic audio representing the first audio signal (Nesta: p[0016] discloses multiple audio signals (sub ban audio signal for a plurality of frequency bands k) are received from an audio source, one of these frequency bands could be noise from a television as described in the background of invention p[0003] playing a movie or any other content with noise);
capturing a second audio signal using a microphone, the second audio signal comprising a mixture of a first acoustic audio source comprising the first acoustic audio and a second acoustic audio source comprising a target audio (Nesta: p[0022] the system captures source pickup through another microphone which contains residual echo);
obtaining a representation of the first audio signal, wherein the representation of the first audio signal lacks a well-defined synchronization with the first audio signal (Nesta: p[0054] explicitly discloses obtaining of unsynchronized references with non-linear or misaligned data);
and processing the second audio signal to provide a processed audio signal, by using a semi-blind source separation process to separate the target audio from the first acoustic audio and determine at least one audio signal component comprising the target audio and in which the first acoustic audio is suppressed, wherein the semi-blind source separation process uses the representation of the first audio signal to guide the separation of the target audio from the first acoustic audio (Nesta: p[0018]-[0020] discloses semi-blind ICA, weighted gradient, producing near-end source and residual echo which matches semi-blind separation yielding suppressed-echo speech).
Regarding Claim 19:
Nesta further discloses the method of claim 18 wherein the semi-blind source separation process comprises:
converting the second audio signal to time-frequency domain data comprising a succession of time-frequency data frames for a succession of time windows, each having a plurality of frequency bands (Nesta: p[0032] discloses an algorithm that does subband analysis which takes multiple frequency bins which shows how a frequency changes over time and stacks them on top of one of another to show how these frequencies change over time altogether);
determining a set of de-mixing matrices, one for each of the frequency bands, to apply to each time-frequency data frame to separate the target audio from the first acoustic audio (Nesta: p[0045]-[0046] discloses/defines demixing/rotation matrices per subband and filter vectors);
and using the representation of the first audio signal to resolve an ambiguity between one or both of i) an allocation de-mixed sources to first acoustic audio, and ii) an allocation of de-mixed frequency bands to the sources (Nesta: p[0019], and p[0043] disclose the permutation problems are fixed through this method which are essentially the internal permutation problem and the external permutation problem. The internal permutation problem is that ICA cannot can separate sources categorically but it cannot actually guarantee that the rows are properly demixed, the external permutation problem is the ability to solve the latter. In other words it properly allocates which output belongs to speech and how to keep that assignment consistent across all frequency bands by using weighting algorithms based on the guiding from echo probability to ensure each frequency ends up in the correct bin).
Regarding Claim 20:
Nesta discloses a non-transitory storage medium storing processor control code to implement the method of any one of claim 1 (Nesta: p[0015] discloses a program memory storing logic for instructing a processor to perform the audio processing and echo cancellation methods of the disclosure).
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
6. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Nesta (US 2016/0029120) in view of Yang (US 2023/0186936).
Regarding Claim 15:
Nesta further discloses the method of claim 13 except wherein the sending comprises sending over an HDMI audio return channel. However, Yang discloses this limitation (Yang: p[0043]-[0044] discloses that the interface may support certain protocols such as HDMI for an audio interface).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included the use of HDMI to send audio signals. One of ordinary skill in the art would find that there was a clear motivation to use such technology. Primarily that HDMI is a commonly used and widely successful form of data transmission that is well established and tested. Therefore, it would be understood that there would be a reasonable expectation of success using the HDMI format without undue experimentation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN SCOTT MCLEAN whose telephone number is (703)756-4599. The examiner can normally be reached "Monday - Friday 8:00-5:00 EST, off Every 2nd Friday".
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN SCOTT MCLEAN/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654