Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed August 10, 2026, has been entered. Claims 1 – 30 are pending in the application. Applicant’s amendments to the Drawings, Specification, and Claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed May 8, 2026.
Response to Arguments
Applicant's arguments, filed August 10, 2026, with respect to claims 1 and 17 – 19 being interpreted under 35 U.S.C. 112(f) have been fully considered but they are not persuasive.
On page 12 of Applicant’s response, Applicant argues “To the extent the Office has interpreted certain claim features under 35 U.S.C. § 112(f), Applicant respectfully submits that the amended claims recite sufficient structure to avoid invocation of § 112(f). In particular, the claims now expressly recite structural elements such as "a convolutional learnable layer configured to...by sliding a 1x1 kernel," "a recurrent learnable layer" (e.g., GRU), and "a format definer" (i.e., rolling window transform). These are well-understood structural terms in the art, not generic nonces. To the extent any feature is nonetheless construed under § 112(f), the specification discloses corresponding structure in detail (see, e.g., Figs. 1B and 8; Section 2.1, pages 53-54).”.
However, in the limitations “a format definer configured to define” in claim 1, “at least one learnable layer configured to encode” in claim 1, “a format definer configured to define” in claim 17, “a first learnable layer which is a convolutional learnable layer configured to encode” in claim 17, “a second learnable layer which is a recurrent learnable layer configured to encode” in claim 17, “a third learnable layer which is a convolutional learnable layer configured to generate” in claim 17, “a quantizer to encode” in claim 18, and “a quantizer to encode” in claim 19, the terms “definer”, “layer”, and “quantizer” are modified by functional language but are not modified by sufficient structure, material, or acts for performing the claimed function. The terms “definer”, “layer”, and “quantizer” are not modified as being performed by hardware, software, or as part of a machine learning algorithm.
Applicant's following arguments, filed August 10, 2026, with respect to the rejections of claims 1 – 30 under 35 U.S.C. 112(b) have been fully considered but they are not persuasive.
On page 12 of Applicant’s response, Applicant argues “In claim 9, a comma has been inserted to clarify that "the kernel" refers to the kernel of "the at least one convolutional learnable layer" introduced in parent claim 8, thereby establishing proper antecedent basis.”. However, claim 8 does not introduce a kernel of the at least one convolutional learnable layer.
On page 13 of Applicant’s response, Applicant argues “Antecedent basis for "the at least one codebook" in claims 22 and 23 is now properly established through the dependency on claim 18, which recites "a quantizer to encode a bitstream from the output encoded audio signal representation."”. However, claim 20, not claim 18, recites "a quantizer to encode a bitstream from the output encoded audio signal representation", so the insufficient antecedent basis for the limitation "the at least one codebook" in claim 22, lines 1-2, and for the limitation "the at least one codebook" in claim 23, lines 1-2, has not been resolved. Claim 21 has been amended to depend from claim 20, resolving the insufficient antecedent basis for the limitation "the learnable quantizer" in claim 21, lines 1-2, and the limitation "the at least one codebook" in claim 21, line 2, but not resolving the insufficient antecedent basis for the limitation "the codebook" in claim 21, lines 3, 4, and 5.
On page 13 of Applicant’s response, Applicant argues “In claim 24, "the second residual portions of frames" has been corrected to "second residual portions of the first output encoded multi-dimensional audio signal representation of the input audio signal."”. However, claim 24 recites the limitation "the second residual portions of frames" in line 9.
On page 13 of Applicant’s response, Applicant argues “In claim 27, "the at least one learnable block" has been amended to "the at least one learnable layer" to correct the terminology.”. However, there is insufficient antecedent basis for the limitation "the encoded fourth multi-dimensional audio signal representation" recited in claim 27, line 3. Claim 17 recites the limitation "an encoded fourth multi-dimensional audio signal representation", but claim 27 does not depend from claim 17. This claim 27 insufficient antecedent basis issue can be resolved by amending claim 27 to depend from claim 19 instead of claim 18.
Applicant’s arguments, filed August 10, 2026, with respect to the 35 U.S.C. 102 rejection of claim 1, have been fully considered but they are not persuasive.
On page 15 of Applicant’s response, Applicant argues “More specifically, Luo segments a sequential input into overlapping chunks of length K with hop size P (see Luo at Section 2.1.1), processes the resulting 3-D tensor T through a stack of B DPRNN blocks (see Luo at Section 2.1.2), applies masks for source separation (see Luo at Section 3.1), and reconstructs separated output signals through overlap-add processing (see Luo at Section 2.1.3). Although those operations transform intermediate data, the rejection does not establish that Luo's identified tensors constitute the particular encoded representations required by amended claim 1, or that Luo's components perform the claimed sequence of encoding operations. That Luo generates intermediate tensors during processing does not, without more, disclose an encoder whose learnable layers are configured to encode an input audio signal into a first multi-dimensional representation, or a further encoded version thereof, so as to generate an encoded output audio signal representation as now required by claim 1.”.
However, Luo et al. ("Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation"), hereinafter Luo, recites, in section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape.", and recites, in section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”, disclosing “at least one learnable layer configured to encode the input audio signal into the first multidimensional audio signal representation of the input audio signal, or further encoded version of the first multi-dimensional audio signal representation, to generate the output encoded audio signal representation of the input audio signal”, where tensor T reads on a multidimensional audio signal representation of the input audio signal, output Q reads on an output encoded audio signal representation of the input audio signal, and a stack of dual-path recurrent neural network blocks reads on at least one learnable layer. Claim 1 does not provide any limitation on generating an output encoded audio signal representation of an input audio signal from a multidimensional audio signal representation of the input audio signal that precludes the method of Luo of generating the multidimensional output Q from the multidimensional representation of the input T.
On page 15 of Applicant’s response, Applicant further argues “To try to justify the rejection, the Office Action maps Luo's segmentation stage to the claimed format definer and Luo's DPRNN blocks to the claimed learnable layers. (See Office Action at pages 9-12.) However, Luo's output Q ϵ IR^(NxL) is a full-resolution sequential output obtained through overlap-add, as it has the same dimensionality as the input W. (See Luo at Section 2.1.3: "Denote the output of the last DPRNN block as T {B+1} ϵ IR^{NxKxS}. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ IR^{NxL}.") However, the Office Action does not explain how this output, or any intermediate tensor in Luo, constitutes the "encoded output audio signal representation" now required by claim 1 or how any layer in Luo produces a "further encoded version" of the first multi-dimensional representation as now claimed. At most, Luo discloses that convolutional and recurrent processing were generally known. It does not disclose the particular claimed arrangement in which an encoder's learnable layers generate successive encoded representations from a format-definer-organized input.”.
However, Luo recites, in section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”), disclosing “a format definer configured to define a first multi-dimensional audio signal representation of the input audio signal, the first multi-dimensional audio signal representation of the input audio signal comprising at least: a first dimension, so that a plurality of mutually subsequent frames is ordered according to the first dimension; and a second dimension so that a plurality of samples of at least one frame are ordered according to the second dimension”, where a segmentation stage reads on a format definer, generating S equal size chunks reads on a plurality of mutually subsequent frames ordered according to a first dimension, and splitting a sequential input W into chunks of length K reads on a plurality of samples of at least one frame ordered according to the second dimension. Luo further recites, in section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape.", and recites, in section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”, disclosing “at least one learnable layer configured to encode the input audio signal into the first multidimensional audio signal representation of the input audio signal, or further encoded version of the first multi-dimensional audio signal representation, to generate the output encoded audio signal representation of the input audio signal”, where tensor T reads on a multidimensional audio signal representation of the input audio signal, output Q reads on an output encoded audio signal representation of the input audio signal, and a stack of dual-path recurrent neural network blocks reads on at least one learnable layer. Claim 1 does not provide any limitation on generating an output encoded audio signal representation of an input audio signal from a multidimensional audio signal representation of the input audio signal that precludes the method of Luo of generating the multidimensional output Q from the multidimensional representation of the input T.
Applicant’s remaining arguments, filed August 10, 2026, with respect to claims 1 – 30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a format definer configured to define” in claim 1
“at least one learnable layer configured to encode” in claim 1
“a format definer configured to define” in claim 17
“a first learnable layer which is a convolutional learnable layer configured to encode” in claim 17
“a second learnable layer which is a recurrent learnable layer configured to encode” in claim 17
“a third learnable layer which is a convolutional learnable layer configured to generate” in claim 17
“a quantizer to encode” in claim 18
“a quantizer to encode” in claim 19.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. Sufficient structure to perform the recited functions is described in the specification on page 103, lines 13-15, “Generally, examples may be implemented as a computer program product with program instructions, the program instructions being operative for performing one of the methods when the computer program product runs on a computer.”.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9, 21 – 24 and 27 – 30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 9 recites the limitations "the kernel" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 21 recites the limitation "the codebook ze" in line 3, the limitation "the codebook re" in line 4, and the limitation "the codebook qe" in line 5. There is insufficient antecedent basis for these limitations in the claim.
Claim 22 recites the limitation "the at least one codebook" in lines 1-2. There is insufficient antecedent basis for this limitation in the claim.
Claim 23 recites the limitation "the at least one codebook" in lines 1-2. There is insufficient antecedent basis for this limitation in the claim.
Claim 24 recites the limitation "the second residual portions of frames" in line 9. There is insufficient antecedent basis for this limitation in the claim.
Claim 27 recites the limitation "the encoded fourth multi-dimensional audio signal representation" in line 3. There is insufficient antecedent basis for this limitation in the claim. Claim 17 recites the limitation "an encoded fourth multi-dimensional audio signal representation", but claim 27 does not depend from claim 17. This claim 27 insufficient antecedent basis issue can be resolved by amending claim 27 to depend from claim 19 instead of claim 18.
Claims 28 – 30 are also rejected as they depend from claim 27 and thus recite the limitations of claim 27, and do not resolve the indefinite language from claim 27.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 – 3, 5, 10 – 13 and 15 – 16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Luo et al. ("Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation"), hereinafter Luo.
Regarding claim 1, Luo discloses an encoder for encoding an input audio signal comprising a sequence of input audio signal frames into an encoded output audio signal representation, each input audio signal frame comprising a sequence of input audio signal samples (Section 1, lines 36-38, "In this paper, we propose a simple network architecture, which we refer to as dual-path RNN (DPRNN), that organizes any kinds of RNN layers to model long sequential inputs in a very simple way."; Section 3.1, lines 1-5, "Although DPRNN can be applied to any systems that require long-term sequential modeling, we investigate its application to the time-domain audio separation network (TasNet) [1, 4, 22], an adaptive front-end method that achieves high speech separation performance on a benchmarking dataset."), the encoder comprising:
a format definer configured to define a first multi-dimensional audio signal representation of the input audio signal (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”), the first multi-dimensional audio signal representation of the input audio signal comprising at least:
a first dimension, so that a plurality of mutually subsequent frames is ordered according to the first dimension (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Generating S equal size chunks reads on a plurality of mutually subsequent frames ordered according to a first dimension.);
and a second dimension so that a plurality of samples of at least one frame are ordered according to the second dimension (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Splitting a sequential input W into chunks of length K reads on a plurality of samples of at least one frame ordered according to the second dimension.),
at least one learnable layer configured to encode the input audio signal into the first multidimensional audio signal representation of the input audio signal, or further encoded version of the first multi-dimensional audio signal representation, to generate the output encoded audio signal representation of the input audio signal (Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; Section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”; Tensor T reads on a multidimensional audio signal representation of the input audio signal, output Q reads on an output encoded audio signal representation of the input audio signal, and a stack of dual-path recurrent neural network blocks reads on at least one learnable layer.);
wherein the format definer is configured to insert, along the second dimension of the first multi-dimensional audio signal representation of the input audio signal, additional input audio signal samples of one or more additional frames immediately successive to or immediately preceding at least one frame (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Concatenating S equal size chunks to form a 3-D tensor T reads on inserting additional input audio signal samples of one or more additional frames immediately successive to or immediately preceding a frame, where S equal size chunks of a sequential input W reads on frames.).
Regarding claim 2, Luo discloses the encoder as claimed in claim 1.
Luo further discloses:
wherein the format definer is configured to insert, along the second dimension of the first multidimensional audio signal representation of the input audio signal, input audio signal samples of each given frame (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Section 3.1, lines 1-5, "Although DPRNN can be applied to any systems that require long-term sequential modeling, we investigate its application to the time-domain audio separation network (TasNet) [1, 4, 22], an adaptive front-end method that achieves high speech separation performance on a benchmarking dataset."); Concatenating S equal size chunks to form a 3-D tensor T reads on inserting additional input audio signal samples of each given frame, where S equal size chunks of a sequential input W reads on frames.).
Regarding claim 3, Luo discloses the encoder as claimed in claim 1.
Luo further discloses:
wherein the at least one learnable layer comprises at least one recurrent learnable layer (Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; A stack of dual-path recurrent neural network blocks reads on a recurrent learnable layer.).
Regarding claim 5, Luo discloses the encoder as claimed in claim 3.
Luo further discloses:
wherein the at least one recurrent learnable layer is operated along the first dimension (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; A stack of dual-path recurrent neural network blocks transforming 3-D tensor T reads on a recurrent learnable layer being operated along the first dimension.).
Regarding claim 10, Luo discloses the encoder as claimed in claim 1.
Luo further discloses:
wherein at least one or more of the at least one learnable layer is a residual learnable layer (Section 2.1.2, lines 42-43, "A residual connection is also added between the output and
T
̂
b
to form the output for DPRNN block b."; A dual-path recurrent neural network block with a residual connection reads on a residual learnable layer.).
Regarding claim 11, Luo discloses the encoder as claimed in claim 10.
Luo further discloses:
wherein at least one learnable layer is a residual learnable layer, a main portion of the first multidimensional audio signal representation of the input audio signal bypassing the at least one learnable layer, and/or the at least one learnable layer is applied to at least a residual portion of the first multidimensional audio signal representation of the input audio signal (Section 2.1.2, lines 42-43, "A residual connection is also added between the output and
T
̂
b
to form the output for DPRNN block b."; A dual-path recurrent neural network block with a residual connection reads on a residual learnable layer.).
Regarding claim 12, Luo discloses the encoder as claimed in claim 3.
Luo further discloses:
wherein the recurrent learnable layer operates along a series of time steps each comprising at least one state, in such a way that each time step is conditioned by the output and/or state of the preceding time step (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; Section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”; A stack of dual-path recurrent neural network blocks operating on a tensor T that includes a sequential input W separated into chunks of length K reads on a recurrent learnable layer operating along a series of time steps each comprising at least one state, in such a way that each time step is conditioned by the output and/or state of the preceding time step.).
Regarding claim 13, Luo discloses the encoder as claimed in claim 12.
Luo further discloses:
wherein the step and/or output of each step is recursively provided to a subsequent time step (Section 2.1, lines 1-6, "A dual-path RNN (DPRNN) consists of three stages: segmentation, block processing, and overlap-add. The segmentation stage splits a sequential input into overlapped chunks and concatenates all the chunks into a 3-D tensor. The tensor is then passed to stacked DPRNN blocks to iteratively apply local (intra-chunk) and global (inter-chunk) modeling in an alternate fashion."; A stack of dual-path recurrent neural network blocks operating on a tensor T that includes a sequential input W separated into chunks of length K by iteratively applying intra-chunk and inter-chunk modeling reads on a recurrent learnable layer operating along a series of time steps where each step is recursively provided to a subsequent time step.).
Regarding claim 15, Luo discloses the encoder as claimed in claim 3.
Luo further discloses:
wherein the recurrent learnable layer generates the output for a given time instant by keeping into account the output and/or a state of a preceding time instant (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; Section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”; A stack of dual-path recurrent neural network blocks operating on a tensor T that includes a sequential input W separated into chunks of length K reads on a recurrent learnable layer generating an output for a given time instant by keeping into account the output and/or a state of a preceding time instant.),
wherein the relevance of the output and/or state of a preceding time instant is obtained during training (Section 3.3, lines 1-8, "We train all models for 100 epochs on 4-second long segments. The learning rate is initialized to 1
e
-
3
and decays by 0.98 for every two epochs. Early stopping is applied if no best model is found in the validation set for 10 consecutive epochs. Adam [28] is used as the optimizer. Gradient clipping with maximum L2-norm of 5 is applied for all experiments. All models are trained with utterance-level permutation invariant training (uPIT) [29] to maximize scale-invariant SNR (SI-SNR) [8]."; Training on segments of sequential data reads on the relevance of the output and/or state of a preceding time instant being obtained during training.).
Regarding claim 16, Luo discloses the encoder as claimed in claim 1.
Luo further discloses:
the format definer being configured to order mutually subsequent samples, one after the other one according to the second dimension (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Splitting a sequential input W into chunks of length K reads on ordering mutually subsequent samples one after the other one according to the second dimension.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Luo in view of Jiang et al. ("End-to-End Neural Speech Coding for Real-Time Communications"), hereinafter Jiang.
Regarding claim 4, Luo discloses the encoder as claimed in claim 3, but does not specifically disclose: wherein the at least one recurrent learnable layer comprises a gated recurrent unit, GRU.
Jiang teaches:
wherein the at least one recurrent learnable layer comprises a gated recurrent unit, GRU (Abstract, lines 3-5, "This paper proposes the TFNet, an end-to-end neural speech codec with low latency for RTC."; Section 2.3, lines 1-3, "We utilize two types of temporal filtering blocks as shown in Fig.2, the dilated temporal convolution module (TCM) and the group-wise gated recurrent unit (G-GRU).").
Jiang is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo to incorporate the teachings of Jiang to use a gated recurrent unit. Doing so would allow for representing audio in speech coding and regression tasks (Jiang; Section 5, lines 1-9).
Regarding claim 6, discloses the encoder as claimed in claim 1, but does not specifically disclose: further comprising at least one first convolutional learnable layer between the format definer and the at least one recurrent learnable layer.
Jiang teaches:
further comprising at least one first convolutional learnable layer between the format definer and the at least one recurrent learnable layer (Section 2.1, lines 1-4, "The TFNet-based codec takes the time-frequency spectrum (20ms window with a 5ms hop length) as input with a power-law compression on the amplitude before feeding into the network."; Section 2.2, lines 1-3, "The encoder consists of several causal 2D convolutional layers, each followed by a batch normalization (BN) and a parametric ReLU (PRelu) for nonlinearity."; Section 2.3, lines 1-3, "We utilize two types of temporal filtering blocks as shown in Fig.2, the dilated temporal convolution module (TCM) and the group-wise gated recurrent unit (G-GRU).").
Jiang is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo to incorporate the teachings of Jiang to perform encoding with convolutional layers followed by temporal filtering with a gated recurrent unit. Doing so would allow for representing audio in speech coding and regression tasks (Jiang; Section 5, lines 1-9).
Regarding claim 8, Luo discloses the encoder as claimed in claim 1, but does not specifically disclose: further comprising at least one convolutional learnable layer downstream to the at least one recurrent learnable layer.
Jiang teaches:
further comprising at least one convolutional learnable layer downstream to the at least one recurrent learnable layer (Section 2.3, lines 1-7, "We utilize two types of temporal filtering blocks as shown in Fig.2, the dilated temporal convolution module (TCM) and the group-wise gated recurrent unit (G-GRU). Both are causal and low-complexity. The TCM module as that in [14] consists of two convolutions with a kernel size of 1 X 1 to change channel dimensions and dilated depthwise convolutions to exploit temporal correlations with low complexity.").
Jiang is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo to incorporate the teachings of Jiang to perform temporal filtering with a gated recurrent unit followed by a convolution. Doing so would allow for representing audio in speech coding and regression tasks (Jiang; Section 5, lines 1-9).
Claims 7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Luo in view of Jiang, and further in view of Zeghidour et al. ("SoundStream: An End-to-End Neural Audio Codec"), hereinafter Zeghidour.
Regarding claim 7, Luo in view of Jiang discloses the encoder as claimed in claim 6, but does not specifically disclose: wherein in the at least one first convolutional learnable layer a kernel is slid along the second dimension of the first multi-dimensional audio signal representation of the input audio signal.
Zeghidour teaches:
wherein in the at least one first convolutional learnable layer a kernel is slid along the second dimension of the first multi-dimensional audio signal representation of the input audio signal (Section III, lines 1-5, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: an encoder, which maps x to a sequence of embeddings (see Section III-A),"; Section III-A, lines 1-12, "The encoder architecture is illustrated in Fig. 3 and follows the same structure as the streaming SEANet encoder described in [2], but without skip connections. It consists of a 1D convolution layer (with
C
e
n
c
channels), followed by
B
e
n
c
convolution blocks. Each of the blocks consists of three residual units, containing dilated convolutions with dilation rates of 1, 3, and 9, respectively, followed by a down-sampling layer in the form of a strided convolution. The number of channels is doubled whenever down-sampling, starting from
C
e
n
c
. A final 1D convolution layer with a kernel of length 3 and a stride of 1 is used to set the dimensionality of the embeddings to D (D = 256 in our experiments).").
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Jiang to incorporate the teachings of Zeghidour to perform encoding with a convolution layer operating on a sequence of audio data samples. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 9, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Jiang discloses the encoder as claimed in claim 8, but does not specifically disclose: wherein in the at least one convolutional learnable layer, the kernel is slid along the second dimension of the first multi-dimensional audio signal representation of the input audio signal.
Zeghidour teaches:
wherein in the at least one convolutional learnable layer, the kernel is slid along the second dimension of the first multi-dimensional audio signal representation of the input audio signal (Section III, lines 1-5, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A),"; Section III-A, lines 1-12, "The encoder architecture is illustrated in Fig. 3 and follows the same structure as the streaming SEANet encoder described in [2], but without skip connections. It consists of a 1D convolution layer (with
C
e
n
c
channels), followed by
B
e
n
c
convolution blocks. Each of the blocks consists of three residual units, containing dilated convolutions with dilation rates of 1, 3, and 9, respectively, followed by a down-sampling layer in the form of a strided convolution. The number of channels is doubled whenever down-sampling, starting from
C
e
n
c
. A final 1D convolution layer with a kernel of length 3 and a stride of 1 is used to set the dimensionality of the embeddings to D (D = 256 in our experiments).").
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Jiang to incorporate the teachings of Zeghidour to perform encoding with a convolution layer operating on a sequence of audio data samples. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Claims 14, 18, 20 – 24 and 26 – 28 are rejected under 35 U.S.C. 103 as being unpatentable over Luo in view of Zeghidour.
Regarding claim 14, Luo discloses the encoder as claimed in claim 12, but does not specifically disclose: comprising a plurality of feedforward modules, each providing the state and/or output to the subsequent module.
Zeghidour teaches:
comprising a plurality of feedforward modules, each providing the state and/or output to the subsequent module (Section III-A, lines 1-12, "The encoder architecture is illustrated in Fig. 3 and follows the same structure as the streaming SEANet encoder described in [2], but without skip connections. It consists of a 1D convolution layer (with
C
e
n
c
channels), followed by
B
e
n
c
convolution blocks. Each of the blocks consists of three residual units, containing dilated convolutions with dilation rates of 1, 3, and 9, respectively, followed by a down-sampling layer in the form of a strided convolution. The number of channels is doubled whenever down-sampling, starting from
C
e
n
c
. A final 1D convolution layer with a kernel of length 3 and a stride of 1 is used to set the dimensionality of the embeddings to D (D = 256 in our experiments)."; A 1D convolution layer followed by
B
e
n
c
convolution blocks reads on a plurality of feedforward modules, where each module provides an output to the subsequent module.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo to incorporate the teachings of Zeghidour to perform encoding with a convolution layer followed by convolution blocks. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 18, Luo discloses the encoder as claimed in claim 1, but does not specifically disclose: comprising a quantizer to encode a bitstream from the output encoded audio signal representation.
Zeghidour teaches:
comprising a quantizer to encode a bitstream from the output encoded audio signal representation (Section III, lines 1-9, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A), ∙ a residual vector quantizer, which replaces each embedding by the sum of vectors from a set of finite codebooks, thus compressing the representation with a target number of bits (see Section III-C)"; A residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits reads on a quantizer to encode a bitstream from the output encoded audio signal representation.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo to incorporate the teachings of Zeghidour to implement a residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 20, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Zeghidour further teaches:
wherein the quantizer is a learnable quantizer configured to associate, to each frame of the output encoded multi-dimensional audio signal representation of the input audio signal, or a further encoded version of the first multi-dimensional audio signal representation, indexes of at least one codebook, so as to generate the bitstream (Section III, lines 1-9, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A), ∙ a residual vector quantizer, which replaces each embedding by the sum of vectors from a set of finite codebooks, thus compressing the representation with a target number of bits (see Section III-C)"; A residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits reads on a quantizer configured to associate indexes of at least one codebook to each frame of the first multi-dimensional audio signal representation of the input audio signal to generate a bitstream.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 21, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 20.
Zeghidour further teaches:
wherein the learnable quantizer uses the at least one codebook associating indexes
i
z
,
i
r
,
i
q
, with the index
i
z
representing a code z approximating E(x) and being taken from the codebook
z
e
, the index
i
r
representing a code r approximating E(x)-z and being taken from the codebook
r
e
, and the index
i
q
representing a code q approximating E(x)-z-r and being taken from the codebook
q
e
to be encoded in the bitstream, wherein x represents the input audio signal and E(x) represents an encoded representation of the input audio signal (Section III, lines 1-9, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A), ∙ a residual vector quantizer, which replaces each embedding by the sum of vectors from a set of finite codebooks, thus compressing the representation with a target number of bits (see Section III-C)"; Section III-C, lines 20-26, "Residual Vector Quantizer: To address this issue we adopt a Residual Vector Quantizer (a.k.a. multi-stage vector quantizer [39]), which cascades
N
q
layers of VQ as follows. The unquantized input vector is passed through a first VQ and quantization residuals are computed. The residuals are then iteratively quantized by a sequence of additional
N
q
− 1 vector quantizers, as described in Algorithm 1."; A residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits, where the residuals are iteratively quantized by a sequence of additional vector quantizers, reads on a quantizer using the at least one codebook associating indexes
i
z
,
i
r
,
i
q
, with the index
i
z
representing a code z approximating E(x) and being taken from the codebook
z
e
, the index
i
r
representing a code r approximating E(x)-z and being taken from the codebook
r
e
, and the index
i
q
representing a code q approximating E(x)-z-r and being taken from the codebook
q
e
to be encoded in the bitstream.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits, where the residuals are iteratively quantized by a sequence of additional vector quantizers. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 22, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Zeghidour further teaches:
wherein the at least one codebook comprises at least one base codebook associating, to indexes to be encoded in the bitstream, multidimensional tensors of the output encoded multi-dimensional audio signal representation of the input audio signal (Section III-A, lines 9-12, "A final 1D convolution layer with a kernel of length 3 and a stride of 1 is used to set the dimensionality of the embeddings to D (D = 256 in our experiments)."; Section III-C, lines 7-11, "This vector quantizer learns a codebook of N vectors to encode each D-dimensional frame of enc(x). The encoded audio enc(x) ∈
R
S
x
D
is then mapped to a sequence of one-hot vectors of shape S × N, which can be represented using S log2 N bits."; D-dimensional embeddings read on multidimensional tensors of the first multi-dimensional audio signal representation, and a codebook encoding D-dimensional embeddings reads on a base codebook associating multidimensional tensors of the first multi-dimensional audio signal representation to indexes to be encoded in the bitstream.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a codebook encoding D-dimensional embeddings. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 23, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Zeghidour further teaches:
wherein the at least one codebook comprises at least one residual codebook associating, to indexes to be encoded in the bitstream, multidimensional tensors of the output encoded multi-dimensional audio signal representation of the input audio signal (Section III-A, lines 9-12, "A final 1D convolution layer with a kernel of length 3 and a stride of 1 is used to set the dimensionality of the embeddings to D (D = 256 in our experiments)."; Section III-C, lines 7-11, "This vector quantizer learns a codebook of N vectors to encode each D-dimensional frame of enc(x). The encoded audio enc(x) ∈
R
S
x
D
is then mapped to a sequence of one-hot vectors of shape S × N, which can be represented using S log2 N bits."; Section III-C, lines 20-26, "Residual Vector Quantizer: To address this issue we adopt a Residual Vector Quantizer (a.k.a. multi-stage vector quantizer [39]), which cascades
N
q
layers of VQ as follows. The unquantized input vector is passed through a first VQ and quantization residuals are computed. The residuals are then iteratively quantized by a sequence of additional
N
q
− 1 vector quantizers, as described in Algorithm 1."; D-dimensional embeddings read on multidimensional tensors of the first multi-dimensional audio signal representation, and a codebook encoding D-dimensional embeddings, where residuals are iteratively quantized, reads on a residual codebook associating multidimensional tensors of the first multi-dimensional audio signal representation to indexes to be encoded in the bitstream.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a codebook encoding D-dimensional embeddings, where residuals are iteratively quantized. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 24, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Zeghidour further teaches:
a second residual codebook associates, to indexes to be encoded in the bitstream, multidimensional tensors representing second residual portions of the output encoded multi-dimensional audio signal representation of the input audio signal, a first residual codebook associates, to indexes to be encoded in the bitstream, multidimensional tensors representing first residual portions of frames of the output encoded multi-dimensional audio signal representation, wherein the second residual portions of frames are residual with respect to the first residual portions of frames (Section III, lines 1-9, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A), ∙ a residual vector quantizer, which replaces each embedding by the sum of vectors from a set of finite codebooks, thus compressing the representation with a target number of bits (see Section III-C)"; Section III-C, lines 20-26, "Residual Vector Quantizer: To address this issue we adopt a Residual Vector Quantizer (a.k.a. multi-stage vector quantizer [39]), which cascades
N
q
layers of VQ as follows. The unquantized input vector is passed through a first VQ and quantization residuals are computed. The residuals are then iteratively quantized by a sequence of additional
N
q
− 1 vector quantizers, as described in Algorithm 1."; A residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits, where the residuals are iteratively quantized by a sequence of additional vector quantizers, reads on a first residual codebook associating multidimensional tensors representing first residual portions of frames of the first multi-dimensional audio signal representation to indexes to be encoded in the audio signal representation, and a second residual codebook associating multidimensional tensors representing second residual portions of the first multi-dimensional audio signal representation to indexes to be encoded in the audio signal representation, where the second residual portions of frames are residual with respect to the first residual portions of frames.).
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits, where the residuals are iteratively quantized by a sequence of additional vector quantizers. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 26, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Zeghidour further teaches:
wherein at least one codebook is a fixed-length codebook (Section III, lines 1-9, "We consider a single channel recording x ∈
R
T
, of duration T and sampled at
f
s
. The SoundStream model consists of a sequence of three building blocks, as illustrated in Fig. 2: ∙ an encoder, which maps x to a sequence of embeddings (see Section III-A), ∙ a residual vector quantizer, which replaces each embedding by the sum of vectors from a set of finite codebooks, thus compressing the representation with a target number of bits (see Section III-C)"; Section III-A, lines 49-52, "Residual vector quantization provides a convenient framework for controlling the bitrate. For a fixed size N of each codebook, the number of VQ layers Nq determines the bitrate.").
Zeghidour is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to further incorporate the teachings of Zeghidour to implement a residual vector quantizer replacing each embedding by the sum of vectors from a set of finite codebooks to compress the representation with a target number of bits, where the codebooks are a fixed-size. Doing so would allow for implementing an audio codec that outperforms state-of-the-art audio codecs over a wide range of bitrates and content types (Zeghidour; Section VI, lines 1-7).
Regarding claim 27, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 18.
Luo further discloses:
further comprising at least one further learnable block downstream to the at least one learnable layer to generate, from the encoded fourth multi-dimensional audio signal representation or another version of the input audio signal, an encoded fifth audio signal representation of the input audio signal with multiple samples for each frame (Section 2.1.1, lines 1-7, "For a sequential input W ϵ
R
N
x
L
where N is the feature dimension and L is the number of time steps, the segmentation stage splits W into chunks of length K and hop size P. The first and last chunks are zero-padded so that every sample in W appears and only appears in K/P chunks, generating S equal size chunks
D
s
ϵ
R
N
x
K
, s = 1, …, S. All chunks are then concatenated together to form a 3-D tensor T = [
D
1
, …,
D
s
] ϵ
R
N
x
K
x
S
.”; Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; Section 2.1.3, lines 1-3, "Denote the output of the last DPRNN block as
T
B
+
1
ϵ
R
N
x
K
x
S
. To transform it back to a sequence, the overlap-add method is applied to the S chunks to form output Q ϵ
R
N
x
L
.”; A stack of dual-path recurrent neural network blocks reads on at least one further learnable block downstream to the at least one learnable block, each block transforming an input 3-D tensor into another tensor with the same shape reads on generate a fifth audio signal representation of the input audio signal from the fourth multi-dimensional audio signal representation, and splitting a sequential input W into chunks of length K reads on an audio signal representation of the input audio signal with multiple samples for each frame.).
Regarding claim 28, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 27.
Luo further discloses:
wherein the at least one further learnable block comprises: at least one residual learnable layer (Section 2.1.2, lines 1-3, "The segmentation output T is then passed to the stack of B DPRNN blocks. Each block transforms an input 3-D tensor into another tensor with the same shape."; Section 2.1.2, lines 42-43, "A residual connection is also added between the output and
T
̂
b
to form the output for DPRNN block b."; A stack of dual-path recurrent neural network blocks reads on at least one further learnable block downstream to the at least one learnable block, and a dual-path recurrent neural network block with a residual connection reads on the at least one further learnable block downstream to the at least one learnable block being a residual learnable layer.).
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Luo in view of Zeghidour, and further in view of Olivieri et al. (US Patent No. 12,073,842), hereinafter Olivieri.
Regarding claim 25, Luo in view of Zeghidour discloses the encoder as claimed in claim 18, but does not specifically disclose: configured to signal, in the bitstream, whether indexes associated to residual frames are encoded or not.
Olivieri teaches:
configured to signal, in the bitstream, whether indexes associated to residual frames are encoded or not (Column 26, lines 4-11, "A vector quantizer 1108 may be configured to quantize the residual vector 1118. In one example, the vector quantizer 1108 may quantize the residual vector using a quantization process to produce the residual ID 1124. Instead of quantizing each sample separately (e.g., scalar quantization), the vector quantizer 1108 may be configured to quantize a block of samples included in the residual vector 1118 (e.g., a shape vector)."; Column 26, lines 57-61, "The audio encoder 1000A may be further configured to encode the coarse energy 1120, the fine energy 1122, and the residual ID 1124 using a bitstream encoder 1110 to create the encoded audio data 31 (which is another way to refer to the bitstream 31)."; Generating a residual ID when a vector quantizer quantizes a residual vector and encoding the residual ID in a bitstream reads on signaling, in the bitstream, whether indexes associated to residual frames are encoded or not.).
Olivieri is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to incorporate the teachings of Olivieri to generate a residual ID when a vector quantizer quantizes a residual vector and encode the residual ID in a bitstream. Doing so would allow for improving compression efficiency (Olivieri; Column 27, lines 13-19).
Claims 29 – 30 are rejected under 35 U.S.C. 103 as being unpatentable over Luo in view of Zeghidour, and further in view of Jiang.
Regarding claim 29, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 27, but does not specifically disclose: wherein the at least one further learnable block comprises: at least one convolutional learnable layer.
Jiang teaches:
wherein the at least one further learnable block comprises: at least one convolutional learnable layer (Section 2.3, lines 1-7, "We utilize two types of temporal filtering blocks as shown in Fig.2, the dilated temporal convolution module (TCM) and the group-wise gated recurrent unit (G-GRU). Both are causal and low-complexity. The TCM module as that in [14] consists of two convolutions with a kernel size of 1 X 1 to change channel dimensions and dilated depthwise convolutions to exploit temporal correlations with low complexity.").
Jiang is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to incorporate the teachings of Jiang to perform temporal filtering with a gated recurrent unit followed by a convolution. Doing so would allow for representing audio in speech coding and regression tasks (Jiang; Section 5, lines 1-9).
Regarding claim 30, as best understood based on the 35 U.S.C. 112(b) issues identified above, Luo in view of Zeghidour discloses the encoder as claimed in claim 27, but does not specifically disclose: wherein the at least one further learnable block comprises: at least one learnable layer activated by an activation function.
Jiang teaches:
wherein the at least one further learnable block comprises: at least one learnable layer activated by an activation function (Section 2.2, lines 1-3, "The encoder consists of several causal 2D convolutional layers, each followed by a batch normalization (BN) and a parametric ReLU (PRelu) for nonlinearity.").
Jiang is considered to be analogous to the claimed invention because it is in the same field of audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Luo in view of Zeghidour to incorporate the teachings of Jiang to implement an encoder consisting of convolutional layers followed by a rectified linear unit. Doing so would allow for representing audio in speech coding and regression tasks (Jiang; Section 5, lines 1-9).
Allowable Subject Matter
Claims 17 and 19 are allowable.
The following is a statement of reasons for the indication of allowable subject matter:
The primary reason claims 17 and 19 are allowable, is the combination of the limitations, in all the claims, to define a first multi-dimensional audio signal representation of the input audio signal with a first dimension and a second dimension, the second dimension representing an axis of audio signal samples within each frame and presenting samples from a subsequent frame, encode the first multi-dimensional audio signal representation of the input audio signal into an encoded second multi-dimensional audio signal representation of the input audio signal with a convolutional learnable layer by sliding a 1x1 kernel along a second dimension of the first multi-dimensional audio signal representation of the input audio signal, encode the encoded second multi-dimensional audio signal representation into an encoded third multi-dimensional audio signal representation of the input audio signal with a recurrent learnable layer configured by operating along a first dimension of the first multi-dimensional audio signal representation of the input audio signal, generate an encoded fourth multi-dimensional audio signal representation of the input audio signal with a convolutional learnable layer by sliding a 1x1 kernel along the second dimension of the encoded third multi-dimensional audio signal representation of the input audio signal, and obtain the output encoded audio signal representation from the encoded fourth multi-dimensional audio signal representation of the input audio signal.
Luo discloses an audio signal representation generator for generating an output audio signal representation from an input audio signal comprising a sequence of input audio signal frames, each input audio signal frame comprising a sequence of input audio signal samples (Section 1, lines 36-38; Section 3.1, lines 1-5) the audio signal representation generator comprising: a format definer configured to define a first multi-dimensional audio signal representation of the input audio signal (Section 2.1.1, lines 1-7); a second learnable layer which is a recurrent learnable layer configured to generate a third multi-dimensional audio signal representation of the input audio signal by operating along a first direction of the first multi-dimensional audio signal representation, or of a processed version thereof which is a second multi-dimensional audio signal representation, of the input audio signal (Section 2.1.1, lines 1-7; Section 2.1.2, lines 1-3; Section 2.1.3, lines 1-3); a third learnable layer configured to generate a fourth multi-dimensional audio signal representation of the input audio signal by sliding along the second direction of the third multi-dimensional audio signal representation of the input audio signal (Section 2.1.2, lines 1-3), so as to obtain the output audio signal representation from the fourth multi-dimensional audio signal representation of the input audio signal (Section 2.1.3, lines 1-3).
Jiang discloses: a third learnable layer which is a convolutional learnable layer (Section 2.1, lines 1-4; Section 2.2, lines 1-3).
Zeghidour discloses: wherein in the at least one first convolutional learnable layer the kernel is slid along the second direction of the first multi-dimensional audio signal representation of the input audio signal (Section III, lines 1-5; Section III-A, lines 1-12).
However, it would not have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Luo, Jiang, and Zeghidour to reach the combination of the limitations to define a first multi-dimensional audio signal representation of the input audio signal with a first dimension and a second dimension, the second dimension representing an axis of audio signal samples within each frame and presenting samples from a subsequent frame, encode the first multi-dimensional audio signal representation of the input audio signal into an encoded second multi-dimensional audio signal representation of the input audio signal with a convolutional learnable layer by sliding a 1x1 kernel along a second dimension of the first multi-dimensional audio signal representation of the input audio signal, encode the encoded second multi-dimensional audio signal representation into an encoded third multi-dimensional audio signal representation of the input audio signal with a recurrent learnable layer configured by operating along a first dimension of the first multi-dimensional audio signal representation of the input audio signal, generate an encoded fourth multi-dimensional audio signal representation of the input audio signal with a convolutional learnable layer by sliding a 1x1 kernel along the second dimension of the encoded third multi-dimensional audio signal representation of the input audio signal, and obtain the output encoded audio signal representation from the encoded fourth multi-dimensional audio signal representation of the input audio signal.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James Boggs whose telephone number is (571)272-2968. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES BOGGS/Examiner, Art Unit 2657
Prosecution Insights