DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 15, 2026, has been entered.
Response to Arguments
Applicant’s arguments, filed April 15, 2026, regarding the 35 U.S.C. 103 rejections of claims 1 – 16, 18 – 19 and 22 – 23 have been considered but they are not persuasive.
On pages 21-22 of Applicant’s response, Applicant argues “Atti et al. discloses an encoder for speech audio and an encoder for speech audio. However, these encoders are associated with content types; one encoder is configured to more efficiently encode an audio stream comprising speech content, while another encoder is configured to more efficiently encode an audio stream comprising non-speech content. The representation format of input, or the structure of input, does not determine to which encoder the input is provided, and a speech signal does not inherently have a different input representation format than a non-speech signal. Atti et al. does not disclose or suggest an association between either encoder and an input representation format, as claimed in Claim 1.”.
However, Atti et al. (US Patent No. 10,854,209), hereinafter Atti, recites, in column 6, lines 20-31, “The IVAS codec 102 is configured to encode the multi-stream formatted audio data 122 to generate the bitstream 126. The IVAS codec 102 is configured to perform encoding of the multi-stream audio data 122 using one or more encoders within the IVAS codec 102, such as an algebraic code-excited linear prediction (ACELP) encoder for speech and a frequency domain (e.g., modified discrete cosine transform (MDCT)) encoder for non-speech audio. The IVAS codec 102 is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, one or more other formats, or any combination thereof.”, disclosing multiple encoders that encode audio data received in a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, or a combination of formats, where a stereo format, an SBA format, an independent streams (IS) format, and a multi-channel format read on input representation formats. Atti discloses “using one or more encoders within the IVAS codec”, where an algebraic code-excited linear prediction (ACELP) encoder and a modified discrete cosine transform (MDCT) encoder are examples. When more than one encoder is used in the IVAS codec, each encoder is associated with a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, or a combination of formats.
On page 22 of Applicant’s response, Applicant further argues “While Atti et al. discloses a switch of an IVAS codec that can receive multiple streams of audio data, it is not disclosed or suggested that the encoders comprised by the codec are associated with different input representation formats. Even if the IVAS codec can be used for a stereo format, an SBA format, etc., the use of one encoder or the other is based on the priority of the audio signal and/or the content of the audio signal, not based on the representation format of the audio signal. For example, even if the IVAS codec takes as input a stereo format audio stream and a SBA format audio stream, the choice of using the ACELP encoder or the MDCT encoder will depend on the content of the audio stream, not on whether the audio stream has a stereo format or an SBA format.”.
However, Atti recites, in column 6, lines 20-31, “The IVAS codec 102 is configured to encode the multi-stream formatted audio data 122 to generate the bitstream 126. The IVAS codec 102 is configured to perform encoding of the multi-stream audio data 122 using one or more encoders within the IVAS codec 102, such as an algebraic code-excited linear prediction (ACELP) encoder for speech and a frequency domain (e.g., modified discrete cosine transform (MDCT)) encoder for non-speech audio. The IVAS codec 102 is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, one or more other formats, or any combination thereof.”, disclosing multiple encoders associated with a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, or a combination of formats, where a stereo format, an SBA format, an independent streams (IS) format, and a multi-channel format read on input representation formats. Atti discloses “using one or more encoders within the IVAS codec”, where using an algebraic code-excited linear prediction (ACELP) encoder for speech and a modified discrete cosine transform (MDCT) encoder for non-speech audio is an example of a configuration with more than one encoder. In the example of using an ACELP encoder and a MDCT encoder, the encoders are associated with the one or more input representation formats of the received audio data, regardless of the ACELP encoder being used for speech and the MDCT encoder being used for non-speech audio.
On page 22 of Applicant’s response, Applicant further argues “Atti et al. discloses encoding based on priority in order to allocate bitrates; a high priority stream is allocated a high bitrate, while a low priority stream is allocated a low bitrate. Atti et al. also discloses that encoding of audio streams depends on the spatial closeness between the streams. In contrast, Claim 1 claims that an encoded multichannel audio signal is generated to comprise at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format. In Claim 1, it is the input representation format that differentiates coders, not a priority or a spatial closeness.”.
However, Atti recites, in column 6, lines 52-58, “The IVAS codec 102 is configured to determine, based on the priority of each of the multiple streams, an analysis and encoding sequence of the multiple streams (e.g., an encoding sequence of frames of each of the multiple streams). In a particular implementation, the streams having higher priority are encoded prior to encoding streams having lower priority.”, disclosing that the priority of each of the multiple streams determines an encoding sequence, not the representation format of the encoders. The limitations of claim 1 do not preclude the first coder associated with at least one first input representation format and the second part encoded with a second coder associated with at least one second input representation format being encoded in a sequence based on priority.
On page 23 of Applicant’s response, Applicant further argues “Accordingly, Atti et al. does not disclose or suggest generating an encoded multichannel audio signal as claimed in Claim 1.”.
However, Atti et al. (US Patent No. 10,854,209), hereinafter Atti, recites, in column 6, lines 20-31, “The IVAS codec 102 is configured to encode the multi-stream formatted audio data 122 to generate the bitstream 126. The IVAS codec 102 is configured to perform encoding of the multi-stream audio data 122 using one or more encoders within the IVAS codec 102, such as an algebraic code-excited linear prediction (ACELP) encoder for speech and a frequency domain (e.g., modified discrete cosine transform (MDCT)) encoder for non-speech audio. The IVAS codec 102 is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, one or more other formats, or any combination thereof.”, disclosing “wherein the encoded multichannel audio signal comprises at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format”, where an IVAS codec configured to perform encoding of multi-stream audio data using one or more encoders, where the IVAS codec is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, and a multi-channel format, reads on the encoded multichannel audio signal comprising at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format, under the broadest reasonable interpretation of the terms “associated with at least one first input representation format” and “associated with at least one second input representation format”.
Therefore, the rejections of Claims 1 – 16, 18 – 19 and 22 – 23 under 35 U.S.C. 103 are maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 16, 18 and 22 – 23 are rejected under 35 U.S.C. 103 as being unpatentable over Vilkamo (PCT Patent Application Publication No. WO2017/182714) in view of Sheaffer et al. (US Patent Application Publication No. 2022/0059123), hereinafter Sheaffer, and Atti et al. (US Patent No. 10,854,209), hereinafter Atti.
Regarding claim 1, Vilkamo discloses an apparatus comprising:
at least one processor (Page 47, lines 10-11, "In some embodiments the device 1200 comprises at least one processor or central processing unit 1207.");
and at least one memory storing instructions that, when executed with the at least one processor (Page 47, lines 15-19, "In some embodiments the device 1200 comprises a memory 1211. In some embodiments the at least one processor 1207 is coupled to the memory 1211. The memory 1211 can be any suitable storage means. The memory 1211 may comprise a program code section for storing program codes implementable upon the processor 1207."), cause the apparatus at least to:
receive at least one channel voice audio signal and metadata associated with the at least one channel voice audio signal, the at least one channel voice audio signal and the associated metadata generated from at least one microphone audio signal (Page 5, line 20 - Page 6, line 2, "There is provided according to a first aspect an apparatus for mixing at least two audio signals, the at least two audio signals associated with at least one parameter, and at least one second audio signal further associated with at least one second parameter, wherein the at least two audio signals and the at least one second audio signal are associated with a sound scene and wherein the at least two audio signals represent spatial audio capture microphone channels and the at least one second audio signal represents an external audio channel separate from the spatial audio capture microphone channels, the apparatus comprising: a processor configured to generate a combined parameter output based on the at least one second parameter and the at least one parameter; and a mixer configured to generate a combined audio signal with a same number or fewer number of channels as the at least one audio signal based on the at least two audio signals and the at least one second audio signal, wherein the combined audio signal is associated with the combined parameter."; Page 6, lines 15-23, "The at least one second parameter may comprise at least one of: at least one direction associated with the at least one second audio signal; at least one direction associated with a spectral band portion of the at least one second audio signal; at least one signal energy associated with the at least one second audio signal; at least one signal energy associated with a spectral band portion of the at least one second audio signal; at least one signal energy ratio associated with the at least one second audio signal; at least one metadata associated with the at least one second audio signal; and at least one signal energy ratio associated with a spectral band portion of the at least one second audio signal."; The second audio signal reads on the at least one channel voice audio signal, and the metadata associated with the second audio signal reads on the metadata associated with the at least one channel voice audio signal.);
receive at least one channel ambience audio signal and metadata associated with the at least one channel ambience audio signal, wherein the at least one channel ambience audio signal and the associated metadata are generated based on an analysis of at least one microphone audio signal, and the at least one channel ambience audio signal is associated with the at least one channel voice audio signal (Page 5, line 20 - Page 6, line 2, "There is provided according to a first aspect an apparatus for mixing at least two audio signals, the at least two audio signals associated with at least one parameter, and at least one second audio signal further associated with at least one second parameter, wherein the at least two audio signals and the at least one second audio signal are associated with a sound scene and wherein the at least two audio signals represent spatial audio capture microphone channels and the at least one second audio signal represents an external audio channel separate from the spatial audio capture microphone channels, the apparatus comprising: a processor configured to generate a combined parameter output based on the at least one second parameter and the at least one parameter; and a mixer configured to generate a combined audio signal with a same number or fewer number of channels as the at least one audio signal based on the at least two audio signals and the at least one second audio signal, wherein the combined audio signal is associated with the combined parameter."; Page 6, lines 7-14, "The at least one parameter may comprise at least one of: at least one direction associated with the at least two audio signals; at least one direction associated with a spectral band portion of the at least two audio signals; at least one signal energy associated with the at least two audio signals; at least one signal energy associated with a spectral band portion of the at least two audio signals; at least one metadata associated with the at least two audio signals; and at least one signal energy ratio associated with a spectral band portion of the at least two audio signals."; The at least two audio signals reads on the least one channel ambience audio signal, and the metadata associated with the at least two audio signals reads on the metadata associated with the at least one channel ambience audio signal.);
and generate an encoded multichannel audio signal based on the at least one channel voice audio signal and the associated metadata and further the at least one channel ambience audio signal and the associated metadata, such that the encoded multichannel audio signal enables spatial presentation of the at least one channel voice audio signal spatially independent of the at least one channel ambience audio signal (Page 8, lines 20-22, "The mixer may be configured to generate the combined audio signal based on adding the at least one second audio signal to one or more channels of the at least two audio signals."; Page 9, line 20 - Page 10, line 3, "According to a second aspect there is provided a method for mixing at least two audio signals, the at least two audio signals associated with at least one parameter, and at least one second audio signal further associated with at least one second parameter, wherein the at least two audio signals and the at least one second audio signal are associated with a sound scene and wherein the at least two audio signals represent spatial audio capture microphone channels and the at least one second audio signal represents an external audio channel separate from the spatial audio capture microphone channels, the method comprising: generating a combined parameter output based on the at least one second parameter and the at least one parameter; and generating a combined audio signal with a same number or fewer number of channels as the at least one audio signal based on the at least two audio signals and the at least one second audio signal, wherein the combined audio signal is associated with the combined parameter."; Page 11, lines 1-5, "The method may comprise appending the at least one direction associated with the at least one second audio signal and/or the spectral band portion of the at least one second audio signal to the at least one direction associated with the at least two audio signals and/or the spectral band portion of the at least two audio signals to generate combined spatial audio information."; Generating a combined audio signal based on adding the at least one second audio signal to one or more channels of the at least two audio signals, where the at least two audio signals are associated with at least one parameter and the at least one second audio signal is associated with at least one second parameter, reads on generating an encoded multichannel audio signal based on the at least one channel voice audio signal and metadata and further the at least one channel ambience audio signal and metadata, and generating combined spatial audio information by appending at least one direction associated with the at least one second audio signal to at least one direction associated with the at least two audio signals reads on the encoded multichannel audio signal enabling the spatial presentation of the at least one channel voice audio signal spatially independent of the at least one channel ambience audio.).
Vilkamo does not specifically disclose: generate an encoded multichannel audio signal such that the encoded multichannel audio signal enables spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal.
Sheaffer teaches:
generate an encoded multichannel audio signal such that the encoded multichannel audio signal enables spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal (Paragraph 0039, lines 1-6, "A spatial renderer 82 having a spatial engine can process the primary speech signal and the one or more ambience audio signals based on the spatial parameters to produce a plurality of time domain channel signals, wherein the one or more ambience audio signals are spatialized in the plurality of time domain channel signals 7."; Paragraph 0040, lines 1-9, "In one aspect, the renderer can select an appropriate rendering of the ambience and/or the clean voice signal based on factors including a) ratio of speech to noise level, b) the content of the data, c) the noise environment of the renderer, or d) user input (e.g., a slider to adjust the ambience level). As a specific example, if the noise environment of the render is such that the user is already immersed in a very high noise level, the system may decide automatically to reduce significantly or not play at all the ambience stream."; Processing the primary speech signal and the one or more ambience audio signals based on spatial parameters to produce a plurality of time domain channel signals reads on a multichannel audio signal enabling spatial presentation of the at least one channel voice audio signal with the at least one channel ambience audio signal, and selecting an appropriate rendering of the ambience and/or the clean voice signal, where the system can significantly reduce the ambience stream, or not play the ambience stream at all, reads on a spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal.).
Vilkamo and Sheaffer are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo to incorporate the teachings of Sheaffer to process a primary speech signal and one or more ambience audio signals based on spatial parameters to produce a plurality of time domain channel signals, and select an appropriate rendering of the ambience and/or the clean voice signal, where the system can significantly reduce the ambience stream, or not play the ambience stream at all. Doing so would allow for speech and ambience to be rendered in a manner that increases the intelligibility of the speech while making the ambience sound more recognizable, yet detached from the primary speaker in a spatially distant manner (Sheaffer; Paragraph 0004, lines 1-19).
Vilkamo in view of Sheaffer does not specifically disclose: wherein the encoded multichannel audio signal comprises at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format.
Atti teaches:
wherein the encoded multichannel audio signal comprises at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format (Column 6, lines 20-31, “The IVAS codec 102 is configured to encode the multi-stream formatted audio data 122 to generate the bitstream 126. The IVAS codec 102 is configured to perform encoding of the multi-stream audio data 122 using one or more encoders within the IVAS codec 102, such as an algebraic code-excited linear prediction (ACELP) encoder for speech and a frequency domain (e.g., modified discrete cosine transform (MDCT)) encoder for non-speech audio. The IVAS codec 102 is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, one or more other formats, or any combination thereof.”; An IVAS codec configured to perform encoding of multi-stream audio data using one or more encoders, where the IVAS codec is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, and a multi-channel format, reads on the encoded multichannel audio signal comprising at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer to incorporate the teachings of Atti to perform encoding of multi-stream audio data using one or more encoders and encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, or one or more other formats. Doing so would allow for encoding multiple audio signals captured concurrently in time using multiple recording devices and synthetically generating multi-channel audio by multiplexing several audio channels that are recorded at the same time or at different times (Atti; Column 3, lines 42-57).
Regarding claim 2, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses: wherein the instructions, when executed with the at least one processor, cause the apparatus to: receive at least one further audio object audio signal, wherein the encoded multichannel audio signal is further based on the at least one further audio object audio signal such that the encoded multichannel audio signal enables spatial presentation of the at least one further audio object audio signal spatially independent of the at least one channel voice audio signal and the at least one channel ambience audio signal (Page 22, line 25 - Page 23, line 6, "In some embodiments the audio and metadata generator 151 comprises a metadata processor 161. The metadata processor 161 may be configured to receive the metadata associated with the SPAC device audio signal and furthermore the metadata associated with the audio object signal. The metadata processor 161 may thus receive, for example from the metadata generator 147, the directional parameters such as the identified SPAC (modelled audio source) direction per time-frequency instance and the energy parameters such as the N identified SPAC direction (modelled audio source) energy ratios. The metadata processor 161 may furthermore receive from the energy/direction analyser 157 the audio object signal energy parameter value(s) and the audio object directional parameters. From these inputs the metadata processor 161 may be configured to generate a suitable combined parameter (or metadata) output which includes the SPAC and the audio object parameter information."; Page 25, lines 3-9, "Although the example above describes the SPAC metadata related to the microphone-array signals having one direction at each time-frequency instance other examples may have more than one direction at each time-frequency instance. Similarly although the above describes a process for merging one audio object signal (and its associated metadata) with the SPAC audio signal and associated metadata other examples may merge more than one audio object signal (and associated metadata)."; Merging more than one audio object signal and associated metadata with the spatial audio capture (SPAC) audio signal and associated metadata reads on the generated encoded multichannel audio signal being based on at least one further audio object, and directional parameters for the audio objects and directional parameters for the SPAC audio signal read on the further audio object audio signal being spatially independent of the at least one channel voice audio signal and the at least one channel ambience audio signal.).
Regarding claim 3, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses:
wherein the at least one microphone audio signal from which is generated the at least one channel voice audio signal and the associated metadata; and the at least one microphone audio signal from which is generated the at least one channel ambience audio signal and the associated metadata, comprise microphone audio signals from one of: separate groups of microphones with no microphones in common; or groups of microphones with at least one microphone in common (Page 9, lines 8-11, "The at least two audio signals representing spatial audio capture microphone channels may be received from a microphone array and the at least one second audio signal representing an external audio channel separate from the spatial audio capture microphone channels may be received from a further microphone array.").
Regarding claim 4, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: receive an input configured to control the generation of the encoded multichannel audio signal (Page 23, lines 1-6, "The metadata processor 161 may furthermore receive from the energy/direction analyser 157 the audio object signal energy parameter value(s) and the audio object directional parameters. From these inputs the metadata processor 161 may be configured to generate a suitable combined parameter (or metadata) output which includes the SPAC and the audio object parameter information."; Receiving the audio object signal energy parameter values and the audio object directional parameters reads on receiving an input configured to control the generation of the encoded multichannel audio signal.).
Regarding claim 5, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: modify a position parameter of the metadata associated with the at least one channel voice audio signal or change a near-channel rendering-channel allocation associated with the at least one channel voice audio signal based on a determined mismatch between the position parameter of the metadata associated with the at least one channel voice audio signal and the allocated near-channel rendering-channel (Page 41, line 25 - Page 42, line 5, "The examples may furthermore be implemented by methods and apparatus configured to combine microphone (or more generally an audio object) signals with the spatial microphone-array originating signals (or other spatially configured audio signals) while modifying the spatial metadata (associated with the spatial microphone array originating signals). The procedure allows transmission of both signals in the same audio signal, which has a lesser number of channels than the original signals had combined. The modification of the spatial metadata means that the spatial information related to the merged signals are combined to a single set of spatial metadata, enabling that the overall spatial reproduction at the receiver end remains very accurate. As is described herein, this property is achieved by the expansion of the spatial metadata as in particular allowed by the present VR/AR audio format."; Modifying spatial metadata to allow transmission of the audio signal with a lesser number of channels read on modifying a position parameter of the metadata based on a determined mismatch between the position parameter of the metadata and an allocated rendering channel.).
Regarding claim 6, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Atti further teaches:
wherein generating the encoded multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: obtain an encoder bit rate (Column 11, line 65 - Column 12, line 3, "The core encoder 204 performs priority-based encoding as described in FIG. 1 to the pre-processed audio streams and generates the bitstream 126. The bitstream 126 may have a bit rate that is determined based on a transmission bit rate between the IVAS codec 102 and the receiving codec 210 via the network 216."); Determining a bit rate based on a transmission bit rate between the immersive voice and audio services (IVAS) codec and the receiving codec reads on obtaining an encoder bit rate.);
select embedded coding levels and allocate a bit rate respective ones of the selected embedded coding levels, wherein a first level of the selected embedded coding levels is associated with the at least one channel voice audio signal and the associated metadata, a second level of the selected embedded coding levels is associated with the at least one channel ambience audio signal, and a third level of the selected embedded coding levels is associated with the metadata associated with the at least one channel ambience audio signal; and encode the at least one channel voice audio signal and the associated metadata, and the at least one channel ambience audio signal and the associated metadata associated with the at least one channel ambience audio signal, based on the allocated bit rates (Column 8, line 59 - Column, line 1, "The IVAS codec 102 is configured to combine the encoded portions of the streams 131-133 to generate the bitstream 126. In a particular implementation, the bitstream 126 has a frame structure in which each frame of the bitstream 126 includes an encoded frame of each of the streams 131-133. In an illustrative example, an i-th frame of the bitstream 126 includes the encoded i-th frame of each of the streams 131-133, along with metadata such as a frame header, stream priority information or bit rate information, location metadata, etc."; Column 9, lines 12-27, "The format of the streams 131-133 may be determined based on the positions of the microphone 106-109, the types of microphones, or a combination thereof. In some implementations, the stream format is configured by a manufacturer of the device 101. In some implementations, the stream format is controlled or configured by the front end audio processor 104 to the IVAS codec 102 based on an application scenarios (e.g., 2-way conversational, conferencing) of the device 101. In other cases, the stream format may also be negotiated between the device 101 and a corresponding bitstream 126 recipient device (e.g., a device containing an IVAS decoder which decodes the bitstream 126) in case of streaming or conversational communication use cases. The spatial metadata 124 is generated and provided to the IVAS codec 102 in certain circumstances, such as e.g., when the streams 121-124 have the independent streams (IS) format."; Column 14, lines 12-17, "In some implementations, the stream priority module 110 is configured to determine whether each stream corresponds to a background audio source or to a foreground audio source and to assign higher priority to streams corresponding to foreground sources and lower priority to streams corresponding to background sources."; Combining the encoded portions of the streams 131-133 along with the metadata to generate a bitstream, where the stream format is negotiated between the device and a corresponding bitstream recipient device and the streams are assigned priorities, reads on selecting embedded coding levels and allocating a bit rate to each of the selected embedded coding levels.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer and Atti to further incorporate the teachings of Atti to combine the encoded portions of audio streams and metadata to generate a bitstream, where the stream format is negotiated between the device and a corresponding bitstream recipient device and the streams are assigned priorities. Doing so would allow for encoding each audio stream based on its priority to allocate higher bit rates to streams having higher priority and lower bit rates to streams having lower priority (Atti; Column 9, lines 41-54).
Regarding claim 7, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Atti further teaches:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: determine a capability parameter, the capability parameter being determined based on at least one of: a transmission channel capacity; or a rendering apparatus capacity, wherein the encoded multichannel audio signal is further generated based on the capability parameter (Column 11, line 63 - Column 12, line 13, "The format pre-processor 202 performs format pre-processing on the audio streams and provides the pre-processed audio streams to the core encoder 204. The core encoder 204 performs priority-based encoding as described in FIG. 1 to the pre-processed audio streams and generates the bitstream 126. The bitstream 126 may have a bit rate that is determined based on a transmission bit rate between the IVAS codec 102 and the receiving codec 210 via the network 216. For example, the IVAS codec 102 and the receiving codec 210 may negotiate a bit rate of the bitstream 126 based on a channel condition of the network 216, and the bit rate may be adjusted during transmission of the bitstream 126 in response to changing network conditions. The IVAS codec 102 may apportion bits to carry encoded information of each of the pre-processed audio streams based on the relative priority of the audio streams, such that the combined encoded audio streams in the bitstream 126 do not exceed the negotiated bit rate."; Negotiating a bit rate of a bitstream based on a channel condition of the network reads on determining a capability parameter based on a transmission channel capacity.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer and Atti to further incorporate the teachings of Atti to negotiating a bit rate of a bitstream based on a channel condition of the network. Doing so would allow for adjusting the bit rate during the transmission of a bitstream in response to changing network conditions (Atti; Column 11, line 63 - Column 12, line 13).
Regarding claim 8, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 7.
Atti further teaches:
wherein generating the encoded multichannel audio signal further based on the capability parameter comprises the instructions, when executed with the at least one processor, to cause the apparatus to: select embedded coding levels and allocate a bit rate to respective ones of the selected embedded coding levels based on the at least one of the transmission channel capacity or the rendering apparatus capacity (Column 11, line 63 - Column 12, line 13, "The format pre-processor 202 performs format pre-processing on the audio streams and provides the pre-processed audio streams to the core encoder 204. The core encoder 204 performs priority-based encoding as described in FIG. 1 to the pre-processed audio streams and generates the bitstream 126. The bitstream 126 may have a bit rate that is determined based on a transmission bit rate between the IVAS codec 102 and the receiving codec 210 via the network 216. For example, the IVAS codec 102 and the receiving codec 210 may negotiate a bit rate of the bitstream 126 based on a channel condition of the network 216, and the bit rate may be adjusted during transmission of the bitstream 126 in response to changing network conditions. The IVAS codec 102 may apportion bits to carry encoded information of each of the pre-processed audio streams based on the relative priority of the audio streams, such that the combined encoded audio streams in the bitstream 126 do not exceed the negotiated bit rate."; Negotiating a bit rate of a bitstream based on a channel condition of the network reads on selecting embedded coding levels and allocating a bit rate to each of the selected embedded coding levels based on the transmission channel capacity.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer and Atti to further incorporate the teachings of Atti to negotiating a bit rate of a bitstream based on a channel condition of the network. Doing so would allow for adjusting the bit rate during the transmission of a bitstream in response to changing network conditions (Atti; Column 11, line 63 - Column 12, line 13).
Regarding claim 9, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses: wherein the at least one microphone audio signal used to generate the at least one channel ambience audio signal and the associated metadata comprises at least two microphone audio signals (Page 8, lines 1-9, "The apparatus may further comprise a microphone arrangement configured to generate the at least two audio signals, wherein locations of the microphone may be defined relative to a defined location. The at least one of the processor or the mixer or the further processor for audio signal mixing may be configured to generate the at least one mix audio signal to simulate a sound wave arriving at the locations of the microphones from the at least one direction associated with the at least one second audio signal and/or spectral band portion of the at least one second audio signal relative to the defined location.").
Regarding claim 10, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 1.
Vilkamo further discloses:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: output the encoded multichannel audio signal (Page 24, lines 19-20, "The combined at least one audio signals may then be output. For example the audio signals may be stored for later processing or passed to the audio renderer.").
Regarding claim 11, Vilkamo discloses an apparatus comprising:
at least one processor (Page 47, lines 10-11, "In some embodiments the device 1200 comprises at least one processor or central processing unit 1207.");
and at least one memory stores instructions that, when executed with the at least one processor (Page 47, lines 15-19, "In some embodiments the device 1200 comprises a memory 1211. In some embodiments the at least one processor 1207 is coupled to the memory 1211. The memory 1211 can be any suitable storage means. The memory 1211 may comprise a program code section for storing program codes implementable upon the processor 1207."), cause the apparatus at least to:
receive an embedded encoded audio signal, the embedded encoded audio signal comprising, at least, at least one channel voice audio signal and associated metadata, and at least one channel ambience audio signal, wherein the embedded encoded audio signal is encoded according to at least one of the following levels of embedded audio signal: a first level for rendering the at least one channel voice audio signal and the associated metadata as a spatial voice scene; a second level for rendering the at least one channel voice audio signal and the associated metadata, and the at least one channel ambience audio signal as a near-far stereo scene; or a third level for rendering the at least one channel voice audio signal and the associated metadata, and the at least one channel ambience audio signal and the associated spatial metadata as a spatial audio scene (Page 5, line 20 - Page 6, line 2, "There is provided according to a first aspect an apparatus for mixing at least two audio signals, the at least two audio signals associated with at least one parameter, and at least one second audio signal further associated with at least one second parameter, wherein the at least two audio signals and the at least one second audio signal are associated with a sound scene and wherein the at least two audio signals represent spatial audio capture microphone channels and the at least one second audio signal represents an external audio channel separate from the spatial audio capture microphone channels, the apparatus comprising: a processor configured to generate a combined parameter output based on the at least one second parameter and the at least one parameter; and a mixer configured to generate a combined audio signal with a same number or fewer number of channels as the at least one audio signal based on the at least two audio signals and the at least one second audio signal, wherein the combined audio signal is associated with the combined parameter."; Page 6, lines 7-14, "The at least one parameter may comprise at least one of: at least one direction associated with the at least two audio signals; at least one direction associated with a spectral band portion of the at least two audio signals; at least one signal energy associated with the at least two audio signals; at least one signal energy associated with a spectral band portion of the at least two audio signals; at least one metadata associated with the at least two audio signals; and at least one signal energy ratio associated with a spectral band portion of the at least two audio signals."; Page 6, lines 15-23, "The at least one second parameter may comprise at least one of: at least one direction associated with the at least one second audio signal; at least one direction associated with a spectral band portion of the at least one second audio signal; at least one signal energy associated with the at least one second audio signal; at least one signal energy associated with a spectral band portion of the at least one second audio signal; at least one signal energy ratio associated with the at least one second audio signal; at least one metadata associated with the at least one second audio signal; and at least one signal energy ratio associated with a spectral band portion of the at least one second audio signal."; The second audio signal and the metadata associated with the second audio signal containing direction parameters read on at least one channel voice audio signal and associated metadata to be rendered as a spatial voice scene, and the at least two audio signals and the metadata associated with the at least two audio signals containing direction parameters read on the least one channel ambience audio signal and associated spatial metadata to be rendered as a spatial audio scene.);
and decode the embedded encoded audio signal and output a multichannel audio signal representing a scene, such that the multichannel audio signal enables spatial presentation of the at least one channel voice audio signal independent of the at least one channel ambience audio signal (Page 11, lines 1-5, "The method may comprise appending the at least one direction associated with the at least one second audio signal and/or the spectral band portion of the at least one second audio signal to the at least one direction associated with the at least two audio signals and/or the spectral band portion of the at least two audio signals to generate combined spatial audio information."; Page 41, lines 25-31, "The examples may furthermore be implemented by methods and apparatus configured to combine microphone (or more generally an audio object) signals with the spatial microphone-array originating signals (or other spatially configured audio signals) while modifying the spatial metadata (associated with the spatial microphone array originating signals). The procedure allows transmission of both signals in the same audio signal, which has a lesser number of channels than the original signals had combined."; Page 42, lines 1-3, "In the examples described herein the audio signal may be rendered into a suitable binaural form, where the spatial sensation may be created using rendering such as by head-related-transfer-function (HRTF) filtering a suitable audio signal."; Page 48, lines 21-24, "The device 1200 may be employed as a render apparatus. As such the transceiver 1209 may be configured to receive the audio signals and positional information from the capture apparatus, and generate a suitable audio signal rendering by using the processor 1207 executing suitable code."; The at least two audio signals with an associate direction and the at least one second audio signal with an associate direction reads on a multichannel audio signal enabling the spatial presentation of the at least one channel voice audio signal independent of the at least one channel ambience channel audio signal, and receiving the audio signals and positional information from the capture apparatus and generating a suitable audio signal rendering reads on decoding the embedded encoded audio signal and outputting a multichannel audio signal.).
Vilkamo does not specifically disclose: wherein the embedded encoded audio signal enables spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal.
Sheaffer teaches:
wherein the embedded encoded audio signal enables spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal (Paragraph 0039, lines 1-6, "A spatial renderer 82 having a spatial engine can process the primary speech signal and the one or more ambience audio signals based on the spatial parameters to produce a plurality of time domain channel signals, wherein the one or more ambience audio signals are spatialized in the plurality of time domain channel signals 7."; Paragraph 0040, lines 1-9, "In one aspect, the renderer can select an appropriate rendering of the ambience and/or the clean voice signal based on factors including a) ratio of speech to noise level, b) the content of the data, c) the noise environment of the renderer, or d) user input (e.g., a slider to adjust the ambience level). As a specific example, if the noise environment of the render is such that the user is already immersed in a very high noise level, the system may decide automatically to reduce significantly or not play at all the ambience stream."; Processing the primary speech signal and the one or more ambience audio signals based on spatial parameters to produce a plurality of time domain channel signals reads on a multichannel audio signal enabling spatial presentation of the at least one channel voice audio signal with the at least one channel ambience audio signal, and selecting an appropriate rendering of the ambience and/or the clean voice signal, where the system can significantly reduce the ambience stream, or not play the ambience stream at all, reads on a spatial presentation of the at least one channel voice audio signal with and at least substantially without spatial presentation of the at least one channel ambience audio signal.).
Vilkamo and Sheaffer are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo to incorporate the teachings of Sheaffer to process a primary speech signal and one or more ambience audio signals based on spatial parameters to produce a plurality of time domain channel signals, and select an appropriate rendering of the ambience and/or the clean voice signal, where the system can significantly reduce the ambience stream, or not play the ambience stream at all. Doing so would allow for speech and ambience to be rendered in a manner that increases the intelligibility of the speech while making the ambience sound more recognizable, yet detached from the primary speaker in a spatially distant manner (Sheaffer; Paragraph 0004, lines 1-19).
Vilkamo in view of Sheaffer does not specifically disclose: wherein the embedded encoded audio signal comprises at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format.
Atti teaches:
wherein the embedded encoded audio signal comprises at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format (Column 6, lines 20-31, “The IVAS codec 102 is configured to encode the multi-stream formatted audio data 122 to generate the bitstream 126. The IVAS codec 102 is configured to perform encoding of the multi-stream audio data 122 using one or more encoders within the IVAS codec 102, such as an algebraic code-excited linear prediction (ACELP) encoder for speech and a frequency domain (e.g., modified discrete cosine transform (MDCT)) encoder for non-speech audio. The IVAS codec 102 is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, one or more other formats, or any combination thereof.”; An IVAS codec configured to perform encoding of multi-stream audio data using one or more encoders, where the IVAS codec is configured to encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, and a multi-channel format, reads on the encoded multichannel audio signal comprising at least a first part encoded with a first coder associated with at least one first input representation format, and a second part encoded with a second coder associated with at least one second input representation format.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer to incorporate the teachings of Atti to perform encoding of multi-stream audio data using one or more encoders and encode data that is received via one or more of a stereo format, an SBA format, an independent streams (IS) format, a multi-channel format, or one or more other formats. Doing so would allow for encoding multiple audio signals captured concurrently in time using multiple recording devices and synthetically generating multi-channel audio by multiplexing several audio channels that are recorded at the same time or at different times (Atti; Column 3, lines 42-57).
Regarding claim 12, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 11.
Vilkamo further discloses:
wherein the embedded encoded audio signal is encoded according to the third level, wherein the embedded encoded audio signal further comprises at least one further audio object audio signal and associated metadata and wherein the instructions, when executed with the at least one processor, cause the apparatus to: decode and output the multichannel audio signal representing the scene, such that spatial presentation of the at least one further audio object audio signal is spatially independent of the at least one channel voice audio signal and the at least one channel ambience audio signal (Page 22, line 25 - Page 23, line 6, "In some embodiments the audio and metadata generator 151 comprises a metadata processor 161. The metadata processor 161 may be configured to receive the metadata associated with the SPAC device audio signal and furthermore the metadata associated with the audio object signal. The metadata processor 161 may thus receive, for example from the metadata generator 147, the directional parameters such as the identified SPAC (modelled audio source) direction per time-frequency instance and the energy parameters such as the N identified SPAC direction (modelled audio source) energy ratios. The metadata processor 161 may furthermore receive from the energy/direction analyser 157 the audio object signal energy parameter value(s) and the audio object directional parameters. From these inputs the metadata processor 161 may be configured to generate a suitable combined parameter (or metadata) output which includes the SPAC and the audio object parameter information."; Page 25, lines 3-9, "Although the example above describes the SPAC metadata related to the microphone-array signals having one direction at each time-frequency instance other examples may have more than one direction at each time-frequency instance. Similarly although the above describes a process for merging one audio object signal (and its associated metadata) with the SPAC audio signal and associated metadata other examples may merge more than one audio object signal (and associated metadata)."; Merging more than one audio object signal and associated metadata with the spatial audio capture (SPAC) audio signal and associated metadata reads on the generated encoded multichannel audio signal being based on at least one further audio object, and directional parameters for the audio objects and directional parameters for the SPAC audio signal read on outputting the multichannel audio signal such that the spatial presentation of the at least one further audio object audio signal is spatially independent of the at least one channel voice audio signal and the at least one channel ambience audio signal.).
Regarding claim 13, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 11.
Vilkamo further discloses:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: receive an input configured to control the decoding of the embedded encoded audio signal and output of the multichannel audio signal (Page 44, lines 10-19, "The audio renderer may be configured to control the azimuth, elevation, and distance of the determined sources or objects within the combined spatial audio signals based on the metadata. Moreover, the user may be allowed to adjust the gain and/or spatial position of any determined source or object based on the output from the head-tracker. Thus the processing/rendering may be dependent on the relative direction (position or orientation) of the external microphone source and the spatial microphones and the orientation of the head as measured by the head-tracker. In some embodiments the user input may be any suitable user interface input, such as an input from a touchscreen indicating the listening direction or orientation."; The user being able to adjust the gain and spatial position of any determined source or object reads on receiving an input configured to control the decoding of the embedded encoded audio signal and output of the multichannel audio signal.).
Regarding claim 14, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 13.
Vilkamo further discloses:
wherein the input comprises a switch of capability, wherein decoding the embedded encoded audio signal and outputting the multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: update the decoding and outputting based on the switch of capability (Page 43, lines 23-30, "In some embodiments the audio signals generated by the object inserter may be passed to a render apparatus comprising a head tracker. The head tracker may be any suitable means for generating a positional or rotational input, for example a sensor attached to a set of headphones or integrated to a head-mounted display configured to monitor the orientation of the listener, with respect to a defined or reference orientation and provide a value or input which can be used by the render apparatus. The head tracker may be implemented by at least one gyroscope and/or digital compass."; Page 44, lines 10-19, "The audio renderer may be configured to control the azimuth, elevation, and distance of the determined sources or objects within the combined spatial audio signals based on the metadata. Moreover, the user may be allowed to adjust the gain and/or spatial position of any determined source or object based on the output from the head-tracker. Thus the processing/rendering may be dependent on the relative direction (position or orientation) of the external microphone source and the spatial microphones and the orientation of the head as measured by the head-tracker. In some embodiments the user input may be any suitable user interface input, such as an input from a touchscreen indicating the listening direction or orientation."; The processing/rendering being dependent on the relative direction of the external microphone source and the spatial microphones and the orientation of the head as measured by the head-tracker reads on updating the decoding and outputting based on a switch of capability.).
Regarding claim 15, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 14.
Vilkamo further discloses:
wherein the switch of capability comprises at least one of: a determination of earbud/earphone configuration; a determination of headphone configuration; or a determination of speaker output configuration (Page 43, lines 23-30, "In some embodiments the audio signals generated by the object inserter may be passed to a render apparatus comprising a head tracker. The head tracker may be any suitable means for generating a positional or rotational input, for example a sensor attached to a set of headphones or integrated to a head-mounted display configured to monitor the orientation of the listener, with respect to a defined or reference orientation and provide a value or input which can be used by the render apparatus. The head tracker may be implemented by at least one gyroscope and/or digital compass."; Page 44, lines 10-19, "The audio renderer may be configured to control the azimuth, elevation, and distance of the determined sources or objects within the combined spatial audio signals based on the metadata. Moreover, the user may be allowed to adjust the gain and/or spatial position of any determined source or object based on the output from the head-tracker. Thus the processing/rendering may be dependent on the relative direction (position or orientation) of the external microphone source and the spatial microphones and the orientation of the head as measured by the head-tracker. In some embodiments the user input may be any suitable user interface input, such as an input from a touchscreen indicating the listening direction or orientation."; “A sensor attached to a set of headphones or integrated to a head-mounted display configured to monitor the orientation of the listener providing a value or input which can be used by the render apparatus reads on the switch of capability comprising a determination of headphone configuration.).
Regarding claim 16, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 13.
Atti further teaches:
wherein the input comprises at least one of: a determination of a change of embedded level, wherein decoding the embedded encoded audio signal and outputting the multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: update the decoding and outputting based on the change of embedded level; or a determination of a change of bit rate for the embedded level, wherein decoding the embedded encoded audio signal and outputting the multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: update the decoding and outputting based on the change of bit rate for the embedded level (Column 11, line 63 - Column 12, line 13, "The format pre-processor 202 performs format pre-processing on the audio streams and provides the pre-processed audio streams to the core encoder 204. The core encoder 204 performs priority-based encoding as described in FIG. 1 to the pre-processed audio streams and generates the bitstream 126. The bitstream 126 may have a bit rate that is determined based on a transmission bit rate between the IVAS codec 102 and the receiving codec 210 via the network 216. For example, the IVAS codec 102 and the receiving codec 210 may negotiate a bit rate of the bitstream 126 based on a channel condition of the network 216, and the bit rate may be adjusted during transmission of the bitstream 126 in response to changing network conditions. The IVAS codec 102 may apportion bits to carry encoded information of each of the pre-processed audio streams based on the relative priority of the audio streams, such that the combined encoded audio streams in the bitstream 126 do not exceed the negotiated bit rate."; Determining a bit rate based on a transmission bit rate between the immersive voice and audio services (IVAS) codec and the receiving codec reads on updating the decoding and outputting based on a determination of a change of bit rate for the embedded level.).
Vilkamo, Sheaffer, and Atti are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer and Atti to further incorporate the teachings of Atti to update the decoding and outputting based on a determination of a change of bit rate for the embedded level. Doing so would allow for adjusting the bit rate during the transmission of a bitstream in response to changing network conditions (Atti; Column 11, line 63 - Column 12, line 13).
Regarding claim 18, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 13.
Vilkamo further discloses:
wherein the instructions, when executed with the at least one processor, cause the apparatus to: control the decoding of the embedded encoded audio signal and outputting of the multichannel audio signal to modify at least one channel voice audio signal position or change a near-channel rendering-channel allocation associated with the at least one channel voice audio signal based on a determined mismatch between the at least one channel voice audio signal position and the allocated near-channel rendering-channel (Page 41, line 25 - Page 42, line 5, "The examples may furthermore be implemented by methods and apparatus configured to combine microphone (or more generally an audio object) signals with the spatial microphone-array originating signals (or other spatially configured audio signals) while modifying the spatial metadata (associated with the spatial microphone array originating signals). The procedure allows transmission of both signals in the same audio signal, which has a lesser number of channels than the original signals had combined. The modification of the spatial metadata means that the spatial information related to the merged signals are combined to a single set of spatial metadata, enabling that the overall spatial reproduction at the receiver end remains very accurate. As is described herein, this property is achieved by the expansion of the spatial metadata as in particular allowed by the present VR/AR audio format."; Modifying spatial metadata to allow transmission of the audio signal with a lesser number of channels read on modifying a channel voice audio signal position based on a determined mismatch between the audio signal detected position and an allocated rendering channel.).
Regarding claim 22, arguments analogous to claim 1 are applicable.
Regarding claim 23, arguments analogous to claim 11 are applicable.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Vilkamo in view of Sheaffer and Atti, and further in view of Rumsey ("Spatial Audio: Channels, Objects, and Ambisonics").
Regarding claim 19, Vilkamo in view of Sheaffer and Atti discloses the apparatus as claimed in claim 13.
Vilkamo further discloses:
wherein the input comprises a determination of correlation between the at least one channel voice audio signal and the at least one channel ambience audio signal (Page 21, lines 27-30, "In some embodiments the audio and metadata generator 151 is configured to receive the spatial audio signals and associated metadata from the SPAC device 141. The audio and metadata generator 151 may furthermore be configured to receive at least one audio object signal."; Page 22, line 25 - Page 23, line 16, "In some embodiments the audio and metadata generator 151 comprises a metadata processor 161. The metadata processor 161 may be configured to receive the metadata associated with the SPAC device audio signal and furthermore the metadata associated with the audio object signal. The metadata processor 161 may thus receive, for example from the metadata generator 147, the directional parameters such as the identified SPAC (modelled audio source) direction per time-frequency instance and the energy parameters such as the N identified SPAC direction (modelled audio source) energy ratios. The metadata processor 161 may furthermore receive from the energy/direction analyser 157 the audio object signal energy parameter value(s) and the audio object directional parameters. From these inputs the metadata processor 161 may be configured to generate a suitable combined parameter (or metadata) output which includes the SPAC and the audio object parameter information. Thus for example where the SPAC device metadata comprises 1 direction and 1 energy ratio parameter (and 1 overall energy parameter for the merging process) and the audio object (external microphone) metadata comprises 1 direction parameter (and 1 overall energy parameter for the merging process), the output metadata may comprise 2 directions where the audio object signal direction is treated as an additional identified direction. Furthermore in some embodiments the output metadata may comprise 2 energy (such as the energy ratio) parameters, which may be the ratio of the power in the SPAC device direction relative to the total energy of the merged audio signals and the other may be the ratio of the audio object audio signal relative to the total energy of the merged audio signals."; Generating a suitable combined parameter output from the spatial audio capture (SPAC) directional parameters and the audio object directional parameters reads on determining a correlation between the at least one channel voice audio signal and the at least one channel ambience audio signal.);
rotating the ambient spatial scene, based on the at least one channel ambience audio signal, according to the obtained rotation parameter (Page 22, lines 16-24, "In some embodiments the audio and metadata generator 151 comprises an energy/direction analyser 157. The energy/direction analyser 157 may be configured to analyse frequency-band signals. The energy/direction analyser 157 may be configured to receive the at least one audio object signal and determine an energy parameter value associated with the at least one audio object signal. The energy parameter value may then be passed to a metadata processor 161. The energy/direction analyser 157 may be configured to determine a direction parameter value associated with the at least one audio object signal. The direction parameter value may then be passed to the metadata processor 161."; Page 43, lines 23-30, "In some embodiments the audio signals generated by the object inserter may be passed to a render apparatus comprising a head tracker. The head tracker may be any suitable means for generating a positional or rotational input, for example a sensor attached to a set of headphones or integrated to a head-mounted display configured to monitor the orientation of the listener, with respect to a defined or reference orientation and provide a value or input which can be used by the render apparatus. The head tracker may be implemented by at least one gyroscope and/or digital compass."; The head tracker providing a positional or rotational input reads on rotating the scene according to an obtained rotation parameter.).
Vilkamo in view of Sheaffer and Atti does not specifically disclose: wherein decoding and outputting the multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: when the correlation is less than a determined threshold then: control a position associated with the at least one channel voice audio signal, and control an ambient spatial scene formed with the at least one channel ambience audio signal with rotating the ambient spatial scene, based on the at least one channel ambience audio signal, according to an obtained rotation parameter or compensating for a rotation of a further device with applying a corresponding opposite rotation to the ambient spatial scene; and when the correlation is greater than or equal to the determined threshold then: control the position associated with the at least one channel voice audio signal, and control the ambient spatial scene formed with the at least one channel ambience audio signal with compensating for a rotation of the further device with applying the corresponding opposite rotation to the ambient spatial scene while letting the rest of the scene rotate or rotating the ambient spatial scene, based on the at least one channel ambience audio signal, according to the obtained rotation parameter.
Rumsey teaches:
wherein decoding and outputting the multichannel audio signal comprises the instructions, when executed with the at least one processor, to cause the apparatus to: when the correlation is less than a determined threshold then: control a position associated with the at least one channel voice audio signal, and control an ambient spatial scene formed with the at least one channel ambience audio signal with rotating the ambient spatial scene, based on the at least one channel ambience audio signal, according to an obtained rotation parameter or compensating for a rotation of a further device with applying a corresponding opposite rotation to the ambient spatial scene; and when the correlation is greater than or equal to the determined threshold then: control the position associated with the at least one channel voice audio signal, and control the ambient spatial scene formed with the at least one channel ambience audio signal with compensating for a rotation of the further device with applying the corresponding opposite rotation to the ambient spatial scene while letting the rest of the scene rotate or rotating the ambient spatial scene, based on the at least one channel ambience audio signal, according to the obtained rotation parameter (Page 991, column 2, line 5 - Page 991, column 3, line 20; "Ambisonics would seem to lend itself rather well to cinematic VR or 360 video because it comes with the ready ability for spatial scenes to be rotated, while retaining all the relative positions of the sounds within the scene, including the reverberant environment. Head tracking can be used to control the decoding of ambisonics to binaural HRTFs (head-related transfer functions) in order that the scene can be rendered according to the current rotation of the user’s head, as shown in Fig. 6. As will be seen, some of the source content can be so-called “head-locked stereo,” which can be used for positionally steady sounds such as narration that don’t need to be affected by head movements."; Rotating the spatial scene while retaining all the relative positions of the sounds within the scene based on head tracking so that the scene can be rendered according to the current rotation of the user’s head reads on rotating the at least one channel voice audio signal and the ambient spatial scene based on compensating for a rotation of the further device when the correlation is high, and not rotating some of the source content that are positionally steady sounds such as narration that do not need to be affected by head movements reads on controlling an ambient spatial scene formed with the at least one channel ambience audio signal by compensating for a rotation of the further device by applying a corresponding opposite rotation to the ambient spatial scene while letting the rest of the scene rotate when the correlation is low.).
Vilkamo, Sheaffer, Atti, and Rumsey are considered to be analogous to the claimed invention because they are in the same field of multi-channel audio coding. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Vilkamo in view of Sheaffer and Atti to incorporate the teachings of Rumsey to rotate the spatial scene while retaining all the relative positions of the sounds within the scene based on head tracking so that the scene can be rendered according to the current rotation of the user’s head, and not rotate some of the source content that are positionally steady sounds such as narration that do not need to be affected by head movements. Doing so would allow for rendering the scene according to the current rotation of the user’s head, while head locking positionally steady sounds such as narration (Rumsey; Page 991, column 2, line 5 - Page 991, column 3, line 20).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James Boggs whose telephone number is (571)272-2968. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES BOGGS/Examiner, Art Unit 2657
Prosecution Insights