DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to preliminary amendment filed on August 19, 2024 and wherein claims 1-15 canceled and claims 16-35 added.
In virtue of this communication, claims 16-35 are currently pending in this Office Action.
In the response to this office action, the Examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the Examiner in prosecuting this application.
Claim Objections
Claims 16-35 are objected to because of the following informalities:
Claim 16 recited “the metadata” and this term appears to be first time recited at this point and thus, should be -- associated metadata --. Claims 17-24 are objected due to the dependencies to claim 1.
Claim 25 is objected for the at least similar reason as described in claim 16 above since claim 25 recited similar deficient limitation as recited in claim 16. Claims 26-28 are objected due to the dependencies to claim 25.
Claim 29 is objected for the at least similar reason as described in claim 16 above since claim 29 recited similar deficient limitation as recited in claim 16. Claims 30-35 are objected due to the dependencies to claim 29.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 16-35 are rejected under 35 U.S.C. 103 as being unpatentable over Laaksonen et al. (US 20220254355 A1, hereinafter Laaksonen) and in view of reference Keiler et al. (US 20130216070 A1).
Claim 16: Laaksonen teaches an apparatus (title and abstract, ln 1-14, an encoder with a decoder in fig. 5) comprising:
at least one processor (at least one processor in the apparatus, para 44); and
at least one memory storing instructions (at least one memory including a computer program code in the apparatus, para 44) that, when executed by the at least one processor, cause the apparatus at least to:
capture spatial audio signals (spatial audio captured in fig. 1, para 116) by a plurality of microphones (one of groups of microphones with no microphone in common and groups of microphones with at least one microphone in common, para 27) using a first capture setting (capturing on the ear of the first user 104 at an angle and a time moment 121 or UE-on-ear capture mode 221 in fig. 2);
generate a set of audio encoder input format data (binaural input type 502, mono input type 501, MASA input type 503, ambisonics type 504, etc., in fig. 5) comprising a representation of the spatial audio signals (input to stereo and spatial encoder and/or mono to EVS in fig. 5) and associated metadata (input to metadata quantizer 517 in fig. 5), the metadata including, in part, a set of descriptive metadata indicative of one or more capture parameters (e.g., “indicates user voice signal with highest priority” for voice priority metadata in table, para 105, 147, and “object position including w-y-z or azimuth/elevation/distance” for position metadata in table, para 147, “object position preference in two-channel” for position metadata in table, para 148) associated with the first capture setting (voice object position or an angle of the first user 104 who made the call, in fig. 1 and “rotation compensation angle information” for side-information of the metadata, para 118);
provide the set of audio encoder input format data to an audio encoder (IVAS encoder in fig. 5, including EVS encoder 513, and stereo and spatial encoder 515, and metadata quantizer 517) for encoding the representation of the spatial audio signals (encoding audio signals in the formats of mono 501, stereo & binaural 502 , MASA 503, ambisonics 504, channel-based 505 ana audio objects 506, para 134-135) and the associated metadata (side information/metadata with input types to be encoded into the bitstream 521, para 136) to a bitstream for a current streaming session (generating bitstream 521 in fig. 5, para 134-136 and at a time period associated with the metadata, para 150 and during the call, e.g., changing from left to the right of the audio object in fig. 22, para 252);
transmit the bitstream to one or more remote devices for rendering the representation of the spatial audio signals (IVAS decoder 531, rendered by using left and right channel speakers 1111-1113, 1131-1133 in fig. 11b-11d) based, at least in part, on the associated metadata (decoded metadata from metadata dequantizer 537 in fig. 5 and for modifying MASA metadata according to voice object position, para 106); and
determine, during the current streaming session, that the first capture setting changes to a second capture setting (changed the capture angle 121 to another angle of the another time moment 123, and then continuation capture of further angle of the further time moment 125 in fig. 1, para 116 or changed the UE-on-ear capture mode 221 to handheld hands-free mode at consequent moment 223 and then to the UE on the table at the further moment 225 in fig. 2, para 117 or capturing ambience or channel voice at different environments, para 46, para 183), wherein, in response to the determination that the first capture setting changes to a second capture setting (capture mode determined based on sensor information or user selection via suitable user interface UI, e.g., the hands-free mode, the handheld hands-free mode, and the handheld mode, para 196, and transition among the capture modes in figs. 1, 2), and while transmitting the bitstream, capturing spatial audio signals comprises capturing spatial audio signals using the second capture setting (settings associated with 123/125 different from 121 due to rotation of the capture device with head turns in fig. 1 and settings associated with 223, 225 different from initial setting 221 due to movement of the sound capture device 202 in fig. 2) and generating the audio encoder input format data comprises changing at least one of the one or more capture parameters of the set of descriptive metadata to be associated with the second capture setting (through IVAS encoder 511 to generate updated bitstream 521 to the IVAS decoder in fig. 5 and for continuation of the call of fig. 1 or 2, e.g., relative positions of the microphone to the sound source are changed in figs. 1-2).
However, Laaksonen does not explicitly teach the descriptive metadata is for rendering the representation of the spatial audio signals.
Keiler teaches an analogous field of endeavor by disclosing an apparatus (title and abstract, ln 1-8 and a HOA content creation system in fig. 3) and wherein capture spatial audio signals by a plurality of microphones using a first capture setting is disclosed (one of the capture settings a-d in fig. 3) and wherein a set of descriptive metadata indicative of one or more capture parameters associated with a first capture setting is disclosed (side information and metadata for Ambisonics, para 7, and represented by MetaDataChunk with metadata information packet to XML format or any user-defined format, and XML format as a descriptive language written inherently, para 128, 140, and indicative of relative directions and characteristics for one or more microphones as sound capture device, para 53, claim 19, and recorded in meta field including coordinate system, microphone, source and virtual listener positions, microphone directional characteristics, room and source information, para 19-20) and encoded/decoded (recorded or encoded, para 4 or represented by an encoding vector of Ambisonics coefficients, para 14 and decoded in figs. 5, 13) for rendering the representation of the spatial audio signals (through encoded vector ψ, para 53, and the loudspeaker array in fig. 2, e.g., through HOA Panner in fig. 2) for benefits of improving the efficiency of spatial audio encoding and decoding (by integrating different formats of HOA descriptions, integrating encoded spatial field with variety of parameters, and allowing extension of flexible and economic of storage data, and metadata being ready for special decoding, etc., para 109 and using HOAs representing both ambient sounds and foreground action sounds, para 47).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have applied the set of descriptive metadata indicative of the one or more capture parameters associated with the first capture setting and used for rendering the representation of the spatial audio signals, as taught by Keiler, to the associated metadata used for rendering the representation of the spatial audio signals in the apparatus, as taught by Laaksonen, for the benefits discussed above.
Claim 25 recited an opposite features of the encoder of the claim 16 and thus, the combination of Laaksonen and Keiler further teaches an apparatus (Laaksonen, IVAS decoder 531 in fig. 5 and Keiler, decoder in fig. 2) comprising: at least one processor (Laaksonen, at least one processor, para 54); and at least one memory storing instructions that, when executed by the at least one processor (at least one memory storing a computer program code, para 54), cause the apparatus at least to:
receive a bitstream for a current streaming session (Laaksonen, bitstream 521 received to IVA decoder in fig. 5 and receiving HOA file for decoding the HOA in fig. 2, para 58);
decode the bitstream for the current streaming session, to determine a representation of spatial audio signals and associated metadata (Laaksonen, via decoders 533, 535 and metadata dequantizer 537 in fig. 5 and Keiler, decoding performed in fig. 2, para 58), the metadata including, in part, a set of descriptive metadata indicative of one or more capture parameters associated with a first capture setting (Laaksonen, the discussion in claim 16 above and Keiler, the direction is about directions of microphones, para 46, 53 and in XML written in descriptive language, 128, 140 and discussed in claim 16 above);
configure an audio renderer according to the decoded descriptive metadata (Laaksonen, through modifying the metadata regarding at least one channel voice audio signal position of the metadata if the mismatch, para 61 and discussed in claim 16 above and Keiler, e.g., through HOA panner in fig. 2);
provide the representation of the spatial audio signals to the audio renderer for rendering to produce a rendered audio output signal (Laaksonen, through the processed audio signals to headsets or speakers with renderer 1311 in figs. 11a-11d, and renderers 1211, 1311 in figs. 12, 13, respectively and Keiler, providing the signal L to the PCM mixer in fig. 2);
output the rendered audio output signal by a plurality of speakers (Laaksonen, figs. 11a-11d, 12-13 discussed above and outputting the PCM mixed audio signal to the loudspeaker array in fig. 2); and
receive an updated bitstream for the current streaming session and decode the updated bitstream for the current streaming session, to determine an updated representation of spatial audio signals and associated metadata, the metadata including, in part, a set of descriptive metadata indicative of one or more capture parameters associated with a second capture setting, wherein configuring the audio renderer according to the decoded descriptive metadata comprises changing at least one of the one or more capture parameters of the set of descriptive metadata to be associated with the second capture setting (Laaksonen, discussed in claim 16 above, application of the first and the second capture settings and opposite to the generation of the first and the second capture setting discussed in claim 16 above and Keiler, the integrating different formats of HOA upon the capture settings in fig. 3 and Keiler, adaptively to future of HOA microphones, para 8 and upon a change of the sample format and limitation of the available bandwidth, para 146 or indicated by PacketDirectionFlag due to a source position moved, para 179).
Claim 29 recited a method of encoding functions that is essentially consistent with claim 16 and thus, has been analyzed and rejected according to claim 16 above.
Claim 17: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein determining that the first capture setting changes to the second capture setting comprises determining that the orientation of the apparatus has changed from a first orientation to a second orientation (Laaksonen, changing orientation of the sound capture device due to turning user’s head 104 in fig. 1 and discussed in claim 16 above and Keiler, the sound source is moving, para 87 and indicative in source moving Position Track Header Condition in the track header, para 138).
Claim 18: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein determining that the first capture setting changes to the second capture setting comprises determining that an input request has been received to change the first capture setting to the second capture setting (Laaksonen, rotating user’s head with the device is determined, para 116 and the discussed in claim 16 above and Keiler, moving sound object is detect, and discussed in claims 16, 17 above).
Claim 19: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein the apparatus is further caused to transmit the set of descriptive metadata indicative of one or more capture parameters associated with the first capture setting to the one or more remote devices (Laaksonen, the descriptive metadata, and Keiler, metadata written in XML or descriptive language, and discussed in claims 16 above) except explicitly teaching via an out-of-band signal.
It has been a recognized problem and need in the art in the audio encoding/decoding field, which may include a design need to solve the problem for efficiently and effectively transmission of the metadata stream and there had been a finite number of identified, predictable potential solutions to the target of transmission of the metadata as such:
1. transmitting the set of descriptive metadata via in-band signal for fast and synchronized with encoded audio stream,
2. transmitting the set of descriptive metadata via out-of-band signal for saving bandwidth for encoded audio stream and real-time transmission of audio load,
it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to have pursued the known potential solutions with a reasonable expectation of success or obvious to try, see MPEP 2141, III.
Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to have applied via out of band signal to transmit the disclosed set of descriptive metadata, as taught by obvious to try above, to transmit the disclosed set of descriptive metadata in the apparatus, as taught by the combination of Laaksonen and Keiler, for the benefits discussed above.
Claim 20: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein the one or more capture parameters comprises (Laaksonen, IVAS with metadata-assisted spatial audio MASA, para 1, including MASA metadata to metadata quantizer in fig. 5, Note: prior art applied under Markush, MPEP2117) at least one of:
a directional element comprising a number of directions described by the spatial metadata (Laaksonen, MASA metadata to metadata quantizer 517 in fig. 5, and thus, inherently, MASA metadata includes MASA format descriptive common metadata parameters which inherently includes number of directions, number of channels, etc.);
a channel element comprising a number of transport channels supported by the apparatus; a source format describing a configuration of the apparatus (Laaksonen, MASA metadata in fig. 5, and thus, inherency for MASA format descriptive common metadata parameters); or
a variable description describing at least one of a capture type, an angle between two microphones of the plurality of microphones, an apparatus size and a microphone polar pattern including omnidirectional, cardiod, hypercardioid, or supercardioid patterns (MASA metadata in fig. 5, and thus, MASA descriptive metadata parameters includes variable description field with 12 bits and indicative of microphone grid, including cardioid, supercardioid, hypercardioid, dipole, binaural, etc.).
Claim 21: the combination of Laaksonen and Keiler further teaches, according to claim 20 above, wherein the bitstream for the current streaming session comprises the one or more capture parameters (Laaksonen, the voice priority metadata and w-y-z or azimuth/elevation/distance for position metadata in table, para 147-148, para 105 and Keiler, side information and metadata, para 7 and including parameters defining the Ambisonics data, para 15, TrackCodingType, in table of para 140, and bandwidth reduction side information, para 146, etc., TrackSourceType, etc., in table of para 137).
Claim 22: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein generating the set of audio encoder input format data comprises using a metadata-assisted spatial audio, MASA, format (Laaksonen, MASA format 503 in fig. 5).
Claim 23: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein the audio encoder is an immersive voice and audio services, IVAS, codec (IVAS codecs including IVAS encoder 511 and IVAS decoder 531 and immersive voice and audio by MASA ambient audio scene or waveforms, para 155 and enhanced voice service EVS 513 in fig. 5 and Keiler, with full immersive reproductions, para 48).
Claim 24: the combination of Laaksonen and Keiler further teaches, according to claim 16 above, wherein the apparatus comprises a user device (Laaksonen, UEs as mobile handsets or devices, para 115, e.g., smartphones, para 4).
Claim 26: the combination of Laaksonen and Keiler further teaches, according to claim 25 above, wherein receiving a bitstream for a current streaming session comprises receiving the bitstream from the apparatus of claim 16 (Laaksonen, fig. 5, including IVAS encoder and decoder in fig. 5 and Keiler, through the generated HOA file at the encoder storage in figs. 3-4 and retrieved from the decoder in fig. 5 ).
Claim 27 has been analyzed and rejected according to claims 25, 16 the combination of Laaksonen and Keiler further teaches, according to claim 25 above, wherein the apparatus is further caused to receive the set of descriptive metadata indicative of one or more capture parameters associated with the first capture setting to the one or more remote devices via an out-of-band signal (Laaksonen, and Keiler, discussed in claim 19 above); and wherein configuring the audio renderer further comprises configuring the audio renderer according to the set of descriptive metadata indicative of one or more capture parameters associated with the first capture setting (Laaksonen, via modifying the metadata due to dismatch for meeting the playback configuration or allocated near-channel rendering-channel channel, para 23, and discussed in claim 16 above, and Keiler, two types of HOA audio data for meeting the dense and less dense loudspeaker arrangement and surrounding, para 24).
Claim 28 has been analyzed and rejected according to claims 25, 24 above (Laaksonen, the smartphone embedded with encoder and decoder, and discussed in claim 24 above).
Claim 30 has been analyzed and rejected according to claims 29, 17 above.
Claim 31 has been analyzed and rejected according to claims 29, 18 above.
Claim 32 has been analyzed and rejected according to claims 29, 19 above.
Claim 33 has been analyzed and rejected according to claims 29, 20 above.
Claim 34 has been analyzed and rejected according to claims 29, 21 above.
Claim 35 has been analyzed and rejected according to claims 29, 22 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LESHUI ZHANG whose telephone number is (571)270-5589. The examiner can normally be reached Monday-Friday 6:30amp-4:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vivian Chin can be reached at 571-272-7848. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LESHUI ZHANG/
Primary Examiner,
Art Unit 2695
Prosecution Insights