DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
IDS received on 08/05/2024 has be considered.
Priority
Foreign Priority documents retrieval on 04/03/2025 was unsuccessful. Applicant is requested to submit the documents.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over Salehin et al. (US 2021/0099825) in view of Laaksonen (US 2024/0071394).
Claim 1
Salehin teaches a method of playing a sound source, the method comprising:
identifying a playback area for at least one higher order ambisonic (HOA) sound source existing in a target space ([0086] As such, the renderer generation unit 32 may obtain the indication of a boundary separating an interior area from an exterior area. The renderer generation unit 32 may also obtain the listener location 17 indicating of a location of the content consumer device 14 relative to the interior area. See HOA renderer 922 of audio playback device in Fig. 6F for example, and [0122]);
determining a coordinate calculation standard for playing the HOA sound source based on a positional relationship between the playback area of the HOA sound source and a listener existing in the target space ([0085] The audio playback system 16A may also interface with the tracking device 306 to obtain the listener position 17, where the boundary, the listener position 17, and the indications 31 are provided to the renderer generation unit 32; [0088] Determining whether to configure the current renderer 22 as either an interior rendering or an exterior render may depend on where the content consumer device 14 resides relative to the boundary in the XR scene. For example, when the content consumer device 14 is, in the XR scene and per the listener location 17, outside of the interior area defined by the boundary, the renderer generation unit 32 may configure the current renderer 22 to operate as an exterior renderer. When the content consumer device 14 is, in the XR scene and per the listener location 17, inside of the interior area defined by the boundary, the renderer generation unit 32 may configure the current renderer 22 to operate as an interior renderer. Examiner notes based on location of listener position with respect to the geometric boundary, calculation for rendering HOA components changes.); and
calculating a head related transfer function (HRTF) rendering angle of the HOA sound source according to the determined coordinate calculation standard and playing the HOA sound source ([0060] When outputting the speaker feeds 25 to headphones, the audio playback system 16A may utilize one of the renderers 22 that provides for binaural rendering using head-related transfer functions (HRTF) or other functions capable of rendering to left and right speaker feeds 25 for headphone speaker playback. [0085], The audio playback system 16A may also interface with the tracking device 306 to obtain the listener position 17, where the boundary, the listener position 17, and the indications 31 are provided to the renderer generation unit 32; [0107], a binaural renderer 102 capable of performing binaural rendering using one or more HRTFs or the other functions capable of rendering to left and right speaker feeds 103; [0118], Each of audio object renderer 912, 3DOF audio renderer 914, and 6DOF audio renderer 22B may receive listener position 17 and HRTF 23. In this example, the output of audio object renderer 912, 3DOF audio renderer 914, or the output of 6DOF audio renderer may be sent to binauralizer 916 which may perform a binaural rendering. In some examples, each of audio object renderer 912, 3DOF audio renderer 914, and 6DOF audio renderer 22B may output ambisonics. ).
Still Salehin may not clearly detail the coordinate calculation standard.
Laaksonen teaches [0130] 1. Global orientation. This is shown in FIG. 4a by references 441 and 443 and can be representative of the world coordinate system or any service high-level coordinate system that can be considered for the placement and orientation of content. For example, it could be combined inputs (e.g., audio streams) from various geographical or user locations or users based on their GPS location data and orientation or to achieve a specific virtual constellation based on the combined inputs. It is understood a mapping from the GPS location to a global orientation would be performed for the placement in the virtual environment. [0133] 4. Capturing user orientation. This is shown in FIG. 4a by reference 411 and 413 and represents the orientation of the user relative to the audio scene. While in some cases the capturing user orientation is of no interest for the scene understanding and rendering, in others it can be of great interest. For example, in some implementations of a UE spatial capture, the capturing user orientation may be indicative of whether capture device orientation is part of the scene interpretation or “accidental”. It can be noted that for head-worn AR device spatial capture, the capturing user orientation and the capture device orientation are typically the same (at least for current device form factors). Furthermore, capturing user orientation may be disconnected of the device orientation in some capture modes. User orientation can also be of interest for 6DoF scene rendering, where a virtual user (avatar) orientation may be based on the real capturing user orientation. One potential such system is, e.g., Social VR in the scope of MPEG-I 6DoF Audio.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate enhanced orientation signaling for immersive communications as taught by Laaksonen with the rendering control system of Salehin because doing so would have provided signaling of capture device orientation to allow for rendering orientation adaptation within a signaling framework for controlling the full freedom of orientation change between capture and user-controlled presentation ([0113] of Laaksonen).
Claim 2
Salehin teaches the method of claim 1, wherein the determining of the coordinate calculation standard comprises, when the listener is located within the playback area of the HOA sound source, determining a reference coordinate system based on a frontal direction in which the listener looks forward as the coordinate calculation standard for playing the HOA sound source ([0102] Salehin, When the operator 202 moves fully within interior area 206, the renderer generation unit 32 may generate the current renderer 22 to render normally. That is, the renderer generation unit 32 may generate the current renderer 22 to fully render the ambisonic audio data 15 residing within the interior area 206 using all of the ambisonic coefficients of the ambisonic audio data 15 and in a manner that correctly places each of the audio sources in the soundfield (e.g., not locating all of the sources in the same location, such as the center 210 of the interior area 206 shown in the example of FIG. 2); See also for example 461 Fig. 4c of Laaksonen).
Claim 3
Salehin of the combination teaches the method of claim 1, wherein the determining of the coordinate calculation standard comprises, when the listener is located outside the playback area of the HOA sound source, determining a global coordinate system as the coordinate calculation standard for playing the HOA sound source ([0100] Salehin, When configuring the current renderer 22 for external rendering using high complexity, the renderer generation unit 32 may configure, in all cases (e.g., opaque or not opaque), the current renderer 22 to render the ambisonic audio data 15 such that the soundfield represented by the ambisonic audio data 15 is spread out depending on a distance between the listener location 17 and the boundary 204. The spread out degree is denoted as theta (θ) degrees in the example of FIG. 1. The distance is shown by the two dashed lines 226A and 226B, resulting in the theta degrees of spread. See also for example global orientation 465 Fig. 4c of Laaksonen).
Claim 4
Salehin of the combination teaches the method of claim 1, wherein the calculating of the HRTF rendering angle and the playing of the HOA sound source comprise, when a reference coordinate system based on a frontal direction in which the listener looks forward is determined as the coordinate calculation standard for playing the HOA sound source, playing the HOA sound source in a predetermined direction regardless of movement of the listener ([0102] Salehin, When the operator 202 moves fully within interior area 206, the renderer generation unit 32 may generate the current renderer 22 to render normally. That is, the renderer generation unit 32 may generate the current renderer 22 to fully render the ambisonic audio data 15 residing within the interior area 206 using all of the ambisonic coefficients of the ambisonic audio data 15 and in a manner that correctly places each of the audio sources in the soundfield (e.g., not locating all of the sources in the same location, such as the center 210 of the interior area 206 shown in the example of FIG. 2).).
Claim 5
Salehin of the combination teaches the method of claim 1, wherein the calculating of the HRTF rendering angle and the playing of the HOA sound source comprise, when a global coordinate system is determined as the coordinate calculation standard for playing the HOA sound source, playing the HOA sound source in different directions according to movement of the listener ([0100] Salehin, the current renderer 22 to render the ambisonic audio data 15 such that the soundfield represented by the ambisonic audio data 15 is spread out depending on a distance between the listener location 17 and the boundary 204. The spread out degree is denoted as theta (θ) degrees in the example of FIG. 1. The distance is shown by the two dashed lines 226A and 226B, resulting in the theta degrees of spread).
Claim 6
Salehin of the combination teaches the method of claim 5, wherein the calculating of the HRTF rendering angle and the playing of the HOA sound source comprise, when the global coordinate system is determined as the coordinate calculation standard for playing the HOA sound source, mapping the HOA sound source into a vertical channel and a horizontal channel according to a relative area with respect to the listener and playing the HOA sound source ([0030] Salehin, Further, it can be shown (since the above is a linear and orthogonal decomposition) that the A.sub.n.sup.m(k) coefficients for each object are additive. In this manner, a number of PCM objects can be represented by the A.sub.n.sup.m(k) coefficients (e.g., as a sum of the coefficient vectors for the individual objects). The coefficients may contain information about the soundfield (the pressure as a function of 3D coordinates), and the above represents the transformation from individual objects to a representation of the overall soundfield, in the vicinity of the observation point {r.sub.r, θ.sub.r, φ.sub.r}. [0031], For example, ambisonic coefficients may represent a soundfield in three dimensions in a manner that potentially enables accurate three-dimensional (3D) localization of sound sources within the soundfield. As such, XR devices may render the ambisonic coefficients to speaker feeds that, when played via one or more speakers, accurately reproduce the soundfield. [0039], Although described primarily with respect to the ambisonic coefficients 11, the techniques may be performed with respect to other types of audio streams, including pulse code modulated (PCM) audio streams, channel-based audio streams, object-based audio streams, etc.).
Claim 7
Salehin teaches a method of playing a sound source, the method comprising: when a listener is located within a playback area of a higher order ambisonic (HOA) sound source (See 202 within interior area in Fig. 3 Salehin), calculating a head related transfer function (HRTF) rendering angle of the HOA sound source according to a first coordinate calculation standard and playing the HOA sound source ([0088] Salehin, Determining whether to configure the current renderer 22 as either an interior rendering or an exterior render may depend on where the content consumer device 14 resides relative to the boundary in the XR scene; [0107], a binaural renderer 102 capable of performing binaural rendering using one or more HRTFs or the other functions capable of rendering to left and right speaker feeds 103; [0118], Each of audio object renderer 912, 3DOF audio renderer 914, and 6DOF audio renderer 22B may receive listener position 17 and HRTF 23. In this example, the output of audio object renderer 912, 3DOF audio renderer 914, or the output of 6DOF audio renderer may be sent to binauralizer 916 which may perform a binaural rendering. In some examples, each of audio object renderer 912, 3DOF audio renderer 914, and 6DOF audio renderer 22B may output ambisonics.); and
when the listener is located outside the playback area of the HOA sound source, calculating an HRTF rendering angle of the HOA sound source according to a second coordinate calculation standard and playing the HOA sound source (See 202 outside of interior area 206 in Fig. 3 of Salehin),
wherein the first coordinate calculation standard and the second coordinate calculation standard include different coordinate systems ([0088] Salehin, For example, when the content consumer device 14 is, in the XR scene and per the listener location 17, outside of the interior area defined by the boundary, the renderer generation unit 32 may configure the current renderer 22 to operate as an exterior renderer. When the content consumer device 14 is, in the XR scene and per the listener location 17, inside of the interior area defined by the boundary, the renderer generation unit 32 may configure the current renderer 22 to operate as an interior renderer.).
Still Salehin may not clearly detail the coordinate calculation standard.
Laaksonen teaches [0130] 1. Global orientation. This is shown in FIG. 4a by references 441 and 443 and can be representative of the world coordinate system or any service high-level coordinate system that can be considered for the placement and orientation of content. For example, it could be combined inputs (e.g., audio streams) from various geographical or user locations or users based on their GPS location data and orientation or to achieve a specific virtual constellation based on the combined inputs. It is understood a mapping from the GPS location to a global orientation would be performed for the placement in the virtual environment. [0133] 4. Capturing user orientation. This is shown in FIG. 4a by reference 411 and 413 and represents the orientation of the user relative to the audio scene. While in some cases the capturing user orientation is of no interest for the scene understanding and rendering, in others it can be of great interest. For example, in some implementations of a UE spatial capture, the capturing user orientation may be indicative of whether capture device orientation is part of the scene interpretation or “accidental”. It can be noted that for head-worn AR device spatial capture, the capturing user orientation and the capture device orientation are typically the same (at least for current device form factors). Furthermore, capturing user orientation may be disconnected of the device orientation in some capture modes. User orientation can also be of interest for 6DoF scene rendering, where a virtual user (avatar) orientation may be based on the real capturing user orientation. One potential such system is, e.g., Social VR in the scope of MPEG-I 6DoF Audio.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate enhanced orientation signaling for immersive communications as taught by Laaksonen with the rendering control system of Salehin because doing so would have provided signaling of capture device orientation to allow for rendering orientation adaptation within a signaling framework for controlling the full freedom of orientation change between capture and user-controlled presentation ([0113] of Laaksonen).
Claim 8
Salehin of the combination teaches the method of claim 7, wherein the first coordinate calculation standard includes a reference coordinate system based on a frontal direction in which the listener looks forward ([0102] Salehin, When the operator 202 moves fully within interior area 206, the renderer generation unit 32 may generate the current renderer 22 to render normally. That is, the renderer generation unit 32 may generate the current renderer 22 to fully render the ambisonic audio data 15 residing within the interior area 206 using all of the ambisonic coefficients of the ambisonic audio data 15 and in a manner that correctly places each of the audio sources in the soundfield (e.g., not locating all of the sources in the same location, such as the center 210 of the interior area 206 shown in the example of FIG. 2). Examiner notes the coordinate system as used in the claim appears to carry a different meaning from a typical “coordinate system” like cartesian or polar coordinates system, but the claim does not provide the specifics.).
Claim 9
Salehin of the combination teaches the method of claim 7, wherein the second coordinate calculation standard includes a global coordinate system ([0100] Salehin, When configuring the current renderer 22 for external rendering using high complexity, the renderer generation unit 32 may configure, in all cases (e.g., opaque or not opaque), the current renderer 22 to render the ambisonic audio data 15 such that the soundfield represented by the ambisonic audio data 15 is spread out depending on a distance between the listener location 17 and the boundary 204. The spread out degree is denoted as theta (θ) degrees in the example of FIG. 1. The distance is shown by the two dashed lines 226A and 226B, resulting in the theta degrees of spread. Examiner notes claim does not further describe the specifics of the “global coordinate system”.).
Claim 10
This claim recites substantially the same limitations as those provided in claim 1 above, and therefore it is rejected for the same reasons.
Claims 11-16
These claims recite substantially the same limitations as those provided in claims 1-6 above, and therefore they are rejected for the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS H MAUNG whose telephone number is (571)270-5690. The examiner can normally be reached Monday-Friday, 9am-6pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carolyn R. Edwards can be reached at 1-(571) 2707136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS H MAUNG/Primary Examiner, Art Unit 2692
/CAROLYN R EDWARDS/Supervisory Patent Examiner, Art Unit 2692