DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-14, 16-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-8, 14, 16 of U.S. Patent No. 11,381,797 (herein, ‘797) in view of US 2021/0004201 by Munoz et al. and US 2021/0247668 by VanBlon et al.
Regarding claim 1 of instant application
Claim 1 of instant application
Claim 1 of ‘797
A method comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A method comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
determining a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content, the temporal relationship determined based on the context; and
presenting the visual content with the selected audio characteristics.
presenting synthesized audio-visual content based on the temporal relationship.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 1 of instant application in comparison to the limitation as recited in claim 1 of ‘797.
However, claim 1 of ‘797 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 1 of ‘797, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 1 of ‘797 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 1 of ‘797 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Claim 2 of the instant application corresponds to claim 8 of ‘797 Patent.
Claim 3 of the instant application corresponds to claim 16 of ‘797 Patent.
Claim 4 of the instant application corresponds to claim 8 of ‘797 Patent.
Claim 5 of the instant application corresponds to claim 16 of ‘797 Patent.
Claim 6 of the instant application corresponds to claim 2 of ‘797 Patent.
Claim 7 of the instant application corresponds to claim 3 of ‘797 Patent.
Claim 8 of the instant application corresponds to claim 4 of ‘797 Patent.
Claim 9 of the instant application corresponds to claim 5 of ‘797 Patent.
Claim 10 of the instant application corresponds to claim 6 of ‘797 Patent.
Claim 11 of the instant application corresponds to claim 7 of ‘797 Patent.
Claim 12 of the instant application corresponds to claim 8 of ‘797 Patent.
Claim 13 of the instant application corresponds to claim 8 of ‘797 Patent.
Claim 14 of the instant application corresponds to claim 8 of ‘797 Patent.
Claim 16 of the instant application corresponds to claim 16 of ‘797 Patent.
Claim 17 of the instant application corresponds to claim 14 of ‘797 Patent.
Claim 18 of the instant application corresponds to claim 16 of ‘797 Patent.
Regarding claim 19 of instant application
Claim 19 of instant application
Claim 19 of ‘797
A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
determining a context for presenting the audio-visual content, wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
determining a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content, the temporal relationship determined based on the context; and
presenting the visual content with the selected audio characteristics.
presenting synthesized audio-visual content based on the temporal relationship.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 19 of instant application in comparison to the limitation as recited in claim 19 of ‘797.
However, claim 19 of ‘797 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 19 of ‘797, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 19 of ‘797 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 19 of ‘797 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Regarding claim 20 of instant application
Claim 20 of instant application
Claim 20 of ‘797
A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
determining a context for presenting the audio-visual content, wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
determining a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content, the temporal relationship determined based on the context; and
presenting the visual content with the selected audio characteristics.
presenting synthesized audio-visual content based on the temporal relationship.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 20 of instant application in comparison to the limitation as recited in claim 20 of ‘797.
However, claim 20 of ‘797 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 19 of ‘797, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 20 of ‘797 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 20 of ‘797 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Claims 1-14, 16-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-8, 11-13 of U.S. Patent No. 11,729,363 (herein, ‘363) in view of US 2021/0004201 by Munoz et al. and US 2021/0247668 by VanBlon et al.
Regarding claim 1 of instant application
Claim 1 of instant application
Claim 1 of ‘363
A method comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A method comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
in accordance with a first context, presenting a first audio loop and a static representation of the visual content; and
presenting the visual content with the selected audio characteristics.
in accordance with a second context, presenting a second audio loop and a looping representation of the visual content.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 1 of instant application in comparison to the limitation as recited in claim 1 of ‘363.
However, claim 1 of ‘363 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 1 of ‘363, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 1 of ‘363 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 1 of ‘363 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Claim 2 of the instant application corresponds to claim 2 of ‘363 Patent.
Claim 3 of the instant application corresponds to claim 1 of ‘363 Patent.
Claim 4 of the instant application corresponds to claim 3 of ‘363 Patent.
Claim 5 of the instant application corresponds to claim 3 of ‘363 Patent.
Claim 6 of the instant application corresponds to claim 2 of ‘363 Patent.
Claim 7 of the instant application corresponds to claim 3 of ‘363 Patent.
Claim 8 of the instant application corresponds to claim 4 of ‘363 Patent.
Claim 9 of the instant application corresponds to claim 5 of ‘363 Patent.
Claim 10 of the instant application corresponds to claim 6 of ‘363 Patent.
Claim 11 of the instant application corresponds to claim 7 of ‘363 Patent.
Claim 12 of the instant application corresponds to claim 8 of ‘363 Patent.
Claim 13 of the instant application corresponds to claim 8 of ‘363 Patent.
Claim 14 of the instant application corresponds to claim 8 of ‘363 Patent.
Claim 16 of the instant application corresponds to claim 11 of ‘363 Patent.
Claim 17 of the instant application corresponds to claim 11 of ‘363 Patent.
Claim 18 of the instant application corresponds to claim 7 of ‘363 Patent.
Regarding claim 19 of instant application
Claim 19 of instant application
Claim 12 of ‘363
A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
determining a context for presenting the audio-visual content, wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
in accordance with a first context, presenting a first audio loop and a static representation of the visual content; and
presenting the visual content with the selected audio characteristics.
in accordance with a second context, presenting a second audio loop and a looping representation of the visual content.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 19 of instant application in comparison to the limitation as recited in claim 12 of ‘363.
However, claim 12 of ‘363 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 12 of ‘363, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 12 of ‘363 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 12 of ‘363 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Regarding claim 20 of instant application
Claim 20 of instant application
Claim 13 of ‘363
A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio-visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content;
determining a context for presenting the audio-visual content;
determining a context for presenting the audio-visual content, wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context; and
in accordance with a first context, presenting a first audio loop and a static representation of the visual content; and
presenting the visual content with the selected audio characteristics.
in accordance with a second context, presenting a second audio loop and a looping representation of the visual content.
It should be noted that the table above distinguishes the equivalent limitations as recited claim 20 of instant application in comparison to the limitation as recited in claim 13 of ‘363.
However, claim 13 of ‘363 fails to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual context.
Munoz et al. teaches based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include selecting an audio characteristic for presenting the plurality of audio portions with the visual content, as taught by Munoz et al. into claim 13 of ‘363, because such incorporation would allow for the benefit of accurately performing playback of the content using selection, thus increase user accessibility of the system.
However, claim 13 of ‘363 and Munoz et al. fail to teach wherein determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user;
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into claim 13 of ‘363 and Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 19, 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-14, 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0004201 by Munoz et al. in view of US 2021/0247668 by VanBlon et al.
Regarding claim 1, Munoz et al. disclose a method comprising:
at an electronic device having a processor:
obtaining audio-visual content of a physical environment, wherein the audio- visual content comprises visual content and audio content comprising a plurality of audio portions corresponding to the visual content (paragraph 0068 teaches “Video, audio, and other sensory data may play important roles in the VR experience. To participate in a VR experience, the user 402 may wear the VR device 400 (which may also be referred to as a VR client device 400) or other wearable electronic device…The displayed world may refer to a virtual world (in which all of the world is simulated), an augmented world (in which portions of the world are augmented by virtual objects), or a physical world (in which a real world image is virtually navigated)..”, paragraph 0044 teaches “The content capture device 20 or the content editing device 22 may, in some examples, be configured to wirelessly communicate with the soundfield representation generator 24. In some examples, the content capture device 20 or the content editing device 22 may communicate, via one or both of a wireless connection or a wired connection, with the soundfield representation generator 24. Via the connection between the content capture device 20 or the content editing device 22 and the soundfield representation generator 24, the content capture device 20 or the content editing device 22 may provide content in various forms of content, which, for purposes of discussion, are described herein as being portions of the audio data 19.”, 0143, 0148-0149);
determining a context for presenting the audio-visual content (in addition to discussion above, paragraph 0034 teaches “While described in the context of the source device 12A and the content consumer device 14A, the techniques may be implemented in any context in which any representation of a soundfield is encoded to form a bitstream representative of the audio data. Moreover, the source device 12A may represent any form of computing device capable of generating the representation of a soundfield, and is generally described herein in the context of being a VR content creator device. Likewise, the content consumer device 14A may represent any form of computing device capable of implementing rendering metadata techniques described in this disclosure as well as audio playback, and is generally described herein in the context of being a VR client device.”, paragraph 0051, 0070-0071, 0077-0079), wherein determining the context comprises:
based on the context, selecting an audio characteristic for presenting the plurality of audio portions with the visual content (in addition to discussion above, paragraph 0079-0081 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data……the audio decoding device 34 may adaptively select between audio streams available via the bitstream 27 (which are represented by the bitstream 27 and hence the bitstream 27 may be referred to as “audio streams 27”). The audio decoding device 34 may select between different audio streams of the audio streams 27 based on audio location information (ALI) (e.g., 45A in FIGS. 1A-1C), such as capture location information or location information relating to a synthesized audio source included as metadata accompanying the audio streams 27, where the audio location information may define coordinates in the displayed world for the microphones that capture the respective audio streams 27 or coordinates in an acoustical space. The ALI 45A may be representative of a capture location (or synthesize location) in a displayed world (or an acoustical space) at which the corresponding one of the audio streams 27 was captured or synthesized. The audio decoding device 34 may select, based on the ALI 45A, a subset of the audio streams 27, where the subset of the audio streams 27 excludes at least one of the audio streams 27. The audio decoding device 34 may output the subset of audio streams 27 as audio data 19′ (which may also be referred to as “audio streams 19′”). In some examples, the audio decoding device 34 may only decode the subset of the audio streams in response to the selection.”, in addition to discussion above, paragraph 0033 that “While described in this disclosure with respect to the VR device, various aspects of the techniques may be performed in the context of other devices, such as a mobile device. In this instance, the mobile device (such as a so-called smartphone) may present the displayed world via a screen, which may be mounted to the head of the user or viewed as would be done when normally using the mobile device. As such, any information on the screen can be part of the mobile device. The mobile device may be able to provide tracking information and thereby allow for both a VR experience (when head mounted) and a normal experience to view the displayed world, where the normal experience may still allow the user to view the displayed world providing a VR-lite-type experience (e.g., holding up the device and rotating or translating the device to view different portions of the displayed world).” Fig. 5D, paragraph 0138-0139 teaches “For example, if listener wants to hear the performers close up (e.g., near stage 444), but wants to see more of the stage (e.g., a wider view than a location closer to the stage) such that position 430 of the listener which is further away from the stage, then the listener may bias the audio source distance threshold of the snapping towards stage audio element S.sub.2 448 instead of closest audio element R.sub.2 456 (distance a>distance b). In some examples, with this bias towards the stage audio elements, the listener stays snapped to S.sub.2 448 as they move towards position 432. At position 432 the listener may snap to audio element S.sub.3 450 because the listener's distance (c) to audio element S.sub.3 450 is less than the listener's distance (d) to audio element S.sub.2 448. Without the bias, the listener would have snapped to audio elements R.sub.2 456 and R.sub.3 458 as the listener moved from position 430 to position 432.”); and
presenting the visual content with the selected audio characteristics (in addition to discussion above, paragraph 0118 teaches “The one or more processors of the content consumer device may select a subset of the plurality of audio streams based on the device location information and the audio location information (880). For example, the one or more processors of the content consumer device 14 may determine a proximity distance based on the device location information and the audio location information for at least one of the plurality of audio streams and select, based on the proximity distance, the subset of the plurality of audio streams. In another example, the one or more processors of the content consumer device 14 may compare the proximity distance to a threshold proximity distance and select, when the proximity distance is less than or equal to a threshold proximity distance, a larger number of the plurality of audio streams compared to when the proximity distance is greater than the threshold proximity distance to obtain the subset of the plurality of audio streams. In another example, the one or more processors of the content consumer device 14 may compare the proximity distance to a threshold proximity distance and select, when the proximity distance is greater than the threshold proximity distance, a smaller number of the plurality of audio streams compared to when the proximity distance is less than or equal to the threshold proximity distance to obtain the subset of the plurality of audio streams.”, paragraph 0148-0149 teaches “In the example of FIG. 7A, the content consumer device 14C also includes the VR device 400. It will be understood that, in various implementations, the VR device 400 may be included in, or externally coupled to, the content consumer device 14C. The VR device 400 includes display hardware and speaker hardware for outputting video data (e.g., as associated with various viewports) and for rendering audio data.”).
Munoz et al. fails to disclose determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user.
VanBlon et al. discloses determining the context comprises determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user (paragraph 0029, 0039, 0044 teaches “In some embodiments, the projection management apparatus 104 determines whether a predetermined condition is satisfied for projecting audiovisual content to a user in a selected viewing context. In various embodiments, the projection management apparatus communicates with at least one sensor 108 to determine one or more parameters of a field of view of the user for the selected viewing context and one or more characteristics of a target viewing surface within the field of view.”, paragraph 0067 teaches “In certain embodiments, the content selection module 312 receives a content selection command from the user 208, for example via the I/O interface 310, such as a voice command or a gesture that is detectable by the at least one sensor 108. Such content selection commands from the user 208 beneficially allow a user to continue performing manual tasks with the benefit of guidance provided by the projected audiovisual content without having to manually input commands to a cell phone, tablet or computer.”, paragraph 0052, 0072-0073)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to determining that a user has selected the visual content via user input or that the user is looking at the visual content based on a gaze of the user, as taught by VanBlon et al., into the system of Munoz et al., because such incorporation would allow for the benefit of providing more options to a user to have select visual content, thus increase user flexibility of the system.
Regarding claim 2, the method wherein selecting the audio characteristic comprises selecting spatialized audio (in addition to discussion above, Munoz et al., paragraph 0082 teaches “In this respect, the audio decoding device 34 may adaptively select a subset of the audio streams 27 to obtain the audio streams 19′ that may result in a more immersive experience (compared to selecting a single audio stream or all of the audio streams 19′). As such, various aspects of the techniques described in this disclosure may improve operation of the audio decoding device 34 (and the audio playback system 16A and the content consumer device 14) itself by possibly enabling the audio decoding device 34 to better spatialize sound sources within the soundfield and thereby improve immersion.”, paragraph 0096 teaches “In this respect, the stream selection unit 44 may increase audio spatialization accuracy when the listener is at the location 300B. Furthermore, when the listener is at the location 300A, the stream selection unit 44 may reduce a bitrate, as only the audio stream of audio element 302A is used to reproduce the soundfield rather than multiple audio streams of audio elements 302B-302J.”).
Regarding claim 3, the method wherein selecting the audio characteristic comprises selecting a portion of the plurality of audio portions, wherein, for the visual content, different portions of the plurality of audio portions of the audio content are selected for different contexts (as discussed above, Munoz et al., paragraph 0081, 0118 teaches selecting audio characteristic).
Regarding claim 4, the method wherein selecting the audio content comprises selecting a portion of audio of the plurality of audio portions based on metadata for different portions of the plurality of audio portions identifying sources of the different portions (in addition to discussion above, Munoz et al., paragraph 0172-0173 teaches “A device configured to play one or more of a plurality of audio streams, the device comprising: a memory configured to store the plurality of audio streams and corresponding audio metadata, each of the audio streams representative of a soundfield, and the audio metadata includes capture location information representative of capture coordinates in a displayed world at which the corresponding one of the plurality of audio streams was captured; and one or more processors coupled to the memory, and configured to: determine location information representative of a device coordinates of the device in the displayed world; select, based on the location information and the capture location information, a subset of the plurality of audio streams, the subset of the plurality of audio streams excluding at least one of the plurality of audio streams; and output, based on the subset of the plurality of audio streams, one or more speaker feeds…..Example 2A. The device of example 1A, wherein the one or more processors are configured to: determine a proximity distance based on the location information and the capture location information included in the metadata for at least one of the plurality of audio streams; and select, based on the proximity distance, the subset of the plurality of audio streams.”).
Regarding claim 5, the method wherein selecting the audio content comprises selecting a portion of the plurality of audio portions based on metadata for different portions of the plurality of audio portions identifying types of the different portions of audio content (in addition to discussion above, Munoz et al., paragraph 0079-0082 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data.”, paragraph 0167).
Regarding claim 6, the method wherein the plurality of audio portions comprises audio of a user of an audio-visual (AV) capture device, low frequency audio, ambient audio, or a plurality of spatialized audio streams (in addition to discussion above, Munoz et al., paragraph 0072-0073, 0079, 0082, 0096), wherein the visual content comprises at least a 2D image, a 3D image, a 2D sequence of images or a 3D sequence of images, a 3D photo, or a 3D video including corresponding audio (as discussed above, paragraph 0068, 0148-0149).
Regarding claim 7, the method further comprising semantically labelling sections of the plurality of audio portions based on metadata included with the audio-visual content of the physical environment or scene analysis of the corresponding visual content (in addition to discussion above, Munoz et al., paragraph 0034 teaches “While described in the context of the source device 12A and the content consumer device 14A, the techniques may be implemented in any context in which any representation of a soundfield is encoded to form a bitstream representative of the audio data. Moreover, the source device 12A may represent any form of computing device capable of generating the representation of a soundfield, and is generally described herein in the context of being a VR content creator device. Likewise, the content consumer device 14A may represent any form of computing device capable of implementing rendering metadata techniques described in this disclosure as well as audio playback, and is generally described herein in the context of being a VR client device”).
Regarding claim 8, the method wherein the metadata comprises: information related to the AV capture device including pose, movement, sensors, and sensor data of the AV capture device; information related to a user of the AV capture device including gaze, body movement, and operational inputs; information related to an environment of the AV capture device during capture; or information related to a scene or the visual content (in addition to discussion above, Munoz et al., paragraph 0080, 0167 teaches “audio metadata indicating privacy restrictions and/or password-based privacy data to toggle, mask, and/or null various audio streams and/or audio sources”).
Regarding claim 9, the method further comprising semantically labelling at least one section of the plurality of audio portions based on analysis of the audio content, wherein semantically labelling at least one section of the plurality of audio portions is performed by the AV capture device, a processing electronic device, or the electronic device (in addition to discussion above, Munoz et al., paragraph 0056-0060, paragraph 0079 teaches “In the context of streaming application (live or recorded), there may be a large number of audio streams associated with varying levels of quality and/or content. The audio streams may represent any type of audio data, including scene-based audio data (e.g., ambisonic audio data, including FOA audio data, MOA audio data and/or HOA audio data), channel-based audio data, and object-based audio data.”, 0095).
Regarding claim 10, the method wherein the audio content is decoupled from the visual content (in addition to discussion above, Munoz et al., fig. 1 shows audio content is decoupled from video content).
Regarding claim 11, the method wherein determining the context for presenting the audio- visual content is based on actions of a user in an extended reality (XR) environment including a representation of the audio-visual content (in addition to discussion above, Munoz et al., paragraph 0140 teaches “In various examples, the wearable device 500 may represent a VR headset (such as the VR device 400 described above), an AR headset, an MR headset, or any other type of extended reality (XR) headset. Augmented Reality “AR” may refer to computer rendered image or data that is overlaid over the real world where the user is actually located. Mixed Reality “MR” may refer to computer rendered image or data that is world locked to a particular location in the real world, or may refer to a variant on VR in which part computer rendered 3D elements and part photographed real elements are combined into an immersive experience that simulates the user's physical presence in the environment. Extended Reality “XR” may represent a catchall term for VR, AR, and MR. More information regarding terminology for XR can be found in a document by Jason Peterson, entitled “Virtual Reality, Augmented Reality, and Mixed Reality Definitions,” and dated Jul. 7, 2017.”).
Regarding claim 12, the method wherein determining the context for presenting the audio- visual content comprises determining at least whether the audio-visual content is selected based on user actions and determining a spatial distance between the user and a representation of the audio-visual content (in addition to discussion above, Munoz et al., paragraph 0068, 0079-0086, 0118 teaches selection of content).
Regarding claim 13, the method wherein determining the context comprises determining that a user has selected the visual content via the user input (in addition to discussion above, Munoz et al., paragraph 0068, 0079-0086, 0118, 0311 teaches selection of content, paragraph 0047 teaches “The bitstream 27 may represent a compressed version of the audio data 19 and any other different types of the content 21 (such as a compressed version of spherical video data, image data, or text data)”; VanBlon et al., paragraph 0029, 0039, 0044).
The motivation for combining references has been discussed in independent claim above.
Regarding claim 14, the method wherein determining the context comprises determining that the user is looking at the visual content based on the gaze of the user (in addition to discussion above, Munoz et al., paragraph 0068, 0079-0086, 0118, 0311 teaches selection of content; VanBlon et al., paragraph 0031 teaches “In various embodiments, the at least one sensor is selected from the group consisting of a distance sensor, an eye-gaze sensor, an imaging sensor, and combinations thereof.”, paragraph 0052 teaches “In certain embodiments, the apparatus 200 includes at least one sensor 108 the senses one or more parameters of a field of view 218 (such as for example an area surrounding a gaze point 206) of a user 208. In some embodiments, the at least one sensor 108 may include a camera for capturing images of a face 220 and eye(s) 222 of a user 208 for determining the field of view 218, a gaze direction 216, gaze point 206, and/or other similar viewing context parameters related to the user 208.”, paragraph 0066, 0072-0073).
The motivation for combining references has been discussed in independent claim above.
Regarding claim 16, the method wherein determining the context comprises determining that the user is within a threshold distance of the visual content in an extended reality (XR) environment (in addition to discussion above, Munoz et al., paragraph 0090-0091 teaches “The stream selection unit 44 may determine an audio source distance 306A. In some examples, stream selection unit 44 may determine the audio source distance 306A as a distance between an audio source location in the acoustical space or virtual acoustical space and the device coordinates. In some examples, stream selection unit 44 may determine the audio source distance as a distance between the audio stream coordinates and the device coordinates. In some examples, stream selection unit 44 may determine the audio source distance as the distance between an audio source location, e.g., audio source location 304 and the device coordinates, (e.g., virtual location 300A of the VR device 400). The stream selection unit 44 may compare the audio source distance 306A to an audio source distance threshold. The stream selection unit 44 may, in some examples, derive the audio source distance threshold based on the energy of the audio source 308. That is, when the audio source 308 has a higher energy (or, in other words, when the audio source 308 is louder), the stream selection unit 44 may increase the audio source distance threshold. When the audio source 308 has a lower energy (or, in other words, when the audio source 308 is quieter), the stream selection unit 44 may decrease the audio source distance threshold. In other examples, the stream selection unit 44 may obtain a statically defined audio source distance threshold, which may be statically defined or specified by the user 402….In any event, the stream selection unit 44 may select, when the audio source distance 306A is greater than the audio source distance threshold (which is assumed in this example for purposes of illustration), a single audio stream of the audio streams 27 of the audio elements 302A-302J (“audio elements 302”). For example, the stream selection unit 44 may select the audio stream of the audio element (e.g., audio element 302A) having the shortest distance to virtual location 300A. The stream selection unit 44 may output the corresponding one of the audio streams 27, which the audio decoding device 34 may decode and output as the audio data 19′.”, paragraph 0093-0095).
Regarding claim 17, the method wherein determining the context comprises determining that the user is more than a threshold distance from the visual content in an extended reality (XR) environment (in addition to discussion above, Munoz et al., paragraph 0090-0091, paragraph 0093-0095 teaches “…The stream selection unit 44 may also determine one or more proximity distances between the virtual location 300A and one or more (and possibly each) of the capture locations (or synthesize locations) represented by the ALI 45A to obtain one or more proximity distances. The stream selection unit 44 may then compare the one or more proximity distances to a threshold proximity distance. The stream selection unit 44 may select, when the one or more proximity distances are greater than the threshold proximity distance, a smaller number of the audio streams 27 compared to when the one or more proximity distances are less than or equal to the threshold proximity distance to obtain the audio streams 19′. However, the stream selection unit 44 may select, when the one or more of the proximity distances are less than or equal to the threshold proximity distance, a larger number of the audio streams 27 compared to when the one or more proximity distances are greater than the threshold proximity distance to obtain the audio streams 19′….”).
Regarding claim 18, the method wherein determining the context comprises determining that the user is moving towards the visual content in an extended reality (XR) environment (as discussed in claim 16, claim 17 above).
Claim 19 rejected for the same reason as discussed in the corresponding claim 1 above.
Claim 20 rejected for the same reason as discussed in the corresponding claim 1 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIGAR CHOWDHURY whose telephone number is (571)272-8890. The examiner can normally be reached Monday-Friday 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached on 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NIGAR CHOWDHURY/Primary Examiner, Art Unit 2484
Prosecution Insights