Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 7/19/24, 12/23/24 and 1/21/26 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8, 9, 11-17, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mattsson et al., United States Patent Publication 2018/0267691 (hereinafter “Mattsson”), in view of Li et al., United States Patent Publication 2023/0128422 (hereinafter “Li”).
Claim 1:
Mattson discloses:
A method for obtaining an audio signal during a traversal event associated with a user interface (UI), the method comprising:
capturing, during the traversal event indicative of performing navigation on the UI, at least one audio stream associated with content, and at least one UI frame (see paragraph [0096]). Mattson teaches capturing an audio stream and UI frame when the user is interacting with objects;
obtaining a set of audio features based on the at least one audio stream, and obtaining one or more UI elements from a set of images generated based on the at least one UI frame (see paragraph [0065], [0096], [0097]). Mattson teaches obtaining audio features and characteristics from the audio stream and UI elements from the images;
determining an audio gain based on a prioritization and a weighted average of the set of audio features, wherein the audio gain indicates an intensity of the at least one audio associated with the UI (see paragraph [0065]). Mattson teaches determining an audio gain that indicates an intensity of the audio based on a calculated measurement of the of the features;
obtaining an audio map comprising at least one biased audio file, wherein the at least one biased audio file is determined based on the set of audio features (see paragraphs [0049] and [0050]). Mattson teaches obtaining an audio map comprising the audio based on the features and user interface; and
obtaining the audio signal based on a priority order assigned to the at least one biased audio file associated with the audio map, the set of audio features, and the audio gain, such that the audio signal is rendered during the traversal event on the UI (see paragraph [0049] and [0050]). Mattson teaches obtaining the mapped audio signal based on the intensity and selected audio file of a plurality of audio file. The priority is from the user or developers.
Mattson fails to expressly disclose an ambient audio stream and determining the context of the audio stream.
Li discloses:
an ambient audio stream (see paragraphs [0153]). Li teaches receiving an audio input from a user;
determining a UI context based on a UI contextual score of the one or more UI elements, wherein the UI context indicates context of activities during the traversal event in the UI (see paragraph [0177] and [0211])). Li teaches determining the context of the actions of the user interface element during the event and basing the contexted action on a threshold;
obtaining an audio map comprising at least one unbiased audio file, wherein the at least one unbiased audio file is determined based on the UI context (see paragraphs [0176] and [0050]). Li teaches obtaining a mapping based on the context of the voice input. The output can be an audio output based on the context; and
obtaining the audio signal based on a priority order assigned to the at least one unbiased audio file associated with the audio map, the set of audio features, such that the audio signal is rendered during the traversal event on the UI (see paragraph [0049] and [0133]). Li teaches obtaining the audio signal response based on the context, the confidence score of the intent and the mapping.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include an ambient audio signal and determining the context of the audio for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claim 2:
Mattson fails to expressly disclose audio output channel.
Li discloses:
further comprising rendering the audio signal via at least one audio channel on a user device corresponding to the UI (see paragraph [0149]). Li teaches content may include video, audio, haptic feedback, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to the viewer).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include an audio channel for output for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claim 3:
Mattsson discloses:
wherein the set of audio features corresponds to a content gain, a content frequency, and a duration of the at least one audio, based on the at least one audio stream being associated with the content (see paragraph [0065]). Mattson teaches receiving a set of audio features extracted from the audio such as, but not limited to, volume, frequency, directionality, channel etc, and
Mattsson fails to expressly disclose the audio being associated with the content.
Li discloses:
wherein the set of audio features corresponds to an ambience gain and a vocal gain of the at least one audio, based on the at least one audio being associated with the ambient audio stream (see paragraph [0101], [0135]. [0154]). Li teaches the ambient audio corresponds to corresponding content based on intent and features.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include extracting features from the audio signal for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claim 4:
Mattson discloses:
wherein the one or more UI elements indicate interactive components of the UI, and wherein the interactive components comprise at least one of a text, an object, a hierarchal tree, a logo, and an icon (see paragraph [0038]). Mattson teaches interactive user interface elements.
Claim 5:
Mattsson discloses:
obtaining a set of audio features based on the at least one audio stream (see paragraph [0065], [0096], [0097]). Mattson teaches obtaining audio features and characteristics from the audio stream and UI elements from the images;
determining a gain value corresponding to the set of audio features (see paragraph [0065]). Mattson teaches determining an audio gain that indicates an intensity of the audio based on a calculated measurement of the of the features;
obtaining the audio signal based on a priority order assigned to the at least one biased audio file associated with the audio map, the set of audio features, and the audio gain, such that the audio signal is rendered during the traversal event on the UI (see paragraph [0049] and [0050]). Mattson teaches obtaining the mapped audio signal based on the intensity and selected audio file of a plurality of audio file. The priority is from the user or developers .
determining the audio gain based on the prioritization and the weighted average such that the change in volume between the audio signal to be generated for the UI and the at least one audio associated with the content being played in the background, is performed at a rate below a change rate threshold (see paragraphs [0065], [0068], [0071]). Mattson teaches determining an audio gain that indicates an intensity of the audio based on a calculated measurement of the of the features. The changes in volume are based on the user interaction and played in the background.
Mattsson fails to teach the ambient audio stream.
Li discloses:
obtaining the set of audio features from the ambient audio stream (see paragraph [0071]). Li teaches processing the ambient audio by extract features from the voice command.
obtaining a user voice input (see paragraph [0071]). Li teaches receiving voice input;
obtaining a weight associated with each of the set of audio features based on a pre-stored weight values (see paragraph [0135]). Li teaches obtaining weights associated with features extracted from the audio.
determining the weighted average corresponding to the set of audio features based on a smoothing factor and the weight associated with each audio feature of the set of audio features (see paragraph [0071] and [0135]). Li teaches
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include extracting features from the audio signal for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claim 6:
Mattson discloses:
performing an element filtering based on the one or more UI elements, wherein the element filtering comprises identifying each of the one or more UI elements separated based on respective positions within the at least one UI frame (see paragraph [0039], [0041] and [0059]). Mattson teaches performing element filtering based on the position of the element in the user interface;
Mattson fails to expressly disclose determining UI context based on score.
Li discloses:
determining a bias score associated with each of the one or more UI elements, wherein the bias score indicates a probability of each of the one or more UI elements matching with a context corresponding to the UI (see paragraph [0107]). Li teaches determining a probability of each UI elements matching the context/intent corresponding to the user interface;
determining a contextual classification of an application corresponding to the UI, wherein the contextual classification indicates a category of the application determined using a pre-trained language machine learning model (see paragraph [0107]). Li teaches determining a contextual classification indicating a category using a pre-trained machine learning model;
obtaining the UI contextual score based on the bias score and the contextual classification for each of the one or more UI elements (see paragraph [0107]). Li teaches calculate a probability of the input being associated with a particular predefined intent; and
determining the UI context based on the UI contextual score (see paragraph [0107]). Li teaches determining the context based on the score.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include a method for determining intent for the purpose of efficiently determining correct intent of the input, as taught by Li.
Claim 8:
Mattson fails to expressly disclose determining UI context based on score.
Li discloses:
wherein determining the at least one unbiased audio file based on the UI context comprises:
obtaining the UI contextual score for each of the one or more UI elements (see paragraph [0107]). Li teaches calculate a probability of the input being associated with a particular predefined intent; and
determining the at least one unbiased audio file using a recurring neural network based on the at least one audio stream associated with the content and a corresponding contextual audio indicating a semantic content of the at least one audio (see paragraph [0108]). Li teaches determining the content to be output based on a recurring neural network based on the audio stream and context.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include a method for determining intent for the purpose of efficiently determining correct intent of the input, as taught by Li.
Claim 9:
Mattson discloses:
at least one biased audio file (see paragraphs [0049] and [0050]). Mattson teaches obtaining an audio map comprising the audio based on the features and user interface; and
Mattson fails to expressly disclose an ambient audio stream and determining the context of the audio stream.
Li discloses:
aggregating the at least one biased audio file and the at least one unbiased audio file to obtain the audio map, wherein the audio map indicates a sequence of the at least one biased audio file and the at least one unbiased audio file along with associated audio-type scores (see paragraph [0108] and [0115]). Li teaches aggregating the data associate with the user interface and context to determine the audio map and associated data.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include an ambient audio signal and determining the context of the audio for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claim 11:
Mattsson discloses:
capturing, while a user device corresponding to the UI is operated, the at least one audio stream associated with a physical event (see paragraph [0096]). Mattson teaches capturing an audio stream when the user is interacting with objects;
extracting the set of audio features based on the at least one audio stream, (see paragraph [0065], [0096], [0097]). Mattson teaches obtaining audio features and characteristics from the audio stream and UI elements from the images;
determining an audio gain based on a prioritization and a weighted average of the set of audio features, (see paragraph [0065]). Mattson teaches determining an audio gain that indicates an intensity of the audio based on a calculated measurement of the of the features;
Mattson fails to expressly disclose an ambient audio stream and determining the context of the audio stream.
Li discloses:
capturing, while a user device corresponding to the UI is operated the ambient audio stream for the physical event (see paragraphs [0153]). Li teaches receiving an audio input from a user;
determining an event context based on the physical event (see paragraph [0177] and [0211])). Li teaches determining the context of the actions of the user interface element during the event and basing the contexted action on a threshold;
obtaining the audio signal based on the event context and the audio gain such that the generated audio signal corresponds to the physical event (see paragraph [0049] and [0133]). Li teaches obtaining the audio signal response based on the context, the confidence score of the intent and the mapping.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson to include an ambient audio signal and determining the context of the audio for the purpose of providing efficient voice control for user interfaces, as taught by Li.
Claims 12-17, 19:
Although Claims 12-17 and 19 are device claims, they are interpreted and rejected for the same reasons as the method of Claims 1-6, 8, respectively.
Claim 20:
Although Claim 20 is a device claim, it is interpreted and rejected for the same reasons as the method of Claim 1.
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mattsson, in view of Li, in further view Elyada et al., United States Patent Publication 20230153061 (hereinafter “Elyada”).
Claim 7:
Mattson and Li fail to expressly disclose scaling and modulating the audio.
Elyada discloses:
scaling a pre-generated audio based on the audio gain; and modulating the pre-generated audio based on the set of audio features to determine the at least one biased audio file, wherein the at least one biased audio file aligns with the UI context (see paragraphs [0031] and [0032]). Elyada teaches scaling and modulating the audio stream to only audio related to the context of the user interface element.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson and Li to include scaling and modulating the audio signal for the purpose of performing actions based on ambient sounds, as taught by Elyada.
Claim 18:
Although Claim 18 is a device claim, it is interpreted and rejected for the same reasons as the method of Claim 7.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Mattsson, in view of Li and Elyada, in further view Gulcehre et al., United States Patent Publication 20250036958 (hereinafter “Gulcehre”).
Claim 10:
Mattson discloses:
obtaining the priority order of the audio map, the set of audio features, and the audio gain (see paragraph [0049] and [0050]). Mattson teaches obtaining the priority based on the audio map, features and based on the intensity of selected audio file of a plurality of audio files. The priority is from the user or developers.
generating one or more contextual audio files using a diffusion model based on the priority order of the audio map, wherein the one or more contextual audio files indicate (see paragraph [0049] and [0050]). Mattson teaches generating the contextual audio file of a plurality of audio files based on the priority and audio map.
Mattsson fails to expressly disclose aggregating data for best results.
Li discloses:
generating one or more mixed audio files based on mixing the at least one biased audio file of the audio map with the at least one unbiased audio file of the audio map (see paragraph [0097]). Li teaches aggregating results based on the context and associated data and combining the data to give the best result.
obtaining the audio signal based on the one or more audio files and gain (see paragraph [0049] and [0133]). Li teaches obtaining the audio signal response based on the processing methods.
Mattsson and Li fails to expressly disclose generating a context of background audio based on modulation.
Elyada discloses:
generating a context of background audio based on a modulation of the audio map with the one or more contextual audio files (see paragraphs [0028], [0031] and [0032]). Elyada teaches scaling and modulating the audio stream to only audio related to the context of the user interface element.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson and Li to include scaling and modulating the audio signal for the purpose of performing actions based on ambient sounds, as taught by Elyada.
Mattson, Li and Elyada fail to expressly disclose generating an audio using a diffusion model.
Gulcehre discloses:
generating one or more contextual audio files using a diffusion model (see paragraph [0036]). Gulcehre teaches generating a contextual audio file using a diffusional model.
semantic content of the audio map (see paragraphs [0031] and [0032]). Elyada teaches scaling and modulating the audio stream to only audio related to the context of the user interface element.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method by Mattson, Li and Elyada to include generating contextual audio using a diffusion model for the purpose of efficiently performing functions based on context, as taught by Elyada.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIONNA M BURKE whose telephone number is (571)270-7259. The examiner can normally be reached M-F 8a-4p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571)272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIONNA M BURKE/Examiner, Art Unit 2178 6/26/26
Prosecution Insights