DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/5/2026 has been entered.
Election/Restrictions
Applicant’s election without traverse of claims 1-5, 7-10, 13, 15, 22, and 24 in the reply filed on 03/17/2025 is acknowledged.
Claim 23 is withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 3/17/2025.
Claims 11-12 are correctly indicated as withdrawn, because they were indicated as belonging to a non-elected group, or group 3. Group 3, claims 11-12 and 23, are drawn to a method and/or system for generating an audio presentation for a user comprising means to generate a third audio signal based on semantic content of the one or more events and/or data indicative of the one or more events. Claim 23 is incorrectly indicated, where claim 23 should be indicated as a withdrawn claim.
The amendments to claims 1, 4, and 22 are drawn to this non-elected group, or group 3.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-5, 7-10, 13, 15, 22, and 24 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-5, 9-10, 13, 22, and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Norris et al. (US 2017/0359467 A1, previously cited, and hereinafter Norris) in view of Hussain et al. (US 2021/0158814 A1, and hereafter Hussain).
Regarding claim 1, Norris teaches a handheld portable device, such as headphones or earphones, comprising a processor (see Norris, ¶ 0003 and 0020), where one or more sensors are used monitor and/or detect the location or direction of physical environment sound around the listener of the wearable electronic device (see Norris, ¶ 0024, 0034, and 0094-0096, and also see Norris, ¶ 0036, 0038, 0040-0041, 0123, 0127, and 0185-0187).
Norris anticipates:
“A method for generating an audio presentation for a user, comprising:
obtaining, by a portable user device comprising one or more processors, data indicative of an acoustic environment of the user, the acoustic environment of the user comprising at least one of a first audio signal playing on the portable user device or a second audio signal associated with a surrounding environment of the user that is detected via one or more microphones that form part of, or are communicatively coupled with, the portable user device” because the handheld portable device, such as a smartphone coupled to earphones with outward facing microphones, monitors the acoustic environment of the user (see Norris, ¶ 0034, 0093-0095, and 0252-0253, figure 3, step 300, and figure 10, units 1002 and 1003); [and]
“obtaining, by the portable user device, data indicative of one or more events, the one or more events comprising at least one of information to be conveyed by the portable user device to the user or at least a portion of the second audio signal associated with the surrounding environment of the user” because the device detects an event, such as a lull in a conversation, causes the device to convey information to the user associated with the audio in the surrounding environment so that the user can distinguish between remote persons and local persons (i.e., people in the same room as the user) talking (see Norris, ¶ 0024, 0076-0081, 0189-0191, 0193, and 0195 and figure 5).
Norris teaches the above features for generating an audio presentation, however Norris does not appear to teach all the features for “generating, by an on-device artificial intelligence system of the portable user device, an audio presentation for the user … , wherein the audio presentation comprises a summary of the one or more events based at least in part on semantic content of the one or more events”.
Hussain teaches dynamic interfacing with applications, such as providing a natural language interface for different applications (see Hussain, abstract). Herein, Hussain teaches a computing device, such as a laptop, tablet, smart phone, etc., that has a data processing system and further has a microphone as part of a user interface (see Hussain, ¶ 0021 and figure 1, units 100, 102, and 128). Hussain teaches that a natural language processing (NLP) component uses a variety of techniques to parse and process an input audio signal to provide an audible output (see Hussain, ¶ 0049, 0051, 0055, and 0062, and figure 1, units 108 and 112). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to modify Norris with the teachings of Hussain for the purpose of providing a digital assistant that outputs an audible summary based on an event, such as responding to queries with informational retrieval (see Norris, ¶ 0266 in view of Hussain, ¶ 0049 and 0062).
Therefore, the combination of Norris and Hussain makes obvious the features:
“generating, by an on-device artificial intelligence system of the portable user device, an audio presentation for the user based at least in part on the data indicative of the one or more events and the data indicative of the acoustic environment of the user, wherein generating the audio presentation comprises determining a particular time to incorporate a third audio signal associated with the one or more events into the acoustic environment, wherein the audio presentation comprises a summary of the one or more events based at least in part on semantic content of the one or more events” because the portable device uses, in part, a SLP (sound localization point) selector (see Norris, ¶ 0259 and figure 11, units 1102 and 1138), where the SLP selector predicts actions desired by the user based on analysis of information such as historical events, user profiles, etc., and the analysis provides probability and/or likelihood that a user would desire the action, such that the SLP selector predicts times, locations, etc. of sound sources or virtual sound sources (see Norris, ¶ 0269), such that the system inserts an alert when there is a lull in the incoming voice and present the audio to the user (see Norris, ¶ 0121, 0195, 0259, 0266, 0269, and 0299 and figures 1-3), where Hussain makes obvious that the audio presentation includes a computer generated voice to provide the audio presentation that comprises these features (Norris, ¶ 0266 in view of Hussain, ¶ 0049, 0051, 0055, and 0062, and figure 1, units 108 and 112); and
“presenting, by portable user device, the audio presentation to the user” because Norris teaches the earphones present the processed audio to the user (see Norris, ¶ 0186-0187 and 0191 and figure 1, steps 110 and 120, figure 2, step 220, and figure 3, step 320).
Regarding claim 2, see the preceding rejection with respect to claim 1 above. The combination makes obvious the “method of claim 1, wherein the audio presentation is presented to the user via one or more wearable speaker devices, and optionally wherein:
the first audio signal is being played to the user via the one or more head mounted speaker devices and/or at least one of the one or more microphones form part of the one or more head mounted speaker devices” because Norris teaches a head-mounted display with speakers to play the voice of the physically present person, where the voice was received by the microphone in the head-mounted display (see Norris, ¶ 0194, 0197-0198, and ¶ 0259, figure 5, units 530, 550, and 562, and figure 11, units 1102, 1128, and 1140).
Regarding claim 3, see the preceding rejection with respect to claim 1 above. The combination makes obvious the “method of claim 1, wherein the one or more wearable speaker devices comprise one or more head-mounted wearable speaker devices” because Norris teaches a binaural output (i.e., at least one speaker for each ear) (see Norris, ¶ 0020-0021).
Regarding claim 4, see the preceding rejection with respect to claim 1 above. The combination makes obvious:
“A method for generating an audio presentation for a user, comprising:
obtaining, by a computing system comprising one or more processors, data indicative of an acoustic environment for the user, the acoustic environment for the user comprising at least one of a first audio signal playing on the computing system or a second audio signal associated with a surrounding environment of the user” because the handheld portable device, such as a smartphone coupled to earphones with outward facing microphones, monitors the acoustic environment of the user (see Norris, ¶ 0034, 0093-0095, and 0252-0253, figure 3, step 300, and figure 10, units 1002 and 1003);;
“obtaining, by the computing system, data indicative of one or more events, the one or more events comprising at least one of information to be conveyed by the computing system to the user or at least a portion of the second audio signal associated with the surrounding environment of the user” because the device detects an event, such as a lull in a conversation, causes the device to convey information to the user associated with the audio in the surrounding environment so that the user can distinguish between remote persons and local persons (i.e., people in the same room as the user) talking (see Norris, ¶ 0024, 0076-0081, 0189-0191, 0193, and 0195 and figure 5);
“generating, by an artificial intelligence system via the computing system, an audio presentation for the user based at least in part on the data indicative of the one or more events and the data indicative of the acoustic environment for the user, wherein the audio presentation comprises a summary of the one or more events based at least in part on semantic content of the one or more events” because the portable device uses, in part, a SLP (sound localization point) selector (see Norris, ¶ 0259 and figure 11, units 1102 and 1138), where the SLP selector predicts actions desired by the user based on analysis of information such as historical events, user profiles, etc., and the analysis provides probability and/or likelihood that a user would desire the action, such that the SLP selector predicts times, locations, etc. of sound sources or virtual sound sources (see Norris, ¶ 0269), such that the system inserts an alert when there is a lull in the incoming voice and present the audio to the user (see Norris, ¶ 0121, 0195, 0259, 0266, 0269, and 0299 and figures 1-3), where Hussain makes obvious that the audio presentation includes a computer generated voice to provide the audio presentation that comprises these features (Norris, ¶ 0266 in view of Hussain, ¶ 0049, 0051, 0055, and 0062, and figure 1, units 108 and 112); and
“presenting, by the computing system, the audio presentation to the user” because Norris teaches the earphones present the processed audio to the user (see Norris, ¶ 0186-0187 and 0191 and figure 1, steps 110 and 120, figure 2, step 220, and figure 3, step 320)
“wherein generating, by the artificial intelligence system, the audio presentation comprises determining, by the artificial intelligence system, a particular time to incorporate a third audio signal associated with the one or more events into the acoustic environment” because the system determines when a lull in a conversation occurs and incorporates the third audio signal as an alert (see Norris, ¶ 0024 and 0195 and figure 5, unit 519).
Regarding claim 5, see the preceding rejection with respect to claim 4 above. The combination makes obvious the “method of claim 4, wherein determining, by the artificial intelligence system, the particular time to incorporate the third audio signal associated with the one or more events into the acoustic environment comprises:
identifying a lull in the acoustic environment” because the system determines when a lull in a conversation occurs (see Norris, ¶ 0195 and 0299); and
“selecting the lull as the particular time” because the system incorporates the third audio signal as an alert at that identified lull (see Norris, ¶ 0195 and 0299).
Regarding claim 9, see the preceding rejection with respect to claim 4 above. The combination makes obvious the “method of claim 4, wherein generating, by the artificial intelligence system, the audio presentation further comprises incorporating, by the artificial intelligence system, the third audio signal into the acoustic environment at the particular time” because the system incorporates the third audio signal as an alert at the particular time of a lull (see Norris, ¶ 0195 and 0299).
Regarding claim 10, see the preceding rejection with respect to claim 4 above. The combination makes obvious the “method of claim 4, wherein generating, by the artificial intelligence system, the audio presentation further comprises:
generating, by the artificial intelligence system, the third audio signal based at least in part on the data indicative of the one or more events” because the system generates the third audio signal as an alert based on data that indicates the event of a lull (see Norris, ¶ 0195 and 0299).
Regarding claim 13, see the preceding rejection with respect to claim 4 above. The combination makes obvious the “method of claim 4, wherein the one or more events comprise at least one of a communication to the user received by the computing system, an external audio signal received by the computing system comprising at least a portion of the second audio signal associated with the surrounding environment of the user, a notification from an application operating on the computing system, or a prompt from an application operating on the computing system” because Norris teaches the communication of external audio as speech from the remote user (see Norris, ¶ 0194-0195 and figure 5, units 510, 516, and 518), and also teaches that the local speech of a physically present person in the surrounding environment of the user (see Norris, ¶ 0197-0198 and figure 5, units 550 and 552).
Regarding claim 22, see the preceding rejection with respect to claim 1 above. The combination makes obvious:
“A system, comprising:
an artificial intelligence system that comprises one or more machine-learned models” (see Norris, ¶ 0251, 0258, 0266, 0269, and 0299, figure 10, units 1002 and 1004, and figure 11, units 1102, 1104, 1108, and 1138);
“one or more processors” (see Norris, ¶ 0251-0252, 0256, 0259-0261, figure 10, units 1024 and 1064, and figure 11, units 1124, 1152, 1154, 1164, and 1174); and
“one or more non-transitory computer-readable media that collectively store instructions that when executed by the one or more processors cause a computing system to perform operations” (see Norris, ¶ 0252, 0256, and 0259-0261, figure 10, units 1020 and 1060, and figure 11, units 1120, 1150, and 1160),
“the operations comprising:
obtaining data indicative of an acoustic environment for the user, the acoustic environment for the user comprising at least one of a first audio signal playing on the computing system or a second audio signal associated with a surrounding environment of the user” because the handheld portable device, such as a smartphone coupled to earphones with outward facing microphones, monitors the acoustic environment of the user (see Norris, ¶ 0034, 0093-0095, and 0252-0253, figure 3, step 300, and figure 10, units 1002 and 1003);
“obtaining, data indicative of one or more events, the one or more events comprising at least one of information to be conveyed by the computing system to the user or at least a portion of the second audio signal associated with the surrounding environment of the user” because the device detects an event, such as a lull in a conversation, causes the device to convey information to the user associated with the audio in the surrounding environment so that the user can distinguish between remote persons and local persons (i.e., people in the same room as the user) talking (see Norris, ¶ 0024, 0076-0081, 0189-0191, 0193, and 0195 and figure 5);
“generating, by the artificial intelligence system, an audio presentation for the user based at least in part on the data indicative of the one or more events and the data indicative of the acoustic environment for the user, wherein the audio presentation comprises a summary of the one or more events based at least in part on semantic content of the one or more events” because the portable device uses, in part, a SLP (sound localization point) selector (see Norris, ¶ 0259 and figure 11, units 1102 and 1138), where the SLP selector predicts actions desired by the user based on analysis of information such as historical events, user profiles, etc., and the analysis provides probability and/or likelihood that a user would desire the action, such that the SLP selector predicts times, locations, etc. of sound sources or virtual sound sources (see Norris, ¶ 0269), such that the system inserts an alert when there is a lull in the incoming voice and present the audio to the user (see Norris, ¶ 0121, 0195, 0259, 0266, 0269, and 0299 and figures 1-3), where Hussain makes obvious that the audio presentation includes a computer generated voice to provide the audio presentation that comprises these features (Norris, ¶ 0266 in view of Hussain, ¶ 0049, 0051, 0055, and 0062, and figure 1, units 108 and 112); and
“presenting the audio presentation to the user” because Norris teaches the earphones present the processed audio to the user (see Norris, figure 1, steps 110 and 120, figure 2, step 220, and figure 3, step 320);
“wherein generating, by the artificial intelligence system, the audio comprises:
determining a particular time to incorporate a third audio signal associated with the one or more events into the acoustic environment” because the system determines when a lull in a conversation occurs (see Norris, ¶ 0024 and 0195); and
“incorporating the third audio signal into the acoustic environment at the particular time” because the system incorporates the third audio signal as an alert (see Norris, ¶ 0195 and figure 5, unit 519).
Regarding claim 24, see the preceding rejection with respect to claim 22 above. The combination makes obvious the “system of claim 22, wherein the system further comprises a wearable device comprising a speaker” because the system includes a wearable device, such as earbuds or the head-mounted display with speakers (see Norris, ¶ 0021, 0038, 0187, 0194, 0253, and 0258-0259, figure 5, units 514 and 562, figure 10, units 1003 and 1040, and figure 11, units 1102 and 1128); and
“wherein presenting the audio presentation to the user to the user comprises playing the audio presentation via the wearable device” because the wearable device presents the audio presentation comprising the audio from the physically present persons and the audio from the remote persons (see Norris, ¶ 0185-0198).
Claim(s) 7-8 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Norris and Hussain as applied to claim 4 above, and further in view of Lyon et al., US 2020/0007993 A1 (previously cited and hereafter Lyon).
Regarding claim 7, see the preceding rejection with respect to claim 4 above. The combination makes obvious the method of claim 4, wherein the third audio signal is associated with a first event of the one or more events. However, the combination does not appear to teach the features of determining by the system to not incorporate an audio signal associated with a second event of the one or more events.
Lyon teaches an augmented auditory experience for a user, where new sounds detected in the user’s environment are detected, classified, and adjusted for presentation to the user through devices such as smart glasses or earbuds (see Lyon, ¶ 0029, 0032, 0034, 0037, 0043, and 0052, and figure 1). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to modify the combination of Norris and Hussain with the teachings of Lyon for the purpose of filtering auditory sources in the user’s environment to help with concentration (see Lyon, ¶ 0002).
Therefore, the combination of Norris, Hussain, and Lyon makes obvious the “method of claim 4, wherein the third audio signal is associated with a first event of the one or more events and wherein the method further comprises:
determining, by the artificial intelligence system, to not incorporate an audio signal associated with a second event of the one or more events into the acoustic environment” because Lyon makes obvious the detection of various sounds in the environment, such as a ringing phone, sneezing sounds, etc., and using artificial intelligence systems (e.g., machine-learning techniques) to recognize and/or classify the newly detected sounds (see Lyon, ¶ 0041, 0046, 0057 and 0061-0062), where the artificial intelligence systems learn over time to not incorporate the audio signal associated with the second event based on the user’s reactions and/or instructions (see Lyon, ¶ 0090-0091).
Regarding claim 8, see the preceding rejection with respect to claim 4 above. The combination makes obvious the method of claim 4, but does not appear to teach the features related to “noise-cancelling at least a portion of the second audio signal associated with the surrounding environment of the user”.
For the same reasons as stated above with respect to claim 7, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to modify the combination of Norris and Hussain with the teachings of Lyon for the purpose of filtering auditory sources in the user’s environment to help with concentration (see Lyon, ¶ 0002).
The combination of Norris, Hussain, and Lyon makes obvious the “method of claim 4, wherein obtaining the data indicative of the acoustic environment for the user comprises obtaining the second audio signal associated with a surrounding environment of the user” because the combination makes obvious obtaining the second audio signal from the surrounding environment (see Norris, ¶ 0197-0198, and see Lyon, ¶ 0034 and 0037); and
“wherein generating, by the artificial intelligence system, the audio presentation for the user comprises noise-cancelling at least a portion of the second audio signal associated with the surrounding environment of the user” because Lyon makes obvious to use noise-cancelling techniques to reduce background noises with the second audio signal, and to reduce distractions by presenting one audio source at a time (see Lyon, ¶ 0002, 0017, 0028, 0034, 0036, and 0057-0058).
Regarding claim 15, see the preceding rejection with respect to claim 4 above. The combination makes obvious the method of claim 4, but does not appear to teach the features related to “determining, by the artificial intelligence system, not incorporate the third audio signal into the acoustic environment”.
For the same reasons as stated above with respect to claim 7, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to modify the combination of Norris and Hussain with the teachings of Lyon for the purpose of filtering auditory sources in the user’s environment to help with concentration (see Lyon, ¶ 0002).
The combination of Norris, Hussain, and Lyon makes obvious the “method of claim 4, wherein determining, by the artificial intelligence system, the particular time to incorporate the third audio signal associated with the one or more events into the acoustic environment comprises determining, by the artificial intelligence system, to not incorporate the third audio signal into the acoustic environment” because Lyon makes obvious the detection of various sounds in the environment, such as a ringing phone, sneezing sounds, etc., and using artificial intelligence systems (e.g., machine-learning techniques) to recognize and/or classify the newly detected sounds (see Lyon, ¶ 0041, 0046, 0057 and 0061-0062), where the artificial intelligence systems learn over time to not incorporate the audio signal associated with the third event based on the user’s reactions and/or instructions (see Lyon, ¶ 0090-0091).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rosenberg, US 2007/0189544 A1 (previously cited), teaches an ambient sound responsive media player, where the output of the media player is adjusted based on detected characteristics of a captured ambient audio signal (see Rosenberg, abstract).
Park, US 2016/0095083 A1 (previously cited), teaches methods to determine when to present different notifications to a user when the device is in a media playback state (see Park, ¶ 0090-0091, 0117-0118, and 0147-0157, and figures 8-9).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniel R Sellers whose telephone number is (571)272-7528. The examiner can normally be reached Mon - Fri 10:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fan S Tsang can be reached at (571)272-7547. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniel R Sellers/Primary Examiner, Art Unit 2694
Prosecution Insights