DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The reply filed on 19 March 2026 has been entered. Applicant’s arguments with respect to claims 1-3, 5-12, 14-18 and 20-23 have been considered but are moot in view of new ground(s) of rejection caused by the amendments.
Claims 1-3, 5-12, 14-18 and 20-23 are pending in this application and have been considered below. Claims 4, 13 and 19 are canceled by the applicant.
Information Disclosure Statement
The IDSs dated 23 January 2024 and 21 April 2025 that have been previously considered remain placed in the application file.
Specification - Drawings
Acknowledgement is made of the color drawings submitted 23 January 2024 in this application. Applicants are reminded that, absent a successful petition, the black and white drawings submitted on 23 January 2024 will be used. No petition is currently on file.
Claim Rejections - 35 USC § 101
Claim 16 has been amended. The rejection of claims 16-20 under 35 USC 101 is withdrawn.
1st Claim Interpretation
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claims 7 recite “information selected from the group of.” Since “information selected from the group of” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
2nd Claim Interpretation
Claims 1, 3-5, 10, 12-14, 16 and 18-20 have been amended. The claim interpretation is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 5-11, 14-17 and 20-23 (all claims not objected to below) are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2023 0104683 A1, (Recker et al.) in view of US Patent Publication 2023 0300532 A1, (Spittle). The references are listed in a PTO-892 from the Office Action in which they are first used.
PNG
media_image1.png
489
343
media_image1.png
Greyscale
Claim 1
Regarding Claim 1, Recker et al. teach a non-transitory computer-implemented method ("a system, including an image sensor, a hearing device and a controller. The image sensor may be configured to sense optical information of an environment and produce image data indicative of the sensed optical information," paragraph [0003]) comprising:
[AltContent: textbox (Recker et al. Fig. 2, showing a hearing assistance device with sensors.)]receiving a first image of a first portion of an environment at a first location of a location type that is pre-associated with at least one ambient noise condition ("Hearing devices may classify a limited number of acoustic environments ( e.g., speech, noise, speech in noise, music, machine noise, and wind noise) and physical activities (e.g., walking, jogging, biking, lying down, standing, etc.)," paragraph [0018]), from a camera associated with a mobile device ("An image sensor may capture optical or visual details of an environment in which the user is situated. This may include information such as, e.g., a geographic location of the user (e.g., Global Positioning System coordinates), a building type (e.g., home, office, coffee shop, etc.), whether the user is indoors or outdoors, the size of the room, the brightness of the room, furnishings (e.g., carpeted or not, curtains or not, presence of furniture, etc.)," paragraph [0034]);
determining that an output location by the machine-learning model fails to meet a confidence threshold to identify the first location ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold);
generating, with the machine-learning model, a first audio preset ("if the user makes adjustments to the hearing device settings in an environment, such information can be used to make recommendations to others in similar environments," paragraph [0020] where hearing device settings are audio presets) corresponding to the location type of the first location ("Audio objects may include additional information about the object or activity. For example, the audio objects may include information such as, e.g., a location, position, object brand, activity intensity, etc. Audio objects may be linked to a specific person or environment," paragraph [0206]); and
transmitting the first audio preset to an auditory device, wherein the auditory device uses the first audio preset to modify sounds at the first location ("indicate the presence of an assistive listening technology and adjust the hearing device to connect to the assistive listening technology," paragraph [0017] and "Such an understanding can be used to improve the hearing device settings assigned to the user (e.g., increase wind noise reduction)," paragraph [0022]).
[AltContent: textbox (Spittle Fig. 46, showing using a cell phone for data collection.)]
PNG
media_image2.png
523
686
media_image2.png
Greyscale
Recker et al. is not relied upon to explicitly teach all of machine-learning models.
However, Spittle teaches providing the first image as input to a machine-learning model, wherein the machine-learning model is trained to identify locations based on the input images ("In some embodiments, multiple sensors may be coupled to and may be programmed to send data streams to a processor, such as a DSP. The raw measured sensor data, processed sensor data, or both may be transmitted to one or more processors and/or plugins. Embodiments of the disclosure may include using machine learning and NNs to process and analyze the sensor data" paragraph [0550]).
Therefore, taking the teachings of Recker et al. and Spittle as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “Using a Camera for Hearing Device Algorithm Training” as taught by Recker et al. to use “Fully Customizable Ear Worn Devices and Associated Development Platform as taught by Spittle. The suggestion/motivation for doing so would have been that, “What is needed is an architecture comprising a main processor that performs some functions (less than all of the processing) and an audio subsystem that offload certain processing functions from the main processor.” as noted by the Spittle disclosure in paragraph [XXXXX], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that hearing aids and devices will need to adapt to ambient noise, which is variable; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of system claim 10 and apparatus claim 16 while noting that the rejection above cites to both device and method disclosures. Claims 10 and 16 are mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Recker et al. teach the method of claim 1, further comprising:
detecting that the mobile device has changed location by a threshold distance ("One advantage of the system being able to remember situations, people, foods, and activities is that as a user enters an environment, performs an activity, or encounters someone associated with a known audio class, the hearing device parameters can be automatically configured to settings for that audio class without waiting for a detailed analysis of the current environment," paragraph [0028] where entering an environment teaches changing location by a threshold distance);
in response to detecting the changed location, receiving a second image at a second location from the mobile device ("a user using the system described herein may enter a living room with a fan in it. The image sensor may identify the fan as a potential sound source, and the hearing devices of the system may confirm that noise was detected," paragraph [0037] where the living room is different than the kayaking above);
providing the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]);
outputting, with the machine-learning model, an identification of the second location associated with the second image ("If the user enters this same living room again or enters another room with the same type of fan, the sound may be identified more quickly using the generated audio object associated with the fan. Additional information about the user may also be captured in this environment," paragraph [0037]); and
providing the second location to an auditory device, wherein the auditory device applies a second audio preset to the auditory device based on the second location ("if the user adjusts the hearing devices, such information may be recorded and used to make recommendations to others in similar situations or environments. Alternatively, if others have made adjustments to their hearing devices in similar environments to the user, the user may receive a recommendation to make similar changes to the user's hearing devices," paragraph [0038]).
Claim 5
Regarding claim 5, Recker et al. teach the method of claim 1, wherein the mobile device is a smartphone and further comprising, prior to generating the first audio preset:
displaying, with the smartphone, a user interface that instructs a user to move the smartphone in a particular direction until the user moves a predetermined distance ("the computing device 154 may include any one or more devices configured to assist in collecting or processing data such as, e.g., a mobile compute device, a laptop, a tablet, a personal digital assistant, a smart speaker system, a smart car system, a smart watch, smart ring, chest strap a TV streamer device, wireless audio streaming device, cell phone or landline streamer device, Direct Audio Input (DAI) gateway device, auxiliary audio input gateway device, telecoil/magnetic induction receiver device, hearing device programmer, charger, hearing device storage/drying box, smartphone, and wearable or implantable health monitor, etc.," paragraph [0193] and "In one example, user input may also be used to identify activities of the user." paragraph [0041]);
receiving a second image at the first location from the camera associated with the smartphone ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022] where user is kayaking teaches multiple images));
providing the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]); and
determining that the output location by the machine-learning model fails to meet the confidence threshold to identify the first location associated with the second image ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold).
Claim 6
Regarding claim 6, Recker et al. teach the method of claim 1, further comprising, prior to generating the first audio preset:
providing a second image at the first location as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]);
identifying a second audio preset associated with the first location ("if the user makes adjustments to the hearing device settings in an environment, such information can be used to make recommendations to others in similar environments," paragraph [0020] where the second setting is the user adjusted setting); and
determining, based on background noise, that the least one ambient noise condition is different from a corresponding ambient noise condition associated with the second audio preset ("If the user enters this same living room again or enters another room with the same type of fan, the sound may be identified more quickly using the generated audio object associated with the fan. Additional information about the user may also be captured in this environment," paragraph [0037]), wherein generating the first audio preset is responsive to the determining ("Furthermore, the user's own history can be used to inform the probability of different audio classes for that individual (e.g., by taking into consideration factors such as the time of day that the user typically enters certain acoustic environments or performs certain activities, the frequency with which the user enters these environments or performs these activities, the amount of time that the user normally spends in such environments or performing such activities, etc.) and for the population( s) to which the user belongs," paragraph [0025] which teaches different sound environments).
Claim 7
Regarding claim 7, Recker et al. teach the method of claim 1, further comprising, prior to generating the first audio preset:
receiving an identification of the first location based on information selected from the group of global positioning system (GPS) coordinates, Bluetooth, Wi-Fi, Near Field Communication (NFC), Radio Frequency Identification (RFID), Ultra-Wideband (UWB), infrared, or combinations thereof ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022] where user is kayaking teaches multiple images);
determining that the first image is associated with the first location ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199]); and
determining that there is no audio preset associated with the first location ("providing a system including an image sensor and a hearing device as described herein can provide for audio classes to be updated without significant user (e.g., user) interaction. Additionally, new audio classes can be identified, generated, and provided to any hearing device user," paragraph [0025]).
Claim 8
Regarding claim 8, Recker et al. teach the method of claim 1, wherein generating the first audio preset comprises:
sampling a background noise for a period of time ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022]); and
outputting, with the machine-learning model, the first audio preset for the ambient noise condition that modifies adjustments in sound levels based on patterns associated the ambient noise condition ("Such an understanding can be used to improve the hearing device settings assigned to the user (e.g., increase wind noise reduction)," paragraph [0022] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]).
Claim 9
Regarding claim 9, Recker et al. teach the method of claim 1, wherein the machine-learning model is trained by:
providing training data that includes different ambient noise conditions, information about how the different ambient noise conditions change as a function of time, and a set of presets that reduce or block background noise associated with the different ambient noise conditions ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022] and "Such an understanding can be used to improve the hearing device settings assigned to the user (e.g., increase wind noise reduction)," paragraph [0022]);
generating feature embeddings from the training data that group features of the different ambient noise conditions based on similarity ("All such data collected by the system may be captured for analysis (real-time or off-line) to improve an understanding of how the acoustic environment varies in real-time based on the user's actions and environment," paragraph [0022]);
providing training ambient noise conditions as input to the machine-learning model ("Receiving user input to identify sources of sound and associate the sound with the source may provide systems or methods that can quickly build a large and accurate database of objects and activities and their associated acoustic characteristics," paragraph [0040] where the large and accurate database is a machine learning model);
outputting one or more training presets that correspond to each training ambient noise condition ("Receiving user input to identify sources of sound and associate the sound with the source may provide systems or methods that can quickly build a large and accurate database of objects and activities and their associated acoustic characteristics," paragraph [0040]);
comparing the one or more training presets to ground truth data ("capture visual, acoustic, physical, and physiological data associated with that activity. Such data may be compared to data associated with other activities to determine unique characteristics of the activity the user is engaged in," paragraph [0041]); and
modifying parameters of the machine-learning model based on a loss function that identifies a difference of the one or more training presets to the ground truth data ("If two activities have similar acoustic properties, but they can be differentiated based on physiological and/or geographical information, then this information may help with accurate audio classification," paragraph [0058] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]).
Claim 10
Regarding claim 10, Recker et al. teach a system ("a system, including an image sensor, a hearing device and a controller. The image sensor may be configured to sense optical information of an environment and produce image data indicative of the sensed optical information," paragraph [0003]) comprising:
one or more processors ("The hearing device 100 includes a processor 104 operatively coupled to memory 106," paragraph [0181]); and
logic encoded in one or more non-transitory media for execution by the one or more processors ("The processor 104 can include or be operatively coupled to memory 106, such as RAM, SRAM, ROM, or flash memory," paragraph [0181]) and when executed are operable to:
receive a first image of a first portion of an environment at a first location of a location type that is pre-associated with at least one ambient noise condition ("Hearing devices may classify a limited number of acoustic environments ( e.g., speech, noise, speech in noise, music, machine noise, and wind noise) and physical activities (e.g., walking, jogging, biking, lying down, standing, etc.)," paragraph [0018]), from a camera associated with a mobile device ("An image sensor may capture optical or visual details of an environment in which the user is situated. This may include information such as, e.g., a geographic location of the user (e.g., Global Positioning System coordinates), a building type (e.g., home, office, coffee shop, etc.), whether the user is indoors or outdoors, the size of the room, the brightness of the room, furnishings (e.g., carpeted or not, curtains or not, presence of furniture, etc.)," paragraph [0034]);
determine that that an output location by the machine-learning model fails to meet a confidence threshold to identify the first location ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold);
generate, with the machine-learning model, a first audio preset ("if the user makes adjustments to the hearing device settings in an environment, such information can be used to make recommendations to others in similar environments," paragraph [0020] where hearing device settings are audio presets) corresponding to the location type of the first location ("Audio objects may include additional information about the object or activity. For example, the audio objects may include information such as, e.g., a location, position, object brand, activity intensity, etc. Audio objects may be linked to a specific person or environment," paragraph [0206]); and
transmit the first audio preset to an auditory device, wherein the auditory device uses the first audio preset to modify sounds at the first location ("indicate the presence of an assistive listening technology and adjust the hearing device to connect to the assistive listening technology," paragraph [0017] and "Such an understanding can be used to improve the hearing device settings assigned to the user (e.g., increase wind noise reduction)," paragraph [0022]).
Recker et al. is not relied upon to explicitly teach all of machine learning models.
However, Spittle teaches provide the first image as input to a machine-learning model, wherein the machine-learning model is trained to identify locations associated with input images ("In some embodiments, multiple sensors may be coupled to and may be programmed to send data streams to a processor, such as a DSP. The raw measured sensor data, processed sensor data, or both may be transmitted to one or more processors and/or plugins. Embodiments of the disclosure may include using machine learning and NNs to process and analyze the sensor data" paragraph [0550]).
Recker et al. and Spittle are combined as per claim 1.
Claim 11
Regarding claim 11, Recker et al. teach the system of claim 10, wherein the logic is further operable to:
detecting that the mobile device has changed location by a threshold distance ("One advantage of the system being able to remember situations, people, foods, and activities is that as a user enters an environment, performs an activity, or encounters someone associated with a known audio class, the hearing device parameters can be automatically configured to settings for that audio class without waiting for a detailed analysis of the current environment," paragraph [0028] where entering an environment teaches changing location by a threshold distance);
in response to detecting the changed location, receive a second image at a second location from the mobile device ("a user using the system described herein may enter a living room with a fan in it. The image sensor may identify the fan as a potential sound source, and the hearing devices of the system may confirm that noise was detected," paragraph [0037] where the living room is different than the kayaking above);
provide the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]);
output, with the machine-learning model, an identification of the second location associated with the second image ("If the user enters this same living room again or enters another room with the same type of fan, the sound may be identified more quickly using the generated audio object associated with the fan. Additional information about the user may also be captured in this environment," paragraph [0037]); and
provide the second location to an auditory device, wherein the auditory device applies a second audio preset to the auditory device based on the second location ("if the user adjusts the hearing devices, such information may be recorded and used to make recommendations to others in similar situations or environments. Alternatively, if others have made adjustments to their hearing devices in similar environments to the user, the user may receive a recommendation to make similar changes to the user's hearing devices," paragraph [0038]).
Claim 14
Regarding claim 14, Recker et al. teach the system of claim 10, wherein the mobile device is a smartphone and the logic is further operable to, prior to generating the first audio preset:
displaying, with the smartphone, a user interface that instructs a user to move the smartphone in a particular direction until the user moves a predetermined distance ("the computing device 154 may include any one or more devices configured to assist in collecting or processing data such as, e.g., a mobile compute device, a laptop, a tablet, a personal digital assistant, a smart speaker system, a smart car system, a smart watch, smart ring, chest strap a TV streamer device, wireless audio streaming device, cell phone or landline streamer device, Direct Audio Input (DAI) gateway device, auxiliary audio input gateway device, telecoil/magnetic induction receiver device, hearing device programmer, charger, hearing device storage/drying box, smartphone, and wearable or implantable health monitor, etc.," paragraph [0193] and "In one example, user input may also be used to identify activities of the user." paragraph [0041]);
receiving a second image at the first location from the camera associated with the smartphone ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022] where user is kayaking teaches multiple images));
providing the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]); and
determine that the output location by the machine-learning model fails to meet the confidence threshold to identify the first location associated with the second image ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold).
Claim 15
Regarding claim 15, Recker et al. teach the system of claim 10, wherein the logic is further operable to, prior to generating the first audio preset:
provide a second image at the first location as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]);
identify a second audio preset associated with the first location ("if the user makes adjustments to the hearing device settings in an environment, such information can be used to make recommendations to others in similar environments," paragraph [0020] where the second setting is the user adjusted setting); and
determine, based on background noise, that at least one ambient noise condition is different from a corresponding ambient noise condition associated with the second audio preset ("If the user enters this same living room again or enters another room with the same type of fan, the sound may be identified more quickly using the generated audio object associated with the fan. Additional information about the user may also be captured in this environment," paragraph [0037]), wherein generating the first audio preset is responsive to the determining ("Furthermore, the user's own history can be used to inform the probability of different audio classes for that individual (e.g., by taking into consideration factors such as the time of day that the user typically enters certain acoustic environments or performs certain activities, the frequency with which the user enters these environments or performs these activities, the amount of time that the user normally spends in such environments or performing such activities, etc.) and for the population( s) to which the user belongs," paragraph [0025] which teaches different sound environments).
Claim 16
Regarding claim 16, Recker et al. teach software encoded in one or more non-transitory computer-readable media for execution by the one or more processors of an auditory device ("a system, including an image sensor, a hearing device and a controller. The image sensor may be configured to sense optical information of an environment and produce image data indicative of the sensed optical information," paragraph [0003])and when executed is operable to:
receive a first image of a first portion of an environment at a first location of a location type that is pre-associated with at least one ambient noise condition ("Hearing devices may classify a limited number of acoustic environments ( e.g., speech, noise, speech in noise, music, machine noise, and wind noise) and physical activities (e.g., walking, jogging, biking, lying down, standing, etc.)," paragraph [0018]), from a camera associated with a mobile device ("An image sensor may capture optical or visual details of an environment in which the user is situated. This may include information such as, e.g., a geographic location of the user (e.g., Global Positioning System coordinates), a building type (e.g., home, office, coffee shop, etc.), whether the user is indoors or outdoors, the size of the room, the brightness of the room, furnishings (e.g., carpeted or not, curtains or not, presence of furniture, etc.)," paragraph [0034]);
determine that an output location by the machine-learning model fails to meet the confidence threshold to identify the first location ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold);
generate, with the machine-learning model, a first audio preset ("if the user makes adjustments to the hearing device settings in an environment, such information can be used to make recommendations to others in similar environments," paragraph [0020] where hearing device settings are audio presets) corresponding to the location type of the first location ("Audio objects may include additional information about the object or activity. For example, the audio objects may include information such as, e.g., a location, position, object brand, activity intensity, etc. Audio objects may be linked to a specific person or environment," paragraph [0206]); and
transmit the first audio preset to an auditory device, wherein the auditory device uses the first audio preset to modify sounds at the first location ("indicate the presence of an assistive listening technology and adjust the hearing device to connect to the assistive listening technology," paragraph [0017] and "Such an understanding can be used to improve the hearing device settings assigned to the user (e.g., increase wind noise reduction)," paragraph [0022]).
Recker et al. is not relied upon to explicitly teach all of machine learning models.
However, Spittle teaches provide the first image as input to a machine-learning model, wherein the machine-learning model is trained to identify locations based on the input images ("In some embodiments, multiple sensors may be coupled to and may be programmed to send data streams to a processor, such as a DSP. The raw measured sensor data, processed sensor data, or both may be transmitted to one or more processors and/or plugins. Embodiments of the disclosure may include using machine learning and NNs to process and analyze the sensor data" paragraph [0550]);
Recker et al. and Spittle are combined as per claim 1.
Claim 17
Regarding claim 17, Recker et al. teach the software of claim 16, wherein the logic is further operable to:
detect that the mobile device has changed location by a threshold distance ("One advantage of the system being able to remember situations, people, foods, and activities is that as a user enters an environment, performs an activity, or encounters someone associated with a known audio class, the hearing device parameters can be automatically configured to settings for that audio class without waiting for a detailed analysis of the current environment," paragraph [0028] where entering an environment teaches changing location by a threshold distance);
in response to detecting the changed location, receive a second image at a second location from the mobile device ("a user using the system described herein may enter a living room with a fan in it. The image sensor may identify the fan as a potential sound source, and the hearing devices of the system may confirm that noise was detected," paragraph [0037] where the living room is different than the kayaking above);
provide the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]);;
output, with the machine-learning model, an identification of the second location associated with the second image ("If the user enters this same living room again or enters another room with the same type of fan, the sound may be identified more quickly using the generated audio object associated with the fan. Additional information about the user may also be captured in this environment," paragraph [0037]); and
provide the second location to an auditory device, wherein the auditory device applies a second audio preset to the auditory device based on the second location ("if the user adjusts the hearing devices, such information may be recorded and used to make recommendations to others in similar situations or environments. Alternatively, if others have made adjustments to their hearing devices in similar environments to the user, the user may receive a recommendation to make similar changes to the user's hearing devices," paragraph [0038]).
Claim 20
Regarding claim 20, Recker et al. teach the software of claim 16, wherein the mobile device is a smartphone and the logic is further operable to, prior to generating the first audio preset:
displaying, with the smartphone, a user interface that instructs a user to move the smartphone in a particular direction until the user moves a predetermined distance ("the computing device 154 may include any one or more devices configured to assist in collecting or processing data such as, e.g., a mobile compute device, a laptop, a tablet, a personal digital assistant, a smart speaker system, a smart car system, a smart watch, smart ring, chest strap a TV streamer device, wireless audio streaming device, cell phone or landline streamer device, Direct Audio Input (DAI) gateway device, auxiliary audio input gateway device, telecoil/magnetic induction receiver device, hearing device programmer, charger, hearing device storage/drying box, smartphone, and wearable or implantable health monitor, etc.," paragraph [0193] and "In one example, user input may also be used to identify activities of the user." paragraph [0041]);
receiving a second image at the first location from the camera associated with the smartphone ("the image sensor may detect that the user is kayaking. The system may capture acoustic properties of the environment, what is happening during the activity (e.g., wind blowing, waves hitting the kayak, paddle noise, etc.), positional and movement data from an inertial measurement unit (IMU), and data (e.g., heart rate, GPS and temperature, etc.) from other sensors of the hearing device or operably coupled computing devices (e.g., mobile device, smart watch, wearables, etc.)," paragraph [0022] where user is kayaking teaches multiple images));
providing the second image as input to the machine-learning model ("the data 148 (e.g., image data, sound data, voice data, audio classes, audio objects, optical components, hearing impairment settings, hearing device settings, an array, a mesh, a digital file, etc.) may be analyzed by a user, used by another machine that provides output based thereon, etc.," paragraph [00199] and "An algorithm or process (e.g. artificial intelligence, machine learning, deep neural network, etc.)" paragraph [0044]); and
determine that the output location by the machine-learning model fails to meet the confidence threshold to identify the first location associated with the second image ("a confidence value may be determined using the one or more audio objects. For example, as more data is collected (e.g., image data and sound data) regarding the same or similar audio objects a confidence value associated with such audio objects may increase. The audio class may be adjusted in response to the determined confidence value exceeding a threshold," paragraph [0207] where exceeding a threshold teaches failing to meet the threshold).
Claim 21
Regarding claim 21, Recker et al. teach the method of claim 1, as noted above.
Recker et al. is not relied upon to explicitly teach all of a predefined pattern of change as a function of time.
However, Spittle teaches wherein the at least one ambient noise condition is associated with a predefined pattern of change as a function of time ("The usual audio system patterns may be matched. The usual audio system patterns may be a mix of fixed core functions and some user configurable functions, for example. A typical audio processing plugin is likely to use some core audio library functions such as standard filters, gain manipulation functions, FFT/IFFT, other filter banks, etc. These DSP functions could be stored within the silicon and are accessible to one or more plugins. In addition, the plugin is likely to have some custom audio processing functions that deliver the special performance and audio experience," paragraph [0600])and wherein generating the first audio present includes the machine learning model predicting the pattern of the change ("In some embodiments, the binaural scene manipulation algorithm may perform binaural auditory cue prediction. The binaural auditory cue prediction may predict which auditory cues may help with the processing of the real-time audio data in the sound signals. The prediction can be provided from neural networks that are trained to determine which spatial cues provide the optimal benefit to listeners based on the signal content," paragraph [0789]).
Recker et al. and Spittle are combined as per claim 1.
Claim 22
Regarding claim 22, Recker et al. teach the system of claim 10, as noted above.
Recker et al. is not relied upon to explicitly teach all of a predefined pattern of change as a function of time.
However, Spittle teaches wherein the at least one ambient noise condition is associated with a predefined pattern of change as a function of time ("The usual audio system patterns may be matched. The usual audio system patterns may be a mix of fixed core functions and some user configurable functions, for example. A typical audio processing plugin is likely to use some core audio library functions such as standard filters, gain manipulation functions, FFT/IFFT, other filter banks, etc. These DSP functions could be stored within the silicon and are accessible to one or more plugins. In addition, the plugin is likely to have some custom audio processing functions that deliver the special performance and audio experience," paragraph [0600])and wherein generating the first audio present includes the machine learning model predicting the pattern of the change ("In some embodiments, the binaural scene manipulation algorithm may perform binaural auditory cue prediction. The binaural auditory cue prediction may predict which auditory cues may help with the processing of the real-time audio data in the sound signals. The prediction can be provided from neural networks that are trained to determine which spatial cues provide the optimal benefit to listeners based on the signal content," paragraph [0789]).
Recker et al. and Spittle are combined as per claim 1.
Claim 23
Regarding claim 23, Recker et al. teach the software of claim 16, as noted above.
Recker et al. is not relied upon to explicitly teach all of a predefined pattern of change as a function of time.
However, Spittle teaches wherein the at least one ambient noise condition is associated with a predefined pattern of change as a function of time ("The usual audio system patterns may be matched. The usual audio system patterns may be a mix of fixed core functions and some user configurable functions, for example. A typical audio processing plugin is likely to use some core audio library functions such as standard filters, gain manipulation functions, FFT/IFFT, other filter banks, etc. These DSP functions could be stored within the silicon and are accessible to one or more plugins. In addition, the plugin is likely to have some custom audio processing functions that deliver the special performance and audio experience," paragraph [0600])and wherein generating the first audio present includes the machine learning model predicting the pattern of the change ("In some embodiments, the binaural scene manipulation algorithm may perform binaural auditory cue prediction. The binaural auditory cue prediction may predict which auditory cues may help with the processing of the real-time audio data in the sound signals. The prediction can be provided from neural networks that are trained to determine which spatial cues provide the optimal benefit to listeners based on the signal content," paragraph [0789]).
Recker et al. and Spittle are combined as per claim 1.
Allowable Subject Matter
Claims 3, 12 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2021 0377643 A1 to Igarashi et al. discloses a sound device that includes a main body installed on a medial surface of an auricle, a holding portion having an annular hollow structure arranged to be coupled to an intertragic notch of an ear near an entrance of an ear canal, a sound guide portion formed as a pipe structure having one end communicating with the main body and another end communicating with the holding portion, an open/close operation unit configured to open or close an earhole, and a control unit configured to control driving of the open/close operation unit.
US Patent Publication 2021 0043049 A1 to Moura et al. discloses a visual frame associated with an environment may be received. The visual frame may be segmented into a plurality of regions of interest. A first position for a first region of the plurality of regions of interest is determined. A determination may be made that the first position intersects with a projected area associated with a user. The projected area is to include one or more areas that are outside of a current direction of movement of the user. One or more characteristics of a first auditory stimulus are selected based on the first position and the determination that the first position is within the projected area.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATH E WELLS whose telephone number is (703)756-4696. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.E.W/Examiner, Art Unit 2664
Date: 7 April 2026
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights