Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are pending. Claims 1, 8, and 16 are independent.
This Application was published as US 20250006206.
Apparent priority is 1 July 2023.
The instant Application is directed to a device which detects hypophonia in a user’s speech and outputs a notification to the user.
Applicant’s amendments and arguments are considered but are either unpersuasive or moot in view of the new grounds of rejection that, if presented, were necessitated by the amendments to the Claims.
This action is Final.
Response to Amendments
Applicant’s amendments to the specification and claims have overcome each and every objection and 112(b) rejection previously set forth in the Non-Final Office Action mailed 5/28/26.
Response to Arguments
35 USC 101
Applicant's arguments have been fully considered and are persuasive. The rejection under 35 UISC 101 is withdrawn.
35 USC 103
Applicant’s arguments with respect to two microphones spaced apart by different distances from the mouth have been fully considered but are not persuasive. Applicant provides arguments in regards to configuration 220a of Fig. 2B, which are persuasive in relation to that configuration. However, configuration 220b of Fig. 2B discloses acoustic sensors behind each ear, and in front of the right ear. 223b is farther from the mouth than 221b. See [0064] also. [0024] discloses that the acoustic sensors can be microphones. This configuration is fully supported by [0021] of the provisional application (63/501,131). Therefore, these limitations are rejected under Anderson, although the rejection is changed to 103 for other amended limitations.
PNG
media_image1.png
348
403
media_image1.png
Greyscale
Extract from Anderson, Fig. 2B
Applicant’s arguments with respect to determining user’s speech based on different time of arrival values have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3, 5, 7-10, 15-17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson et al. (US 20240379093 A1) in view of Sekiya et al. (US 20170280239 A1).
Regarding claim 1, Anderson discloses: 1. A wearable speech therapy device to be worn by a user, ("[0004] An exemplary system and method are disclosed for a head-worn device for augmenting speech therapy or mitigating Parkinson's effect on speech and other speech-impairing conditions by providing haptic feedback, vibratory feedback, audio feedback, or other stimulations to a wearer by isolating and analyzing vocal/speech output of the user for signal-associated assessment, including, e.g., based on timing, direction, loudness, and/or speaking rate..." – ref [0003] of provisional application (63/501,131))
the wearable speech therapy device comprising: a housing including a first end and a second end spaced apart from the first end; (Fig. 2C shows a housing with two ends – ref Fig. 2 of provisional application (63/501,131))
a first microphone located closer to the first end than the second end, (Fig. 2C shows a first microphone 250a closer to one end – ref Fig. 2 of provisional application (63/501,131))
wherein, when the wearable speech therapy device is worn by the user, the first microphone and the second microphone are spaced apart by different distances from a mouth of the user; (see Fig. 2B, 220b. 223b and 222b are spaced different distances from the user’s mouth. ref [0021] of provisional 63/501,131)
a second microphone located closer to the second end than the first end; (Fig. 2C shows a second microphone 250b. Fig. 2B, 220b shows a configuration with a second microphone on the other side of the user's head, which corresponds to the second end of the housing. – ref Fig. 2 of provisional application (63/501,131))
a speaker; ("[0140]...Output device(s) 512, such as a display, speakers, printer, etc., may also be included..." – ref [0037] of provisional application (63/501,131))
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the wearable speech therapy device to perform acts comprising: ("[0139] In its most basic configuration, computing device 500 typically includes at least one processing unit 506 and system memory 504…" – ref [0034] of provisional application (63/501,131))
receiving, from the first microphone, first audio data associated with audio captured within an environment, receiving, from the second microphone, second audio data associated with the audio, ("[0071] At step 310, the method 300 includes receiving an audio signal, for example, via a plurality of acoustic sensors." – ref Fig. 1 of provisional application (63/501,131))
determining, based at least in part on the first audio data and the second audio data and different time of arrival values of the audio at the first microphone and the second microphone, that the audio is associated with user speech of the user within the environment, ("[0102]...Two MEMS microphones and an accelerometer were used to determine if the user was speaking. This speech data was filtered to reduce environmental noise and then compared to the vocal sound pressure level thresholds." – ref [0026] of provisional application (63/501,131)) (different time of arrival values are not explicitly disclosed)
determining, based at least in part on the audio being associated with the user speech, one or more biomarkers associated with the user speech, ("[0074] At step 316, the method 300 includes determining whether the wearer's speech signal satisfies one or more speech parameters, including at least one of an acoustic intensity parameter or a speech rate parameter…" – ref [0026] of provisional application (63/501,131))
determining that the one or more biomarkers satisfy a threshold associated with the user speech containing hypophonia, and ("[0074]... In some implementations, the method 300 includes determining whether the wearer's speech signal satisfies an acoustic intensity parameter or speech rate parameter based on an amount of energy in the wearer's speech signal. By way of example, the at least one threshold for the acoustic intensity parameter can include a low acoustic intensity level and a high acoustic intensity level. Similarly, the at least one predetermined threshold for the speech rate parameter can include a low speech rate level and a high speech rate level." – the low acoustic intensity level is associated with hypophonia – ref [0026] and [0014] of provisional application (63/501,131))
causing, based at least in part on the one or more biomarkers satisfying the threshold, output of a notification via the speaker. ("[0075] At step 318, in response to detecting that the wearer's speech signal fails to satisfy at least one of the speech parameters, the method 300 includes outputting haptic biofeedback and/or other stimulation (e.g., audio, visual) to the wearer." – failing to satisfy the speech parameters is satisfying a threshold for low acoustic intensity – ref [0011] of provisional application (63/501,131))
Anderson does not explicitly disclose: determining user speech based on different time of arrival values of the audio at the first microphone and the second microphone.
Sekiya discloses: a housing including a first end and a second end spaced apart from the first end; a first microphone located closer to the first end than the second end, wherein, when the wearable speech therapy device is worn by the user, the first microphone and the second microphone are spaced apart by different distances from a mouth of the user; (Fig. 1 shows a housing with voice acquisition units 110B and 110D at opposite ends, spaced at different distances from the user’s mouth. See also [0037]. [0058] discloses that the voice acquisition unit can be a microphone.)
PNG
media_image2.png
529
484
media_image2.png
Greyscale
Sekiya Fig. 1
Sekiya further discloses: determining user speech based on different time of arrival values of the audio at the first microphone and the second microphone. (“[0067] In one example, the controller 160 performs beamforming processing of forming the directivity to acquire a voice coming from the direction of the user's mouth using a plurality of voice data items acquired by the voice acquisition unit 110. The beamforming processing is a process of changing the degree of enhancement for each area where sound arrives. More specifically, the beamforming processing performed by the controller 160 may include a process of reducing sound coming from a specific area, or may include a process of enhancing sound coming from a desired orientation. In one example, the controller 160 may regard a voice coming from a direction other than the direction of the user's mouth as noise to be reduced. Furthermore, the controller 160 may enhance a voice coming from the direction of the user's mouth. As described above, the voice acquisition unit 110 does not necessarily have its own directivity. The controller 160 controls the directivity by performing the beamforming processing on the voice data acquired by each of the voice acquisition units 110. The controller 160 can perform the beamforming processing using the phase difference between the voice data items acquired by each of the voice acquisition units 110.” – [0044] is explicit that the phase difference is a time difference.)
Anderson and Sekiya are considered analogous art to the claimed invention because they disclose speech processing devices. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Anderson with the beamforming method disclosed by Sekiya. Doing so would have been beneficial to enhance the sound coming from a specific area and to reduce a voice coming from another direction from the user’s mouth. (Sekiya [0067]) Anderson discloses in Fig. 3B that the first beamforming operation is performed before detecting user speech; therefore replacing the beamforming of Anderson with the beamforming of Sekiya would result in using the different time of arrival values to detect the user’s speech. This combination falls under combining prior art elements according to known methods to yield predictable results or simple substitution of one known element for another to obtain predictable results. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Regarding claim 3, Anderson discloses: 3. The wearable speech therapy device of claim 1, further comprising at least one of a lighting element or a haptic motor, the acts further comprising causing at least one of a second notification to be output on the at least one of the lighting element or the haptic motor. ("[0010] In some embodiments, at least one of the feedback elements comprises at least one haptic biofeedback element, wherein the processor is further configured to output haptic biofeedback via the at least one haptic biofeedback element." – ref [0011] of provisional application (63/501,131))
Regarding claim 5, Anderson discloses: 5. The wearable speech therapy device of claim 1, wherein when the wearable speech therapy device is worn by the user, the first microphone and the second microphone are substantially aligned with an anatomical axis of the user. (Fig. 2B shows multiple configurations which align the microphones with an axis of the user. For example, 220a shows a configuration where the microphones are aligned on the axis of the user's ears. As another example, 220g shows a configuration where the microphones are aligned to the axis of the "front" of the user. – ref Fig. 2 of provisional application (63/501,131))
Regarding claim 7, Anderson discloses: 7. The wearable speech therapy device of claim 1, wherein the one or more biomarkers include at least one of a pitch of the user speech, an intonation in the user speech, a tone associated with the user speech, a pause in the user speech, a phonation associated with the user speech, or an amplitude of the user speech. ("[0061] In some implementations, as depicted in FIG. 2A, the wearable speech-assisting device 108 is configured to perform a noise reduction operation to reduce ambient noise, and determine whether the wearer's speech signal satisfies one or more speech parameters, for example, by comparing an amplitude of the wearer's speech signal to a predetermined threshold." – ref [0026] of provisional application (63/501,131))
Regarding claim 8, Anderson discloses: 8. A speech therapy device comprising: one or more sensors comprising a first microphone and a second microphone spaced apart from one another; (Fig. 2C shows two microphones spaced apart from one another and an accelerometer – ref Fig. 2 of provisional application (63/501,131))
one or more output components; ("[0140]...Output device(s) 512, such as a display, speakers, printer, etc., may also be included..." – ref [0037] of provisional application (63/501,131))
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the speech therapy device to perform acts comprising: ("[0139] In its most basic configuration, computing device 500 typically includes at least one processing unit 506 and system memory 504…" – ref [0034] of provisional application (63/501,131))
receiving, from the one or more sensors, data, ("[0071] At step 310, the method 300 includes receiving an audio signal, for example, via a plurality of acoustic sensors." ) – ref Fig. 1 of provisional application (63/501,131))
determining, based at least in part on different time of arrival values at the first microphone and the second microphone, that the data is indicative of speech of a user, ("[0102]...Two MEMS microphones and an accelerometer were used to determine if the user was speaking. This speech data was filtered to reduce environmental noise and then compared to the vocal sound pressure level thresholds." – ref [0026] of provisional application (63/501,131)) (Anderson does not explicitly disclose that it is based on time of arrival values.)
determining, based at least in part on the data being indicative of the speech, one or more biomarkers associated with the speech, ("[0074] At step 316, the method 300 includes determining whether the wearer's speech signal satisfies one or more speech parameters, including at least one of an acoustic intensity parameter or a speech rate parameter…" ref [0026] and [0014] of provisional application (63/501,131))
the one or more biomarkers including at least an amplitude associated with the speech, ("[0061] In some implementations, as depicted in FIG. 2A, the wearable speech-assisting device 108 is configured to perform a noise reduction operation to reduce ambient noise, and determine whether the wearer's speech signal satisfies one or more speech parameters, for example, by comparing an amplitude of the wearer's speech signal to a predetermined threshold." ref [0026] of provisional application (63/501,131))
determining that the one or more biomarkers fail to satisfy a threshold associated with hypophonia, and ("[0074]... In some implementations, the method 300 includes determining whether the wearer's speech signal satisfies an acoustic intensity parameter or speech rate parameter based on an amount of energy in the wearer's speech signal. By way of example, the at least one threshold for the acoustic intensity parameter can include a low acoustic intensity level and a high acoustic intensity level. Similarly, the at least one predetermined threshold for the speech rate parameter can include a low speech rate level and a high speech rate level." – the low acoustic intensity level is associated with hypophonia- ref [0026] of provisional application (63/501,131))
causing, based at least in part on the one or more biomarkers failing to satisfy the threshold, output of a notification via the one or more output components. ("[0075] At step 318, in response to detecting that the wearer's speech signal fails to satisfy at least one of the speech parameters, the method 300 includes outputting haptic biofeedback and/or other stimulation (e.g., audio, visual) to the wearer." ref [0011] of provisional application (63/501,131))
Anderson does not explicitly disclose: determining, based at least in part on different time of arrival values at the first microphone and the second microphone, that the data is indicative of speech of a user.
Sekiya discloses: determining, based at least in part on different time of arrival values at the first microphone and the second microphone, that the data is indicative of speech of a user. (“[0067] In one example, the controller 160 performs beamforming processing of forming the directivity to acquire a voice coming from the direction of the user's mouth using a plurality of voice data items acquired by the voice acquisition unit 110. The beamforming processing is a process of changing the degree of enhancement for each area where sound arrives. More specifically, the beamforming processing performed by the controller 160 may include a process of reducing sound coming from a specific area, or may include a process of enhancing sound coming from a desired orientation. In one example, the controller 160 may regard a voice coming from a direction other than the direction of the user's mouth as noise to be reduced. Furthermore, the controller 160 may enhance a voice coming from the direction of the user's mouth. As described above, the voice acquisition unit 110 does not necessarily have its own directivity. The controller 160 controls the directivity by performing the beamforming processing on the voice data acquired by each of the voice acquisition units 110. The controller 160 can perform the beamforming processing using the phase difference between the voice data items acquired by each of the voice acquisition units 110.” – [0044] is explicit that the phase difference is a time difference.)
Anderson and Sekiya are considered analogous art to the claimed invention because they disclose speech processing devices. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Anderson with microphones placed as disclosed by Sekiya, and to use the corresponding beamforming method disclosed by Sekiya. Doing so would have been beneficial to enhance the sound coming from a specific area and to reduce a voice coming from another direction from the user’s mouth. (Sekiya [0067]) Anderson discloses in Fig. 3B that the first beamforming operation is performed before detecting user speech; therefore replacing the beamforming of Anderson with the beamforming of Sekiya would result in using the different time of arrival values to detect the user’s speech.
Regarding claim 9, Anderson discloses: 9. The speech therapy device of claim 8, wherein the one or more sensors further comprise: at least one of an internal measurement unit (IMU), an accelerometer, a gyroscope, or a piezoelectric sensor. ("[0059] In some embodiments, the additional sensor(s) 204 include an accelerometer that facilitates accelerometer-based voice monitoring…" ref [0033] of provisional application (63/501,131))
Regarding claim 10, Anderson discloses: 10. The speech therapy device of claim 8, wherein the one or more output components comprise at least one of: a lighting element; a speaker; or a haptic motor. ("[0140]...Output device(s) 512, such as a display, speakers, printer, etc., may also be included..."; see also "[0010] In some embodiments, at least one of the feedback elements comprises at least one haptic biofeedback element, wherein the processor is further configured to output haptic biofeedback via the at least one haptic biofeedback element." ref [0011] of provisional application (63/501,131))
Regarding claim 15, Anderson discloses: 15. The speech therapy device of claim 8, further comprising a housing including a first end and a second end, (Fig. 2C shows a housing with two ends. ref Fig. 2 of provisional application (63/501,131))
wherein: the one or more sensors include a first microphone and a second microphone, the first microphone being located closer to the first end as compared to the second microphone, the second microphone being located closer to the second end as compared to the first microphone; and (Fig. 2C shows microphone 250a is closer to the first end than microphone 250b. 250b is closer to the second end than 250a. ref [0021] of provisional 63/501,131)
when the speech therapy device is worn by the user, the first microphone is located closer to a mouth of the user as compared to the second microphone. (Fig. 2A shows that the microphone at the first end is closer to the user's mouth than the microphone behind the user's ear. ref [0021] of provisional 63/501,131)
Regarding claim 16, Anderson discloses: 16. A speech therapy device configured to be worn by a user, ("[0004] An exemplary system and method are disclosed for a head-worn device for augmenting speech therapy or mitigating Parkinson's effect on speech and other speech-impairing conditions by providing haptic feedback, vibratory feedback, audio feedback, or other stimulations to a wearer by isolating and analyzing vocal/speech output of the user for signal-associated assessment, including, e.g., based on timing, direction, loudness, and/or speaking rate..." ref [0003] of provisional application (63/501,131))
the speech therapy device comprising: a first microphone; (Fig. 2C shows a first microphone 250a ref Fig. 2 of provisional application (63/501,131))
a second microphone, (Fig. 2C shows a second microphone 250b ref Fig. 2 of provisional application (63/501,131))
wherein the first microphone and the second microphone are configured to be spaced apart by different distances from a mouth of the user when the speech therapy device is worn by the user; (see Fig. 2B, 220b. 223b and 222b are spaced different distances from the mouth of the user. ref [0021] of provisional 63/501,131)
one or more output components; ("[0140]...Output device(s) 512, such as a display, speakers, printer, etc., may also be included..." - ref [0037] of provisional application (63/501,131))
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the speech therapy device to perform acts comprising: ("[0139] In its most basic configuration, computing device 500 typically includes at least one processing unit 506 and system memory 504…" – ref [0034] of provisional application (63/501,131))
receiving, from the first microphone, first audio data associated with a sound captured in an environment, receiving, from the second microphone, second audio data associated with the sound, ("[0071] At step 310, the method 300 includes receiving an audio signal, for example, via a plurality of acoustic sensors." – ref Fig. 1 of provisional application (63/501,131))
determining, based at least in part on the first audio data and the second audio data and different time of arrival values of the sound at the first microphone and the second microphone, that the sound is associated with user speech of the user, ("[0102]...Two MEMS microphones and an accelerometer were used to determine if the user was speaking. This speech data was filtered to reduce environmental noise and then compared to the vocal sound pressure level thresholds." – ref [0026] of provisional application (63/501,131)) (Different time of arrival values are not explicitly disclosed)
determining, based at least in part on the sound being associated with the user speech, one or more characteristics associated with the user speech, ("[0074] At step 316, the method 300 includes determining whether the wearer's speech signal satisfies one or more speech parameters, including at least one of an acoustic intensity parameter or a speech rate parameter…" – ref [0026] of provisional application (63/501,131))
determining that the one or more characteristics are indicative of hypophonia, and ("[0074]... In some implementations, the method 300 includes determining whether the wearer's speech signal satisfies an acoustic intensity parameter or speech rate parameter based on an amount of energy in the wearer's speech signal. By way of example, the at least one threshold for the acoustic intensity parameter can include a low acoustic intensity level and a high acoustic intensity level. Similarly, the at least one predetermined threshold for the speech rate parameter can include a low speech rate level and a high speech rate level." ref [0026] and [0014] of provisional application (63/501,131))
causing, based at least in part on the one or more characteristics being indicative of hypophonia, output of a notification via the one or more output components. ("[0075] At step 318, in response to detecting that the wearer's speech signal fails to satisfy at least one of the speech parameters, the method 300 includes outputting haptic biofeedback and/or other stimulation (e.g., audio, visual) to the wearer." – ref [0011] of provisional application (63/501,131))
Anderson does not explicitly disclose: determining user speech based on different time of arrival values of the audio at the first microphone and the second microphone.
Sekiya further discloses: determining user speech based on different time of arrival values of the sound at the first microphone and the second microphone. (“[0067] In one example, the controller 160 performs beamforming processing of forming the directivity to acquire a voice coming from the direction of the user's mouth using a plurality of voice data items acquired by the voice acquisition unit 110. The beamforming processing is a process of changing the degree of enhancement for each area where sound arrives. More specifically, the beamforming processing performed by the controller 160 may include a process of reducing sound coming from a specific area, or may include a process of enhancing sound coming from a desired orientation. In one example, the controller 160 may regard a voice coming from a direction other than the direction of the user's mouth as noise to be reduced. Furthermore, the controller 160 may enhance a voice coming from the direction of the user's mouth. As described above, the voice acquisition unit 110 does not necessarily have its own directivity. The controller 160 controls the directivity by performing the beamforming processing on the voice data acquired by each of the voice acquisition units 110. The controller 160 can perform the beamforming processing using the phase difference between the voice data items acquired by each of the voice acquisition units 110.” – [0044] is explicit that the phase difference is a time difference.)
Anderson and Sekiya are considered analogous art to the claimed invention because they disclose speech processing devices. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Anderson with the beamforming method disclosed by Sekiya. Doing so would have been beneficial to enhance the sound coming from a specific area and to reduce a voice coming from another direction from the user’s mouth. (Sekiya [0067]) Anderson discloses in Fig. 3B that the first beamforming operation is performed before detecting user speech; therefore replacing the beamforming of Anderson with the beamforming of Sekiya would result in using the different time of arrival values to detect the user’s speech. This combination falls under combining prior art elements according to known methods to yield predictable results or simple substitution of one known element for another to obtain predictable results. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Regarding claim 17, Anderson discloses: 17. The speech therapy device of claim 16, wherein the one or more output components comprise at least one of a speaker, a lighting element, or a haptic motor. ("[0140]...Output device(s) 512, such as a display, speakers, printer, etc., may also be included..."; see also "[0010] In some embodiments, at least one of the feedback elements comprises at least one haptic biofeedback element, wherein the processor is further configured to output haptic biofeedback via the at least one haptic biofeedback element." – ref [0011] of provisional application (63/501,131))
Regarding claim 20, Anderson discloses: 20. The speech therapy device of claim 16, wherein the one or more characteristics comprise at least one of a pitch of the user speech, an intonation in the user speech, a tone associated with the user speech, a pause in the user speech, a phonation associated with the user speech, or an amplitude of the user speech. ("[0061] In some implementations, as depicted in FIG. 2A, the wearable speech-assisting device 108 is configured to perform a noise reduction operation to reduce ambient noise, and determine whether the wearer's speech signal satisfies one or more speech parameters, for example, by comparing an amplitude of the wearer's speech signal to a predetermined threshold." – ref [0026] of provisional application (63/501,131))
Claim(s) 2, 11-12, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson in view of Sekiya as applied in claim 1 above, further in view of McNaney et al. ("LApp: A Speech Loudness Application for People with Parkinson’s on Google Glass").
Regarding claim 2, Anderson discloses: 2. The wearable speech therapy device of claim 1, the acts further comprising: receiving, from the first microphone, third audio data associated with second audio captured within the environment; receiving, from the second microphone, fourth audio data associated with the second audio; determining, based at least in part on the second audio being associated with the second user speech, one or more second biomarkers associated with the second user speech; determining that the one or more second biomarkers fail to satisfy the threshold associated with the second user speech containing hypophonia; and ("[0048]... The system 100 is configured to monitor a wearer's 111 (i.e., subject, user, patient) speech/voice as audio signals 101 via multiple acoustic sensors and continuously perform real-time analysis of the audio signals 101. " – see claim 1; [0048] discloses that the steps are performed continuously which reads on the limitations – ref [0026] and [0006]of provisional application (63/501,131))
causing, based at least in part on the one or more second biomarkers failing to satisfy the threshold, output of a second notification via the speaker, the second notification being different than the notification. (not explicitly disclosed)
Anderson does not explicitly disclose a second notification when the threshold for hypophonia is not met. Neither does Sekiya.
McNaney discloses: causing, based at least in part on the one or more second biomarkers failing to satisfy the threshold, output of a second notification via the speaker, the second notification being different than the notification. ("To this end we redesigned the cue as a large ‘thumbs up’ symbol that could be more easily seen peripherally and to provide positive reinforcement that appropriate volume levels were being met. " pg. 499, first para)
Anderson, Sekiya, and McNaney are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with a separate notification that the user’s speech is in the target volume zone as disclosed by McNaney. Doing so would have been beneficial to provide positive reinforcement. (McNaney pg. 499, first para)
Regarding claim 11, Anderson discloses: 11. The speech therapy device of claim 8, the acts further comprising: receiving, from the one or more sensors, second data; determining that the second data is indicative of second speech of the user; determining, based at least in part on the second data being indicative of the second speech, one or more second biomarkers associated with the second speech, the one or more second biomarkers including at least a second amplitude associated with the second speech; determining that the one or more second biomarkers satisfy the threshold associated with hypophonia; and ("[0048]... The system 100 is configured to monitor a wearer's 111 (i.e., subject, user, patient) speech/voice as audio signals 101 via multiple acoustic sensors and continuously perform real-time analysis of the audio signals 101. " – see claim 1; [0048] discloses that the steps are performed continuously which reads on the limitations – ref [0026] and [0006]of provisional application (63/501,131))
causing, based at least in part on the one or more second biomarkers satisfying the threshold, output of a second notification via the one or more output components, the second notification being different than the notification. (not explicitly disclosed)
Anderson does not explicitly disclose a second notification when the threshold associated with hypophonia is met (in this case the threshold being met indicates no hypophonia). Neither does Sekiya.
McNaney discloses: causing, based at least in part on the one or more second biomarkers satisfying the threshold, output of a second notification via the one or more output components, the second notification being different than the notification. ("To this end we redesigned the cue as a large ‘thumbs up’ symbol that could be more easily seen peripherally and to provide positive reinforcement that appropriate volume levels were being met. " pg. 499, first para)
Anderson, Sekiya, and McNaney are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with a separate notification that the user’s speech is in the target volume zone as disclosed by McNaney. Doing so would have been beneficial to provide positive reinforcement. (McNaney pg. 499, first para)
Regarding claim 12, Anderson discloses: 12. The speech therapy device of claim 11, the acts further comprising: receiving, from the one or more sensors, third data; determining that the third data is indicative of third speech of the user; determining, based at least in part on the third data being indicative of the third speech, one or more third biomarkers associated with the third speech, the one or more third biomarkers including at least a third amplitude associated with the third speech; determining that the one or more third biomarkers fail to satisfy the threshold associated with hypophonia; and causing, based at least in part on the one or more third biomarkers failing to satisfy the threshold, output of the notification via the one or more output components. ("[0048]... The system 100 is configured to monitor a wearer's 111 (i.e., subject, user, patient) speech/voice as audio signals 101 via multiple acoustic sensors and continuously perform real-time analysis of the audio signals 101. " – see claim 1; [0048] discloses that the steps are performed continuously which reads on the limitations – ref [0026] and [0006]of provisional application (63/501,131))
Regarding claim 19, Anderson discloses: 19. The speech therapy device of claim 16, the acts further comprising: receiving, from the first microphone, third audio data associated with a second sound captured in the environment; receiving, from the second microphone, fourth audio data associated with the second sound, determining, based at least in part on the third audio data and the fourth audio data, that the second sound is associated with second user speech of the user, determining, based at least in part on the second sound being associated with the second user speech of the user, one or more second characteristics associated with the second user speech; determining that the one or more second characteristics are not indicative of hypophonia; and ("[0048]... The system 100 is configured to monitor a wearer's 111 (i.e., subject, user, patient) speech/voice as audio signals 101 via multiple acoustic sensors and continuously perform real-time analysis of the audio signals 101. " – see claim 1; [0048] discloses that the steps are performed continuously which reads on the limitations – ref [0026] and [0006]of provisional application (63/501,131))
causing, based at least in part on the one or more second characteristics not being indicative of hypophonia, output of a second notification via the one or more output components, the second notification being different than the notification. (not explicitly disclosed)
Anderson does not explicitly disclose a second notification when hypophonia is not indicated. Neither does Sekiya.
McNaney discloses: causing, based at least in part on the one or more second characteristics not being indicative of hypophonia, output of a second notification via the one or more output components, the second notification being different than the notification. ("To this end we redesigned the cue as a large ‘thumbs up’ symbol that could be more easily seen peripherally and to provide positive reinforcement that appropriate volume levels were being met. " pg. 499, first para)
Anderson, Sekiya, and McNaney are considered analogous art to the claimed invention because they disclose devices for notifying the user of low speech volume. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with a separate notification that the user’s speech is in the target volume zone as disclosed by McNaney. Doing so would have been beneficial to provide positive reinforcement. (McNaney pg. 499, first para)
Claim(s) 4, 6, 13-14, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderson in view of Sekiya as applied in claim 1 above, further in view of Berisha et al. (US 20230045078 A1).
Regarding claim 4, Anderson discloses: 4. The wearable speech therapy device of claim 1, further comprising a sensor, the acts further comprising receiving, from the sensor, sensor data, ("[0059] In some embodiments, the additional sensor(s) 204 include an accelerometer that facilitates accelerometer-based voice monitoring…" – ref Fig. 2 of provisional application (63/501,131))
wherein: determining that the audio corresponds to the user speech is based at least in part on the sensor data; and ("[0102]...Two MEMS microphones and an accelerometer were used to determine if the user was speaking. This speech data was filtered to reduce environmental noise and then compared to the vocal sound pressure level thresholds." – ref Fig. 2 of provisional application (63/501,131))
determining the one or more biomarkers is based at least in part on the sensor data. ("[0137]... VoxLog uses both an accelerometer and an Air microphone, while APM solely relies on an accelerometer [2]. These latter two devices measure both SPL and the fundamental frequency (F0) of the patient's speech to derive additional measurements relating to vocal dose [3]..." – ref Fig. 2 of provisional application (63/501,131))
Anderson discloses that VoxLog can determine biomarkers based on accelerometer and microphone data, but does not explicitly disclose that they use this method for their system. Sekiya also does not disclose this.
Berisha discloses: determining the one or more biomarkers is based at least in part on the sensor data. (“[0058] In certain embodiments, such machine learning algorithms (or other signal processing approaches) may compare the multi-dimensional statistical signature against one or more baseline statistical signatures of speech production and respiratory abilities by comparing each of several features (e.g., articulation precision, respiratory support, nasality, prosody, and phonatory control) to corresponding baseline speech and respiration feature of one or more baseline statistical signatures of speech production and respiration abilities. In certain embodiments, the machine learning algorithms may also take into account additional data, such as sensor data (e.g., from an accelerometer or environmental sensor), a time of day, an ambient light level, and/or a device usage pattern of the user.”)
Anderson, Sekiya, and Berisha are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination to use a machine learning algorithm that takes into account accelerometer data as disclosed by Berisha. Doing so would have been beneficial to provide more efficient and objective results to the user. (Berisha [0005])
Regarding claim 6, Anderson does not disclose the additional limitations. Neither does Sekiya.
Berisha discloses: 6. The wearable speech therapy device of claim 1, wherein determining the one or more biomarkers is based at least in part on: providing, as an input to a machine-learned (ML) model ("[0017]...In some embodiments, the comparing the multi-dimensional statistical signature against the one or more baseline statistical signatures of speech production ability comprises applying a machine learning algorithm to the multi-dimensional statistical signature. In some embodiments, the machine learning algorithm is trained with past comparisons for other users. In some embodiments, extracting the multi-dimensional statistical signature of speech production abilities of the user from the input signal comprises measuring speech features across one or more of the following perceptual dimensions: articulation, prosodic variability, phonation changes, rate, and rate variation; and comparing the multi-dimensional statistical signature against the one or more baseline statistical signatures of speech production ability comprises comparing each speech feature to a corresponding baseline speech feature of the one or more baseline statistical signatures of speech production ability." )
trained to identify hypophonia, (Table 1 shows hypophonia is one of the conditions evaluated.)
the first audio data and the second audio data; and ("[0054] The audio input circuitry 108 may comprise at least one microphone. In certain embodiments, the audio input circuitry 108 may comprise a bone conduction microphone, a near field air conduction microphone array, or a combination thereof..." )
receiving, as an output from the ML model, an indication associated with the one or more biomarkers. ("[0101] In some embodiments, the systems, devices, and methods disclosed herein utilize one or algorithms or models configured to evaluate or assess speech and/or respiration, which may include generating an output indicative of a physiological state or condition or change (e.g., congestion, smoking cessation, etc.) corresponding to the speech and/or respiration evaluation." )
Anderson, Sekiya, and Berisha are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination to use a machine learning algorithm to detect if the speech is associated with hypophonia as disclosed by Berisha. Doing so would have been beneficial to provide more efficient and objective results to the user. (Berisha [0005])
Regarding claim 13, Anderson discloses: 13. The speech therapy device of claim 8, wherein the one or more biomarkers further include at least one of a pitch of the speech, an intonation in the speech, a tone associated with the speech, a pause in the speech, or a phonation associated with the speech. ("[0009] In some embodiments, the one or more speech parameters include at least one of an acoustic intensity parameter, a speech rate parameter, pitch, speech duration, voice quality, or response time." )
Anderson discloses pitch as a biomarker; however, this does not appear to be supported in Anderson’s provisional application. The instant application’s provisional application also does not appear to support the limitations of claim 13. However, for the purposes of compact prosecution, claim 13 is rejected over Anderson in view of Berisha. Sekiya does not disclose these limitations.
Berisha discloses: 13. The speech therapy device of claim 8, wherein the one or more biomarkers further include at least one of a pitch of the speech, an intonation in the speech, a tone associated with the speech, a pause in the speech, or a phonation associated with the speech. (“[0016]… In some embodiments, the signal processing circuitry is configured to process the input signal by measuring speech features represented in the input signal, the speech features comprising one or more of articulation rate, articulation entropy, vowel space area, energy decay slope, phonatory duration, and average pitch…”; see also “[0017]… In some embodiments, extracting the multi-dimensional statistical signature of speech production abilities of the user from the input signal comprises measuring speech features across one or more of the following perceptual dimensions: articulation, prosodic variability, phonation changes, rate, and rate variation; and comparing the multi-dimensional statistical signature against the one or more baseline statistical signatures of speech production ability comprises comparing each speech feature to a corresponding baseline speech feature of the one or more baseline statistical signatures of speech production ability.”)
Anderson, Sekiya, and Berisha are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination to use additional biomarkers to detect if the speech is associated with hypophonia as disclosed by Berisha. Doing so would have been beneficial so that the biomarkers could be compared to baseline speech signatures. (Berisha [0016])
Regarding claim 14, Anderson does not disclose the additional limitations. Neither does Sekiya.
Berisha discloses: 14. The speech therapy device of claim 8, wherein determining the one or more biomarkers is based at least in part on: providing, as an input to a machine-learned (ML) model trained to identify hypophonia, the data; ("[0017]...In some embodiments, the comparing the multi-dimensional statistical signature against the one or more baseline statistical signatures of speech production ability comprises applying a machine learning algorithm to the multi-dimensional statistical signature. In some embodiments, the machine learning algorithm is trained with past comparisons for other users. In some embodiments, extracting the multi-dimensional statistical signature of speech production abilities of the user from the input signal comprises measuring speech features across one or more of the following perceptual dimensions: articulation, prosodic variability, phonation changes, rate, and rate variation; and comparing the multi-dimensional statistical signature against the one or more baseline statistical signatures of speech production ability comprises comparing each speech feature to a corresponding baseline speech feature of the one or more baseline statistical signatures of speech production ability." )
and receiving, as an output from the ML model, an indication associated with the one or more biomarkers. ("[0101] In some embodiments, the systems, devices, and methods disclosed herein utilize one or algorithms or models configured to evaluate or assess speech and/or respiration, which may include generating an output indicative of a physiological state or condition or change (e.g., congestion, smoking cessation, etc.) corresponding to the speech and/or respiration evaluation." )
Anderson, Sekiya, and Berisha are considered analogous art to the claimed invention because they disclose devices for processing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination to use a machine learning algorithm to detect if the speech is associated with hypophonia as disclosed by Berisha. Doing so would have been beneficial to provide more efficient and objective results to the user. (Berisha [0005])
Regarding claim 18, Anderson discloses: 18. The speech therapy device of claim 16, further comprising one or more sensors that include at least one of an accelerometer, a gyroscope, an internal measurement unit (IMU), or a piezoelectric sensor, ("[0059] In some embodiments, the additional sensor(s) 204 include an accelerometer that facilitates accelerometer-based voice monitoring…" – ref [0033] of provisional application (63/501,131))
the acts further comprising receiving, from the one or more sensors, data, ("[0059] In some embodiments, the additional sensor(s) 204 include an accelerometer that facilitates accelerometer-based voice monitoring…" – ref [0033] of provisional application (63/501,131))
wherein: determining that the sound is associated with the user speech is based at least in part on the data; and ("[0102]...Two MEMS microphones and an accelerometer were used to determine if the user was speaking. This speech data was filtered to reduce environmental noise and then compared to the vocal sound pressure level thresholds." – ref [0033] of provisional application (63/501,131))
determining the one or more characteristics associated with the user speech is based at least in part on the data. ("[0137]... VoxLog uses both an accelerometer and an Air microphone, while APM solely relies on an accelerometer [2]. These latter two devices measure both SPL and the fundamental frequency (F0) of the patient's speech to derive additional measurements relating to vocal dose [3]..." – ref [0033] of provisional application (63/501,131))
Anderson discloses that VoxLog can determine characteristics based on accelerometer and microphone data, but does not explicitly disclose that they use this method for their system. Sekiya does not disclose the additional limitations.
Berisha discloses: determining the one or more characteristics associated with the user speech is based at least in part on the data. (“[0058] In certain embodiments, such machine learning algorithms (or other signal processing approaches) may compare the multi-dimensional statistical signature against one or more baseline statistical signatures of speech production and respiratory abilities by comparing each of several features (e.g., articulation precision, respiratory support, nasality, prosody, and phonatory control) to corresponding baseline speech and respiration feature of one or more baseline statistical signatures of speech production and respiration abilities. In certain embodiments, the machine learning algorithms may also take into account additional data, such as sensor data (e.g., from an accelerometer or environmental sensor), a time of day, an ambient light level, and/or a device usage pattern of the user.”)
Anderson, Sekiya, and Berisha are considered analogous art to the claimed invention because they disclose devices for notifying the user of low speech volume. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination to use a machine learning algorithm that takes into account accelerometer data as disclosed by Berisha. Doing so would have been beneficial to provide more efficient and objective results to the user. (Berisha [0005])
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JON C MEIS whose telephone number is (703)756-1566. The examiner can normally be reached Monday - Thursday, 8:30 am - 5:30 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JON CHRISTOPHER MEIS/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights