DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding applicant’s arguments filed 09 March 2026 that Mese, Lal, and Yu do not teach the newly amended claim language, the rejection below has been modified in response to the amended language.
Applicant argues that Yu does not teach “determining that the language settings correspond to the first language; determining that the cadence of the speech of the first user indicates a limited a limited understanding of the first language more familiar to the first user; and determining that a cadence of the speech of the one or more second users when speaking the second language indicates a better understanding of the second language”. In the prior rejection, Lal is invoked to read on the concept of determining a cadence of speech indicating a limited understanding of a language ([0003], [0014], [0017]: use speed and tone information to determine a user’s command of the language). Yu uses more than characteristics of the speaker’s utterance to determine how well the listener understands the utterance. Per [0194]: “For example, the electronic device 400 may determine whether the second user understands the first speech signal in the second language based on the first information including at least one of the second user input information 561, the language understanding history 522, language proficiency 524 of the second user, and the user profiling 523”. Lal’s use of a user’s speaking cadence to measure language proficiency makes an obvious combination with Yu’s use of user profiling/data collection to measure language proficiency, such as in [0180], language understanding history of a user is updated “when the second user uses (listen, write, speak, etc.) the first language”. The rejection has been adjusted accordingly to reflect this reasoning, see below for details.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-9, and 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mese et al. (US 20210043109 A1) in view of Yu (US 20220148567 A1) in view of Lal (US20190187952 A1).
Regarding claim 1, Mese et al. discloses a method comprising:
receiving, by a computing device, audio data corresponding to speech of a first user of a plurality of users and speech of one or more second users of the plurality of users; ([0045]: audio receivers that provide audio input based on user provided microphone input)
processing the audio data to determine one or more properties of the speech of the first user and one or more properties of the speech of the one or more second users, wherein the one or more properties of the speech of the first user comprise a cadence of the speech of the first user and a first language spoken by the first user; ([0057]: detecting a language spoken by the user)
comparing the one or more properties of the speech of the first user and the one or more properties of the speech of the one or more second users to language settings of the computing device; and ([0058]: comparing language of the user speech to language of the content)
determining, based on the comparing, that the language settings of the computing device correspond to a different language than a second language more familiar to the one or more second users by determining that the language settings correspond to the first language ([0008]: when a language is detected, user may be profiled as preferring that language;)
modifying, based on the determining that the language settings of the computing device correspond to the different language than the second language more familiar to the one or more second users, the language settings of the computing device to process subsequent speech based on a second language of the one or more different languages; ([0025]: detect when a user conversing with another in a language; [0058]: convert displayed content to the new language; [0057]: use multi-language or language-specific speech-to-text models)
Yu discloses, (but Mese et al. does not explicitly disclose):
determining based on the comparing that the language settings of the computing device correspond to a different language than a second language more familiar to the one or more second users by:
determining that the speech of the first user indicates a limited understanding of the first language spoken by the first user; ([0194]: determine whether a user understands a speech signal based on profile language proficiency; Fig 5, [0202]: adjust a translation level setting based on the relative levels of understanding of the two languages)
determining that the speech of the one or more second users when speaking the second language indicates a better understanding of the second language. ([0194]: determine whether a user understands a speech signal based on profile language proficiency; Fig 5, [0202]: adjust a translation level setting based on the relative levels of understanding of the two languages)
Mese tracks user language preferences, but does not directly track the level of language proficiency a user has. However, Yu does teach this feature. It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate the techniques of Yu into the method of Mese because by analyzing the relative proficiencies of a plurality of users, language settings such as for translation can be optimized for a smoother experience (see Yu [0063]).
Lal discloses, (but Mese and Yu do not explicitly disclose):
one or more properties of the speech comprising a cadence of speech of the user; ([0017]: analyze speed and tone of the user’s speech)
determining that the cadence of the speech of the user indicates a limited understanding of the language spoken by the user; ([0003], [0014], [0017]: use speed and tone information to determine a user’s command of the language)
determining that a cadence of the speech of the one or more second users when speaking the second language indicates a better understanding of the second language; ([0003], [0014], [0017]: use speed and tone information to determine a user’s command of the language)
Mese and Yu track user language preferences, but do not explicitly disclose doing so by analyzing speech cadence. Lal does teach this feature, and it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate the cited features of Lal with Mese et al. and Yu because one’s ability to speak in a language affects how well they are able to understand content in that language (See Lal [0003]).
Regarding claim 2, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Yu further discloses the method wherein modifying the language settings of the computing device comprises:
Determining that the second language is spoken by a majority of the plurality of users by determining for each user of the plurality of users, a language spoken by the user. (Figure 5 operation 553, [0212]: determining a language that each participant is speaking)
Regarding claim 3, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method further comprising, based on the comparing and based on the cadence of the speech of the user:
implementing accessibility features of the computing device. ([0004]: enabling of closed captions; [0057]-[0058]: modifying accessibility settings to display a different language)
Regarding claim 5, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method wherein modifying the language settings of the computing device comprises:
selecting, based on the one or more properties of the speech of the user, content; and
causing display of the content. ([0058]: converting content based on language spoken by user and displaying converted content)
Regarding claim 6, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method wherein modifying the language settings of the computing device comprises one or more of:
modifying subtitle settings of the computing device; or causing the computing device to perform machine translation of text content; ([0059]: in response to detecting a particular language, may display content as machine translated text, or represent audio as translated closed captioning)
Regarding claim 7, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method further comprising:
storing a user profile that indicates the one or more properties of the speech of the user; (Figure 12: settings page containing settings for a particular profile; [0023], [0065]: identifying a particular user based on identified characteristics of the user’s voice)
providing, to one or more second computing devices, the user profile. ([0070]: altering settings based on identified user, history of user)
Regarding claim 8, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method wherein modifying the language settings of the computing device comprises:
modifying display properties of a user interface provided by the computing device. ([0006]-[0007]: identifying a user from input and adjusting an accessibility setting that affects size of text on screen; [0065]: identifying a particular user based on identified characteristics of the user’s voice;)
Regarding claim 9, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method wherein two or more of the plurality of users speak a third language of the one or more different languages. (Figure 12, [0023], [0025]: store multiple policies or user-specific settings; any number of users can be profiled as preferring any number of languages)
Regarding claim 15, Mese et al. discloses a method comprising:
receiving, by a computing device, first audio data corresponding to speech by a first user; ([0045]: audio receivers that provide audio input based on user provided microphone input)
storing, based on one or more first properties of the speech of the first user, a user profile indicating language settings for the computing device; (Figure 12: settings page containing settings for a particular profile; [0023], [0065]: identifying a particular user based on identified characteristics of the user’s voice)
receiving, by the computing device, second audio data indicating speech by the first user in a first language and a speech of one or more second users is one or more different languages; ([0045]: audio receivers that provide audio input based on user provided microphone input; [0023]: continues to detect for a mismatch between the user’s audio input and the accessibility settings currently being used; [0025]: detect if a user is speaking to another in a certain language)
comparing the language settings with one or more second properties of the second audio data; and (Fig 12: Per-user accessibility settings; [0023]: continue to detect for a mismatch between the user’s audio and the current settings, [0057]: detecting a language spoken by the user)
based on the comparing, and based on the one or more second properties of the second audio data, modifying the user profile by adding an indication that causes the computing device to process subsequent speech based on a second language of the one or more different languages. ([0023], [0025]: upon detection of a mismatch, adjust the settings to the new state, i.e. change a user’s preferred language to Spanish when Spanish is detected from the user; [0057]: use multi-language or language-specific speech-to-text models)
Yu discloses, (but Mese et al. does not explicitly disclose):
determining based on the comparing that the language settings of the computing device correspond to a different language than a second language more familiar to the one or more second users by:
determining that the speech of the first user indicates a limited understanding of the first language spoken by the first user; ([0194]: determine whether a user understands a speech signal based on profile language proficiency; Fig 5, [0202]: adjust a translation level setting based on the relative levels of understanding of the two languages)
determining that the speech of the one or more second users when speaking the second language indicates a better understanding of the second language. ([0194]: determine whether a user understands a speech signal based on profile language proficiency; Fig 5, [0202]: adjust a translation level setting based on the relative levels of understanding of the two languages)
Mese tracks user language preferences, but does not directly track the level of language proficiency a user has. However, Yu does teach this feature. It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate the techniques of Yu into the method of Mese because by analyzing the relative proficiencies of a plurality of users, language settings such as for translation can be optimized for a smoother experience (see Yu [0063]).
Lal discloses, (but Mese and Yu do not explicitly disclose):
one or more properties of the speech comprising a cadence of speech of the user; ([0017]: analyze speed and tone of the user’s speech)
determining that the cadence of the speech of the user indicates a limited understanding of the language spoken by the user; ([0003], [0014], [0017]: use speed and tone information to determine a user’s command of the language)
determining that a cadence of the speech of the one or more second users when speaking the second language indicates a better understanding of the second language; ([0003], [0014], [0017]: use speed and tone information to determine a user’s command of the language)
Mese and Yu track user language preferences, but do not explicitly disclose doing so by analyzing speech cadence. Lal does teach this feature, and it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate the cited features of Lal with Mese et al. and Yu because one’s ability to speak in a language affects how well they are able to understand content in that language (See Lal [0003]).
Regarding claim 16, Mese, Lal, and Yu disclose its dependent elements as written above for claim 15. Mese et al. further discloses the method wherein comparing the language settings with the one or more second properties of the second audio data comprises:
determining whether the second audio data is associated with the first user. ([0023], [0065]: associate the input with a user from characteristics of the audio)
Regarding claim 17, Mese, Lal, and Yu disclose its dependent elements as written above for claim 15. Mese et al. further discloses the method wherein receiving the first audio data comprises receiving the first audio data via a first user device associated with the first user, and wherein receiving the second audio data comprises receiving the second audio data via a second user device associated the one or more second users. (Fig 2: various user devices can connect to network 200/processor 214 which performs the method logic in an embodiment; [0065]: the user may be identified by the particular device the input came from)
Regarding claim 18, Mese, Lal, and Yu disclose its dependent elements as written above for claim 15. Mese et al. further discloses the method wherein the modified user profile indicates a plurality of languages, the method further comprising:
causing display of video content corresponding to a first language of the plurality of languages; ([0058]: convert at least some of the content into a language spoken is not necessarily all; [0022], [0024]: content can include audio-visual content)
causing display of subtitles corresponding to a second language of the plurality of languages. ([0059]: enabling closed captioning for a detected language)
Regarding claim 19, Mese, Lal, and Yu disclose its dependent elements as written above for claim 15. Mese et al. further discloses the method wherein modifying the user profile comprises:
adding, to the user profile, an indication of an accessibility feature to be implemented via the computing device. ([0007]: changing a setting from a setting already associated with and tailored to a particular using)
Regarding claim 20, Mese, Lal, and Yu disclose its dependent elements as written above for claim 15. Mese et al. further discloses the method causing a second computing device to display content based on the modified user profile. ([0058]: changing a setting and converting at least some displayed content to that of the new language; [0007]: changing a setting from a setting already associated with and tailored to a particular user; Fig 2, [0065]: a user can be detected from the particular device used, with the method logic being implemented on a remote server, thus settings previously changed will carry over to your user profile which can then be invoked from a different device)
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mese et al. in view of Yu in view of Lal as applied to claim 1 above, and further in view of Hart et al. (US 20120062729 A1).
Regarding claim 4, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al. further discloses the method wherein receiving the audio data corresponding to the speech of the first user comprises receiving the audio data corresponding to the speech of the first user via a user device, ([0045]: audio receivers that provide audio input based on user provided microphone input; Fig 2: Various user devices)
However, Mese, Lal, and Yu do not disclose the method further comprising modifying, based on the one or more properties of the speech of the first user, recording settings of the user device.
Hart et al. does disclose the method further comprising modifying, based on the one or more properties of the speech of the first user, recording settings of the user device. ([0047]: adjust a gain of the audio based on received data from audio capture element; [0044]: determine a location of the user from the audio data, then filter out noise not coming from the user)
It would have been obvious to one with ordinary skill in the art before the effective of the claimed invention to modify recording settings (settings that adjust how the audio is collected, such as filters or gain) upon receiving speech from the user as taught by Hart et al. when performing the method of Mese et al. as it helps, for example, filter out unwanted noise from the speech signal (see Hart et al. [0044], [0047])
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mese et al. in view of Yu in view of Lal as applied to claim 1 above, and further in view of Palaniswami (US 20220141540 A1).
Regarding claim 10, Mese, Lal, and Yu disclose its dependent elements as written above for claim 1. Mese et al., Yu, and Lal do not disclose training, using training data, a machine learning model to identify speech properties of users, wherein the training data comprises associations between audio content corresponding to speech of a plurality of different users and properties of the speech of the plurality of different users;
providing, as input to the trained machine learning model, the audio data; and
receiving, as output from the trained machine learning model, an indication of the one or more properties of the speech of the first user.
However, Palaniswami et al. does disclose the method wherein the processing the audio data to determine one or more properties of the speech of the user comprises:
training, using training data, a machine learning model to identify speech properties of users, wherein the training data comprises associations between audio content corresponding to speech of a plurality of different users and properties of the speech of the plurality of different users; ([0057]: storing data including audio inputs received from users and detection results, using that data to train a model)
providing, as input to the trained machine learning model, the audio data; and ([0052]: using audio data and a trained model)
receiving, as output from the trained machine learning model, an indication of the one or more properties of the speech of the user. ([0052]: the trained model is a language detection model that separates received audio into a particular language)
Use of machine learning models to improve accuracy of speech property detection is a well understood technique in the art. It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate trained machine models from Palaniswami et al. into Mese et al.’s method because such a trained model may more accurately detect the language being spoken (Palaniswami et al. [0057]).
Claim(s) 11-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mese et al. (US 20210043109 A1) in view of Palaniswami et al. (US 20220141540 A1) in view of Lal (US 20190186952 A1) in view of Yu (US 20220148567 A1).
Regarding claim 11, analogous elements of this claim that are present in claim 1 are taught as explained above for claim 1.
Mese, Yu, and Lal fail to disclose training, using training data, a machine learning model to identify speech properties of users, wherein the training data comprises associations between audio content corresponding to speech of a plurality of different users and properties of the speech of the plurality of different users;
providing, as input to the trained machine learning model, the audio data;
receiving, as output from the trained machine learning model, an indication of one or more properties of the speech of the first user;
However, Palaniswami does disclose training, using training data, a machine learning model to identify speech properties of users, wherein the training data comprises associations between audio content corresponding to speech of a plurality of different users and properties of the speech of the plurality of different users; ([0057]: storing data including audio inputs received from users and detection results, using that data to train a model)
providing, as input to the trained machine learning model, the audio data; ([0052]: using audio data and a trained model)
receiving, as output from the trained machine learning model, an indication of one or more properties of the speech of the first user, wherein the one or more properties of the speech of the first user comprise a language spoken by the first user; ([0052]: the trained model is a language detection model that separates received audio into a particular language)
Use of machine learning models to improve accuracy of speech property detection is a well understood technique in the art. It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to incorporate trained machine models from Palaniswami et al. into Mese et al.’s method because such a trained model may more accurately detect the language being spoken (Palaniswami et al. [0057]).
Regarding claim 12, Mese et al., Palaniswami et al., Lal, and Yu disclose its dependent elements as written above for claim 11. Palaniswami et al. further discloses the method further comprising
after modifying the language settings of the computing device, receiving an indication that the first user further modified the language settings of the computing device; (Fig 3C: When Spanish is spoken, system indicates that the audio track will be changed)
causing the trained machine learning model to be further trained based on the indication that the first user further modified the language settings of the computing device. ([0057]: system stores logging data that includes audio data from users and language detection outputs. This logging data is used to further train the learning model)
Regarding claim 13, Mese et al., Palaniswami et al., Lal, and Yu disclose its dependent elements as written above for claim 11. Palaniswami et al. further discloses the method wherein the audio content corresponding to speech of the plurality of different users corresponds to commands spoken by the plurality of different users, and wherein the properties of the speech of the plurality of different users indicates languages of the commands spoken by the plurality of different users. ([0029]: detect a language being spoken by multiple people in a conversation)
Regarding claim 14, Mese et al., Palaniswami et al., Lal, and Yu disclose its dependent elements as written above for claim 11. Mese et al. discloses the method further comprising, based on the cadence of the speech of the first user:
implementing accessibility features of the computing device. ([0004]: enabling of closed captions; [0057]-[0058]: modifying accessibility settings to display a different language)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892.
Mehra (US 20170025116) discloses the identification of commonly spoken languages in a household to help display appropriate media content.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALVIN ISKENDER whose telephone number is (703)756-4565. The examiner can normally be reached M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HAI PHAN can be reached on (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALVIN ISKENDER/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights