DETAILED ACTION
Claims 1, 13 – 19 and 21 – 24 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed on 24 February 2026 has been entered.
Response to Amendment
With regard to the Final Office Action from 28 October 2025 and the Advisory Action from 12 February 2026, the Applicant has filed a response on 24 February 2026.
Claim 23 was objected to for a minor informality. The claim has been amended to rectify this situation, and the Examiner hereby withdraws the claim objection.
Response to Arguments
With regard to the 35 U.S.C. 103 rejection given to the independent claims, the Applicant argues against the Examiner’s use of the Chevrier et al. reference (Remarks: page 10 par 1), indicating that contrary to the Examiner’s indication, ‘performing the transcription of the communication session in real-time is not the same as determining, before the communication ends, whether or not the speech data being acquired during the communication with the first device 104 can be converted into the text data.’ The Applicant further indicates (Remarks: page 10 par 2) that this Chevrier et al. reference does not determine, before a communication with the first terminal ends, whether or not the speech data being acquired during the communication with the device 104 can be converted into the text data, nor determine that the speech data acquired during the communication with the device 104 cannot be converted into the text data due to a deterioration of a communication state of the communication with the device 104 and transmit a display instruction based on a determination result indicating that the speech data cannot be converted into the text data to the device 106 in a case where the audio data is not transmitted from the device 104 for a certain period of time and the communication with the device 104 is determined as being disconnected, where the display content on the state of the environmental sound or the state of the speech acquisition device. The Examiner refers to the previously-applied [0035] of the Chevrier et al. reference to teach real-time speech transcription, indicating a situation whereby it is determined in the process of receiving speech communication, that this communication can be converted into text data. Later in [0096], it is provided that the sending of audio for transcription is stopped through the course of maintaining a communication session when the voice has been inactive for a time period, further leading to a termination of the communication session. This further provides teaching for a situation involving the determination that speech data can no longer be converted to text, the lack of input audio being an example of a deterioration in the speech data since the speech data would not be present.
The Applicant further states (Remarks: page 11 par 1) that the KO reference does not determine whether or not the speech can be converted into the audio signal before the communication ends. The Examiner holds that the reference of KO is not applied to this limitation.
The Applicant argues (Remarks: page 11 par 2–4) that the there is no motivation to combine the references of Chevrier at el. and KO, claiming that it would not have been obvious to combine KO’s system requiring two physically distinct devices with Chevrier’s which includes only the single transcription system while its first and second devices 104 and 106 do not have the transcription function. The Examiner indicates that both systems are in the same field of speech recognition, which, by that alone, makes them suitable for combination, regardless of the other differences between them. Further, the Examiner indicates that the reason for combination here is based on the determination of the environmental sound through an SNR calculation of KO, with the receipt of speech for the purpose of recognition as taught by Chevrier et al. The combination yields a predictable result, as previously indicated, and suitable as a reason for the combination of both references.
The independent claims have introduced subject matter that was not considered previously. In light of these newly-introduced subject, the claims will be addressed by their current presentation.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 21 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Chevrier et al. (US 2019/0228774 A1: hereafter — Chevrier) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand et al. (US 2023/0063839 A1: hereafter — Bromand).
For claim 1, Chevrier discloses a system comprising:
a first terminal (Chevrier: FIG. 1, [0014] — a first device 104);
a second terminal physically distinct from and spatially separated from the first terminal (Chevrier: FIG. 1, [0014]–[0015] — a second device 106 which is physically distinct from the first device, and the network also shows that the type of communication that can occur between both devices would present that both devices are spatially separate from each other); and
a server apparatus configured to perform a bidirectional communication between the first terminal and the second terminal (Chevrier: FIG. 1, [0014]–[0015] — a network 102 as a server apparatus for enabling the bidirectional communication between the first and second terminals), wherein:
the first terminal includes:
a speech acquisition device configured to acquire audio data including speech data and environmental sound in real-time (Chevrier: [0020] — the first device may obtain audio from a microphone of the first device; [0023] — capturing a first user speaking (indicating the capturing of audio data that includes speech data) and sound that does not include the first user speaker such as music and crying and other sounds that are not spoken words (environmental sound); [0035] — the transcription may be performed in real-time (indicating acquiring audio data including speech data and environmental sound in real-time)),
a first processor (Chevrier: [0017] — the first device includes a processor) [[configured to detect a state of the environmental sound and a state of the speech acquisition device and output a detection result indicating the state of the environmental sound and the state of the speech acquisition device]], and
a first communication device configured to transmit the acquired audio data [[and the detection result to the server apparatus]] (Chevrier: [0018]–[0019] — audio communication between the first and second devices (showing that the first device also contains a communication device to transmit acquired speech)),
the server apparatus includes:
a second communication device communicable with the first terminal and the second terminal and configured to receive the audio data [[and the detection result from the first terminal]] (Chevrier: [0028] — enhancing communication sessions between two devices in communication on the network), and
a second processor configured (Chevrier: [0028] — the transcription system at the network may include a processor) to:
convert the speech data of the received audio data into text data in real-time (Chevrier: [0035] — ‘[t]he transcription of the communication session may occur in real-time’ (indicating a transcription of the speech into text));
determine, before a communication with the first terminal ends, whether or not the speech data being acquired during the communication with the first terminal can be converted into the text data (Chevrier: [0035] — the transcription of the communication session my occur in real-time (indicating that this is happening while the communication is in session, before the communication ends, the performing of the transcription also being a determination that speech data can be converted into text data); [0020] — acquiring speech from a microphone of the first device); and
[[determine that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a deterioration of a communication state of the communication with the first terminal and]] transmit, [[a display instruction based on a determination result indicating that the audio data cannot be converted into the text data to the second terminal]] via the second communication device, in a case where the speech data is not transmitted from the first terminal for a certain period of time and the communication with the first terminal is determined as being disconnected (Chevrier: [0093] — the sending of the audio to the transcription system may be stopped in the event that the voice is inactive (the voice being inactive is an indication that the speech data has not been transmitted for a time period); FIG. 4 Steps 408 → 412 — terminating the network in the situation where voice is not detected for a time period (the termination indicating a cessation in transcription); FIG. 5 Step 512 — when the voice is inactive, stop sending the audio to the transcription system while maintaining the communication session; [0039] — a time period of inactivity that is able to determine that the conversation has ended (thereby teaching of a disconnection of the first terminal since the conversation is determined to have been ended); [0069] — the display is configured to receive transcripts from the processor of the server for display at the second device; [0037] — in response to the voice being inactive (the presence of audio that cannot be converted into text), audio does not get sent to the transcription system, (when audio isn’t sent to the transcription system, the system does not display any transcript, which would be a visual or display indication/instruction that received audio data cannot be converted into text)), and
the second terminal includes a third communication device communicable with the server apparatus, a display device, and a third processor [[configured to display the determination result of the server apparatus on the display device in response to the display instruction received]] by the third communication device from the server apparatus (Chevrier: [0017] — the second device includes a processor; [0018] — the second device being able to maintain communication with the first device, with FIG. 1 showing the network server as an intermediary; [0034] — the second device comprises a display (able to display information)).
The reference of Chevrier fails to disclose the further limitations of this claim regarding the determining a state of environmental sound and of the speech acquisition device to output a detection result. This is however not new to the art as the reference of Ko is now introduced to teach as:
a first processor configured to detect a state of the environmental sound and a state of the speech acquisition device and output a detection result indicating the state of the environmental sound and the state of the speech acquisition device (KO: [0055] — determining a signal-to-noise ratio (SNR) after detecting ambient noise level for a determination by the electronic device to perform ASR (teaching of determining an environmental sound state along with the speech acquisition device’s state in form of the SNR, the SNR itself being a measure of both states, such that as is known in the art, a high SNR indicates that the status of the speech acquisition device is in proper form for transcribing speech, and a low SNR would instead indicate that the status of environmental sound is high which would obstruct performing speech transcription) and if the result of SNR is less than a preset value, the electronic device performs ASR (as an outputting of a detection result)), and
a first communication device configured to transmit the acquired audio data and the detection result to the server apparatus (KO: [0055] — when the SNR result is above a preset value, ASR gets performed at the server (teaching of transmitting the speech data and, by this reference, since the speech data gets transmitted to the server for the server to process it, this serves as a transmittal of the detection result of the of the state of the environmental sound and of the device, over to the server, to allow the server perform the ASR instead of the local device)),
the server apparatus includes:
a second communication device communicable with the first terminal and the second terminal and configured to receive the audio data and the detection result from the first terminal (KO: [0055] — when the SNR result is above a preset value, ASR gets performed at the server (teaching of the server receiving the speech data and, by this reference, since the speech data gets transmitted to the server for the server to process it, this serves as a transmittal of the detection result of the of the state of the environmental sound and of the device, over to the server, to allow the server perform the ASR instead of the local device)).
The reference of Chevrier provides teaching for receiving speech for recognition at a first device, but differs from the claimed invention in that the claimed invention further provides obtaining a detection result based on detecting the states of environmental sound and that of the speech acquisition device. This isn’t new to the art as the reference of KO is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of KO which determines the states of environmental sound and that of the acquisition device to determine a detection result associated with received input speech in the form of an SNR, with the teaching of receiving speech data for recognition in real-time, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of being able to evaluate if there’s too much noise in a received speech signal, that speech recognition might not be feasible. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
The combination of Chevrier in view of KO fails to teach the further limitations of this claim, for which the reference of Morita is now introduced to teach as:
determine that the speech data acquired during the communication with the first terminal cannot be converted into the text data due to a deterioration of a communication state of the communication with the first terminal and transmit a display instruction based on a determination result indicating that the speech data cannot be converted into the text data to the second terminal via the second communication device, in a case where the speech data is not transmitted from the first terminal for a certain period of time and the communication with the first terminal is determined as being disconnected]] (Morita: [0057] — a determination of a poor sound pickup condition, leading the (speech) voice recognition unit to instruct the output unit through a display, to have the speaker repeat his/her words, indicating that speech recognition cannot be performed due to poor sound quality).
The combination of Chevrier in view of KO provides teaching for receiving speech for recognition at a first device and converting the speech into text, but differs from the claimed invention in that the claimed invention further provides a determination that the acquired speech data cannot be converted into text because of a deterioration of a communication state, and displaying this to the user. This isn’t new to the art as the reference of Morita is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of Morita which determines that there is a poor sound pick resulting in the received speech being unable to be recognized, and displaying this determination to the speaker, with the teaching of receiving speech data for recognition in real-time, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of not having to allocate processing resources to speech data that is determined to be too corrupt or degraded for ASR to be performed on, leading to the conservation of processing power for an alternative process, while also allowing a user adjust the audio pick-up conditions. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
The combination of Chevrier in view of KO further in view of Morita fails to disclose the limitation of displaying the determination result as received by the server. This isn’t new to the art as the reference of Bromand is now introduced to teach as:
the display instruction including a display content on the state of the environmental sound or the state of the speech acquisition device (Bromand: [0102] — communicating a display message to the user that states ‘Wind noise from the air vents may be impacting voice recognition. Please try lowering the A/C fan speed’ (this being a display instruction regarding the state of the environmental sound));
[[the second terminal includes a third communication device communicable with the server apparatus, a display device, and a third processor]] configured to display the determination result of the server apparatus on the display device in response to the display instruction received by the third communication device from the server apparatus (Bromand: [0100] — a voice interaction server that is able to provide assistance and feedbacks services (to a user device); [0102] — communicating a display message to the user that states ‘Wind noise from the air vents may be impacting voice recognition. Please try lowering the A/C fan speed’).
The combination of Chevrier in view of KO further in view of Morita provides teaching for a second terminal that receives communication from the server, but differs from the claimed invention in that the claimed invention further provides teaching for displaying the state of an environmental sound and a determination result that acquired speech data cannot be converted into text. This is however seen to be taught by the reference of Bromand as provided above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of Bromand which sends information from a server that speech cannot be recognised based on a determined environmental sound status, to a user terminal, with the teaching of the receiving of communication at a second terminal from a server as taught by the combination of Chevrier in view of KO further in view of Morita, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of duly informing a user of the second terminal, that speech recognition of received speech data is not possible, so that the user of the second terminal may inform the user of the first terminal and both parties may attempt to mitigate the situation. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
For claim 21, it is analysed and rejected by the same reasons set forth in the rejection of claim 1 above given that both instant claims have similar limitations. Chevrier provides in FIG. 1, a transcription system located at the server with [0069] providing that the server has a processor, [0066] provides a memory that stores executable programming instructions, these being suitable to read upon the limitations of this claim.
As for claim 22, computer program product claim 22 and system claim 1 are related as computer program product storing executable instructions required for the operation of the system as programmable steps on a computer. Chevrier in [0068] provides teaching for non-transitory computer-readable storage media suitable to read upon the limitations of this claim. Accordingly, claim 22 is similarly rejected under the same rationale as applied above with respect to system claim 1.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), as applied to claim 1, further in view of KANG et al. (US 2017/0116987 A1: hereafter — Kang).
For claim 13, claim 1 is incorporated and the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand provides teaching for displaying an indication of the determination result of whether or not the acquired speech data is able to be converted or not able to be converted into text (Chevrier: [0035] — the second device (a control unit present in the supporter terminal) being able to display a transcript of a real-time audio communication session).
This combination however fails to disclose the further limitations of this claim, regarding an icon to indicate that speech data cannot be converted into text data. This is however seen to be taught by the reference of Kang as:
the system, wherein the third processor included in the second terminal displays the determination result on the display device using an icon indicating a cause the speech data cannot be converted into the text data, or a cause a quality of the converted text data after the conversion deteriorates (Kang: [0163] — in a situation where speech recognition fails due to a network problem or unrecognisable speech input, the electronic device may change the colour of a call icon as illustrated on a screen to indicate a recognition failure).
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the teaching of the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand which displays an indication of the determination result that acquired speech can or cannot be converted into text, with the known teaching of Kang which changes the colour of a call based on a situation that speech recognition fails as a result of a network problem, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of making the determination result that speech can or cannot be converted into text very visible to the user, so that the user can be quickly made aware of the transcription situation. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), as applied to claim 1, and further in view of Thomson et al. (US 10,971,153 B2: hereafter — Thomson).
For claim 14, claim 1 is incorporated and the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand discloses the system, wherein:
the first terminal further includes an output device (Chevrier: [0032] — the first device has a speaker as an output device; [0016] — the first terminal may also be a desktop computer (known to include a display output device)).
The combination of Chevrier in view of KO further in view of Morita and further in view of Bromand fails to disclose the further limitations of this claim, for which the further reference of Thomson is now introduced to teach as:
the first processor included in the first terminal is configured to:
determine, during the acquisition of the speech data by the speech acquisition device, whether or not a quality of the converted text data after the conversion of the speech data being acquired deteriorates (Thomson: page 158 Col 158 line 53 – Col 159 line 4 — obtaining quality measures such as accuracy or error rates regarding speech transcription and being able to make notifications such as “3% below last weeks’ top revoiced ASR system” to show a deterioration in the transcription text); and
output a determination result on the quality of the converted text data to the output device to notify the determination result to a user of the first terminal by at least one of sound, light, and vibration (Thomson: Col 158 Col 158 line 53 – Col 159 line 4 — providing a text notification of the text quality in display form (the text notification on a display is indicative of a light style notification)).
The combination of Chevrier in view of KO further in view of Morita and further in view of Bromand provides teaching for the first terminal having an output device, but differs from the claimed invention in that the claimed invention further provides teaching for determining whether the quality of the recognition text deteriorates and outputting the determination result on the first terminal through light. This isn’t new to the art as the reference of Thomson is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the teaching of Thomson which notifies a user through light, of a deterioration in the quality of recognition text, with the teaching of the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand which includes an output device at the first terminal, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of informing the user of the first device of a declining speech recognition quality leading that’s leading to a deterioration in text quality, so that the user can perform an action to attempt to mitigate the recognition issue, such as speaking more clearly. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Claims 15, 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), as applied to claim 1, and further in view of FURUKAWA et al. (US 2020/0311354 A1: hereafter — Furukawa).
For claim 15, claim 1 is incorporated but the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand fails to disclose the limitation of this claim, for which the reference of Furukawa is now introduced to teach as the system, wherein the second processor included in the server apparatus stores the text data and the determination result in association with each other (Furukawa: [0268] — saving at a cloud server, the text of a transcription of user speeches (the storing of the transcripts at the server indicates the presence of memory as well as well the claimed control unit located at the cloud server; also, the storage of the speech transcription is an indication of the determination that the speech can be transcribed into text, thereby also teaching the determination result); [0269] — the speech recognition being performed at the server; [0273] — the presence of a processor or CPU for performing the needed tasks (to show the presence of a processor at the server)).
The combination of Chevrier in view of KO further in view of Morita and further in view of Bromand provides teaching for the transcription of a live speech communication between participants, but differs from the claimed invention in that the claimed invention further provides teaching for storing the transcription text data and the determination result at the processor included in the server apparatus. This is however not new to the art as the reference of Furukawa is seen above to teach.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to incorporate the known teaching of Furukawa which stores transcription text at a server, with the teaching of performing speech transcription of a conversation between participants as taught by the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of making use of less storage space at either pf the devices, by storing the transcripts at the server instead where there might be better memory allocation. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
For claim 16, claim 15 is incorporated and the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand and further in view of Furukawa discloses the system, wherein the third processor included in the second terminal reads the stored text data and displays the read text data in a display mode according to the determination result (Furukawa: [0269] — transmitting from a cloud server, the text, to a [second terminal] device (indicating a reading of the stored text from the server); FIG. 3 Step S15 — displaying the translated text (displaying the read text based on the determination that the speech can be transcribed)).
For claim 19, claim 15 is incorporated and the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand and further in view of Furukawa discloses the system, wherein the first processor included in the first terminal stores speech data acquired by speech acquisition device during the communication with the server apparatus, and in a case where a request is received by the first communication device from the server apparatus, transmits the speech data in a range according to the request to the server apparatus via the first communication device (Chevrier: [0044] — storing the audio communications locally on the device (indicating a storage of the acquired speech at the first terminal terminal); [0053] — the processing system (indicating the first processor also) from the terminal may provide the speech data to the transcription system which is located at the server (such that the speech data is provided in communication chunks based on each received audio, taking this as the range); FIG. 5 Step 504 — establishing a connection with a transcription system (as an indication of a request for receiving audio from the device based on the establishment of connection with the server)).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), and further in view of Furukawa (US 2020/0311354 A1) as applied to claim 15, and further in view of Gurbani et al. US 2021/0142789 A1: hereafter — Gurbani).
For claim 17, claim 15 is incorporated but the combination of Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Furukawa fails to teach the limitation of this claim, for which the reference of Gurbani is now introduced to teach as the system, wherein the second processor included in the server apparatus corrects a portion that cannot be converted into the text data or a portion where a quality deteriorates, using text data before and after the portion that cannot be converted into the text data or before and after the portion where the quality deteriorates (Gurbani: [0037] — an error resolution component that is able to correct a mis-transcribed word making use of surrounding language characteristics and contextual cues (correcting a portion that is incorrect or that is not properly converted into text, making use of contextual data before and after the incorrect portion, also note that a mis-transcribed word within a collection of text would indicate a deterioration in the quality of the sentence structure); [0022] — a processor included at the server).
The combination of Chevrier in view of KO further in view of Morita, further in view of Bromand and further in view of Furukawa provides teaching for storing the transcribed text at the server, but differs from the claimed invention in that the claimed invention further provides teaching for the correction of a portion that cannot be converted into text data making use of text data before and after the portion. This isn’t new to the art as the reference of Gurbani goes to sow above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to incorporate the known teaching of Gurbani which applies surrounding context information to correcting misrecognised text, with the teaching of the storing of transcribed text at a server as provided by the combination Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Furukawa, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of ensuring that the transcribed text being sent to either of the users at either terminal is reasonably correct and free of errors. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), and further in view of Furukawa (US 2020/0311354 A1) as applied to claim 15, and further in view of Saadatpanah (US 2022/0392435 A1).
For claim 18, claim 15 is incorporated but the combination of Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Furukawa discloses the system, wherein:
the first terminal further includes an imaging device configured to capture an image (Chevrier: [0018] — the first terminal is able to perform a video call (indicating that the device is able to capture an image));
the first processor of the first terminal transmits the image captured by the imaging device to the server apparatus via the first communication device (Chevrier: [0018] — the first device is able to perform video call useful in outgoing communication session (indicating a transferring of captured images to the server); [0025] — the first device and the second device may communicate with each other over a network 102 with [0019] showing that audio and video communications are exchanged between the first and second devices (teaching of transmitting the captured image to the server apparatus)).
The combination of Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Furukawa fails to teach the further limitations of this claim, for which the reference of Saadatpanah is now introduced to teach as:
the second processor included in the server apparatus identifies a work content of the worker from the image device by the second communication device from the first terminal, and stores the identified work content in association with the determination result (Saadatpanah: [0069] — capturing contextual information from the device of a user at a time when the user uttered a speech command, the contextual information involving image data captured by a camera, such that the security system 306 may send the image (to a server according to FIG. 3) with various objects being identified in the captured images (indicating the control unit being able to identify work content from the captured images that are captured at the first terminal); [0023] — performing speech recognition on the command (indicating a determination that speech can be converted into text, thereby associating the determination result with identified content); [0025] — the memory in the server may store data; [0096] — a fingerprint analysis server that is able to recognise scenes from captured video image (obtained content from captured image); [0101] — performing image recognition on obtained image (indicating a storage of the identified content); [0022] — the presence of a hardware processor).
The combination of Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Furukawa provides teaching for providing a determination result that speech can be converted into text, but differs from the claimed invention in that the claimed invention further provides teaching for associating determination with identified content from a captured image at a terminal. This isn’t new to the art as the reference of Saadatpanah is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the teaching of the combination of Chevrier in view of KO further in view of Morita, further in view of Bromand further in view of Furukawa which provides a determination result that speech can be converted into text, with the known teaching of Saadatpanah which captures images from a terminal, obtains content from it and associates it with a received speech command that is recognised, thereby coming up with teaching of the claimed invention. The combination of both prior art elements would have provided the predictable result of obtaining useful contextual information applicable to addressing the recognised speech, able to guide the system in presenting further content aid to the speakers engaged in the conversation. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), and further in view of Thomson (US 10,971,153 B2) as applied to claim 14, further in view of Melendo Casado et al. (US 2018/0061419 A1: hereafter — Melendo), further in view of HORNBERGER et al (US 2023/0419967 A1: hereafter — Hornberger) and further in view of Aher et al. (US 2021/0343271 A1: hereafter — Aher).
For claim 23, claim 14 is incorporated and the combination of Chevrier in view of KO further in view of Morita, further in view of Bromand and further in view of Thomson fails to disclose the limitations of this claim, for which the reference of Melendo is now introduced to teach as: the system wherein:
the first processor included in the first terminal transmits the determination result on the quality of the converted text data to the server apparatus via the first communication device (Melendo: [0040] — the speech recogniser may calculate a speech recognition confidence score and transmit the speech recognition confidence score to the server (the confidence score being an indication of the determination result on the quality of the converted text)).
The combination of Chevrier in view of KO further in view of Morita and further in view of Bromand provides teaching for determining the quality of converted text data, but differs from the claimed invention in that the claimed invention now further provides teaching for the transmission of the determination result on the converted text quality, to the server. This is however not new to the art as the reference of Melendo is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the teaching of the combination of Chevrier in view of KO further in view of Morita, further in view of Bromand, and further in view of Thomson which provides a determination result of the quality of converted text, with the known teaching of Melendo which transmits the determination result to a server as a speech recognition confidence score, thereby coming up with teaching of the claimed invention. The combination of both prior art elements would have provided the predictable result of transferring other processing steps from a local device to a server so that the server can appropriately perform the further processing steps based on the quality score, instead of the further steps being performed locally. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
The combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson and further in view of Melendo fail to disclose the further limitations of this claim, for which the reference of Hornberger is now introduced to teach as:
the second processor included in the server apparatus is configured to:
determine, before the communication with the first terminal ends, whether or not the quality of the converted text data after the conversion of the speech data acquired during the communication with the first terminal deteriorates based on the determination result on the quality of the converted text data received from the first terminal via the second communication device (Hornberger: [0251] — determining a captioning/speech recognition confidence as a measure of the quality of the captioning while a communication is in session (such that having the speech recognition confidence fall below a threshold is an indication of deteriorated quality in text from one of the terminals engaged in the communication session)).
The combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson and further in view of Melendo provides teaching for the determination of if speech can be converted into text, but differs from the claimed invention in that the claimed invention further provides teaching for determining if the quality of the converted text has deteriorated. This isn’t new to the art as the reference of Hornberger goes to show above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of Hornberger which determines a confidence score associated with a captioning/transcription system after the speech-to-text conversion has taken place, as a way of determining the quality of the captioning, with the teaching of the combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson and further in view of Melendo which determines if speech can be converted into text, thereby coming up with teaching of the claimed invention. The combination of both prior art elements would have provided the predictable result of notifying the user that poor transcription quality is currently being encountered, so that the user may be able to find a possible solution to the situation instead of continuing with speech that would result in poor quality transcripts. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
The combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson, further in view of Melendo, and further in view of Hornberger fails to disclose the further limitation of this claim, for which the reference of Aher is now introduced to teach as
determine that the quality of the text data after the conversion deteriorates due to an influence of the environmental sound in a case where a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal is greater than or equal to a predetermined threshold value, or in a case where a ratio between a magnitude of the speech data and the magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal satisfies a predetermined condition (Aher: [0059] — an error determination module able to determine the accuracy of a transcription; [0047] — error determination module may determine an incorrect transcription in any manner, such as in [0050] which is done by determining a noise level of the recorded audio query where high noise levels indicate a greater likelihood of improper transcription (high noise levels are an indication of the ratio between the magnitude of the speech data and the magnitude of the environmental audio data satisfying a predetermined threshold).
The combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson, further in view of Melendo, and further in view of Hornberger provides teaching for the determination of a deterioration in the quality of converted audio into text, but differs from the claimed invention in that the claimed invention further teaches of associating the deterioration of the quality of text to a situation having an SNR that satisfies a condition. This is however not new to the art as the reference of Aher is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of Aher which attributes error determination in the transcription with high noise levels in the recorded audio, with the teaching of the combination of Chevrier in view of KO, further in view of Morita, further in view of Bromand, further in view of Thomson, further in view of Melendo, and further in view of Hornberger which provides a deterioration in the quality of the converted text, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of determining that the reason for the poor transcription quality is due to the presence of a noisy environment, thereby being able to inform the user to move to a less noisy environment so that the speech may be better transcribed. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Chevrier (US 2019/0228774 A1) in view of KO (US 2020/0143807 A1), further in view of Morita (US 2009/0018831 A1) and further in view of Bromand (US 2023/0063839 A1), as applied to claim 1, and further in view of Kagoshima (US 2018/0275951 A1).
For claim 24, claim 1 is incorporated but the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand fails to disclose the limitation of this claim, for which the reference of Kagoshima is now introduced to teach as the system, wherein the second processor included in the server apparatus is configured to:
determine that the speech data cannot be converted into the text data due to a malfunction of the speech acquisition device included in the first terminal, in a case where both a magnitude of the speech data being acquired and a magnitude of the environmental sound included in the audio data acquired during the communication with the first terminal are less than a predetermined threshold value (Kagoshima: [0018] — checking microphone volume levels of a first terminal and when the input audio signal is less than a registered threshold, speech recognition does not occur as the audio signal gets discarded; [0023] — eliminating signals of noise (indicating that the microphone also receives noise/environmental audio data)).
The combination of Chevrier in view of KO further in view of Morita and further in view of Bromand provides teaching for the determination of if acquired speech can be converted into text, but differs from the claimed invention in that the claimed invention further provides teaching for determining that speech cannot be converted into text when the magnitude of speech and magnitude of environmental audio data being acquired during the communication are less than a threshold. This is however not new to the art as the reference of Kagoshima is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to combine the known teaching of Kagoshima which does not perform speech recognition, thereby determining that speech data cannot be converted into text when the volume levels of audio from a microphone is less than a certain threshold, with the teaching of the combination of Chevrier in view of KO further in view of Morita and further in view of Bromand which determines if speech can be converted into text, thereby coming up with teaching of the claimed invention. The combination of both prior art elements would have provided the predictable result of saving energy by not expending the resources to attempt speech recognition when there’s very low or insignificant audio levels not worth the process of speech recognition. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415–421, 82 USPQ2d 1385, 1395-97 (2007).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure.
Sharifi et al. (US 2022/0093104 A1) provides teaching for content metadata indicating a quality of audio associated with speech input being very poor, such that the poor-quality audio can provide an indication that it would be difficult for an ASR module to successfully transcribe audio data into text [0007].
Wang (US 2009/0125299 A1) provides a situation whereby words of a transcript are colour-coded to indicate the quality of speech recognition being performed on the input speech, such that red indicates that the confidence level of the speech recognition of the word is of bad quality [0029].
KONISHI et al. (US 2013/0218575 A1) provides teaching for visually notifying a user that a speech quality is bad, so that the user may make an adjustment improve the speech quality [0053].
Smith et al. (US 2023/0319121 A1) provides teaching for a display indication of a bad audio quality (FIG. 4).
MORITA KUGO (JP 2006208751 A) provides teaching for a situation whereby a voice recognition unit is restricted from performing voice recognition due to a poor sound collection state.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to OLUWADAMILOLA M. OGUNBIYI whose telephone number is (571)272-4708. The Examiner can normally be reached Monday – Thursday (8:00 AM – 5:30 PM Eastern Standard Time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s Supervisor, PARAS D. SHAH can be reached at (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWADAMILOLA M OGUNBIYI/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
04/25/2026
Prosecution Insights