DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the amendment filed July 22, 2026. Claims 1, 7-10, 12-16, and 18-20 have been amended. Claims 3 and 6 have been cancelled. Claims 1, 7-16, and 18-20 remain pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki et al (US Patent Application Publication No. 2022/0027579), hereinafter Suzuki, in view of Boesen (US Patent Application Publication No. 2003/0115068).
Suzuki discloses a translation device and method. Regarding claims 1 and 16, Suzuki teaches an electronic device [Fig. 11, (20)] comprising: at least one microphone (21/23); at least one speaker (22/24); a communication module [fig. 11; para 0399-0400]; a display (25/26); a memory storing instructions [fig. 11; para 0399-0400]; and at least one processor [201], wherein the instructions, when executed by the processor individually or collectively cause the electronic device to:
acquire first audio through the at least one microphone being connected the external device through the communication module, wherein the first audio comprises at least one of a counterpart’s voice, a user’s voice, surrounding noise, and a sound output from the least one speaker [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- assisting conversations between two or more speakers while stably recognizing voices by removing acoustic noise including echo, even in the case where voices of a plurality of speakers and a plurality of synthesized voices are present simultaneously overlapping one another, the synthesized voices being output as a result of recognizing and translating the voice of each speaker into a language on the other end and synthesizing resultant voices];
generate first audio by cancelling the sound output from the acquired first audio via an acoustic echo canceller (AEC) using the sound output from the at least one speaker as a first audio reference [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing];
transmit the first audio data to the external device through the communication module [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0400];
receive second audio data acquired through a microphone of the external device from the external device [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- assisting conversations between two or more speakers while stably recognizing voices by removing acoustic noise including echo, even in the case where voices of a plurality of speakers and a plurality of synthesized voices are present simultaneously overlapping one another, the synthesized voices being output as a result of recognizing and translating the voice of each speaker into a language on the other end and synthesizing resultant voices];
extract, based on the second audio data and based on information of the user’s voice, the counterpart’s voice from the first audio as a first target voice by cancelling the user’s voice and the surrounding noise from the first audio data [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing], where the information of he user’s voice is stored in memory [Suzuki’s facial recognition specifies speaker -- para 0202; 0287-0288 and also para 0053 -- on the basis of the first language notified from first language selection circuit 27 and the second language notified from second language selection circuit 28, control circuit 37 further causes first voice recognition circuit 31 to recognize voices in the first language, causes second voice recognition circuit 32 to recognize voices in the second language, causes first translation circuit 33 to translate the first language into the second language, causes second translation circuit 34 to translate the second language into the first language --where, the user’s stored voice information [user’s language] is used to recognize the speech and extract speech/information for translating the received speech];
obtain first translation information by translating the first audio data and [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- first/second translation circuit];
obtain second translation information by translating the second audio data [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- first/second translation circuit];
transmit the first translation information to the external device [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- assisting conversations between two or more speakers while stably recognizing voices by removing acoustic noise including echo, even in the case where voices of a plurality of speakers and a plurality of synthesized voices are present simultaneously overlapping one another, the synthesized voices being output as a result of recognizing and translating the voice of each speaker into a language on the other end and synthesizing resultant voices]; and
output the second translation information to a user of the electronic device via a speaker or display [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- assisting conversations between two or more speakers while stably recognizing voices by removing acoustic noise including echo, even in the case where voices of a plurality of speakers and a plurality of synthesized voices are present simultaneously overlapping one another, the synthesized voices being output as a result of recognizing and translating the voice of each speaker into a language on the other end and synthesizing resultant voices].
. Additionally, the combination of Suzuki and Boesen teaches based on a second audio for which the AEC is not processed being received from the external device, generate the second audio data from which a portion of echoes is cancelled by processing the AEC [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing having received audio through the communication module that originated from Boesen’s external device].
Regarding claim 8, the combination of Suzuki and Boesen teaches the at least one processor is further configured to: detect a start of the first target voice and an end of the first target voice[Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing]; perform automatic speech recognition (ASR) on the first target voice, based on the start of the first target voice and the end of the first target voice [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- first/second voice recognition circuits]; and acquire first translation information by translating first text for which the ASR has been performed [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- translation circuit].
Regarding claim 9, the combination of Suzuki and Boesen teaches convert the first translation information into a first translation voice by using text-to-speech (TTS) [Suzuki para 0007 -- receives input of a first voice of the first speaker, a first voice recognition circuit that recognizes the first voice to output a first character string, a first translation circuit that translates the first character string output from the first voice recognition circuit into a language of the second speaker to output a third character string, a first voice synthesis circuit that converts the third character string output from the first translation circuit into a first translated voice, a first loudspeaker that amplifies a sound level of the first translated voice, a second microphone that receives input of a second voice of the second speaker, a second voice recognition circuit that recognizes the second voice to output a second character string, a second translation circuit that translates the second character string output from the second voice recognition circuit into a language of the first speaker to output a fourth character string, a second voice synthesis circuit that converts the fourth character string output from the second translation circuit into a second translated voice, a second loudspeaker that amplifies a sound level of the second translated voice]; and output the first translation voice through a speaker of the external device by transmitting through the communication module the first translation voice to the external device [Suzuki Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398].
Regarding claim 10, the combination of Suzuki and Boesen teaches the at least one processor is further configured to receive a second target voice extracted from the second audio data and a start of the second target voice and an end of the second target voice from the external device [Suzuki Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing].
Regarding claim 11, the combination of Suzuki and Boesen teaches wherein the second target voice comprises a user's voice improved by cancelling at least a portion of sounds except for the user's voice from the second audio data [Suzuki Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- echo cancelling/cross talk AEC processing], and wherein the start of the second target voice and the end of the second target voice are detected through a voice pick-up (VPU) sensor of the external device [Boesen’s bone conduction sensor para 0030].
Regarding claim 12, the combination of Suzuki and Boesen teaches the at least one processor is further configured to: perform ASR on the second target voice, based on the start of the second target voice and the end of the second target voice [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- first/second voice recognition circuits]; and acquire second translation information by translating a second text on which the ASR is performed [Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- translation circuit].
Regarding claims 13 and 20, the combination of Suzuki and Boesen teaches the at least one processor is further configured to: convert the second translation information into a second translation voice by using text-to-speech (TTS) [Suzuki para 0007 -- receives input of a first voice of the first speaker, a first voice recognition circuit that recognizes the first voice to output a first character string, a first translation circuit that translates the first character string output from the first voice recognition circuit into a language of the second speaker to output a third character string, a first voice synthesis circuit that converts the third character string output from the first translation circuit into a first translated voice, a first loudspeaker that amplifies a sound level of the first translated voice, a second microphone that receives input of a second voice of the second speaker, a second voice recognition circuit that recognizes the second voice to output a second character string, a second translation circuit that translates the second character string output from the second voice recognition circuit into a language of the first speaker to output a fourth character string, a second voice synthesis circuit that converts the fourth character string output from the second translation circuit into a second translated voice, a second loudspeaker that amplifies a sound level of the second translated voice]; and display the second translation information on the display or output the second translation voice to the at least one speaker [Suzuki para 0390].
Regarding claims 14 and 20, the combination of Suzuki and Boesen teaches the processor is further configured to: acquire third audio through the at least one microphone [Suzuki/Boesen microphone inputs, where the third audio is obtained via ongoing communications between the two parties]; and obtain a first translation voice by translating the third audio is output through the external device [Suzuki Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- translation circuit in combination with Boesen’s external device] .
Regarding claim 15, the combination of Suzuki and Boesen teaches the the at least one processor is further configured to output second translation information obtained by translating the second audio through the at least one speaker [Suzuki Fig 1/Fig 11; para 0012; 0027-0032; 0033-055; 0108-0129; 0134-0156; 0190-0198; 0201-0209; 0390-0398 --- translation circuit that provides for the output via the loudspeakers] when the external device acquires fourth audio [Suzuki/Boesen microphone inputs, where the fourth audio is obtained via ongoing communications between the two parties].
Claims 7 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki in view of Boesen as applied to claims 1 and 16 above, and further in view of Yoshimine (US Patent Application Publication No. 2006/0204033).
Regarding claims 7 and 18-19, the combination of Suzuki and Boesen fails to teach capturing the counterpart through the camera and analyzing lip reading of the captured counterpart image. In a similar field of endeavor, Yoshimine teaches a conversation assisting device and method that implements a lip-reading function, which analyzes a picture of the user in particular the motion of the lips of the user captured by the camera into character information or sound information [para 0034] and specifically teaches the system allows the users to effectively and smoothly converse with each other [para 0006]. One having ordinary skill in the art at the time of the invention would have recognized the advantages of implementing the lip-reading analysis processing suggested by Yoshimine, in the system of Suzuki/Boesen, for the purpose of allowing the users to effectively and smoothly converse with each other, as taught by Yoshimine, thereby enhancing and improving the user’s experience with the translation system.
Response to Arguments
Applicant's arguments filed July 22, 2026 have been fully considered but they are not persuasive.
Applicant argues “Suzuki does not describe a separate "external device" that is worn by the user and that is distinct from the electronic device. Accordingly, Suzuki does not disclose acquiring first audio at the electronic device through a microphone of the electronic device "while a user wears an external device," as required by claim 1.”
The Examiner notes, the feature for which Applicant argues is not taught by Suzuki is no longer recited in the claims. However, in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). IN this instance, Boesen was cited for teaching “a user wears an external device.” Thus, the combination of Suzuki and Boesen provide adequate support for acquiring first audio at the electronic device through a microphone of the electronic device "while a user wears an external device."
Applicant argues that Suzuki fails to disclose or teach "transmit the first audio data to the external device through the communication module; receive, through the communication module, second audio data acquired through a microphone of the external device from the external device, wherein the second audio data comprises the user's voice as a second target voice; extract, based on the second audio data, the counterpart's voice from the first audio data as a first target voice by cancelling the user's voice and the surrounding noise from the first audio data." In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). IN this instance, Boesen was cited for teaching “receive, through the communication module, second audio data acquired through a microphone of the external device from the external device, wherein the second audio data comprises the user's voice as a second target voice.” Thus, the combination of Suzuki and Boesen provide adequate support for limitations as claimed.
Applicant argues “Boesen's architecture is built for a single user wearing an earpiece who communicates with a remote translator-not for two speakers conversing through a shared device.” Applicant argues that modifying Suzuki to incorporate Boesen's wearable earpiece would change the principle of operation of Suzuki's translation device 20 in a manner not contemplated by Suzuki which relies on the speakers being co-located and without the need for a transmission of spoken audio via a communication module. The Examiner respectfully disagrees. Boesen specifically teaches the system can be used locally [para 0013] and with any computer or electronic device, particularly a PDA [para 0025]. Thus, making Boesen’s system capable of participating in a communication session with another participant using Suzuki’s device [Fig. 11]. Additionally, the combination of Suzuki and Boesen would merely allow the participants to easily hear the translation of the other participant’s utterances in a crowded or noisy environment.
Applicant argues modifying Boesen to incorporate Suzuki's dual-microphone, dual-speaker, single-device architecture would render Boesen unsatisfactory for its intended purpose. In response to applicant's argument that “modifying Boesen to incorporate Suzuki's dual-microphone, dual-speaker, single-device architecture would render Boesen unsatisfactory for its intended purpose,” the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Similarly, a text for obviousness does not require the secondary reference be modified to incorporate the structure of the primary reference. Boesen was cited based on concepts and features that do not require changing or physically altering the structure of Suzuki, but merely provides a well-known source or format for users to interact/communicate with a smartphone or other electronic device.
Applicant argues a cross-device reference-based cancellation scheme is absent from both references, and there is no teaching, suggestion, or motivation in either reference to combine them in a way that would result in such arrangement. In response to applicant' s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Boesen teaches a voice communication device with foreign language translations, providing an earpiece [Fig 1] with front, side and rear microphones [para 0022] that transmits signals to a translation unit [that can performed locally [para 0013]] and which translates the voice communication and transmits the resulting translation back to the earpiece, the earpiece receives the translation and transduces the translated voice sound information through a speaker disposed within the earpiece [para 0013; 0025] that can be used with PDAs, embedded computers or other electronic devices [para 0025] and suggests the system is advantageous in allowing reception of voice communications in positions or orientations relative to the microphones and the wearer [para 0013]. Therefore, one having ordinary skill in the art at the time of the invention would have recognized the advantages of implementing the earpiece with multidirectional microphone processing, suggested by Boesen, and the results would have been predictable and resulted in improved communications amongst the speakers allowing the participants to easily hear the translation of the other participant’s utterances in a crowded or noisy environment and also insure translations are clearly heard by a hearing-impaired individual.
Applicant respectfully submits that the Examiner's proposed combination of Suzuki and Boesen would, at best, be reconstructed only with impermissible hindsight from Applicant's own disclosure. In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Applicant argues that Suzuki's facial recognition uses image- based recognition to identify the speaker and look up the speaker's language in a database Suzuki does not use information of the user's voice stored in memory to extract a target voice. The Examiner respectfully disagrees. Suzuki teaches [para 0053] on the basis of the first language notified from first language selection circuit 27 and the second language notified from second language selection circuit 28, control circuit 37 further causes first voice recognition circuit 31 to recognize voices in the first language, causes second voice recognition circuit 32 to recognize voices in the second language, causes first translation circuit 33 to translate the first language into the second language, causes second translation circuit 34 to translate the second language into the first language, causes first voice synthesis circuit 35 to synthesize voices in the second language, and causes second voice synthesis circuit 36 to synthesize voices in the first language. Thus, the user’s stored voice information [user’s language] is used to recognize the speech and extract speech/information for translating the received speech.
Applicant argues Suzuki fails to teach "based on a second audio for which the AEC is not processed being received, through the communication module, from the external device, generate the second audio data from which a portion of echoes is cancelled by processing the AEC.” In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In this instance, Boesen teaches second audio for which the AEC is not processed being received, through the communication module, from the external device. Thus, the combination of Suzuki and Boesen provide adequate support for limitations as claimed.
. Applicant argues claims 7-15 and 18-20 depend from claim 1 or 16 and that these claims should be patentable at least for their dependencies and the novel features recited therein. The Examiner respectfully disagrees. As indicated in the rejections and arguments above, the combination of Suzuki and Boesen teach/suggest the claim limitations of claims 1 and 16. Additionally, the combination of Suzuki and Boesen [provide adequate support for the limitations as claimed in dependent claims 8-15 and 20. Further, as indicated in the rejections above, the combination of Suzuki, Boesen and Yoshimine teach and or suggest the limitations as claimed in claims 7 and 18-19.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGELA A ARMSTRONG whose telephone number is (571)272-7598. The examiner can normally be reached M,T,TH,F 11:30-8:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ANGELA A. ARMSTRONG
Primary Examiner
Art Unit 2659
/ANGELA A ARMSTRONG/Primary Examiner, Art Unit 2659
