DETAILED ACTION
This communication is in response to the Application filed on 02/28/2024 (foreign priority). Claims 1-14 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent European Application No. 24160398.4, filed on February 28, 2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February 26, 2025 was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 7-10, 12, and 14 is/are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Tumkur et al. (US 20230252246 A1).
Regarding claim 1, Tumkur teaches a system for speech-to-text conversion of passenger announcements on board of an aircraft, the system comprising:
a cabin management control configured to provide a speech signal related to an announcement to passengers on board of an aircraft (see [0046], where the aircraft 100 includes a server computer 110, multiple in-flight entertainment systems 120a-120n (e.g., one in-flight entertainment system 120k is shown), a public announcement system 112, a crew microphone 114 having a push-to-talk switch 116, an assist display 118, and multiple wireless access points 119a-119n (one wireless access point 119k is shown). Each wireless access point 119a-119n includes a receiver 142 and a transmitter 144. The crew microphone 114, the push-to-talk switch 116, and the assist display 118 may be located in the galley area 106 of the aircraft 100. In embodiments of the aircraft 100 that include multiple galley areas 106 and/or other areas, a copy of the crew microphone 114, the push-to-talk switch 116, and/or the assist display 118 may be located in each galley area 106 and/or the other areas (e.g., meeting rooms, bar area, and the like). The server computer 110 and the public announcement system 112 may be located within the aircraft 100 based on a configuration of the aircraft 100. The wireless access points 119a-119n may be distributed near to the seats 102a-102n. Also see [0051], where the particular cabin crew member 90k may present one or more crew spoken words 92 into the crew microphone 114. The crew microphone 114 may convert the crew spoken words 92 into a crew electrical signal 136 received by the server computer 110 and the public announcement system 112. The particular cabin crew member 90k may enter one or more crew selections 93 to the touchscreen of the assist display 118. The assist display 118 may transfer the crew selections 93 to the server computer 110 via a crew selection signal 137. Also see [0054], where the public announcement system 112 implements an audio amplifier and multiple speakers. The public announcement system 112 is operational to broadcast a public announcement 139 (e.g., the crew spoken words 92 in the crew electrical signal 136) into the passenger cabin 104 (FIG. 1) in real time (e.g., less than a few millisecond delay) while the push-to-talk switch 116 is in an active position (e.g., the switch is pressed). In some designs, the crew spoken words 92 may be presented from the in-flight entertainment systems 120a-120n and/or the handheld devices 130a-130n. In various embodiments, the public announcement system 112 may also be operational to broadcast spoken words from the flight crew. Also see FIG 2.);
a cabin application server comprising a processing unit configured to convert the speech signal into a text message containing text corresponding to spoken words of the speech signal, wherein the processing unit is configured to convert the speech signal locally without accessing online computing resources (see [0065], where the speech-to-text converters 150a-150n implement software programs stored in the memory circuit 158 and executed by the processors 156. The speech-to-text converters 150a-150n are operational to generate the source text messages by converting the spoken words in the digital signals based on the particular source languages. Generally, each speech-to-text converter 150a-150n is tuned for efficient conversion of the spoken words in a particular source language. In various embodiments, a single speech-to-text converter 150a-150n is implemented for each of the recognized languages. In other embodiments, multiple speech-to-text converters 150a-150n are implemented for one or more of the recognized languages such that one or more conversions in that recognized language may take place concurrently. Also see [0046], where the server computer 110 and the public announcement system 112 may be located within the aircraft 100 based on a configuration of the aircraft 100); and
a network interface configured to provide the text message to at least one passenger device on board of the aircraft (see [0050], where the server computer 110 may generate a passenger video signal 128k received by the particular in-flight entertainment system 120k. The passenger video signal 128k conveys a sequence of passenger target images (e.g., graphical user interface images). The passenger target images may be presented by the passenger screen 124k as passenger images 84k seen by the particular passenger 80k. The server computer 110 may also convert the passenger target images into digital data presented to the particular handheld device 130k via the wireless access point 119k and the wireless link 135k. The particular handheld device 130k may present passenger target images on the handheld passenger screen 134k as the passenger images 84k seen by the particular passenger 80k. Also see [0097], where the same infrastructure may be used to convert cabin crew announcements (ex. safety briefing) into text messages displayed on the backseat screens of the in-flight entertainment systems 120a-120n. For airliners replacing the in-flight entertainment systems 120a-120n with pairings to mobile phones(e.g., handheld devices 130a-130n), the passengers 80a-80n, the text messages may be transmitted to the mobile phones for display to the passengers 80a-80n).
Regarding claim 2, Tumkur teaches the system according to claim 1, wherein the processing unit comprises:
an AI accelerator configured to utilize at least one of AI and machine learning for converting the speech signal (see [0016], where in one or more embodiments of the method, the converting of the crew electrical signal to the crew source text message is performed using an artificial intelligence based speech-to-text conversion. Also see [0041], where the system and method utilize artificial intelligence based speech-to-text conversion techniques and natural language processing to establish communication between the passenger and the cabin crew members. Also see [0053], where the server computer 110 is operational to execute software that provides multiple artificial intelligence based speech-to-text conversions of the passenger electrical signal 126k and the crew electrical signal 136 to create text messages in corresponding source languages. The software may also provide multiple natural language based language conversions that convert source text messages in the source language to target text messages in target languages. The source languages and the target languages may be received by the server computer 110 via the passenger selection signal 127k and the crew selection signal 137).
For purposes of this rejection, the claim term “AI accelerator” is interpreted under broadest reasonable interpretation as a specialized hardware component, or processor capable of handling artificial intelligence tasks within a device or system. While the claim language uses the term “AI accelerator,” Tumkur does not use the same terminology when referring to a specialized hardware component for artificial intelligence tasks and instead uses “server computer.” However, this mere difference in terminology does not create a substantive distinction because the server computer 110 possesses one or more processors capable of handling artificial intelligence tasks for the purpose of speech to text conversion.
Regarding claim 3, Tumkur teaches the system according to claim 1,
wherein the processing unit is configured to translate the speech signal from a source language to a target language as part of the conversion (see [0061], where referring to FIG. 3, a schematic block diagram of an example implementation of the server computer 110 is shown in accordance with one or more exemplary embodiments. The server computer 110 includes multiple passenger input/output (I/O) circuits 140a-140n, multiple language translators 146a-146n, a main buffer circuit 148, multiple speech-to-text converters 150a-150n, a cabin crew input circuit 152, a cabin crew output circuit 154, one or more processors 156 (one shown), and one or more memory circuits 158 (one shown). Also see [0063], where the language translators 146a-146n implement software programs stored in the memory circuit 158 and executed by the processors 156. The language translators 146a-146n are operational to read a source text message written in a source language from the main buffer circuit 148 and convert that text message into a target text message written in a target language. The recognizable languages may include English, German, French, Spanish Dutch, Chinese, and the like. Each language translator 146a-146n is configured to translate between two of the recognizable languages. In some embodiments, a number of language translators 146a-146n may be similar to the number of recognizable languages plus several more for reserves. Therefore, the language translators 146a-146n may translate a cabin crew announcement into each recognizable language concurrently and still allow for some passengers 80a-80n to request service (via source text messages) during the announcement).
wherein the network interface is configured to provide a correspondingly translated text message to the at least one passenger device on board of the aircraft (see [0085], where in the step 312, the crew source text message is read from the main buffer circuit 148. Multiple ones of the language translators 146a-146n in the server computer 110 convert the crew source text message in the step 314 into multiple passenger target text messages concurrently using the natural language based language conversions. In the step 316, the passenger target text messages are displayed to the passengers 80a-80n thru the passenger screens 124a-124n or the handheld passenger screens 134a-134n. Also see [0096], where when a particular passenger 80k requests water, snacks, or other specific requests, he/she may request assistance verbally. The verbal request is converted into a passenger source text message, translated into a crew target text message, and displayed on the assist display 118. A cabin crew member 90a-90n reads the request and walks to the particular passenger 80k with water, snack or other specific request. When an announcement is made over the public announcement system 112, the words may be converted to text and translated in real time to various languages preferred by the various passengers 80a-80n. The text versions of the public announcements also accommodate hearing-challenged passengers 80a-80n by providing the public announcements in readable form).
Regarding claim 5, Tumkur teaches the system according to claim 1, further comprising:
a data storage (see [0061], where the server computer 110 includes multiple passenger input/output (I/O) circuits 140a-140n, multiple language translators 146a-146n, a main buffer circuit 148, multiple speech-to-text converters 150a-150n, a cabin crew input circuit 152, a cabin crew output circuit 154, one or more processors 156 (one shown), and one or more memory circuits 158 (one shown). Also see [0064], where the main buffer circuit 148 implements a memory buffer. The main buffer circuit 148 is operational to temporarily store source text messages, source languages selection, and target language selections generated by the passengers 80a-80n and the cabin crew members 90a-90n concurrently. The source text messages and source language selections are received into the main buffer circuit 148 from the speech-to-text converters 150a-150n. The target language selections are received from the passenger input/output circuits 140a-140n and the cabin crew input circuit 152. The source text messages, the source language selections, and the target language selections are read out to the language translators 146a-146n for conversion into target text messages in the target languages.),
wherein the cabin management control is configured to at least one of:
store at least one of the text message, the translated text message, and the translated speech signal in the data storage (see [0064], where the main buffer circuit 148 is operational to temporarily store source text messages, source languages selection, and target language selections generated by the passengers 80a-80n and the cabin crew members 90a-90n concurrently. Also see [0083], where the crew source text message is buffered in the main buffer circuit 148 in the step 306 for subsequent translation into the various passenger target languages).
The “at least one of” language in claim 5 creates an election between alternatives (A) record a live audio announcement as speech signal and store the live audio announcement in the data storage; and (B) store at least one of the text message, the translated text message, and the translated speech signal in the data storage. Accordingly, only one of the alternatives needs to be met. Tumkur’s storage of the source text message alone satisfies the limitation presented in alternative (B); and therefore, fully satisfies claim 5.
Regarding claim 7, Kumtur teaches an aircraft comprising: the system according to claim 1 (see [0043], where referring to FIG. 1, a schematic diagram of an example implementation of an aircraft 100 is shown in accordance with one or more exemplary embodiments. The aircraft 100 generally includes multiple seats 102a-102n in a passenger cabin 104, and one or more galley areas 106 (one shown). Also see [0046], where the aircraft 100 includes a server computer 110, multiple in-flight entertainment systems 120a-120n (e.g., one in-flight entertainment system 120k is shown), a public announcement system 112, a crew microphone 114 having a push-to-talk switch 116, an assist display 118, and multiple wireless access points 119a-119n (one wireless access point 119k is shown). Each wireless access point 119a-119n includes a receiver 142 and a transmitter 144. The crew microphone 114, the push-to-talk switch 116, and the assist display 118 may be located in the galley area 106 of the aircraft 100. In embodiments of the aircraft 100 that include multiple galley areas 106 and/or other areas, a copy of the crew microphone 114, the push-to-talk switch 116, and/or the assist display 118 may be located in each galley area 106 and/or the other areas (e.g., meeting rooms, bar area, and the like). The server computer 110 and the public announcement system 112 may be located within the aircraft 100 based on a configuration of the aircraft 100. The wireless access points 119a-119n may be distributed near to the seats 102a-102n) and further teaches all of the limitations as recited in claim 1 above for the same reasons.
Regarding claim 8, which recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 1. As detailed in the rejection of claim 1, the disclosed system teaches each step of the method recited in claim 8. Accordingly, claim 8 is rejected for the same reasons set forth in the rejection of claim 1.
Regarding claim 9, which recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 2. As detailed in the rejection of claim 2, the disclosed system teaches each step of the method recited in claim 9. Accordingly, claim 9 is rejected for the same reasons set forth in the rejection of claim 2.
Regarding claim 10, which recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 3. As detailed in the rejection of claim 3, the disclosed system teaches each step of the method recited in claim 10. Accordingly, claim 10 is rejected for the same reasons set forth in the rejection of claim 3.
Regarding claim 12, which recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 5. As detailed in the rejection of claim 5, the disclosed system teaches each step of the method recited in claim 12. Accordingly, claim 12 is rejected for the same reasons set forth in the rejection of claim 5.
Regarding claim 14, Kumtur teaches:
a non-transitory computer readable medium comprising storing a computer program configured to perform the method of claim 8 when the computer program is executed by a processor (see [0053], where the server computer 110 may implement one or more processors, memory, and associated input/output circuitry. In various embodiments, the memory may include non-transitory computer readable memory that stores software. The software is executable by the processors in the server computer 110. The server computer 110 is operational to execute software that provides multiple artificial intelligence based speech-to-text conversions of the passenger electrical signal 126k and the crew electrical signal 136 to create text messages in corresponding source languages. The software may also provide multiple natural language based language conversions that convert source text messages in the source language to target text messages in target languages. The source languages and the target languages may be received by the server computer 110 via the passenger selection signal 127k and the crew selection signal 137) and further teaches all of the limitations as recited in claim 8 above for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tumkur (US 20230252246 A1) in view of Watson et al. (US 10169338 B2).
Regarding claim 4, which depends on claim 3, Tumkur teaches all the limitations of claim 3 but fails to teach outputting the translated speech signal to at least one passenger device.
However, Watson does teach:
a) wherein the processing unit is configured to generate a correspondingly translated speech signal containing spoken words corresponding to text of the translated text message (see Column 11, lines 32-58, where the module 312 includes a receiving module 362 that receives an input 360. The input may be based on pre-recorded audio 319 and/or pre-defined text 321. The input is provided to a processing module 364 that converts the audio into text or text into audio based on the input 360. The processing module 364 may also translate the converted input into one or more languages from a first language to one or more other languages. One or more translation package(s) 317A is used for translating audio/text from the first language to one or more languages. In one aspect, translation package(s) 317A are stored selectively based on the translation languages for a specific flight. The translation languages may be based on a flight itinerary (e.g., origin, destination and layovers) as well as on airline passenger preference defined by data 352. In one aspect, module 312 downloads translation package(s) 317A from a ground system (not shown) based on flight schedules/passenger data. This is efficient because then the aircraft only has to store what it needs for a specific flight. For pre-recorded audio 319 and/or pre-defined text 321, the translation packages 317A include pre-translated versions in each language such that conversion is performed by selecting the translation corresponding to the pre-recorded audio and/or pre-defined text),
b) wherein the network interface is configured to provide the translated speech signal to the at least one passenger device on board of the aircraft (see Column 11, lines 66-67 and Column 12, lines 1-6, where in one aspect, the processing module 364 generates an output 361 that is provided to an output module 370 for transmission to PEDs 302 and/or seat devices 326. The output 361 may be text when audio is converted to text or the output may be audio, when text is converted to audio, as described below in detail. Alternatively, the output 361 causes an application on the PEDs 302 to select text or audio stored within the application for presentation on the PED).
Tumkur and Watson are considered to be analogous to the claimed invention because they are in the same field of speech analysis and data processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date to have modified Tumkur to incorporate the teachings of Watson in order to increase accessibility for passengers who may not understand the cabin crew’s original language, especially those with hearing or visual disabilities (see Column 1, lines 46-55, where in conventional flights, safety messages are often pre-recorded and played for passengers. This is a challenge for hearing impaired passengers because they may not be able to hear the audio. Airline crew member often have to personally check the hearing impaired passengers. Furthermore, when text messages are transmitted to passengers, one or more passenger may be visually impaired that may not be able to properly see a displayed message. Continuous efforts are being made to efficiently transmit messages on aircrafts and other transportation vehicles).
Regarding claim 11, which depends on claim 10 and recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 4. As detailed in the rejection of claim 4, the disclosed system teaches each step of the method recited in claim 11. Accordingly, claim 11 is rejected for the same reasons set forth in the rejection of claim 4.
Claim(s) 6 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tumkur (US 20230252246 A1) in view of Klose et al. (US 20160372114 A1).
Regarding claim 6, which depends on claim 1, Tumkur teaches all the limitations of claim 1 but fails to teach a notification signal that alerts at least one passenger that an announcement is made.
However, Klose does teach:
wherein the cabin management control is configured to provide a notification signal to the at least one passenger device via the network interface notifying the at least one passenger device that an announcement is made (see [0014], where the method comprises, before signaling the text message to the one or more devices, notifying the one or more devices that the text message is to be expected. The notification may include a buzzing sound or the like. In this way, the passengers are informed about the text message and even distracted passengers are more likely to notice the ongoing announcement in audio and/or in text. Various techniques may be used for the signaling. For instance, the signaling may be performed by notifying only the one or more devices that are determined as recipients of the text message and/or notifying all devices within reach by broadcasting, for example. The signaling may be performed via the onboard connectivity system, e.g., an onboard base station, by means of WLAN or Bluetooth beacons, and/or push messages over a data channel via broadcast to all devices within reach or via multicast to all devices that have subscribed to the text message service. The signaling may be performed before or during an announcement. Also see [0043], before or during signaling of a text message to the one or more devices, the one or more devices that are to receive the text message may be notified. The notification may include a buzzing sound or the like. The notification may be transmitted by the signaling component 106 via the text interface 114. For example, PODs may be notified while display devices of the IFE system may not be notified).
Tumkur and Klose are considered to be analogous to the claimed invention because they are in the same field of speech analysis and data processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date to have modified Tumkur to incorporate the teachings of Klose in order to increase accessibility for passengers with physical disabilities, such as hearing impairment, and alert them that an important announcement is being made (see [0003], where several announcements are usually made on board of aircraft by aircraft personnel, e.g., flight attendants or pilots. For example, such announcements are made before or after take-off and landing. In general, two different types of announcements exist on board of aircraft, the first being pre-recorded announcements and the second being live announcements. Pre-recorded announcements are sometimes called Pre-Recorded Announcements and Music (PRAM) and live announcements are often called Passenger Address (PA) announcements. Both of these announcements are usually announced or advertised to passengers by a so-called Chime A chime is a certain tone or jingle indicating that an announcement is going to be made. Further, during an announcement, a message is usually displayed on passenger displays of an In-Flight Entertainment (IFE) system. For example, such a message reads “announcement in progress” or the like. Also see [0004], where for deaf or hearing-impaired passengers it is difficult if not impossible to follow such announcements. Further, passengers might be distracted and might not listen carefully to the announcement made).
Regarding claim 13, which depends on claim 8 and recites a method, this claim is rejected as unpatentable over the same prior art and reasoning applied against claim 6. As detailed in the rejection of claim 6, the disclosed system teaches each step of the method recited in claim 13. Accordingly, claim 13 is rejected for the same reasons set forth in the rejection of claim 6.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Klose et al. (U.S. Patent 10,056,085): Speech Recognition On Board Of An Aircraft.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN HONG FANG-WU whose telephone number is (571)270-0607. The examiner can normally be reached Monday - Friday, 9AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras Shah can be reached at (571)-270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN HONG FANG-WU/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
09/10/2026
Prosecution Insights