DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because Fig. 7 elements 706 and twice in 710 have misspelled the word “transcription.” Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: Fig. 10 elements 1014, 1028, 1034. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Using the subject matter eligibility test from page 74621 of the Federal Register Notice titled “2014 Interim Guidance on Patent Subject Matter Eligibility,” a two-step process is performed. Under step 1, the claims are analyzed to determine if the claim is directed to a process, machine, article of manufacture, or composition of matter. In this case, claims 1-15 are directed to a system, which is a machine or an article of manufacture; claims 16-19 are directed to a method, which is a process; and claim 20 is directed to a storage medium, which is a machine or an article of manufacture. Step 2A (part 1 of the Mayo test), using the guidance from pages 50-57 of the Federal Register Vol. 84 No. 4 from Monday, January 7, 2019, requires applying a two-prong inquiry. In Prong One, examiners evaluate whether the claim recites a judicial exception, determining if the claim is directed to a law of nature, a natural phenomenon, or an abstract idea. In this case, claim 1 recites analyzing audio, generating transcriptions, and fusing transcriptions, which are mental processes. In Prong Two, examiners evaluate whether the judicial exception is integrated into a practical application that imposes a meaningful limit on the judicial exception. In this case, additional elements of receiving and providing data are mere extrasolution activity, while structural elements of processor, memory, storage device, and machine learning models are generic computing components, and do not integrate the abstract ideas into a practical application.
Step 2B (part 2 of the Mayo test) requires analyzing the claims to determine if they recite additional elements that amount to significantly more than the judicial exception. In this case, the claims do not include additional elements that are sufficient to amount to significantly more than the abstract idea itself.
Regarding claims 1, 16, and 20, analyzing audio, generating transcriptions, and fusing transcriptions are mental processes, which is an abstract idea. For example, a human could hear audio and transcribe it, as well as combine multiple transcripts. Additional elements of receiving and providing data are mere extrasolution activity, while structural elements of processor, memory, storage device, and machine learning models are generic computing components, and do not integrate the abstract ideas into a practical application or constitute significantly more.
Regarding claims 2-4, 11-12, 15, and 17-19, the limitations are further clarifications of the above abstract ideas.
Regarding claim 5, generating a prompt is a mental process, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 6, detecting objects, generating details of the objects, and including them in the prompt are mental processes, which is an abstract idea. Camera and head wearable apparatus appear to be generic computing components, and do not integrate the abstract ideas into a practical application or constitute significantly more.
Regarding claim 7, determining a location and generating a prompt are mental processes, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 8, accessing a profile and generating a prompt are mental processes, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 9, generating a prompt is a mental process, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 10, generating a prompt is a mental process, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 13, providing and displaying data is mere extrasolution activity, and does not integrate the abstract ideas into a practical application or constitute significantly more.
Regarding claim 14, displaying data is mere extrasolution activity, and does not integrate the abstract ideas into a practical application or constitute significantly more.
The limitations of the claims, taken alone, do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements individually. Applicable case law cited in the Federal Register includes, but is not limited to: Alice Corp., 134 S. Ct. at 2355-56, Digitech Image Tech., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344 (Fed. Cir. 2014), Benson, 409 U.S. at 63.
See "Preliminary Examination Instructions in view of the Supreme Court Decision in Alice Corporation Pty. Ltd. v. CLS Bank International, et al.," dated June 25, 2014, and the Federal Register notice titled "2014 Interim Guidance on Patent Subject Matter Eligibility" (79 FR 74618).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 11-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Thomson (US 2021/0375288 A1), in view of Thomson et al. (US 10,573,312 B1), hereinafter referred to as Thomson2.
Regarding claim 1, Thomson teaches:
A system comprising:
at least one processor (Fig. 2 element 240, para [0051], where a processor is used);
at least one memory component storing instructions (Fig. 2 element 242, para [0051], where memory is used) that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
receiving audio, via a computing device, and performing in real-time until termination is detected, operations (para [0020], where audio is received during a communication session) comprising:
analyzing the received audio, using a first machine learning model, to generate a first textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning);
providing the first textual transcription of the audio to the computing device (para [0023], where the transcription is provided to the devices);
analyzing the received audio, using a second machine learning model, to generate a second textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning); and
fusing the first textual transcription and the second textual transcription using a third machine learning model to generate a fused transcription (para [0030], [0055], where the transcription generation technique is a fusion or combination of the output of two or more transcriptions, and para [0093], where the transcription systems use machine learning); and
Thomson does not teach:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model.
Thomson2 teaches:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model (Col. 83 lines 16-29, where transcriptions are fused at the end of the communication session when real time operations are not necessary).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson by performing the fusing of the transcriptions of Thomson (Thomson para [0025]) either in real time or after the end of the communication session as taught by Thomson2 (Thomson2 col. 83 lines 16-29), so that the processes do not need to be run as frequently (Thomson2 col. 83 lines 16-29).
Regarding claim 2, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein the audio is analyzed at predefined time intervals as it is received from the computing device and the first textual transcription is generated for each portion of the audio corresponding to each predefined interval (Thomson2 col. 79 lines 15-30, where the fused transcription is recomputed periodically at short intervals).
Regarding claim 3, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein the first machine learning model is a faster, less accurate and lower cost machine learning model than the second machine learning model (Thomson para [0041], [0072], [0075], where one technique is more accurate with higher latency, and higher costs).
Regarding claim 11, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein the first textual transcription is generated in real-time or near real-time and the second textual transcription is generated after the first textual transcription is generated (Thomson2 col. 16 lines 49-60, where one transcription is provided to the user faster than the other transcriptions, and col. 4 lines 39-50, where the transcription is made and provided in real time).
Regarding claim 12, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein termination is detected by determining that no audio is received for more than a predefined period of time (Thomson2 col. 130 lines 13-26, where the fusing decision involves silence detection, such as detecting a period of silence as in col. 85-86 Table 7 line 1).
Regarding claim 13, Thomson in view of Thomson2 teaches:
The system of claim 1, the operations further comprising:
providing the final transcription to the computing device causing display of the final transcription on a display of the computing device (Thomson para [0011], where the transcription is provided to the device to display to the user).
Regarding claim 14, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein the first textual transcription is displayed on the computing device in real-time as it is generated and the displayed first textual transcription is updated with the fused transcription as the fused transcription is generated (Thomson2 col. 4 lines 39-50, where the transcription is made and provided in real time, and col. 12 line 56 - col. 13 line 19, col. 16 lines 41-48, where the displayed transcription is updated with the edited or fused transcription).
Regarding claim 15, Thomson in view of Thomson2 teaches:
The system of claim 1, wherein the first machine learning model and the second machine learning model are analyzing the received audio in parallel (Thomson2 col. 21 table 1 line 6, where the systems operate in parallel).
Regarding claim 16, Thomson teaches:
A computer-implemented method comprising:
receiving audio, via a computing device, and performing in real-time until termination is detected, operations (para [0020], where audio is received during a communication session) comprising:
analyzing the received audio, using a first machine learning model, to generate a first textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning);
providing the first textual transcription of the audio to the computing device (para [0023], where the transcription is provided to the devices);
analyzing the received audio, using a second machine learning model, to generate a second textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning); and
fusing the first textual transcription and the second textual transcription using a third machine learning model to generate a fused transcription (para [0030], [0055], where the transcription generation technique is a fusion or combination of the output of two or more transcriptions, and para [0093], where the transcription systems use machine learning); and
Thomson does not teach:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model.
Thomson2 teaches:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model (Col. 83 lines 16-29, where transcriptions are fused at the end of the communication session when real time operations are not necessary).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson by performing the fusing of the transcriptions of Thomson (Thomson para [0025]) either in real time or after the end of the communication session as taught by Thomson2 (Thomson2 col. 83 lines 16-29), so that the processes do not need to be run as frequently (Thomson2 col. 83 lines 16-29).
Regarding claim 17, Thomson in view of Thomson2 teaches:
The computer-implemented method of claim 16, wherein the audio is analyzed at predefined time intervals as it is received from the computing device and the first textual transcription is generated for each portion of the audio corresponding to each predefined interval (Thomson2 col. 79 lines 15-30, where the fused transcription is recomputed periodically at short intervals).
Regarding claim 18, Thomson in view of Thomson2 teaches:
The computer-implemented method of claim 16, wherein the first machine learning model is a faster, less accurate and lower cost machine learning model than the second machine learning model (Thomson para [0041], [0072], [0075], where one technique is more accurate with higher latency, and higher costs).
Regarding claim 20, Thomson teaches:
A non-transitory computer-readable storage medium storing instructions (Fig. 2 element 242, para [0051], [0057] where memory includes storage media) that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receiving audio, via a computing device, and performing in real-time until termination is detected, operations (para [0020], where audio is received during a communication session) comprising:
analyzing the received audio, using a first machine learning model, to generate a first textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning);
providing the first textual transcription of the audio to the computing device (para [0023], where the transcription is provided to the devices);
analyzing the received audio, using a second machine learning model, to generate a second textual transcription of the audio (para [0025], where the transcription generation technique recognize and transcribe the audio, and para [0093], where the transcription systems use machine learning); and
fusing the first textual transcription and the second textual transcription using a third machine learning model to generate a fused transcription (para [0030], [0055], where the transcription generation technique is a fusion or combination of the output of two or more transcriptions, and para [0093], where the transcription systems use machine learning); and
Thomson does not teach:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model.
Thomson2 teaches:
upon detection of termination of the audio, generating a final transcription by fusing the first textual transcription and the second textual transcription of all audio received, using the third machine learning model (Col. 83 lines 16-29, where transcriptions are fused at the end of the communication session when real time operations are not necessary).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson by performing the fusing of the transcriptions of Thomson (Thomson para [0025]) either in real time or after the end of the communication session as taught by Thomson2 (Thomson2 col. 83 lines 16-29), so that the processes do not need to be run as frequently (Thomson2 col. 83 lines 16-29).
Claim(s) 4-5 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, in view of Thomson2, and further in view of Qian et al. (US 2026/0079606 A1), hereinafter referred to as Qian.
Regarding claim 4, Thomson in view of Thomson2 teaches:
The system of claim 1,
Thomson in view of Thomson2 does not teach:
wherein the third machine learning model is a large language model.
Qian teaches:
wherein the third machine learning model is a large language model (para [0121], where an LLM is used to generate the combined transcript).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 by using the LLM of Qian (Qian para [0121]) to fuse the transcripts of Thomson in view of Thomson2 (Thomson para [0025]), in order to intelligently identify content in data and generate content elements for additional data (Qian para [0020]).
Regarding claim 5, Thomson in view of Thomson2 teaches:
details of the first machine learning model, the second machine learning model (Thomson para [0026], where techniques include the details) and
Thomson in view of Thomson2 does not teach:
wherein fusing the first textual transcription and the second textual transcription using the third machine learning model comprises generating a prompt including the first textual transcription and the second textual transcription.
Qian teaches:
The system of claim 1, wherein fusing the first textual transcription and the second textual transcription using the third machine learning model comprises generating a prompt including the first textual transcription and the second textual transcription (para [0121-122], where an LLM is used to fuse the transcriptions, the prompt including the two transcripts and instructions to generate a combined transcript).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 by using the LLM of Qian (Qian para [0121]) to fuse the transcripts of Thomson in view of Thomson2 (Thomson para [0025]), in order to intelligently identify content in data and generate content elements for additional data (Qian para [0020]).
Regarding claim 19, Thomson in view of Thomson2 teaches:
The computer-implemented method of claim 16,
Thomson in view of Thomson2 does not teach:
wherein the third machine learning model is a large language model.
Qian teaches:
wherein the third machine learning model is a large language model (para [0121], where an LLM is used to generate the combined transcript).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 by using the LLM of Qian (Qian para [0121]) to fuse the transcripts of Thomson in view of Thomson2 (Thomson para [0025]), in order to intelligently identify content in data and generate content elements for additional data (Qian para [0020]).
Claim(s) 6-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, in view of Thomson2, and Qian, and further in view of Park et al. (US 2024/0419701 A1), hereinafter referred to as Park.
Regarding claim 6, Thomson in view of Thomson2 and Qian teaches:
The system of claim 5,
Thomson in view of Thomson2 and Qian does not teach:
wherein the computing device is a head wearable apparatus and the operations further comprising:
detecting, via one or more camera of the head wearable apparatus, one or more objects in a field of view of the head wearable apparatus;
generating contextual details associated with the detected one or more objects; and
generating the prompt to further include the contextual details associated with the detected one or more objects.
Park teaches:
wherein the computing device is a head wearable apparatus (para [0069], where smart glasses are used) and the operations further comprising:
detecting, via one or more camera of the head wearable apparatus, one or more objects in a field of view of the head wearable apparatus (para [0069], where smart glasses gather contextual information about objects of interest using gaze information);
generating contextual details associated with the detected one or more objects (para [0069], where smart glasses gather contextual information about objects of interest using gaze information); and
generating the prompt to further include the contextual details associated with the detected one or more objects (para [0070], where the prompt to the LLM includes the context about the objects).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 and Qian by using the context information of Park (Park para [0069-70]) to form the prompt of Thomson in view of Thomson2 and Qian (Qian para [0121-122]), as the contextual information may be useful to augment the user prompt (Park para [0069]).
Regarding claim 7, Thomson in view of Thomson2 and Qian teaches:
The system of claim 5, further comprising:
Thomson in view of Thomson2 and Qian does not teach:
determining a location corresponding to the computing device; and
generating the prompt to further include the location corresponding to the computing device.
Park teaches:
determining a location corresponding to the computing device (para [0070], where location tracking information is used); and
generating the prompt to further include the location corresponding to the computing device (para [0069-70], where the prompt to the LLM includes the context information including location data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 and Qian by using the context information of Park (Park para [0069-70]) to form the prompt of Thomson in view of Thomson2 and Qian (Qian para [0121-122]), as the contextual information may be useful to augment the user prompt (Park para [0069]).
Regarding claim 8, Thomson in view of Thomson2 and Qian teaches:
The system of claim 5, further comprising:
Thomson in view of Thomson2 and Qian does not teach:
accessing a profile of a user associated with the computing device; and
generating the prompt to further include context details associated with the profile of the user associated with the computing device.
Park teaches:
accessing a profile of a user associated with the computing device (para [0221], where a user profile is managed by the device); and
generating the prompt to further include context details associated with the profile of the user associated with the computing device (para [0221], where the profile information is used to augment interactions between the user and the LLM).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 and Qian by using the context information of Park (Park para [0069-70]) to form the prompt of Thomson in view of Thomson2 and Qian (Qian para [0121-122]), as the contextual information may be useful to augment the user prompt (Park para [0069]).
Regarding claim 9, Thomson in view of Thomson2 and Qian teaches:
The system of claim 5, wherein the computing device is a first computing device and further comprising:
Thomson in view of Thomson2 and Qian does not teach:
generating the prompt to further include contextual information determined from a second computing device coupled with the first computing device.
Park teaches:
generating the prompt to further include contextual information determined from a second computing device coupled with the first computing device (para [0069-70], where glasses and phone are paired and context information from each device is used to generate the prompt).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 and Qian by using the context information of Park (Park para [0069-70]) to form the prompt of Thomson in view of Thomson2 and Qian (Qian para [0121-122]), as the contextual information may be useful to augment the user prompt (Park para [0069]).
Regarding claim 10, Thomson in view of Thomson2 and Qian teaches:
The system of claim 5, wherein the computing device is a first computing device and further comprising:
Thomson in view of Thomson2 and Qian does not teach:
generating the prompt to further include historical transcription data.
Park teaches:
generating the prompt to further include historical transcription data (para [0065], [0067], [0069], where accumulated user context includes speech to text data, and is used in forming the prompt).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Thomson in view of Thomson2 and Qian by using the context information of Park (Park para [0069-70]) to form the prompt of Thomson in view of Thomson2 and Qian (Qian para [0121-122]), as the contextual information may be useful to augment the user prompt (Park para [0069]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 2024/0386813 A1 para [0148] teaches using multiple ASR models simultaneously, with some optimized for streaming real time speech to text while others are optimized for accuracy but cannot stream, and then combining the outputs to increase accuracy; US 2026/0148012 A1 para [0034] teaches using context such as profile and location data, in generating a prompt for an LLM.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRYAN S BLANKENAGEL whose telephone number is (571)270-0685. The examiner can normally be reached 8:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYAN S BLANKENAGEL/Primary Examiner, Art Unit 2658
Prosecution Insights