CTNF 18/862,089 CTNF 80127 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent claims 1, 11, and 16 recite “receive a transcription …”, “receive a summary request …”, “identify … extracted text”, “process the extracted text … to obtain a summary …”, and “display …”, and “providing …”. Independent claim 21 recites “receive a transcription …”, “receive and image …”, “process the transcription stream … to obtain a summary”, “combine the image and the summary …”, and “display the augmented summary”. These limitations, under its broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting “processor”. For example, but for the “processor” language, these steps in the context of this claim encompasses the user manually receiving a transcription and a summary request, summarizing the transcription for displaying on a device screen. The user also manually receiving an image and combine the summary with the image for displaying using the help of a computer. All of these steps can be performed in the mind and/or using a pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements - using a processor to perform these steps. The use of a processor is recited at a high-level of generality (i.e., as a generic computer device performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “a summarization machine learning model” is merely for the purpose of data gathering and/or insignificant extra-solution activity that amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible. Dependent claims 2-10, 12-15, 17-20, and 22-30 includes additional steps or elements under its broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of generic computer components in the context of this claim encompasses the user manually performing these steps. All of these steps can be performed in the mind and/or using a pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. The claims are also not patent eligible. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-5, 7-9, 11, 13-16, and 20 are rejected under 35 U.S.C. 102( a)(1 )as being anticipated by Boekweg et al. (USPG 2020/0357408, hereinafter Boekweg) . Regarding claims 1, 11, and 16, Boekweg discloses a computer program product, device and method, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to: at least one processor ( figure 4 ); at least one memory ( figure 4 ); at least one input device ( figure 4 ); and at least one display, wherein instructions stored using the at least one memory, when executed by the at least one processor ( figure 4 ), cause the device to: receive a transcription stream including transcribed text that has been transcribed from speech ( paragraphs 109-110 and/or figure 5, steps 505-515, obtaining audio and transcribing into text ); receive a summary request for a summary to be provided on a display of a device ( paragraph 77, “the device 200 may send the request to generate the summary 215 in response to obtaining an indication of an event, such as user misunderstanding, and determining that the user misunderstanding satisfies a threshold level of user misunderstanding”, this event can be facial expression of the user as discussed in paragraph 114 ); identify, from the transcribed text and in response to the summary request, extracted text ( paragraph 32, “The summary of the transcription may include a simplified form of the transcription. As examples, the summary may include, based on the transcription: a reduced word count, smaller or less complex words, bullet points, main concepts, paraphrasing, acronyms, slang, key words, or any other suitable type of summary that simplifies the transcription” ); process the extracted text using a summarization machine learning (ML) model to obtain a summary of the extracted text ( paragraph 111, “The summary may be generated using a summary generation system” and/or figure 5, step 520 ); and display the summary on the display of the device ( paragraph 112 and/or figure 5, step 525, displaying a summary of the text ). Regarding claims 2-5 and 14-15, Boekweg further discloses wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: receive the summary request from a user of the device, via an input device of the device ( paragraph 77, “ the device 200 may send the request to generate the summary 215 in response to obtaining an indication of an event, such as user misunderstanding, and determining that the user misunderstanding satisfies a threshold level of user misunderstanding”; this event can be facial expression of the user as discussed in paragraph 114 ); wherein the input device includes at least one of a touchscreen, a gesture recognition device, a scroll bar, a button, or a microphone ( paragraph 77, “ the device 200 may send the request to generate the summary 215 in response to obtaining an indication of an event, such as user misunderstanding, and determining that the user misunderstanding satisfies a threshold level of user misunderstanding”; this event can be facial expression of the user or audio volume received via a microphone as discussed in paragraph 114 ); wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: receive the summary request as a vocal command from a user of the device, via a microphone of the device ( this event can be facial expression of the user or “key phrases in the audio” received via a microphone as discussed in paragraph 114 ); receive the transcription stream from a speech recognition engine ( figure 5, steps 515-520, speech recognizer converts speech to text, which are then forwarded to the text summarizer ). Regarding claims 7-9 and 20, Boekweg further discloses wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: identify the extracted text as including text received at the device prior to the summary request; and extract the extracted text from a transcription buffer ( paragraphs 80-82, text is extracted for the purpose of summary generation; the summary can be made available for presentation upon request ); wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: identify the extracted text as including text received after the summary request; and extract the extracted text from the transcription stream after the summary request ( paragraphs 77, 80-82, and 114, multiple embodiments, where request can be made before and/or after text summarization ); generate at least two summaries using the summarization ML model, including the summary; and select the summary from the at least two summaries based on device characteristics of the device and on user preferences of a user of the device ( paragraphs 84 and 89, “a search of multiple summaries that include the summary 215 may be made on the device 200 ”; a summary can be provided for display based on a received command ). Regarding claim 13, Boekweg further discloses the device of claim 11, wherein the device is configured to receive the transcription stream and the summary from a second device in communication with the device ( figure 1 and/or paragraph 37, “receiving the summary at one or both of the first and second devices 104 and 106 may include receiving the summary of the transcription of the communication session as generated by the transcription system 130” ) . 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15-03-aia AIA Claim 21 is rejected under 35 U.S.C. 102(a)(2) as being anticipated by Baughman et al. (USPG 2022/0027550, hereinafter Baughman) . Regarding claim 21, Baughman discloses a computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to: receive a transcription stream including transcribed text that has been transcribed from speech ( process in figure 5, steps 54-58 together with paragraph 38, “multimedia data can include a video and the audio of the video transcribed to text” ); receive an image associated with receipt of the transcription stream ( figure 5, step 56 and 320-324 ); process the transcription stream using a summarization machine learning (ML) model to obtain a summary stream, including processing the transcribed text to obtain a summary ( figure 5, steps 308-316 and/or paragraph 49, “the method 100 includes summarizing the text information into a text factoid 316 , as in block 112 . The method can use the retrieved text data and text information to generate a text factoid 316 , for example, a text summarization of the text information” ); combine the image and the summary to obtain an augmented summary ( figure 5, step 328 and/or paragraph 54, “combining the text factoid and the current state image into a multimedia factoid 328 ” ); and display the augmented summary ( figure 5, step 332 and/or paragraph 55, “The method 100 includes storing the multimedia factoid 328 for sending the multimedia factoid to a media outlet for presentation on a media format” ) . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 6, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Boekweg in view of Cheung et al. (USPG 2021/0151058, hereinafter Cheung) . Regarding claim 6, 12, and 19, Boekweg fails to explicitly disclose, however, Cheung teaches wherein the device includes a head-mounted display (HMD) and the display includes an HMD display, and wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: display the transcription stream using the HMD display ( paragraph 70, “application engine 440 and rendering engine 422 may generate and render for display a user interface for displaying transcription and/or additional data” ). Since Boekweg and Cheung are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of display a transcription on an HMD display. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Boekweg in view of Shakil et al. (USPG 2014/0222526, hereinafter Shakil) . Regarding claim 10, Boekweg fails to explicitly disclose, however, Shakil teaches computer program product of claim 1, wherein the device includes a head-mounted display (HMD) and the display includes an HMD display, and wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: display the summary using the HMD display ( see claim 15, “at a head-mounted display of a head-mounted computing device worn by the provider, wherein rendering of the summary at the head-mounted display prevents the patient from viewing the summary” ). Since Boekweg and Shakil are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of display a summary on an HMD display. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim s 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Boekweg in view of Asi et al. (USPG 2023/0360640, hereinafter Asi) . Regarding claim 17, Boekweg fails to explicitly disclose, however, Asi teaches the method of claim 16, further comprising: storing the summary with the summary request as labeled training data; and training the summarization ML model using the labeled training data ( paragraph 34, using text and keywords as data labeled with predetermined summaries to train a ML model ). Since Boekweg and Asi are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of training ML model to summarize text. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007). Regarding claim 18, Bowekweg further discloses the method of claim 17, further comprising: detecting a second summary request using the summarization ML model after the training; and summarizing second extracted text using the summarization ML model ( see claim 1 above; second summary request is merely a second request to summarize a second using the trained ML model . 07-21-aia AIA Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Cho et al. (SUPG 2022/0414338, hereinafter Cho) . Regarding claim 22, Baughman fails to explicitly disclose, however, Cho teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: select a time interval based on receiving the image ( paragraph 115, “When a user hovers the mouse pointer over the timeline of a video, a summary preview is shown” ); and extract the summary from a portion of the summary stream corresponding to the time interval ( paragraph 115, “When a user hovers the mouse pointer over the timeline of a video, a summary preview is shown” ). Since Boekweg and Cho are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of selecting a time interval on the video to find a corresponding text summary. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim s 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Dayan et al. (USPG 2015/0332666, hereinafter Dayan) . Regarding claims 23-24, Baughman fails to explicitly disclose, however, Dayan teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: process the transcribed text using a text entity extractor ML model to identify an entity within the transcribed text; and display the augmented summary with the entity visually distinguished therein ( paragraphs 42-57, analyzing text content to determine entities, relation, emotion, etc. to generate a text summary and then present the summary along with image/video as discussed in in figure 1 of Dayan and figure 5 of Baughman discussed in claim 1 above ); and process the transcribed text using an emotion analyzer ML model to identify an emotion associated with the transcribed text; and display the augmented summary with the emotion visually distinguished therein ( paragraphs 42-57, analyzing text content to determine entities, relation, emotion, etc. to generate a text summary and then present the summary along with image/video as discussed in in figure 1 of Dayan and figure 5 of Baughman discussed in claim 1 above ). Since Baughman and Dayan are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of generating text summary based on identified entities and emotion in the text content. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Krupat et al. (USPG 2018/0303397, hereinafter Krupat) . Regarding claim 25, Baughman fails to explicitly disclose, however, Krupat teaches the computer program product of claim 24, wherein the emotion is indicated by inclusion of a corresponding emoji ( paragraphs 144 and 147; also see figure 3 ). Since Baughman and Krupat are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of using emoji to represent detect emotion in a content. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Curtis et al. (USPG 2015/0235264, hereinafter Curtis) . Regarding claim 26, Baughman fails to explicitly disclose, however, Curtis teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: process the image using an image entity extractor ML model to identify an entity within the image; and display the augmented summary with the entity visually distinguished therein ( process in figure 1, identify entities in the video taught by Curtis and present augmented summary taught by Baughman as discussed in claim 1 above ). Since Baughman and Curtis are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of process a video signal to identify entities in it. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Liou (USPG 2002/0059395, hereinafter Liou) . Regarding claim 27, Baughman fails to explicitly disclose, however, Liou teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: process the transcribed text using an intent extractor ML model to identify an intention associated with the transcribed text; and display the augmented summary with the intention visually distinguished therein ( paragraph 59 and/or figure 11 ). Since Baughman and Liou are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of displaying summary and intention. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Nir (USPG 2016/0066055, hereinafter Nir) . Regarding claim 28, Baughman fails to explicitly disclose, however Nir teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: combine the image and the transcribed text to obtain an augmented transcription; and display the augmented transcription ( paragraph 49, converting audio in the audio-visual content to text and then combining the transcribed text to the video for displaying ). Since Baughman and Nir are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of generating closed captions. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Shakil . Regarding claim 29, Baughman fails to explicitly disclose, however, Shakil teaches the computer program product of claim 21, wherein the at least one computing device includes a head-mounted display (HMD), and wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: display the augmented summary using the HMD ( see claim 15, “at a head-mounted display of a head-mounted computing device worn by the provider, wherein rendering of the summary at the head-mounted display prevents the patient from viewing the summary” ). Since Baughman and Shakil are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of display a summary on an HMD display. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . 07-21-aia AIA Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over Baughman in view of Boekweg . Regarding claim 30, Baughman fails to explicitly disclose, however, Boekweg teaches the computer program product of claim 21, wherein the instructions, when executed by the at least one computing device, are further configured to cause the at least one computing device to: display at least one stream of the transcription stream and the summary stream with a scroll bar having a scroll button ( paragraphs 32, 90-95, 112, and/or 120, both transcript and summary have scrolling functions; Also see figures 3A-B ); receive a movement of the scroll button that aligns the scroll button with text of the transcription stream or the summary stream; and generate the augmented summary based on a selection of the scroll button while aligned with the text ( paragraphs 32, 90-95, 112, and/or 120, both transcript and summary have scrolling functions that when move, corresponding sections are aligned; Also see figures 3A-B ). Since Baughman and Boekweg are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of having scrolling functions associated with transcript and text summary such that when moved, corresponding sections are aligned. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007) . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Aoki et al. (USPG 2025/0168276) teach a method for transcribing audio to text and then summarizing the text that is considered pertinent to the claimed invention . Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUYEN X VO whose telephone number is (571)272-7631. The examiner can normally be reached M-F, 8-4. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HUYEN X VO/Primary Examiner, Art Unit 2656 Application/Control Number: 18/862,089 Page 2 Art Unit: 2656 Application/Control Number: 18/862,089 Page 3 Art Unit: 2656 Application/Control Number: 18/862,089 Page 4 Art Unit: 2656 Application/Control Number: 18/862,089 Page 5 Art Unit: 2656 Application/Control Number: 18/862,089 Page 6 Art Unit: 2656 Application/Control Number: 18/862,089 Page 7 Art Unit: 2656 Application/Control Number: 18/862,089 Page 8 Art Unit: 2656 Application/Control Number: 18/862,089 Page 9 Art Unit: 2656 Application/Control Number: 18/862,089 Page 10 Art Unit: 2656 Application/Control Number: 18/862,089 Page 11 Art Unit: 2656 Application/Control Number: 18/862,089 Page 12 Art Unit: 2656 Application/Control Number: 18/862,089 Page 13 Art Unit: 2656 Application/Control Number: 18/862,089 Page 14 Art Unit: 2656 Application/Control Number: 18/862,089 Page 15 Art Unit: 2656 Application/Control Number: 18/862,089 Page 16 Art Unit: 2656 Application/Control Number: 18/862,089 Page 17 Art Unit: 2656 Application/Control Number: 18/862,089 Page 18 Art Unit: 2656 Application/Control Number: 18/862,089 Page 19 Art Unit: 2656
Prosecution Insights