DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments filed 08/03/2026 to claim(s) 1, 9 and 14 have been entered and fully considered in this rejection. Claim 12 is cancelled accordingly.
Response to Arguments
Applicant’s arguments with respect to the 35 U.S.C. 112(b) rejection of claims 14 to 20 have been considered in light of the amended claim. The 35 U.S.C. 112(b) rejection has been withdrawn.
Applicant’s arguments with respect to the 35 U.S.C. 102 rejection claim(s) 1 to 6 and 8 have been considered but are moot because the new ground of rejection.
Applicant’s arguments with respect to the 35 U.S.C. 103 rejection claim(s) 7 and 9 to 20 have been considered but are moot because the new ground of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 to 6, 8 and 14 to 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gardner; Richard et al. (US 12008332 B1) hereinafter GARDNER in view of Lester; Brian David et al. (US 20230325725 A1) hereinafter LESTER in view of Saleh; Mohammad et al. (US 10885436 B1) hereinafter SALEH in further view of GU, Yang et al. (CN 114374813 A) hereinafter GU.
Regarding claim 1, GARDNER teaches:
A method for transcribing audio, the method comprising: receiving audio information uploaded through a scenario entry of a storage service application installed on a client;
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy.” (GARDNER [Column 12 line 24-30]).
“At operation 302, a first content item is identified. The first content item may contain multiple sub-content items, such as sentences or passages. The first content item can be sourced in various ways, such as loading a document file, scraping a website, interfacing with a content repository via API, recording a phone call, transcribing a video, extracting text from images, etc. ... ” (GARDNER [Column 12 line 57-63]).
determining, based on the scenario entry, the scenario type of the audio information;
“In some embodiments, when the content summarization services 120 are operating in an inference mode, such as when a maximum abstractness threshold is exceeded, the services 120 may access external systems and data sources. For example, as illustrated in FIG. 1, external data stores 130 containing supplemental information may be accessed via network 104. These external data stores 130 may include knowledge bases, ontologies, public and proprietary databases, and other structured data. Example external data items include dictionaries, thesauruses, industry-specific taxonomies, company and product databases, maps, academic paper repositories, legal document collections, and financial data. By leveraging these external data items in inference mode, the content summarization services 120 can enrich the representation of the content items 128 to include additional contextual details and derived knowledge. This allows the services 120 to generate abstractive summaries that go beyond the information directly contained within the content items 128.” (GARDNER [Column 5 line 55- Column 6 line 6]).
performing speech recognition on the audio information to obtain text information corresponding to the audio information;
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy. ” (GARDNER [Column 12 line 24-30]).
“ ...Real-time access enables them to leverage the summarization capabilities to extract key information, insights, and context to inform automated decisions and actions. ” (GARDNER [Column 20 line 25-28]).
inputting the text information and a prompt corresponding to the scenario type into a language model to obtain summary information,
“To achieve this functionality, the system may automatically generate prompts designed to produce a desired level of abstraction from the large language model (LLM) that generates the summaries. …” (GARDNER [Column 18 line 25-28]).
“Additionally, for sensitive content like legal contracts or medical records, the system may identify and preserve especially important data even at very high levels of summarization. Techniques may include one or more of the following: Flagging certain data like names, dates, identifiers, obligations as high priority for retention during the natural language processing analysis of the source content; Maintaining a list of entities, terms, and data types that are typically essential for certain document types like contracts or patient records based on best practices: Providing explicit instructions in the prompts generated for the LLM to retain specific highlighted information in the summary, overriding the abstraction process; Programmatically checking generated summaries during training to ensure retention of flagged data, and providing feedback to the LLM on errors; Allowing users to tag certain data as mandatory to keep prior to summarization; Employing conditional logic in the prompts to preserve different data depending on document type, leveraging document classification techniques; and/or Generating multiple summary variations with and without the sensitive data to enable comparison. ” (GARDNER [Column 19 line 1-24]).
wherein the language model is obtained by performing supervised fine-tuning on a pre-trained model using samples corresponding to various scenario types,
“The training module 280 enables fine-tuning or training of the language models 230 on custom datasets relevant to the summarization task and domain. While not required, additional training can improve performance by adapting the models to the terminology, writing style, and abstraction levels preferred for the target use case. For example, legal summarization may benefit from models fine-tuned on legal documents to better handle long, complex content with precise definitions and citations. The training module 280 may be configured to fine-tune language models 230 on domain-specific datasets. Data preprocessing may include techniques like formatting, cleaning, and sampling representative content. Various model architectures like BERT or GPT-3 may be used for abstractive summarization. Models may be trained using maximum likelihood estimation and backpropagation to optimize abstraction performance. ” (GARDNER [Column 9 line 44-59]).
“At operation 602, data is generated. For example, the training data includes large corpora for pretraining foundation models as well as domain-specific document/summary pairs for fine-tuning. In example embodiments, a data generation module may create datasets for training. ” (GARDNER [Column 61 line 65- Column 62 line 2]).
and the prompts corresponding to the various scenario types are obtained by tuning initial prompts corresponding to the various scenario types
“The system calculates appropriate word limits, abstraction levels, and instructions to generate prompts across the zoom level spectrum. It can optimize prompts through iterative testing and feedback. The prompts are designed to control the length, abstraction, inferences, data integration, and other attributes of the LLM-produced summaries. ” (GARDNER [Column 18 line 40-45]).
“The technical advancements will provide widespread utility across information-intensive domains including legal, academic, corporate, and government environments. ” (GARDNER [Column 2 line 30-39]).
wherein each sample comprises sample text information and sample summary information;
“The prompt engineering model may be trained on a diverse dataset of {content, prompt, summary} triplets. The content comprises text samples exhibiting a wide range of linguistic features and document structures. The prompts may be high-quality human-engineered examples written for different abstraction levels. The summaries may be human-generated by domain experts at each target abstraction level.” (GARDNER [Column 14 line 44-50]).
GARDNER does not teach but LESTER teaches:
using the same language model
LESTER [0051] “However, the systems and methods may share across all downstream tasks a single frozen pre-trained language model, in which all weights are fixed. A user can prime the model for a given task through prompt design (i.e., hand-crafting a text prompt with a description or examples of the task at hand). For instance, to condition a model for sentiment analysis, one can attach the prompt, “Is the following movie review positive or negative?” before the input sequence, “This movie was amazing!””
wherein the initial prompts are acquired with respective samples corresponding to the respective scenario type,
LESTER [0005] “One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.”
LESTER [0007] “In some implementations, the operations can include obtaining input text data. The input text data can include one or more words. The operations can include processing the prompt and the input text data with the pre-trained machine-learned model to generate output text data. The output text data can include a plurality of text characters. In some implementations, the operations can include providing the output text data as an output. The operations can include storing the prompt in a prompt database. The prompt database can include a plurality of prompts associated with a plurality of different tasks.”
wherein the initial prompts are tuned based on a corresponding difference between the sample summary information and
LESTER [0117] “At 606, the computing system can determine a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. In some implementations, the prompt gradient can be determined by evaluating a loss function that is evaluated based on a difference between the training output and the one or more training labels. The loss function can include a perceptual loss or another loss function. In some implementations, the labels can include ground truth outputs for the respective training examples.”
LESTER [0118] “At 608, the computing system can adjust one or more prompt parameters of the prompt based on the prompt gradient. In some implementations, the prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels such that the prompt is configured to be input with input data to the pre-trained machine-learned model to generate output data associated with the particular task. The particular task can include determining whether the input data comprises content associated with a positive intent. In some implementations, the input data can include visual data. The visual data can include one or more images. In some implementations, the output data can include output visual data, and the output visual data can include one or more images generated based at least in part on the input data and the prompt.”
LESTER [0243] “The training example (e.g., the example dataset and the respective labels) and the prompt can be processed with a machine-learned model (e.g., a frozen model) to generate one or more prompt gradients. The prompt gradient can be based at least in part on a difference between the label and a predicted label. The predicted label can be generated based on the example.”
wherein the
LESTER [0005] “One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.”
LESTER [0113] “In particular, FIG. 9 can depict a prompt generation system 900 being utilized to tune a prompt 904 for a particular task associated with the one or more training examples 902 and the one or more respective training labels 910. For example, one or more training examples 902 may be processed by the pre-trained machine-learned model 906 to generate output data 908. The output data 908 can be compared to one or more training labels 910 associated with the one or more training examples 910 in order to determine a prompt gradient 912. The prompt gradient 912 can then be utilized to adjust one or more parameters of a prompt 904. In some implementations, the prompt 904 can be input with the one or more training examples 902 to prime the pre-trained machine-learned model 906 for a particular task. The resulting output data 908 can then be utilized to evaluate a loss function to generate a prompt gradient 912 that can be utilized to further tune the prompt 904. The prompt parameter training can involve a plurality of iterations of output generation and comparison. During such training, the parameters of the pre-trained machine-learned model 906 can remain unadjusted, or “frozen.””
LESTER [0116] “At 604, the computing system can process one or more training examples of the plurality of training examples and a prompt with a pre-trained machine-learned model to generate a training output. The plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning (e.g., the pre-trained machine-learned model can be frozen such that the parameters are not adjusted during training of the prompt parameters). In some implementations, the pre-trained machine-learned model can include a model adapted to generate a text prediction output for text that follows an input text (e.g., the input text can include “the sky is” and the output can be “blue”). Alternatively and/or additionally, the pre-trained machine-learned model may have been trained with text masking (e.g., the input text can include “The man old” and the output can be “is”). The pre-trained machine-learned model can include one or more encoder blocks and one or more decoder blocks. For example, the pre-trained machine-learned model can include an encoder-decoder model such as a transformer model. The prompt can be associated with a particular task, and the particular task may be associated with the one or more training examples”
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER the capability to use the same pre-trained model and compare the result obtained from adjusting the prompt parameters as well as to tune an initial prompt using training samples associated with the respective scenario/task. The benefit and motivation of such modification is discussed by LESTER in the following portion: LESTER [Abstract] “Systems and methods for natural language processing can leverage trained prompts to condition a large pre-trained machine-learned model to generate an output for a specific task. For example, a subset of parameters may be trained for the particular task to then be input with a set of input data into the pre-trained machine-learned model to generate the task-specific output. During the training of the prompt, the parameters of the pre-trained machine-learned model can be frozen, which can reduce the computational resources used during training while still leveraging the previously learned data from the pre-trained machine-learned model.”
GARDNER in view of LESTER does not teach but SALEH teaches:
prediction summary information
”In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task.” (SALEH [Column 5 line 58 to Column 6 line 3]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER the capability to use text documents a respective known/ sample summaries in GARDNER’s training/ fine-tuning of its summarization. The benefit and motivation of such modification is discussed by SALEH in the following portion: “The system can pre-train a text summarization neural network using unlabeled text documents which are publicly available or otherwise easily obtainable in massive volumes. The pre-trained text summarization neural network can then be used to effectively adapt to a specific text summarization task using orders of magnitude less data than was used to pre-train the network. For example, while pre-training the network may utilize billions of unlabeled text documents, adapting the network for a specific task may require merely a few thousand labeled text documents.” (SALEH [Column 1 line 43 to 52]).
GARDNER in view of LESTER in further view of SALEH does not teach but GU teaches:
wherein the scenario entry is selectable from a plurality of interface elements displayed in a user interface of the storage service application, the selected interface element corresponding to a respective scenario type;
GU [n0083] “The display unit 130 can also be used to display information input by the user or information provided to the user, as well as the graphical user interface (GUI) of various menus of the recorder 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front of the recorder 100. The display screen 132 can be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 130 can be used to display multiple scene markers on the screen display interface of the recorder in this application and to preview the collected multimedia data.”
GU [n0122] “Therefore, the server can verify the correctness of the user information of the user using the recorder and the user's usage rights based on the user login information in the login request. Based on the verification result, the server will issue multiple scene identifiers that the user who logged in to the recorder can use, which can be displayed on the recorder's interface for the user to select, as shown in Figure 5A. It can ensure the information security of the recorder and avoid information security problems caused by unauthorized users logging in. For example, the default scenarios in Figure 5A are illegal scenarios, such as: scenario 1 is a car accident, scenario 2 is running a red light, scenario 3 is drunk driving, scenario 4 is driving against traffic, scenario 5 is fighting, scenario 6 is driving without a license, scenario 7 is visiting someone, and scenario 8 is a checkpoint inspection. The classification can be further refined according to actual usage, or other scenarios can be set according to actual conditions. This application embodiment does not impose any restrictions on this.”
GU [n0124] “In step 402, in response to the selection operation of the target scene identifier among the plurality of scene identifiers, the target scene identifier and the corresponding multimedia resource are associated and stored.”
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER in further view of SALEH the capability to provide a graphical interface to display a plurality of selectable scene/ scenario identifier entries, as well as to include the storing/uploading capabilities according to the scenario type. The benefit and motivation of such modification is discussed by GU in the following portion: GU [n0077] “The inventive concept of this application can be summarized as follows: the recorder can display multiple scene identifiers; and in response to the selection operation of a target scene identifier among the multiple scene identifiers, the target scene identifier and the corresponding multimedia resource are associated and stored; furthermore, the target scene identifier, the multimedia resource, and the start and end recording times of the multimedia resource can be sent to the server in a file upload request. Therefore, the recorder in this application can have the initiative to divide multimedia information, so that when using the recorder, the user can directly select the target scene identifier to record multimedia information, or directly exit the recording of multimedia information under the current scene identifier when recording another multimedia information, and select the corresponding scene identifier to record multimedia information. This realizes the recorder's active division of multimedia information and the association and storage of the target scene identifier and the corresponding multimedia resources, making the division of multimedia information more accurate, and facilitating accurate subsequent retrieval of multimedia information, thereby improving the management efficiency of multimedia resources.”
Regarding claim 2, the rejection of claim 1 is incorporated, furthermore GARDNER teaches:
The method according to claim 1, wherein the scenario type comprises at least one of: classroom recording, telephone recording, or interview recording.
(301) Students summarize textbooks, papers, lectures, and online materials at appropriate reading levels. ” (GARDNER [Column 28 line 16-19]).
(980) Investigative journalists can use the system to summarize lengthy raw transcripts and audio recordings from interviews with sources into condensed overviews. ” (GARDNER [Column 57 line 40-43]).
(597) Transcript of a 30 minute sales call between a sales representative and prospective client discussing the client's business needs and how the representative's company's software products can address them. ” (GARDNER [Column 40 line 60-64]).
Regarding claim 3, the rejection of claim 2 is incorporated, furthermore GARDNER teaches:
The method according to claim 2, wherein the scenario type is the classroom recording;
“Example Academic Use Cases
Summarizing Lectures
Professors can use the system to summarize lengthy written lectures and textbook chapters into condensed study materials for students. ” (GARDNER [Column 57 line 62-67]).
“This helps create optimized condensed lectures and study guides tailored to the needs of the students. ” (GARDNER [Column 58 line 7-8]).
and the inputting the text information and a prompt corresponding to the scenario type into a language model to obtain summary information, comprises: categorizing the text information based on the prompt corresponding to the classroom recording to obtain a category of the text information;
“The appropriate techniques are selected and combined to construct prompts customized for the content attributes, abstraction level, use case, and user profile. ” (GARDNER [Column 16 line 40-43]).
(301) Students summarize textbooks, papers, lectures, and online materials at appropriate reading levels. ” (GARDNER [Column 28 line 16-19]).
“Classification models can categorize the text subjects for topical filtering and abstraction. Text classifiers may be trained on labeled document corpuses to predict categories and tags. The system may be configured to use classifiers to orient summaries to key topical facets. ” (GARDNER [Column 7 line 27-32]).
and summarizing the text information according to the category using the language model to obtain the summary information.
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy. ” (GARDNER [Column 12 line 24-30]).
Regarding claim 4, the rejection of claim 3 is incorporated, furthermore GARDNER teaches:
The method according to claim 3, wherein the category comprises at least one of: a language learning category, a test-taking learning category, or a pan-learning category.
“A student can upload their professor's lengthy slide decks into the system to create condensed study guides for exam prep on their phone. Important concepts and examples are extracted while removing redundant or peripheral information from the raw slides. ” (GARDNER [Column 53 line 41-46]).
“Students summarize textbooks, papers, lectures, and online materials at appropriate reading levels. Journalists abstract interviews, scientific studies, and background sources into news articles for general audiences. ” (GARDNER [Column 28 line 16-19]).
“Extractive and abstractive summarizers identify and distill key ideas using encoder-decoder networks trained on domain-specific corpora. Classifiers categorize content facets using convolutional neural networks. Question answerers retrieve salient details with transformer architectures.” (GARDNER [Column 27 line 50-55]).
Regarding claim 5, the rejection of claim 2 is incorporated, furthermore GARDNER teaches:
The method according to claim 2, wherein the scenario type is the telephone recording or the interview recording;
“Investigative journalists can use the system to summarize lengthy raw transcripts and audio recordings from interviews with sources into condensed overviews. ” (GARDNER [Column 59 line 39-42]).
“Transcript of a 30 minute sales call between a sales representative and prospective client discussing the client's business needs and how the representative's company's software products can address them. ” (GARDNER [Column 40 line 59-63]).
and the inputting the text information and a prompt corresponding to the scenario type into a language model to obtain summary information, comprises: abstracting the text information into dialogue text information based on the prompt corresponding to the telephone recording or the interview recording;
“The system may be configured to automatically determine the most appropriate zoom levels and/or abstraction levels for the most likely uses of the system with respect to the content item (e.g., by prompting the LLM), generate corresponding prompts for each determined zoom level, and return the responses to each of the prompts. Positive Zoom Levels 100% —Prompt. Full transcript restatement (4,000 words) Response: The full transcript. 75% —Prompt: Restate the key discussion points in the sales call in 500 words: ” (GARDNER [Column 41 line 20-31]).
and summarizing the dialogue text information using the language model to obtain the summary information.
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy.” (GARDNER [Column 12 line 24-30]).
“The system may be configured to automatically determine the most appropriate zoom levels and/or abstraction levels for the most likely uses of the system with respect to the content item (e.g., by prompting the LLM), generate corresponding prompts for each determined zoom level, and return the responses to each of the prompts. Positive Zoom Levels 100% —Prompt. Full transcript restatement (4,000 words) Response: The full transcript. 75% —Prompt: Restate the key discussion points in the sales call in 500 words: ” (GARDNER [Column 41 line 20-31]).
Regarding claim 6, the rejection of claim 1 is incorporated, furthermore GARDNER teaches:
The method according to claim 1, wherein the scenario entry comprises a scenario import entry, for uploading the audio information locally stored on the client.
“A student can upload their professor's lengthy slide decks into the system to create condensed study guides for exam prep on their phone. Important concepts and examples are extracted while removing redundant or peripheral information from the raw slides. ” (GARDNER [Column 53 line 41-46]).
“While some of the descriptions of the system herein focus on text summarization, the system architecture is designed to support multimedia inputs as well, including audio, images, and video. ” (GARDNER [Column 12 line 20-23]).
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy.” (GARDNER [Column 12 line 24-30]).
Regarding claim 8, the rejection of claim 1 is incorporated, furthermore GARDNER teaches:
The method according to claim 1, wherein the method further comprises: receiving a batch processing instruction based on the scenario type sent by a storage service application installed on the client;
“The summarization workflow may begin when a client request is received by the system's API gateway. Robust ingestion prepares the request for processing. ” (GARDNER [Column 22 line 61-63]).
“Validated requests are queued asynchronously for robustness, including using, for example, one or more of the following: Message queues like Kafka, RabbitMQ or SQS buffer requests; Backpressure prevents overloading downstream services; Batching optimizes throughput and overhead; and/or Prioritization ensures VIP requests are processed quickly. Reliable queueing smooths out traffic spikes for efficient processing. ” (GARDNER [Column 23 line 44-53]).
“To improve quality, the system can automatically construct prompts engineered for the LLM vs just passing raw requests. ” (GARDNER [Column 51 line 2-4]).
“For example, the system may be pre-configured with templates for common summarization tasks like: Summarize article into 200 word overview; Extract key points from report in bullet list; and/or Explain concepts from textbook chapter simply. ” (GARDNER [Column 51 line 4-9]).
and processing historical uploaded audio information corresponding to the scenario type, based on the batch processing instruction,
“Initially, the system may be configured to handle simple summarization requests in an automated way with no customization. Clients may send requests like the following: POST/summarize { “content”: “ . . . ”, “abstraction_level”: 50% }” (GARDNER [Column 50 line 53-60]).
“Request batching with dynamic timeouts balances throughput and latency. Caching reduces prompt generation time for common content profiles. ” (GARDNER [Column 16 line 7-10]).
wherein the processing comprises at least one of: filtering the audio information,
“Classification models can categorize the text subjects for topical filtering and abstraction. Text classifiers may be trained on labeled document corpuses to predict categories and tags. The system may be configured to use classifiers to orient summaries to key topical facets. ” (GARDNER [Column 7 line 27-31]).
“Allow filtering and segmentation of analytics by any dimension like user, content type, model etc. ” (GARDNER [Column 37 line 17-18]).
editing the audio information,
“Within the system itself, aggregated behavioral data from user interactions can also be leveraged as a feedback signal for ongoing training. This can include data on which summaries users find most helpful, edits they make, zoom levels used, and other actions reflecting subjective preferences. ” (GARDNER [Column 8 line 30-34]).
“ Assign roles like Viewer, Editor, Reviewer, Admin etc. Each role maps to predefined permissions like summarizing content, editing summaries, configuring system. ” (GARDNER [Column 34 line 47-50]).
exporting the text information corresponding to the audio information,
“ For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy. ” (GARDNER [Column 12 line 24-30]).
“Negative zoom levels (e.g., from 0% to −100%) allow expanding of the summary by incorporating additional inferences made about the content and/or relevant data form one or more internal or external data sources. For example, at 0% abstraction, no additional information may be added beyond restating the original content. ” (GARDNER [Column 17 line 28-33]).
or exporting the summary information corresponding to the audio information.
“Configure target lengths for different user roles and content types. Executives may get 100-word previews while analysts get 500-word overviews. ” (GARDNER [Column 35 line 42-44]).
“The system reduces friction associated with digesting or referencing long, complex documents on small mobile screens. Key information is extracted on demand for easy review and sharing on the go.” (GARDNER [Column 54 line 58-62]).
Regarding claim 14, GARDNER teaches:
An electronic device, comprising: at least one processor;
“The storage unit 1716 includes a machine-readable medium 1722 on which is stored one or more sets of instructions and data structures (e.g., software) 1724 embodying or utilized by any one or more of the methodologies, operations, or functions described herein. The instructions 1724 may also reside, completely or at least partially, within the main memory 1704 and/or within the processor 1702 during execution thereof by the computer system 1100, the main memory 1704 and the processor 1702 also constituting machine-readable media. ” (GARDNER [Column 66 line 9-18]).
and a memory communicatively connected to the at least one processor;
“The storage unit 1716 includes a machine-readable medium 1722 on which is stored one or more sets of instructions and data structures (e.g., software) 1724 embodying or utilized by any one or more of the methodologies, operations, or functions described herein. The instructions 1724 may also reside, completely or at least partially, within the main memory 1704 and/or within the processor 1702 during execution thereof by the computer system 1100, the main memory 1704 and the processor 1702 also constituting machine-readable media. ” (GARDNER [Column 66 line 9-18]).
wherein, the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform first operations for transcribing audio;
“The storage unit 1716 includes a machine-readable medium 1722 on which is stored one or more sets of instructions and data structures (e.g., software) 1724 embodying or utilized by any one or more of the methodologies, operations, or functions described herein. The instructions 1724 may also reside, completely or at least partially, within the main memory 1704 and/or within the processor 1702 during execution thereof by the computer system 1100, the main memory 1704 and the processor 1702 also constituting machine-readable media. ” (GARDNER [Column 66 line 9-18]).
the first operations comprising: receiving audio information uploaded through a scenario entry of a storage service application installed on a client;
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy. ” (GARDNER [Column 112 line 24-30]).
determining, based on the scenario entry, a scenario type of the audio information;
“Various techniques can be used to orchestrate multi-model summarization, including one or more of the following: Generating multiple candidate summaries by providing the same prompt to different models; Then, an ensemble model compares the candidates and selects the best summary using consensus validation; Breaking down the prompt into modular sub-tasks best suited to different models, such as using a legal-focused model for contract analysis and a financial model for financial data; …” (GARDNER [Column 21 line 10-20]).
“By combining multiple diverse models, the system can take advantage of complementary strengths in language understanding, reasoning, data integration, and domain-specific knowledge. The orchestration mechanisms help select, refine, and consolidate model outputs to produce superior summaries compared to relying on a single model alone. ” (GARDNER [Column 21 line 37-44]).
performing speech recognition on the audio information to obtain text information corresponding to the audio information;
“For audio input such as podcasts, speeches, and phone calls, automated speech recognition may first be used to transcribe the audio to text. The resulting text transcript may then summarized using the system's text summarization capabilities. The system may employ optimized speech-to-text models that are fine-tuned on the target audio domains to maximize transcription accuracy. ” (GARDNER [Column 112 line 24-30]).
and inputting the text information and a prompt corresponding to the scenario type into a language model to obtain summary information,
“The present disclosure provides specific technological solutions to these problems through one or more described features or combinations of features. The disclosed system enables users to precisely define a target level of abstraction (e.g., by specifying a percentage of length reduction) for a representation of a summary of a content item. For example, the system gives granular control over depth and brevity. The system may also be configured to engineer iterative prompts to have the LLM summarize its own prior outputs at increasing levels of abstraction. This allows for gradual refinement while preserving substantive information. ” (GARDNER [Column 2 line 5-15]).
wherein the language model is obtained by performing supervised fine-tuning on a pre-trained model using samples corresponding to various scenario types,
“During further training and fine-tuning of the models for the summarization task, curated datasets of document-summary pairs in the target domain can be used. These provide examples for training the models to produce summaries with the appropriate length, perspective, and/or other attributes. ” (GARDNER [Column 8 line 12-16]).
“The prompt engineering model may be trained on a diverse dataset of {content, prompt, summary} triplets. The content comprises text samples exhibiting a wide range of linguistic features and document structures. The prompts may be high-quality human-engineered examples written for different abstraction levels. The summaries may be human-generated by domain experts at each target abstraction level. ” (GARDNER [Column 14 line 44-50]).
“This provides supervised training data to learn associations between content characteristics, prompt structure, and desired summaries. Human-curated data ensures prompts achieve goals without undesirable biases. ” (GARDNER [Column 14 line 51-54]).
and the prompts corresponding to the various scenario types are obtained by tuning initial prompts corresponding to the various scenario types
“The system is pre-trained on a large dataset of {content, prompt, summary} triplets. The content comprises documents of various lengths and complexities. The prompts are engineered with different instructions, keywords, and text manipulation techniques to target different levels of abstraction. The summaries are human-generated at the target abstraction levels. ” (GARDNER [Column 4 line 25-31]).
“This training data enables the system to learn prompt engineering strategies tailored to different abstraction goals. Neural networks learn optimal prompt phrasing, text preprocessing like entity masking, and prompt structure to achieve the target compression rates. ” (GARDNER [Column 4 line 2-36]).
wherein each sample comprises sample text information and sample summary information;
“The prompt engineering model may be trained on a diverse dataset of {content, prompt, summary} triplets. The content comprises text samples exhibiting a wide range of linguistic features and document structures. The prompts may be high-quality human-engineered examples written for different abstraction levels. The summaries may be human-generated by domain experts at each target abstraction level.” (GARDNER [Column 14 line 44-50]).
GARDNER does not teach but LESTER teaches:
using the same language model
LESTER [0051] “However, the systems and methods may share across all downstream tasks a single frozen pre-trained language model, in which all weights are fixed. A user can prime the model for a given task through prompt design (i.e., hand-crafting a text prompt with a description or examples of the task at hand). For instance, to condition a model for sentiment analysis, one can attach the prompt, “Is the following movie review positive or negative?” before the input sequence, “This movie was amazing!””
wherein the initial prompts are acquired with respective samples corresponding to the respective scenario type,
LESTER [0005] “One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.”
LESTER [0007] “In some implementations, the operations can include obtaining input text data. The input text data can include one or more words. The operations can include processing the prompt and the input text data with the pre-trained machine-learned model to generate output text data. The output text data can include a plurality of text characters. In some implementations, the operations can include providing the output text data as an output. The operations can include storing the prompt in a prompt database. The prompt database can include a plurality of prompts associated with a plurality of different tasks.”
wherein the initial prompts are tuned based on a corresponding difference between the sample summary information and
LESTER [0117] “At 606, the computing system can determine a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. In some implementations, the prompt gradient can be determined by evaluating a loss function that is evaluated based on a difference between the training output and the one or more training labels. The loss function can include a perceptual loss or another loss function. In some implementations, the labels can include ground truth outputs for the respective training examples.”
LESTER [0118] “At 608, the computing system can adjust one or more prompt parameters of the prompt based on the prompt gradient. In some implementations, the prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels such that the prompt is configured to be input with input data to the pre-trained machine-learned model to generate output data associated with the particular task. The particular task can include determining whether the input data comprises content associated with a positive intent. In some implementations, the input data can include visual data. The visual data can include one or more images. In some implementations, the output data can include output visual data, and the output visual data can include one or more images generated based at least in part on the input data and the prompt.”
LESTER [0243] “The training example (e.g., the example dataset and the respective labels) and the prompt can be processed with a machine-learned model (e.g., a frozen model) to generate one or more prompt gradients. The prompt gradient can be based at least in part on a difference between the label and a predicted label. The predicted label can be generated based on the example.”
wherein the
LESTER [0005] “One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.”
LESTER [0113] “In particular, FIG. 9 can depict a prompt generation system 900 being utilized to tune a prompt 904 for a particular task associated with the one or more training examples 902 and the one or more respective training labels 910. For example, one or more training examples 902 may be processed by the pre-trained machine-learned model 906 to generate output data 908. The output data 908 can be compared to one or more training labels 910 associated with the one or more training examples 910 in order to determine a prompt gradient 912. The prompt gradient 912 can then be utilized to adjust one or more parameters of a prompt 904. In some implementations, the prompt 904 can be input with the one or more training examples 902 to prime the pre-trained machine-learned model 906 for a particular task. The resulting output data 908 can then be utilized to evaluate a loss function to generate a prompt gradient 912 that can be utilized to further tune the prompt 904. The prompt parameter training can involve a plurality of iterations of output generation and comparison. During such training, the parameters of the pre-trained machine-learned model 906 can remain unadjusted, or “frozen.””
LESTER [0116] “At 604, the computing system can process one or more training examples of the plurality of training examples and a prompt with a pre-trained machine-learned model to generate a training output. The plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning (e.g., the pre-trained machine-learned model can be frozen such that the parameters are not adjusted during training of the prompt parameters). In some implementations, the pre-trained machine-learned model can include a model adapted to generate a text prediction output for text that follows an input text (e.g., the input text can include “the sky is” and the output can be “blue”). Alternatively and/or additionally, the pre-trained machine-learned model may have been trained with text masking (e.g., the input text can include “The man old” and the output can be “is”). The pre-trained machine-learned model can include one or more encoder blocks and one or more decoder blocks. For example, the pre-trained machine-learned model can include an encoder-decoder model such as a transformer model. The prompt can be associated with a particular task, and the particular task may be associated with the one or more training examples”
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER the capability to use the same pre-trained model and compare the result obtained from adjusting the prompt parameters as well as to tune an initial prompt using training samples associated with the respective scenario/task. The benefit and motivation of such modification is discussed by LESTER in the following portion: LESTER [Abstract] “Systems and methods for natural language processing can leverage trained prompts to condition a large pre-trained machine-learned model to generate an output for a specific task. For example, a subset of parameters may be trained for the particular task to then be input with a set of input data into the pre-trained machine-learned model to generate the task-specific output. During the training of the prompt, the parameters of the pre-trained machine-learned model can be frozen, which can reduce the computational resources used during training while still leveraging the previously learned data from the pre-trained machine-learned model.”
GARDNER in view of LESTER does not teach but SALEH teaches:
prediction summary information
”In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task.” (SALEH [Column 5 line 58 to Column 6 line 3]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER the capability to use text documents a respective known/ sample summaries in GARDNER’s training/ fine-tuning of its summarization. The benefit and motivation of such modification is discussed by SALEH in the following portion: “The system can pre-train a text summarization neural network using unlabeled text documents which are publicly available or otherwise easily obtainable in massive volumes. The pre-trained text summarization neural network can then be used to effectively adapt to a specific text summarization task using orders of magnitude less data than was used to pre-train the network. For example, while pre-training the network may utilize billions of unlabeled text documents, adapting the network for a specific task may require merely a few thousand labeled text documents.” (SALEH [Column 1 line 43 to 52]).
GARDNER in view of LESTER in further view of SALEH does not teach but GU teaches:
wherein the scenario entry is selectable from a plurality of interface elements displayed in a user interface of the storage service application, the selected interface element corresponding to a respective scenario type;
GU [n0083] “The display unit 130 can also be used to display information input by the user or information provided to the user, as well as the graphical user interface (GUI) of various menus of the recorder 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front of the recorder 100. The display screen 132 can be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 130 can be used to display multiple scene markers on the screen display interface of the recorder in this application and to preview the collected multimedia data.”
GU [n0122] “Therefore, the server can verify the correctness of the user information of the user using the recorder and the user's usage rights based on the user login information in the login request. Based on the verification result, the server will issue multiple scene identifiers that the user who logged in to the recorder can use, which can be displayed on the recorder's interface for the user to select, as shown in Figure 5A. It can ensure the information security of the recorder and avoid information security problems caused by unauthorized users logging in. For example, the default scenarios in Figure 5A are illegal scenarios, such as: scenario 1 is a car accident, scenario 2 is running a red light, scenario 3 is drunk driving, scenario 4 is driving against traffic, scenario 5 is fighting, scenario 6 is driving without a license, scenario 7 is visiting someone, and scenario 8 is a checkpoint inspection. The classification can be further refined according to actual usage, or other scenarios can be set according to actual conditions. This application embodiment does not impose any restrictions on this.”
GU [n0124] “In step 402, in response to the selection operation of the target scene identifier among the plurality of scene identifiers, the target scene identifier and the corresponding multimedia resource are associated and stored.”
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER in further view of SALEH the capability to provide a graphical interface to display a plurality of selectable scene/ scenario identifier entries, as well as to include the storing/uploading capabilities according to the scenario type. The benefit and motivation of such modification is discussed by GU in the following portion: GU [n0077] “The inventive concept of this application can be summarized as follows: the recorder can display multiple scene identifiers; and in response to the selection operation of a target scene identifier among the multiple scene identifiers, the target scene identifier and the corresponding multimedia resource are associated and stored; furthermore, the target scene identifier, the multimedia resource, and the start and end recording times of the multimedia resource can be sent to the server in a file upload request. Therefore, the recorder in this application can have the initiative to divide multimedia information, so that when using the recorder, the user can directly select the target scene identifier to record multimedia information or directly exit the recording of multimedia information under the current scene identifier when recording another multimedia information, and select the corresponding scene identifier to record multimedia information. This realizes the recorder's active division of multimedia information and the association and storage of the target scene identifier and the corresponding multimedia resources, making the division of multimedia information more accurate, and facilitating accurate subsequent retrieval of multimedia information, thereby improving the management efficiency of multimedia resources.”
Regarding claim 15, the rejection of claim 14 is incorporated, furthermore arguments analogous to the rejection of claim 2 and 13 are applicable.
Regarding claim 16, the rejection of claim 15 is incorporated, furthermore arguments analogous to the rejection of claim 3 are applicable.
Regarding claim 17, the rejection of claim 16 is incorporated, furthermore arguments analogous to the rejection of claim 4 are applicable.
Regarding claim 18, the rejection of claim 15 is incorporated, furthermore arguments analogous to the rejection of claim 5 are applicable.
Regarding claim 19, the rejection of claim 14 is incorporated, furthermore arguments analogous to the rejection of claim 6 are applicable.
Claim(s) 9 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over GARDNER in view of LESTER in view of SALEH .
Regarding claim 9, the rejection of claim 1 is incorporated, furthermore GARDNER teaches:
A method for training a language model, the method comprising: acquiring a first sample corresponding to a scenario type, wherein the first sample comprises first sample text information and first sample summary information;
“The Training Module 280 allows retraining the LLMs and other models on new data to improve summarization accuracy over time. It allows sample selection, data preprocessing, hyperparameter tuning, and model optimization. Adding this module allows the system to leverage new training data to automatically customize models for increased relevance to particular summarization tasks and domains. ” (GARDNER [Column 61 line 12-19]).
“For pretraining, it may sample diverse documents from web crawl data, books, academic papers, news etc., as described herein. This develops the general capabilities of the LLMs. ” (GARDNER [Column 62 line 3-7]).
“For instance, a healthcare company can use abstractive models trained on medical text to generate summaries using proper terminology. Question answering models can provide helpful explanations of clinical concepts. ” (GARDNER [Column 35 line 29-32]).
and adjusting parameters of the pre-trained model based on the first loss to obtain the language model.
“During fine-tuning, the model may be trained to generate summaries similar to the examples using maximum likelihood estimation and backpropagation. ” (GARDNER [Column 10 line 22-28]).
“During training, the model may input each text snippet sequentially and may try to predict the next word using its current internal representations. The weights may be updated through backpropagation to minimize the prediction loss across the entire corpus. ” (GARDNER [Column 6 line 21-25]).
wherein each sample comprises sample text information and sample summary information;
“The prompt engineering model may be trained on a diverse dataset of {content, prompt, summary} triplets. The content comprises text samples exhibiting a wide range of linguistic features and document structures. The prompts may be high-quality human-engineered examples written for different abstraction levels. The summaries may be human-generated by domain experts at each target abstraction level.” (GARDNER [Column 14 line 44-50]).
inputting the sample text information and the initial prompt into
“The prompts can include the original text to summarize along with instructions tailored to elicit the target summary characteristics from the LLM. The system sends the prompts through the API and ingests the LLM-generated summaries to present to the user.” (GARDNER [Column 6 line 45-49]).
“At operation 308, the prompt is provided to the LLM. The LLM can be implemented using machine learning frameworks like TensorFlow or PyTorch. It may run locally or leverage cloud APIs. Summarization occurs autonomously after prompt submission. Generated text in the response is extracted programmatically.” (GARDNER [Column 13 line 37-42]).
GARDNER does not teach but LESTER teaches:
the same language model
LESTER [0051] “However, the systems and methods may share across all downstream tasks a single frozen pre-trained language model, in which all weights are fixed. A user can prime the model for a given task through prompt design (i.e., hand-crafting a text prompt with a description or examples of the task at hand). For instance, to condition a model for sentiment analysis, one can attach the prompt, “Is the following movie review positive or negative?” before the input sequence, “This movie was amazing!””
wherein the method further comprises: acquiring an initial prompt and a sample corresponding to the scenario type,
LESTER [0005] “One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.”
and tuning the initial prompt based on a difference between the sample summary information and the
LESTER [0031] “A prompt gradient can then be determined based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. In some implementations, the prompt gradient can be determined by evaluating a loss function that is evaluated based on a difference between the training output and the one or more training labels. The loss function can include a perceptual loss or another loss function. In some implementations, the labels can include ground truth outputs for the respective training examples.”
LESTER [0028] “…The prompt can be associated with a particular task, and the particular task can be associated with the one or more training examples. A prompt gradient can be determined based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. The systems and methods can include adjusting one or more prompt parameters of the prompt based on the prompt gradient. The prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels such that the prompt is configured to be input with input data to the pre-trained machine-learned model to generate output data associated with the particular task.”
to obtain a prompt corresponding to the scenario type.
LESTER [0028] “…The prompt can be associated with a particular task, and the particular task can be associated with the one or more training examples. A prompt gradient can be determined based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. The systems and methods can include adjusting one or more prompt parameters of the prompt based on the prompt gradient. The prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels such that the prompt is configured to be input with input data to the pre-trained machine-learned model to generate output data associated with the particular task.”
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER the capability to use the same pre-trained model and compare the result obtained from adjusting the prompt parameters as well as to tune an initial prompt using training samples associated with the respective scenario/task. The benefit and motivation of such modification is discussed by LESTER in the following portion: LESTER [Abstract] “Systems and methods for natural language processing can leverage trained prompts to condition a large pre-trained machine-learned model to generate an output for a specific task. For example, a subset of parameters may be trained for the particular task to then be input with a set of input data into the pre-trained machine-learned model to generate the task-specific output. During the training of the prompt, the parameters of the pre-trained machine-learned model can be frozen, which can reduce the computational resources used during training while still leveraging the previously learned data from the pre-trained machine-learned model.”
GARDNER in view of LESTER does not teach, but SALEH teaches:
inputting the first sample text information into a pre-trained model to obtain first prediction summary information;
“In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task. ” (SALEH [Column 5 line 58 – Column 6 line 3]).
calculating a first loss based on the first sample summary information and the first prediction summary information;
“In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task.” (SALEH [Column 5 line 58 – Column 6 line 3]).
prediction summary information
”In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task.” (SALEH [Column 5 line 58 to Column 6 line 3]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER the capability to generate a first prediction summary information and utilize it to compute a loss difference to adjust the parameters of the pre-trained model. The benefit and motivation of such modification is discussed by SALEH in the following portion: “ ...An unlabeled text document generally refers to a text document for which no ground-truth summary is available. In the absence of such ground-truth summaries, the system employs a self-supervised learning scheme which, e.g., unlike conventional supervised learning that relies on using the ground-truth labels as training objectives, allows for determining training objectives from the unlabeled text documents by the system itself. Specifically, for an unlabeled text document, pre-training with self-supervised learning involves extracting (i.e., masking out) one or more segments from the unlabeled text document and then training the text summarization neural network to predict the extracted segments based on processing the remaining, unmasked portion of the unlabeled text document.” (SALEH [Column 3 line 28-41]).
Regarding claim 13, the rejection of claim 9 is incorporated, furthermore arguments analogous to the rejection of claim 2 are applicable.
Claim(s) 10 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over GARDNER in view of LESTER in view of SALEH in further view of Chau; Vi Dinh et al. (US 12381839 B1) hereinafter CHAU.
Regarding claim 10, the rejection of claim 9 is incorporated, furthermore GARDNER teaches:
The method according to claim 9, wherein the method further comprises: acquiring a second sample corresponding to the scenario type, wherein the second sample comprises second sample text information and second sample summary information;
“Periodically, new samples may be aggregated and used to retrain the prompt engineering model. This allows adapting to individual user styles and preferences. ” (GARDNER [Column 15 line 4-7]).
“For pretraining, it may sample diverse documents from web crawl data, books, academic papers, news etc., as described herein. This develops the general capabilities of the LLMs. ” (GARDNER [Column 62 line 3-6]).
“While some of the descriptions of the system herein focus on text summarization, the system architecture is designed to support multimedia inputs as well, including audio, images, and video. ” (GARDNER [Column 12 line 20-23]).
calculating an accuracy of the language model, based on the second sample summary information and the second prediction summary information;
“In RL, an agent (the prompt engineering model) takes actions (generates prompts) in an environment (the LLM) to maximize a reward (the summarization accuracy). The model receives feedback on how well each prompt achieves the target abstraction level. It learns to improve prompts through trial-and-error. ” (GARDNER [Column 15 line 34-40]).
“...Fidelity measures how accurately the summary captures the key points. ... ” (GARDNER [Column 9 line 41-42]).
“Fidelity—semantic similarity metrics like ROUGE, BLEU, and METEOR compare reference summaries to system outputs by n-gram overlap. Higher scores indicate better alignment with human references. ” (GARDNER [Column 10 line 34-37]).
GARDNER in view of LESTER does not teach, but SALEH teach:
inputting the second sample text information into the language model to obtain second prediction summary information;
“In brief, for each second text document, the system processes data representing the second text document using the text summarization neural network and in accordance with current values of the plurality of network parameters to generate a network output that specifies a predicted summary of the second text document. The system then determines, based on a difference between the predicted summary and the known summary, an update to the current values of the plurality of network parameters and then then applies the update to the current network parameter values. In this way, the parameter values learned during pre-training process are adjusted so that they are adapted to the specific text summarization task. ” (SALEH [Column 5 line 58 – Column 6 line 3]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER the capability to generate a second prediction summary information and utilize it to compute the accuracy (or fidelity) of the response to adjust the parameters of the pre-trained model. The benefit and motivation of such modification is discussed by SALEH in the following portion: “...An unlabeled text document generally refers to a text document for which no ground-truth summary is available. In the absence of such ground-truth summaries, the system employs a self-supervised learning scheme which, e.g., unlike conventional supervised learning that relies on using the ground-truth labels as training objectives, allows for determining training objectives from the unlabeled text documents by the system itself. Specifically, for an unlabeled text document, pre-training with self-supervised learning involves extracting (i.e., masking out) one or more segments from the unlabeled text document and then training the text summarization neural network to predict the extracted segments based on processing the remaining, unmasked portion of the unlabeled text document.” (SALEH [Column 3 line 28-41]).
GARDNER in view of LESTER in further SALEH do not teach, but CHAU teach:
and determining, in response to the accuracy of the language model being not less than a preset accuracy threshold, that the language model passes a test.
“To obtain the training output chat summaries, the model training system 902 can employ a master machine learning model 910 that is larger and more complex than the chat summary model 904 and has a higher accuracy (such as an accuracy higher than a threshold value). The model training system 902 can provide the training input chat messages to the master machine learning model 910 to generate a chat summary as the training output summary. In this way, the training data samples 908 can be generated faster than human providing the chat summaries. Further, it is also faster for the chat summary model 904 to learn from the master machine learning model 910 than from the training output provided by human. This also allows the chat summary model 904, which is much smaller and simpler, to learn from the more accurate master machine learning model. The training determines the parameters of the chat summary model 904 by minimizing a loss function defined using the training data samples 908.” (CHAU [Column 24 line 1-18]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER in further view of SALEH the capability to compare the accuracy of the language model output against a predicted second summary information in order to determine if the result meets a threshold of accuracy. The benefit and motivation of such modification is discussed by CHAU in the following portion: “In some examples, the chat summary model is trained based on a pre-trained model using training data samples. Each of the training data samples can include a group of input chat messages and a corresponding chat summary. The chat summary can be generated by applying a master machine learning model that is more complex, resource and time consuming, and accurate than the chat summary model to the group of input chat messages. The parameters of the chat summary model obtained through training are further quantized to reduce the number of bits used to represent each parameter and speed up the generation process. The chat summary model with the quantized parameters is used to generate chat summaries for chat messages.” (CHAU [Column 3 line 15-27]).
Regarding claim 11, the rejection of claim 10 is incorporated, furthermore GARDNER teaches:
The method according to claim 10, wherein the method further comprises: calculating a second loss, in response to the accuracy of the language model being less than the preset accuracy threshold, based on the second sample summary information and the second prediction summary information;
“During training, the model may input each text snippet sequentially and may try to predict the next word using its current internal representations. The weights may be updated through backpropagation to minimize the prediction loss across the entire corpus. ” (GARDNER [Column 6 line 21-26]).
and adjusting parameters of the language model based on the second loss.
“During fine-tuning, the model may be trained to generate summaries similar to the examples using maximum likelihood estimation and backpropagation.” (GARDNER [Column 10 line 23-28]).
“During training, the model may input each text snippet sequentially and may try to predict the next word using its current internal representations. The weights may be updated through backpropagation to minimize the prediction loss across the entire corpus.” (GARDNER [Column 6 line 21-26]).
Claims 7 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over GARDNER in view of LESTER in view of SALEH in view of GU in further view of Rogers; Sherman T. et al. (US 20130085785 A1) hereinafter ROGERS.
Regarding claim 7, the rejection of claim 1 is incorporated, furthermore GARDNER teaches:
The method according to claim 1, wherein the scenario entry comprises a scenario recording entry,
“Investigative journalists can use the system to summarize lengthy raw transcripts and audio recordings from interviews with sources into condensed overviews. ” (GARDNER [Column 57 line 39-42]).
for uploading the audio information recorded in real time by the client,
“With real-time input and summarization, the system can output a condensed overview of the live event as it unfolds to keep analysts instantly informed. ” (GARDNER [Column 57 line 24-26]).
“The external systems can connect to the summarization API to submit documents, specify zoom levels, and receive generated summaries on demand for integration into downstream processes and applications. Real-time access enables them to leverage the summarization capabilities to extract key information, insights, and context to inform automated decisions and actions. ” (GARDNER [Column 20 line 22-28]).
“While some of the descriptions of the system herein focus on text summarization, the system architecture is designed to support multimedia inputs as well, including audio, images, and video. ” (GARDNER [Column 12 line 20-24]).
GARDNER in view of LESTER in further view of SALEH does not teach, but ROGERS teach:
and the audio information is deleted from the client upon completion of the uploading.
“... Additionally or alternatively, client and meeting information can be uploaded to a third-party storage device, for example, one associated with the compliant entity 145. The uploading process can also remove at least the audio recordings and sensitive portions of the data sets from the device 111. ...” (ROGERS [0133]).
It would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to include in the teachings of GARDNER in view of LESTER in further view of SALEH the capability to delete the audio information once it is uploaded as discussed by ROGERS. One benefit and motivation of such modification is discussed by ROGERS in the following portion: “The service layer application 109 provides interaction between the mobile application 113, the management application 131 and associated management application 131, the database 105, and the storage medium 107. The service layer application 109 provides security as well as the communication between the components of the system. Security includes authentication and limitations that protect personal health information (PHI) and other sensitive data in the system. For example, the service layer application 109 can limit the amount of PHI that can be stored on a portable wireless device 111 before uploading transmits the data sets and recordings to the database 105 and/or the storage medium 107. ...” (ROGERS [0133]).
Regarding claim 20, the rejection of claim 14 is incorporated, furthermore arguments analogous to the rejection of claim 7 are applicable.
Summary of references used as prior art
GARDNER teaches the core LLM-based controllable summarization framework, including prompting content to obtain summaries and training prompt-engineering techniques using content/prompt/summary data, as well as listing scenarios like the claimed ones.
LESTER teaches a task-specific prompt tunning of a pre-trained language model, as well as freezing parameters of the model so the model remains the same during the prompt tunning phase.
SALEH teaches training/ fine-tuning a text summarization model using text and corresponding known summaries, including generated a predicted summary and comparing it to a known summary to update model parameters based on the difference.
GU teaches a scenario-based multimedia interface, as well as multiple selectable scene identifiers for a particular multimedia item as well as storage/upload and later retrieval.
CHAU teaches determining the success of a summarization in a test style.
Rogers teaches deleting the information once that media is uploaded.
Prior art not relied in this rejection
Kulshreshtha; Devang et al. (US 20250005063 A1) AUTOMATED EVALUATION OF EVIDENCE MAPPING MODELS: is considered pertinent because it teaches LLM processing and summarization of conversation transcripts, including fine-tuning a pre-trained LLM for summarization using instructions prompt.
PRYZANT; Reid Allen et al. (US 20250111147 A1) AUTOMATIC LANGUAGE MODEL (LM) INPUT OPTIMIZATION USING TEXTUAL GRADIENTS: is considered pertinent because it discloses automatic optimization/ tunning of prompts supplied to an LLM using gradients.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HECTOR J. CRESPO FEBLES whose telephone number is (571)272-4512. The examiner can normally be reached Mon - Fri 7:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HECTOR J. CRESPO FEBLES/ Examiner, Art Unit 2657
/DANIEL C WASHBURN/ Supervisory Patent Examiner, Art Unit 2657
