DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
2. The action is responsive to the Applicant’s Amendment filed on 2/06/2026.
Claims 1-6, 8-15, and 17-21 are pending in the application.
Response to Arguments
3. Applicant’s arguments with respect to the rejections previously made and the amended claims filed on 2/06/2026 have been fully considered but they are not persuasive. In view of the claim amendments, the rejections are being updated accordingly.
In regards to independent claim 1, Applicant argued “none of the cited references, along or in combination, teach or suggest the feature of generating titles at the document level first, and then re-using those generated titles as semantic inputs to select a cluster-level title as recited in claim 1”.
In response to the arguments, it is submitted the feature that the Applicant is arguing is not recited in claim 1. The claim language includes the steps of accessing a plurality of documents, providing the plurality of documents as an input to a trained title generating machine-learning (ML) model, generating a plurality of titles, creating a title embedding, creating a document cluster embedding, generating an average embedding, measuring a similarity, ranking each of the plurality of titles, selecting, based on the ranking, a top one or top few of the plurality of titles, providing the title candidates as an output. The limitation of “re-using those generated titles as semantic inputs to select a cluster-level title” is not recited in claim 1.
However, Cao uses titles for much more than the “document labeling” that the Applicant argues. Cao teaches generating titles at the document level first, and then re-using those generated titles as semantic inputs to select a cluster-level title in paragraph [0003] by stating, “The method may further include, responsive to determining that the document has a title, ranking the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title…. responsive to determining that the document has a title, rank the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title.” See also Fig. 1, and paragraphs [0023]-[0025], which explicitly teach generating a document title and using the title to select a cluster-level title, stating, “The title may be compared with every cluster c.sub.i, and the top-k (e.g., k=1, 2) most similar clusters may be selected as the representative clusters in terms of semantic.” In addition see paragraph [0008], “FIG. 2 is a diagram illustrating cluster and classification utilizing the generated document embedding in one embodiment of the present disclosure”. See also Fig. 6 for more details.
In addition, Agley teaches document-generated titles by teaching, “[0072] FIG. 8 shows a user interface 700 displaying concepts within a content item… text portions are shown as paragraphs, text portions, in various examples, may be analyzed at the sentence level, page level, or in other groupings”. Other groupings include at a document level. Fig. 11 and paragraphs [0082]-[0083] explicitly teach that an uploaded document is associated with a concept, stating, “For example, concept association 135 may identify a listing of candidate keywords and keyword filtering 140 may select the most relevant keywords from those candidate keywords to represent a concept.” A concept is a label, which also corresponds to a title, according to the instant specification, which states in paragraph [0017], “sets of labeled (e.g., titled) documents (e.g., short texts)”. Therefore, Agley teaches a title for the uploaded document.
Thus, for at least the reasons as set forth above, it is submitted that Cao and Agley teach the feature of generating titles at the document level first, and then re-using those generated titles as semantic inputs to select a cluster-level title.
In regards to independent claims 8 and 17, the emphasized limitations that the Applicant argues in claims 8 and 17 are similar to the emphasized limitations of claim 1, which have been addressed above. See the response of claim 1 above for explanation.
Furthermore, it is also submitted that all limitations in pending claims, including those not specifically argued, are properly addressed. The reason is set forth in the rejections. See claim analysis below for detail.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-6, 8-15, and 17-21 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US 20200026767 A1) in view of Cao et al. (US 20180032897 A1), McInerney (US 20230070497 A1), Gilad et al. (US 20220366302 A1), and Agley et al. (US 20240086452 A1).
7. Regarding Claim 1, Chen discloses a data processing system comprising:
a processor; and a memory in communication with the processor, the memory comprising executable instructions (Fig. 8; [0064]: Computing device 805 in computing environment 800 can include one or more processing units, cores, or processors 810, memory 815) that, when executed by the processor, cause the data processing system to perform functions of:
accessing a plurality of documents in a document cluster ([0023]: As illustrated in FIG. 1, a plurality of documents are generated, stored, or received by the system at 105), the plurality of documents being documents that have been categorized as belonging to the document cluster ([Abstract]: The method includes receiving a plurality of documents, each document having associated content features);
providing the plurality of documents as an input to a trained title generating machine- learning (ML) model, the trained title generating ML model being trained for generating a title for a document (Fig. 1; [0024]: At 110, a title generation computer model is applied to each of the documents to generate a title or other short summary; Fig. 5; [0043]: As illustrated, the neural network model 500 is an encoder-decoder RNN model with domain adaptation. Labeled source data (articles 515) is fed to the encoder 505 and the decoder 510 learns to generate summary titles (summary 520));
generating a plurality of titles via the trained title generating ML model, each of the plurality of titles being a title for one of the plurality of documents (Fig. 1; titled documents 115; [0025]: After titles or short summaries have been generated for each of the documents, the documents and titles are provided to a User Interface Controller at 120);
However, Chen does not explicitly teach “creating a title embedding for each of the plurality of titles; creating a document cluster embedding for the document cluster; measuring a similarity between each of the title embeddings and the document cluster embedding to identify titles that are more similar to the embedding for the document cluster; ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the document cluster; selecting, based on the ranking, a top one or top few of the plurality of titles from among the generated plurality of titles as title candidates for the document cluster; and providing the title candidates as an output for user review and selection.”
On the other hand, in the same field of endeavor, Cao teaches
creating a title embedding for each of the plurality of titles ([0003]: ranking… word embeddings of words in the title… the determining of the document embedding are performed for multiple documents. The method may further include labeling each of the multiple documents; [0025]: The title is also represented by the sum embeddings of its words);
Additionally, McInerney teaches
creating a document cluster embedding for the document cluster comprising the plurality of documents ([Abstract]: A system receives a plurality of documents and generates embeddings of the sentences from the plurality of documents; Fig. 3; [0021]: The clustering module 331 may be configured to use an encoder to generate an embedding of the sentences from the source documents, and then cluster those embedded sentences according to their relative distance in the representation space… The generation module is configured to generate a summary of the documents based on the clustered and masked sentences; Fig. 2; [0023]-[0025]: Embeddings of the sentences from the source Documents 210, 220, and 230 may be generated, and then those embeddings may be clustered together based on their relative distance in the representation space… This is shown as Clusters 240, 250, and 260 which show that the similar sentences of each of the source documents 210, 220, and 230 are clustered together; [0028]- [0029]: At step 310, embeddings of sentences from the plurality of documents are generated. At step 315, The sentences from the plurality of documents are clustered, based on the embeddings, into a plurality of clusters);
measuring a similarity between each of the title embeddings and the document cluster embedding to identify titles that are more relevant to the document cluster based on the similarity between each title embedding and the embedding for the document cluster (Fig. 2; [0024]-[0025]: Sentences from the reference summary 270 may also be encoded into embeddings and then those embeddings are compared with embeddings of the clusters 240 250 and 260… Aligning may be performed, for example, by choosing the closest cluster to each reference sentence, using Euclidean distance between the mean sentence embedding of the cluster and the sentence embedding of the reference sentence);
ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the embedding for the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster. Other methods of selecting a subset of sentences from a cluster include “Oracle” ranking);
selecting, based on the ranking, a top one or top few of the plurality of titles from among the generated plurality of titles as title candidates for the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster; Fig. 4; [0032]-[0035]: …the final generated summary 440… For example, the sentence which is at the centroid of a cluster may have a line drawn from that sentence in the display to the sentence(s) associated with that cluster in the summary 440).
Also, Gilad teaches generating an average embedding by generating individual embeddings for portions of the plurality of documents in the cluster and generating an average embedding for the cluster based on the individual embeddings, wherein the topic embedding comprises the average embedding generated for the cluster of documents as a whole, thereby enabling generation of accurate titles based on both content of each of the plurality of documents and an overall topic of the document cluster (Fig. 6; [0124]-[0125]: In step 604, the method 600 can comprise retrieving an initial set of embeddings using the documents obtained using the internal read version… In step 606, the method 600 can comprise combining the retrieved embeddings. In various embodiments, the method 600 can combine the embeddings in any suitable manners, which include, for example, by performing an averaging operation on the set of embeddings via various techniques and algorithms);
Furthermore, Agley teaches providing the title candidates as an output for user review and selection (Fig. 1; [0052]-[0056]: title generation 138 may involve a summarization of cluster text produced using Natural Language Processing models including, for example, deep learning models… In some examples, keyword filtering 140 may utilize a similarity score between cluster text and each topic keyword to rank the topic keywords for each cluster, retaining a number of the most relevant (e.g., highest similarity score) topic keywords… Dominant topics may be then provided to result generation 144… for use by the content management system 102 (e.g., for creation of user interfaces, updating the knowledge base 125, and/or performing other tasks)),
wherein: the plurality of titles generated for the plurality of documents are each generated independently based on content of a corresponding individual document (Figs, 4-8; [0068]-[0069]: Turning to FIG. 4, the user interface 300 displays dominant concepts or topics per page of a document… For example, the user interface 400 includes a chart showing the dominant concept (labeled, e.g., using relevant keywords produced by keyword filtering 140)… FIG. 6 depicts a user interface 500 including graphics showing the distribution of various concepts within a content item… the user interface 500 shows at what pages of a document various concepts appear… [0072] FIG. 8 shows a user interface 700 displaying concepts within a content item… text portions are shown as paragraphs, text portions, in various examples, may be analyzed at the sentence level, page level, or in other groupings [Figs 5- 8 show a plurality of titles generated for the plurality of documents]), and
the document cluster embedding is generated without using the generated titles (Fig. 1; [0050]-[0053]: In the transformer-based approach, concept association 135 may pre-process the text to remove punctuation and convert each paragraph to an embedding, which may be a high-dimensional vector encoding the meaning of the paragraph. The transformer 136 may be used to create the embeddings… title generation 138 may use the transformer 136, or another transformer; [This is nonfunctional descriptive material describing the titles and embedding. None of the claimed steps are depending on any of the information being described. All steps in the claims would be performed the same to achieve a same outcome regardless of this descriptive information]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Chen to incorporate the teachings of Cao, McInerney, and Agley to create a title embedding for each of the plurality of titles, create a document cluster embedding for the document cluster, rank the titles, and select one or more titles as title candidates for the document cluster to provide as an output.
The motivation for doing so would be to rank clusters based on cosine similarity of word embeddings of words in the cluster and word embeddings of words in the title, as recognized by Cao ([0004]: The hardware processor may be further operable to, responsive to determining that the document has a title, rank the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title), summarize multiple documents, as recognized by McInerney ([Abstract]: Embodiments described herein provide methods and systems for summarizing multiple documents), combine the embedding to generate accurate titles, as recognized by Gilad ([0002]: Certain types of ML models can consist of many vector embeddings that describe the semantic similarity of entities modeled by their proximity in vector spaces), and to identify optimal topic clusters, as recognized by Agley ([0051] of Agley: The embeddings representing the paragraphs may be placed in a high-dimensional semantic space. To identify optimal topic clusters, concept association 135 may use k-means (other similar HBDscan) clustering to cluster the embeddings in the high-dimensional semantic space. An optimal number of clusters may be determined by maximizing the silhouette score for the embeddings in the clusters).
Regarding Claim 2, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 1.
Chen further teaches wherein the trained title generating ML model is a trained encoder-decoder language model that generates abstractive titles for a document in the document cluster (Fig. 2; [0031]: At 225, the set of “synthetic” or preliminary titles for the unlabeled target domain is first used to train a neural network to develop a model using the combined expanded vocabulary from 215. In some example implementations, a sequence-to-sequence encoder-decoder model may be used to generate a title).
Regarding Claim 3, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 1.
Cao further teaches wherein creating a title embedding for each of the plurality of titles includes generating numerical vector representations of text for each of the plurality of titles ([0015]: The embedding representation of the present disclosure in one embodiment is a low dimensional and a real-valued vector. An example includes x=[0.1, 0.08, . . . , −0.23, . . . ]).
Regarding Claim 4, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 1.
Agley further teaches wherein creating a document cluster embedding for the document cluster includes creating an averaged embedding for the document cluster (Fig. 2; [0052]: The embedding of the cluster text may, in some examples, be generated by averaging the embeddings of the paragraphs in the cluster).
Regarding Claim 5, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 4.
Agley further teaches wherein creating an averaged embedding includes:
utilizing a model trained for generating topic embeddings from text inputs to generate one or more embeddings for the plurality of documents in the document cluster (Fig. 2; [0050]: The transformer 136 may be used to create the embeddings and may be any type of transformer model, such as the MiniLM-L6-v2, or other similar transformer); and
calculating an average of the generated topic embeddings to generate the averaged embedding for the document cluster ([0052]: The embedding of the cluster text may, in some examples, be generated by averaging the embeddings of the paragraphs in the cluster).
Regarding Claim 6, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 1.
Cao further teaches wherein measuring the similarity between each of the title embeddings and the document cluster embedding includes calculating a similarity score between each of the title embeddings and the document cluster embedding ([0003]: The method may further include, responsive to determining that the document has a title, ranking the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title).
Regarding Claim 8, Chen discloses a method for automatically generating a title for a cluster of documents ([Abstract]: A method and system of generating titles for documents in a storage platform are provided) comprising:
accessing a plurality of documents in the document cluster ([0023]: As illustrated in FIG. 1, a plurality of documents are generated, stored, or received by the system at 105), the plurality of documents being documents that have been categorized as belonging to the document cluster ([Abstract]: The method includes receiving a plurality of documents, each document having associated content features);
providing the plurality of documents as an input to a trained title generating machine- learning (ML) model, the trained title generating ML model being trained for generating a title for a document (Fig. 1; [0024]: At 110, a title generation computer model is applied to each of the documents to generate a title or other short summary; Fig. 5; [0043]: As illustrated, the neural network model 500 is an encoder-decoder RNN model with domain adaptation. Labeled source data (articles 515) is fed to the encoder 505 and the decoder 510 learns to generate summary titles (summary 520));
generating a title for each of the plurality of documents via the trained title generating ML model (Fig. 1; titled documents 115; [0025]: After titles or short summaries have been generated for each of the documents, the documents and titles are provided to a User Interface Controller at 120);
However, Chen does not explicitly teach “creating a title embedding for each of the plurality of titles; creating a document cluster embedding for the document cluster; measuring a similarity between each of the title embeddings and the document cluster embedding to identify titles that are more similar to the embedding for the document cluster; ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the document cluster; selecting, based on the ranking, a top one or top few of the plurality of titles from among the generated plurality of titles as title candidates for the document cluster; and providing the title candidates as an output for user review and selection.”
On the other hand, in the same field of endeavor, Cao teaches
creating a title embedding for each of the plurality of titles ([0003]: ranking… word embeddings of words in the title… the determining of the document embedding are performed for multiple documents. The method may further include labeling each of the multiple documents; [0025]: The title is also represented by the sum embeddings of its words);
Additionally, McInerney teaches
creating a topic embedding for the document cluster based on the plurality of documents of the document cluster ([Abstract]: A system receives a plurality of documents and generates embeddings of the sentences from the plurality of documents; Fig. 3; [0021]: The clustering module 331 may be configured to use an encoder to generate an embedding of the sentences from the source documents, and then cluster those embedded sentences according to their relative distance in the representation space… The generation module is configured to generate a summary of the documents based on the clustered and masked sentences; Fig. 2; [0023]-[0025]: Embeddings of the sentences from the source Documents 210, 220, and 230 may be generated, and then those embeddings may be clustered together based on their relative distance in the representation space… This is shown as Clusters 240, 250, and 260 which show that the similar sentences of each of the source documents 210, 220, and 230 are clustered together; [0028]- [0029]: At step 310, embeddings of sentences from the plurality of documents are generated. At step 315, The sentences from the plurality of documents are clustered, based on the embeddings, into a plurality of clusters);
measuring a similarity between each of the title embeddings and the document cluster embedding to identify titles that are more similar to the embedding for the generated titles and the topic embedding for the document cluster to identify generated titles that are more relevant to the document cluster based on the similarity between each title embedding and the topic embedding for the document cluster (Fig. 2; [0024]-[0025]: Sentences from the reference summary 270 may also be encoded into embeddings and then those embeddings are compared with embeddings of the clusters 240 250 and 260… Aligning may be performed, for example, by choosing the closest cluster to each reference sentence, using Euclidean distance between the mean sentence embedding of the cluster and the sentence embedding of the reference sentence);
ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster. Other methods of selecting a subset of sentences from a cluster include “Oracle” ranking);
selecting, based on the ranking, a top one or top few of the generated plurality of titles as title candidates for the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster; Fig. 4; [0032]-[0035]: …the final generated summary 440… For example, the sentence which is at the centroid of a cluster may have a line drawn from that sentence in the display to the sentence(s) associated with that cluster in the summary 440).
Also, Gilad teaches generating an average embedding by generating individual embeddings for portions of the plurality of documents in the cluster and generating an average embedding for the cluster based on the individual embeddings, wherein the topic embedding comprises the average embedding generated for the cluster of documents as a whole, thereby enabling generation of accurate titles based on both content of each of the plurality of documents and an overall topic of the document cluster (Fig. 6; [0124]-[0125]: In step 604, the method 600 can comprise retrieving an initial set of embeddings using the documents obtained using the internal read version… In step 606, the method 600 can comprise combining the retrieved embeddings. In various embodiments, the method 600 can combine the embeddings in any suitable manners, which include, for example, by performing an averaging operation on the set of embeddings via various techniques and algorithms);
Furthermore, Agley teaches providing the title candidates as an output for user review and selection (Fig. 1; [0052]-[0056]: title generation 138 may involve a summarization of cluster text produced using Natural Language Processing models including, for example, deep learning models… In some examples, keyword filtering 140 may utilize a similarity score between cluster text and each topic keyword to rank the topic keywords for each cluster, retaining a number of the most relevant (e.g., highest similarity score) topic keywords… Dominant topics may be then provided to result generation 144… for use by the content management system 102 (e.g., for creation of user interfaces, updating the knowledge base 125, and/or performing other tasks)).
wherein: the plurality of titles generated for the plurality of documents are each generated independently based on content of a corresponding individual document (Figs, 4-8; [0068]-[0069]: Turning to FIG. 4, the user interface 300 displays dominant concepts or topics per page of a document… For example, the user interface 400 includes a chart showing the dominant concept (labeled, e.g., using relevant keywords produced by keyword filtering 140)… FIG. 6 depicts a user interface 500 including graphics showing the distribution of various concepts within a content item… the user interface 500 shows at what pages of a document various concepts appear… [0072] FIG. 8 shows a user interface 700 displaying concepts within a content item… text portions are shown as paragraphs, text portions, in various examples, may be analyzed at the sentence level, page level, or in other groupings [Figs 5- 8 show a plurality of titles generated for the plurality of documents]), and the document cluster embedding is generated without using the generated titles (Fig. 1; [0050]-[0053]: In the transformer-based approach, concept association 135 may pre-process the text to remove punctuation and convert each paragraph to an embedding, which may be a high-dimensional vector encoding the meaning of the paragraph. The transformer 136 may be used to create the embeddings… title generation 138 may use the transformer 136, or another transformer; [This is nonfunctional descriptive material describing the titles and embedding. None of the claimed steps are depending on any of the information being described. All steps in the claims would be performed the same to achieve a same outcome regardless of this descriptive information]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Chen to incorporate the teachings of Cao, McInerney, and Agley to create a title embedding for each of the plurality of titles, create a document cluster embedding for the document cluster, rank the titles, and select one or more titles as title candidates for the document cluster to provide as an output.
The motivation for doing so would be to rank clusters based on cosine similarity of word embeddings of words in the cluster and word embeddings of words in the title, as recognized by Cao ([0004]: The hardware processor may be further operable to, responsive to determining that the document has a title, rank the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title), summarize multiple documents, as recognized by McInerney ([Abstract]: Embodiments described herein provide methods and systems for summarizing multiple documents), combine the embedding to generate accurate titles, as recognized by Gilad ([0002]: Certain types of ML models can consist of many vector embeddings that describe the semantic similarity of entities modeled by their proximity in vector spaces), and to identify optimal topic clusters, as recognized by Agley ([0051] of Agley: The embeddings representing the paragraphs may be placed in a high-dimensional semantic space. To identify optimal topic clusters, concept association 135 may use k-means (other similar HBDscan) clustering to cluster the embeddings in the high-dimensional semantic space. An optimal number of clusters may be determined by maximizing the silhouette score for the embeddings in the clusters).
Regarding Claim 9, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 8.
Chen further teaches wherein the trained title generating ML model is a trained text to text language model that receives each of the plurality of documents as the input and generates the title for each of the plurality of documents as an output ([0005]: The method includes receiving a plurality of documents… applying a title generation computer model to each of the plurality of documents to generate a title based on the associated content features… wherein the title generation computer model is created by training a neural network using a combination of: a first set of unlabeled data from a first domain related to content features of the plurality of documents; and a second set of pre-labeled data from a second domain different from the first domain; Fig. 2; the model is re-trained at 235 on the source domain, which has title-text pairs).
Regarding Claim 10, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 8.
Chen further teaches wherein the trained title generating ML model is trained by using a publicly available labeled dataset to fine-tune a pretrained ML model (Fig. 2, labeled data 210; [0005]: The method includes… applying a title generation computer model to each of the plurality of documents to generate a title based on the associated content features… wherein the title generation computer model is created by training a neural network using… a second set of pre-labeled data from a second domain different from the first domain; [0028]: training data set 210 may be labeled articles or stores posted to a news platform providing general interest stories (general interest domain)).
Regarding Claim 11, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 10.
Chen further teaches wherein the pretrained ML model is an encoder-decoder deep learning model (Fig. 2; [0031]: In some example implementations, a sequence-to-sequence encoder-decoder model may be used to generate a title).
Regarding Claim 12, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 8.
Cao further teaches wherein creating the title embedding for each of the received titles includes generating numerical vector representations of text for each of the received titles ([0015]: The embedding representation of the present disclosure in one embodiment is a low dimensional and a real-valued vector. An example includes x=[0.1, 0.08, . . . , −0.23, . . . ]).
Regarding Claim 13, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 8.
Agley further teaches wherein creating the topic embedding for the document cluster includes creating an averaged embedding for the document cluster (Fig. 2; [0052]: The embedding of the cluster text may, in some examples, be generated by averaging the embeddings of the paragraphs in the cluster).
Regarding Claim 14, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 13.
Agley further teaches, further comprising: utilizing a model trained for generating topic embeddings from text inputs to generate one or more embeddings for the plurality of documents in the document cluster (Fig. 2; [0053:] In various examples, title generation 138 may involve a summarization of cluster text produced using Natural Language Processing models including, for example, deep learning models such as transformers. For example, title generation 138 may use the transformer 136, or another transformer); and
calculating an average of the generated topic embeddings to generate the topic embedding for the document cluster (Fig. 2; [0052]: The embedding of the cluster text may, in some examples, be generated by averaging the embeddings of the paragraphs in the cluster).
Regarding Claim 15, the combined teachings of Chen, Cao, McInerney, and Agley disclose the method of claim 8.
Cao further teaches wherein measuring the similarity between the title embeddings for the generated titles and the topic embedding for the document cluster includes calculating a similarity score between each of the title embeddings for the generated titles and the topic embedding for the document cluster embedding ([0003]: The method may further include, responsive to determining that the document has a title, ranking the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title).
Regarding Claim 17, Chen discloses a non-transitory computer readable medium on which are stored instructions that, when executed, cause a programmable device to perform functions of (Fig. 8; [0070]: Computing device 805 can use and/or communicate using computer-usable or computer-readable media, including transitory media and non-transitory media):
accessing a plurality of documents in a document cluster ([0023]: As illustrated in FIG. 1, a plurality of documents are generated, stored, or received by the system at 105), the plurality of documents being documents that have been categorized as belonging to a document cluster ([Abstract]: The method includes receiving a plurality of documents, each document having associated content features);
providing the plurality of documents as an input to a trained title generating machine- learning (ML) model, the trained title generating ML model being trained for generating a title for a document (Fig. 1; [0024]: At 110, a title generation computer model is applied to each of the documents to generate a title or other short summary; Fig. 5; [0043]: As illustrated, the neural network model 500 is an encoder-decoder RNN model with domain adaptation. Labeled source data (articles 515) is fed to the encoder 505 and the decoder 510 learns to generate summary titles (summary 520));
generating a title via the trained title generating ML model for each of the plurality of documents (Fig. 1; titled documents 115; [0025]: After titles or short summaries have been generated for each of the documents, the documents and titles are provided to a User Interface Controller at 120);
However, Chen does not explicitly teach “creating a title embedding for each of the plurality of titles; creating a document cluster embedding for the document cluster; measuring a similarity between each of the title embeddings and the document cluster embedding to identify titles that are more similar to the embedding for the document cluster; ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the document cluster; selecting, based on the ranking, a top one or top few of the plurality of titles from among the generated plurality of titles as title candidates for the document cluster; and providing the title candidates as an output for user review and selection.”
On the other hand, in the same field of endeavor, Cao teaches
creating a title embedding for each of the generated titles ([0003]: ranking… word embeddings of words in the title… the determining of the document embedding are performed for multiple documents. The method may further include labeling each of the multiple documents; [0025]: The title is also represented by the sum embeddings of its words);
Additionally, McInerney teaches
creating a topic embedding for the document cluster, the topic embedding representing the plurality of documents of the document cluster ([Abstract]: A system receives a plurality of documents and generates embeddings of the sentences from the plurality of documents; Fig. 3; [0021]: The clustering module 331 may be configured to use an encoder to generate an embedding of the sentences from the source documents, and then cluster those embedded sentences according to their relative distance in the representation space… The generation module is configured to generate a summary of the documents based on the clustered and masked sentences; Fig. 2; [0023]-[0025]: Embeddings of the sentences from the source Documents 210, 220, and 230 may be generated, and then those embeddings may be clustered together based on their relative distance in the representation space… This is shown as Clusters 240, 250, and 260 which show that the similar sentences of each of the source documents 210, 220, and 230 are clustered together; [0028]- [0029]: At step 310, embeddings of sentences from the plurality of documents are generated. At step 315, The sentences from the plurality of documents are clustered, based on the embeddings, into a plurality of clusters);
measuring a similarity between each of the title embeddings for the generated titles and the topic embedding for the document cluster to identify generated titles that are more relevant to the document cluster based on the similarity between each title embedding and the topic embedding for the document cluster (Fig. 2; [0024]-[0025]: Sentences from the reference summary 270 may also be encoded into embeddings and then those embeddings are compared with embeddings of the clusters 240 250 and 260… Aligning may be performed, for example, by choosing the closest cluster to each reference sentence, using Euclidean distance between the mean sentence embedding of the cluster and the sentence embedding of the reference sentence);
ranking each of the plurality of titles based on the measured similarity between each of the title embeddings and the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster. Other methods of selecting a subset of sentences from a cluster include “Oracle” ranking);
selecting, based on the ranking, a top one or top few of the generated plurality of titles as title candidates for the document cluster ([0032]: When generating cluster-wise summaries, the model may only use a subset of the sentences in a cluster, for example the 10 sentences closest to the mean of the cluster; Fig. 4; [0032]-[0035]: …the final generated summary 440… For example, the sentence which is at the centroid of a cluster may have a line drawn from that sentence in the display to the sentence(s) associated with that cluster in the summary 440).
Also, Gilad teaches generating an average embedding by generating individual embeddings for portions of the plurality of documents in the cluster and generating an average embedding for the cluster based on the individual embeddings, wherein the topic embedding comprises the average embedding generated for the cluster of documents as a whole, thereby enabling generation of accurate titles based on both content of each of the plurality of documents and an overall topic of the document cluster (Fig. 6; [0124]-[0125]: In step 604, the method 600 can comprise retrieving an initial set of embeddings using the documents obtained using the internal read version… In step 606, the method 600 can comprise combining the retrieved embeddings. In various embodiments, the method 600 can combine the embeddings in any suitable manners, which include, for example, by performing an averaging operation on the set of embeddings via various techniques and algorithms);
Furthermore, Agley teaches providing the title candidates as an output for user review and selection (Fig. 1; [0052]-[0056]: title generation 138 may involve a summarization of cluster text produced using Natural Language Processing models including, for example, deep learning models… In some examples, keyword filtering 140 may utilize a similarity score between cluster text and each topic keyword to rank the topic keywords for each cluster, retaining a number of the most relevant (e.g., highest similarity score) topic keywords… Dominant topics may be then provided to result generation 144… for use by the content management system 102 (e.g., for creation of user interfaces, updating the knowledge base 125, and/or performing other tasks)).
wherein: the plurality of titles generated for the plurality of documents are each generated independently based on content of a corresponding individual document (Figs, 4-8; [0068]-[0069]: Turning to FIG. 4, the user interface 300 displays dominant concepts or topics per page of a document… For example, the user interface 400 includes a chart showing the dominant concept (labeled, e.g., using relevant keywords produced by keyword filtering 140)… FIG. 6 depicts a user interface 500 including graphics showing the distribution of various concepts within a content item… the user interface 500 shows at what pages of a document various concepts appear… [0072] FIG. 8 shows a user interface 700 displaying concepts within a content item… text portions are shown as paragraphs, text portions, in various examples, may be analyzed at the sentence level, page level, or in other groupings [Figs 5- 8 show a plurality of titles generated for the plurality of documents]), and the document cluster embedding is generated without using the generated titles (Fig. 1; [0050]-[0053]: In the transformer-based approach, concept association 135 may pre-process the text to remove punctuation and convert each paragraph to an embedding, which may be a high-dimensional vector encoding the meaning of the paragraph. The transformer 136 may be used to create the embeddings… title generation 138 may use the transformer 136, or another transformer; [This is nonfunctional descriptive material describing the titles and embedding. None of the claimed steps are depending on any of the information being described. All steps in the claims would be performed the same to achieve a same outcome regardless of this descriptive information]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Chen to incorporate the teachings of Cao, McInerney, and Agley to create a title embedding for each of the plurality of titles, create a document cluster embedding for the document cluster, rank the titles, and select one or more titles as title candidates for the document cluster to provide as an output.
The motivation for doing so would be to rank clusters based on cosine similarity of word embeddings of words in the cluster and word embeddings of words in the title, as recognized by Cao ([0004]: The hardware processor may be further operable to, responsive to determining that the document has a title, rank the clusters based on cosine similarity of word embeddings of words in a cluster and word embeddings of words in the title), summarize multiple documents, as recognized by McInerney ([Abstract]: Embodiments described herein provide methods and systems for summarizing multiple documents), combine the embedding to generate accurate titles, as recognized by Gilad ([0002]: Certain types of ML models can consist of many vector embeddings that describe the semantic similarity of entities modeled by their proximity in vector spaces), and to identify optimal topic clusters, as recognized by Agley ([0051] of Agley: The embeddings representing the paragraphs may be placed in a high-dimensional semantic space. To identify optimal topic clusters, concept association 135 may use k-means (other similar HBDscan) clustering to cluster the embeddings in the high-dimensional semantic space. An optimal number of clusters may be determined by maximizing the silhouette score for the embeddings in the clusters).
Regarding Claim 18, the combined teachings of Chen, Cao, McInerney, and Agley disclose the non-transitory computer readable medium of claim 17.
Chen further teaches wherein the trained title generating ML model is a trained text to text language model that receives the plurality of the documents as the input and generates a title for each of the plurality of documents ([0005]: The method includes receiving a plurality of documents… applying a title generation computer model to each of the plurality of documents to generate a title based on the associated content features… wherein the title generation computer model is created by training a neural network using a combination of: a first set of unlabeled data from a first domain related to content features of the plurality of documents; and a second set of pre-labeled data from a second domain different from the first domain; Fig. 2; the model is re-trained at 235 on the source domain, which has title-text pairs).
Regarding Claim 19, the combined teachings of Chen, Cao, McInerney, and Agley disclose the non-transitory computer readable medium of claim 17.
Chen further teaches wherein the trained title generating ML model is trained by using a publicly available labeled dataset to fine-tune a pretrained ML model (Fig. 2, labeled data 210; [0005]: The method includes… applying a title generation computer model to each of the plurality of documents to generate a title based on the associated content features… wherein the title generation computer model is created by training a neural network using… a second set of pre-labeled data from a second domain different from the first domain; [0028]: training data set 210 may be labeled articles or stores posted to a news platform providing general interest stories (general interest domain)).
Regarding Claim 20, the combined teachings of Chen, Cao, McInerney, and Agley disclose the non-transitory computer readable medium of claim 17.
McInerney further teaches wherein the plurality of documents are concatenated in one document that includes the plurality of documents in the document cluster (Fig. 2; [0023]: Embeddings of the sentences from the source Documents 210, 220, and 230 may be generated, and then those embeddings may be clustered together based on their relative distance in the representation space… This is shown as Clusters 240, 250, and 260 which show that the similar sentences of each of the source documents 210, 220, and 230 are clustered together), and
the topic embedding for the document cluster is generated from the concatenation of the plurality of documents ([0024]-[0025]: Sentences from the reference summary 270 may also be encoded into embeddings and then those embeddings are compared with embeddings of the clusters 240 250 and 260… a pretrained summarization model may be used to generate cluster-wise summaries from clusters 240, 250 and 260, respectively).
Regarding Claim 21, the combined teachings of Chen, Cao, McInerney, and Agley disclose the data processing system of claim 1.
McInerney further teaches wherein the documents in the document cluster are instances of written user feedback regarding a product or service (Fig. 1; [0020]: In some examples, the Summarization module 130, may receive an input 140, e.g., such as a collection of documents on a particular topic, via a data interface 115; [0023]: FIG. 2 is a simplified diagram showing an example method for summarizing multiple documents. Documents 210, 220, and 230 are multiple input documents, e.g., having text that covers similar material… portions of each document may describe the same or similar matter).
Conclusion
26. THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIRLEY D. HICKS whose telephone number is (571)272-3304. The examiner can normally be reached Mon - Fri 7:30 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached on (571) 272-4085. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S D H/Examiner, Art Unit 2168
/CHARLES RONES/Supervisory Patent Examiner, Art Unit 2168
Prosecution Insights