DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-11 and 13-21 are pending. Claims 1, 17 and 19 are independent. Claims have been amended.
Apparent priority 12/07/2022.
Applicant’s amendments and arguments are considered but are either unpersuasive or moot in view of the new grounds of rejection that, if presented, were necessitated by the amendments to the Claims.
This action is Non-Final.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/25/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to the rejection of claims 1, 3-13, 17 and 19 under 35 U.S.C. 103 have been considered but are moot because of the new ground of rejection in view of Lyle and Hennum for claims 1, 3-11, 13, 17, 19 and 21.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-11, 13, 17, 19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Lyle et al. (US Patent 12,288,570; hereinafter “Lyle”) in view of Hennum et al (US Patent 7,689,910; hereinafter “Hennum”).
As per claims 1, 17 and 19, Lyle discloses:
A method for generating a topic index, computing device and non-transitory computer-readable storage medium storing one or more programs configured for execution by a computing device having one or more processors and memory, comprising: one or more processors (Lyle; Fig. 2, item 212; Col. 9, lines 1-17 – one or more processors); memory (Lyle; Fig. 2, item 214; Col. 9, lines 1-17 – memory); and one or more programs stored in the memory and configured for execution by the one or more processors (Lyle; Col. 9, lines 1-17 - Instructions stored within memory 214 may be executed by the one or more processors 212), the one or more programs comprising instructions for: obtaining audio content (Lyle; Col. 6, lines 12-42 - original video content is received… It is to be understood that the inference engine also receives audio data and/or transcription data which can be used in parallel with the outputted synthesized video in cases where the video includes speech or otherwise has sounds present in the video stream; see also Col. 15, lines 22-67 & Col. 16 lines 1-67 – speech transcription & NLP analysis); extracting vocabulary terms from the audio content (Lyle; Col. 15, lines 22-67 & Col. 16 lines 1-67 - …the speech recognition component of the NLP may take the digital or analog audio signal from the call and performs speech recognition analysis to recognize one or more words spoken. Speech recognition (also referred to as automatic speech recognition (ASR), computer speech recognition or voice recognition) technology generally represents a set of technologies that allows computers equipped with a source of sound input, such as a microphone, to transform human speech into a sequence of words recorded in a computer data file…); generating, using a transformer model, a vocabulary embedding from the combined vocabulary (Lyle; Col. 19, lines 48-67 & Col. 20, lines 1-13 – word embeddings; see also Col. 21, lines 55-67 - the topic analysis module 242, which is configured to identify one or more topics, keywords, themes, ideas, items, etc. that were discussed or spoken of during the video); generating one or more topic embeddings from the audio content and the vocabulary embeddings (Lyle; Col. 19, lines 48-67 & Col. 20, lines 1-13 – topic modeling techniques… Latent Dirichlet Allocation (LDA)… TOP2VEC… embedded topic models); generating a topic embedding index for the audio content based on the one or more topic embeddings (Lyle; Col. 21, lines 20-43 - …the topic mapper 260 is configured to map the detected topics to the relevant sections of the transcript, and create an indexable and easily searchable source of data representing the video content as a set of topic-labeled segments…); and storing the embedding index for use with a search engine system (Lyle; Col. 21, lines 20-43 – playback manager uses video topic mapper’s stored data to manage playback through user selections; see also Fig. 3, item 360; Col. 17, lines 9-30 – encoded data being shown to be stored in storage 360). Lyle, however, fails to disclose obtaining a predefined ad hoc vocabulary list, wherein the ad hoc vocabulary list includes terms having a low frequency and high distinctiveness, and after obtaining the predefined ad hoc vocabulary list, generating a combined vocabulary that combines at least a portion of the extracted vocabulary terms with one or more ad hoc vocabulary terms from the predefined ad hoc vocabulary list. Hennum does teach obtaining a predefined ad hoc vocabulary list, wherein the ad hoc vocabulary list includes terms having a low frequency and high distinctiveness (Hennum; Col. 8, lines 34-50 - The knowledge representation of the terms (vocabulary and words) can be predefined 710 as a controlled vocabulary…; see also Col. 8, lines 13-33 for a description of broader-narrower relationships to define vocabulary elements where a broader vocabulary (high frequency, low distinctiveness) of terms is combined with a narrower vocabulary (low frequency, high distinctiveness)), and after obtaining the predefined ad hoc vocabulary list, generating a combined vocabulary that combines at least a portion of the extracted vocabulary terms with one or more ad hoc vocabulary terms from the predefined ad hoc vocabulary list (Hennum; Col. 8, lines 34-50 - The knowledge representation of the terms (vocabulary and words) can be predefined 710 as a controlled vocabulary. This can extend the vocabulary because more information is available compared to the generated knowledge representation from the document words (combined vocabulary)). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle to include obtaining a predefined ad hoc vocabulary list, wherein the ad hoc vocabulary list includes terms having a low frequency and high distinctiveness, and after obtaining the predefined ad hoc vocabulary list, generating a combined vocabulary that combines at least a portion of the extracted vocabulary terms with one or more ad hoc vocabulary terms from the predefined ad hoc vocabulary list, as taught by Hennum, in order to add markup for controlled vocabularies to extensible markup languages, thereby allowing words to be marked up in a document as occurrences of terms from vocabularies so that the marked-up words are processed as occurrences of semantic subjects within a knowledge representation (Hennum; Col. 4, lines 36-42).
As per claim 3, Lyle in view of Hennum discloses: The method of claim 1, wherein the one or more topic embeddings are generated using an Embedded Topic Model (ETM) (Lyle; Col. 19, lines 48-67 & Col. 20, lines 1-13 – topic modeling techniques… Latent Dirichlet Allocation (LDA)… TOP2VEC… embedded topic models).
As per claim 4, Lyle in view of Hennum discloses: The method of claim 1, wherein the one or more topic embeddings are generated using latent Dirichlet allocation (LDA) and Word2vec algorithms (Lyle; Col. 19, lines 48-67 & Col. 20, lines 1-13 – topic modeling techniques… Latent Dirichlet Allocation (LDA)… TOP2VEC… embedded topic models).
As per claim 5, Lyle in view of Hennum discloses: The method of claim 1, wherein the audio content comprises a podcast and a podcast segment, and respective topic embeddings are generated for each of the podcast and the podcast segment (Lyle; Col. 6, lines 61-67 & Col. 7, lines 1-19 - …video streaming comprises digital video broadcasting…).
As per claim 6, Lyle in view of Hennum discloses: The method of claim 1, wherein generating the one or more topic embeddings from the audio content comprises identifying the top N topics for the audio content (Lyle; Col. 21, lines 4-19 - topic ranking which can identify the larger themes in the video that can be used to auto-generate bookmarks or chapters in the video and/or re-arrange a video based on the identified themes).
As per claim 7, Lyle in view of Hennum discloses: The method of claim 1, wherein obtaining the audio content comprises obtaining a transcript of an audio recording (Lyle; Col. 6, lines 12-42 - original video content is received… It is to be understood that the inference engine also receives audio data and/or transcription data which can be used in parallel with the outputted synthesized video in cases where the video includes speech or otherwise has sounds present in the video stream; see also Col. 15, lines 22-67 & Col. 16 lines 1-67 – speech transcription & NLP analysis).
As per claim 8, Lyle in view of Hennum discloses: The method of claim 1, wherein obtaining the audio content comprises extracting audio features from an audio recording (Lyle; Col. 15, lines 46-63 - a featurizer may initially deidentify data and process and convert the data to consumable features).
As per claim 9, Lyle in view of Hennum discloses: The method of claim 1, wherein the topic embedding index is a podcast topic embedding index corresponding to a podcast database; and the method further comprises generating a podcast segment topic embedding index corresponding to a podcast segment database (Lyle; Col. 17, lines 9-30 - In this high-level schematic, original video data (“original data”) 310 can be stored in storage 360. Original data has a first file size “A”. When the original data 310 is processed by embodiments of the video data encoding system, an encoder 320 can replace some, most, or nearly all of the video with a sequence of codes (encoded data 334). Encoded data has a second file size “B”. In addition, preserved original content 332 in the form of reference image(s) and clips for unclassified content will be preserved. Preserved original content 332 has a third file size C. Collectively, compressed video data 330 will have a file size of “B+C”. As a general matter, the file size “A” as compared to the file size “B+C” is significantly larger, and will typically be hundreds to thousands time greater in size. Nevertheless, when the compressed video data 330 is provided to the video synthesizer (“simulator”) and its associated decoder 340, the outputted simulated data 350 can carry the bulk of the information that had been captured as original data 310).
As per claim 10, Lyle in view of Hennum discloses: The method of claim 1, wherein the topic embedding index corresponds to a podcast database that includes entries for full episodes and entries for episode segments (Lyle; Col. 17, lines 9-30 - In this high-level schematic, original video data (“original data”) 310 can be stored in storage 360. Original data has a first file size “A”. When the original data 310 is processed by embodiments of the video data encoding system, an encoder 320 can replace some, most, or nearly all of the video with a sequence of codes (encoded data 334). Encoded data has a second file size “B”. In addition, preserved original content 332 in the form of reference image(s) and clips for unclassified content will be preserved. Preserved original content 332 has a third file size C. Collectively, compressed video data 330 will have a file size of “B+C”. As a general matter, the file size “A” as compared to the file size “B+C” is significantly larger, and will typically be hundreds to thousands time greater in size. Nevertheless, when the compressed video data 330 is provided to the video synthesizer (“simulator”) and its associated decoder 340, the outputted simulated data 350 can carry the bulk of the information that had been captured as original data 310).
As per claim 11, Lyle in view of Hennum discloses: The method of claim 1, wherein the search engine system comprises a semantic search engine (Lyle; Col. 5, lines 19-23 - The incorporation of transcript data further enables the user to search by keywords, or—following processing by a topic analysis module-broader topics (semantic searching) in the video as detected by the video navigation system).
As per claim 13, Lyle in view of Hennum discloses: The method of claim 1, wherein the vocabulary terms include one or more of: a phrase and a sentence (Lyle; Col. 15, lines 47-63 - NLP techniques may be used to process sample speech data as well as to interpret the language, for example by parsing sentences, and determining underlying meanings of the words). As per claim 21, Lyle in view of Hennum disclose: The method of claim 1, upon which claim 21 depends.
And further, Hennum does teach wherein the ad hoc vocabulary list includes terms having a low frequency and high distinctiveness (Hennum; Col. 8, lines 34-50 - The knowledge representation of the terms (vocabulary and words) can be predefined 710 as a controlled vocabulary…; see also Col. 8, lines 13-33 for a description of broader-narrower relationships to define vocabulary elements where a broader vocabulary (high frequency, low distinctiveness) of terms is combined with a narrower vocabulary (low frequency, high distinctiveness)). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle to include wherein the ad hoc vocabulary list includes terms having a low frequency and high distinctiveness, as taught by Hennum, in order to add markup for controlled vocabularies to extensible markup languages, thereby allowing words to be marked up in a document as occurrences of terms from vocabularies so that the marked-up words are processed as occurrences of semantic subjects within a knowledge representation (Hennum; Col. 4, lines 36-42). Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Lyle in view of Hennum, and further in view of Hellman (US PG Pub 20220013023).
As per claim 2, Lyle in view of Hennum discloses: The method of claim 1, upon which claim 2 depends. Lyle in view of Hennum, however, fails to disclose wherein the transformer model comprises a SentenceBERT model. Hellman does teach wherein the transformer model comprises a SentenceBERT model (Hellman; p. 0079 - the disclosed embodiments may consider embedding sentences using SBERT, a version of BERT that has been fine-tuned on the SNLI and Multi-Genre NLI tasks). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle and Hennum to include wherein the transformer model comprises a SentenceBERT model, as taught by Hellman, in order to predict how sentences relate to one another. This means that the SBERT network has been specifically fine-tuned to embed individual sentences into a common space (Hellman; p. 0079).
Claims 14, 15, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Lyle in view of Hennum, and further in view of Asgekar (US PG Pub 20230142718). As per claims 14, 18 and 20, Lyle in view of Hennum discloses: The method, system and non-transitory computer-readable storage medium of claims 1, 17 and 19, upon which claims 14, 18 and 20 depend. Lyle in view of Hennum, however, fails to disclose receiving a query string from a user; converting the query string to a query topic embedding; and obtaining one or more search results by comparing the query topic embedding with the topic embedding index. Asgekar does teach receiving a query string from a user (Asgekar; p. 0100 - the query embeddings generator 235 can generate query embeddings by inputting a set of query terms received from a client device into the transformer model. The query terms can be provided as part of a content request 280. As described herein above, a content request 280 can include one or more words, which can form a question, another type of sentence, phrase, or other text information); converting the query string to a query topic embedding (Asgekar; p. 0100 - Because the content embeddings 275 and the query embeddings are generated using the same transformer model 250, the query embeddings and the content embeddings can share the same embeddings space. Thus, the query embeddings can be compared to any vector in the embeddings space, such as the pivots, to determine a distance (e.g., relatedness) of a query embedding to a content embedding 275 (e.g., and thus a corresponding information resource 270, etc.)); and obtaining one or more search results by comparing the query topic embedding with the topic embedding index (Asgekar; p. 0101 - Once the query embeddings are generated, the information set selector 240 can select a subset of the information resources 270 that are related to the content request 280 from which the query embeddings were generated. The information set selector 240 can determine related information resources by calculating a distance in the embeddings space between the query embeddings and the plurality of pivots). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle and Hennum to include receiving a query string from a user; converting the query string to a query topic embedding; and obtaining one or more search results by comparing the query topic embedding with the topic embedding index, as taught by Asgekar, in order to automatically classify and index teaching media such that it can be easily accessed and provided to best answer a student query or to achieve a requested learning objective (Asgekar; p. 0002). As per claim 15, Lyle in view of Hennum and Asgekar disclose: The method of claim 14, upon which claim 15 depends. And further, Asgekar teaches wherein the query is a word, a phrase, or a sentence (Asgekar; p. 0100 - the query embeddings generator 235 can generate query embeddings by inputting a set of query terms received from a client device into the transformer model. The query terms can be provided as part of a content request 280. As described herein above, a content request 280 can include one or more words, which can form a question, another type of sentence, phrase, or other text information). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle and Hennum, to include wherein the query is a word, a phrase, or a sentence, as taught by Asgekar, in order to automatically classify and index teaching media such that it can be easily accessed and provided to best answer a student query or to achieve a requested learning objective (Asgekar; p. 0002).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Lyle in view of Hennum, and further in view of Tomkins (US PG Pub 20210295822).
As per claim 16, Lyle in view of Hennum discloses: The method of claim 1, upon which claim 16 depends. Lyle in view of Hennum, however, fails to disclose wherein extracting the vocabulary terms from the audio content includes one or more of: removing punctuation and stop words from a transcript, and filtering one or more words from the transcript. Tomkins does teach wherein extracting the vocabulary terms from the audio content includes one or more of: removing punctuation and stop words from a transcript, and filtering one or more words from the transcript (Tomkins; p. 0147 - some embodiments may determine that each word of the query may be used as an n-gram, where one or more of the words may be modified or deleted based on a set of filters that remove stop words, lemmatizes words, stems words, or the like). Therefore, it would have been obvious for one of ordinary skill in the art to modify the method of Lyle and Hennum, to include wherein extracting the vocabulary terms from the audio content includes one or more of: removing punctuation and stop words from a transcript, and filtering one or more words from the transcript, as taught by Tomkins, in order to achieve cross-context natural language model generation (Tomkins; p. 0002).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art made of record and not relied upon includes: Thambiratnam (US PG Pub 20160259782) discloses: Content-based analysis is performed on multimedia content prior to encoding the multimedia content in the rendering chain of processing. A content-based index stream is generated based on the content-based analysis and the content-based index stream is embedded in the multimedia file during rendering. The content-based index stream can be used to generate a content-based searchable index when necessary (Thambiratnam; Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Rodrigo A Chavez whose telephone number is (571)270-0139. The examiner can normally be reached Monday - Friday 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 5712727602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RODRIGO A CHAVEZ/Examiner, Art Unit 2658
/RICHEMOND DORVIL/Supervisory Patent Examiner, Art Unit 2658
Prosecution Insights