Introduction
1. This office action is in response to Applicant’s submission filed on 7/17/2026. Claims 1, 3-13, and 15-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 7/17/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
4. The Amendment filed 5/11/2026 has been entered and fully considered. With regard to the rejections under 35 USC 103, those arguments are found unpersuasive as discussed herebelow.
With respect to Claim 1, that rejection is maintained. The Amendment argues that Gray does not describe or suggest “generating, prior to obtaining a search query, a text summary of the content item that is constructed to represent an entirety of the content item by providing to a large language model (LLM) a model prompt including the text of the content item and text associated with the image.” However, The term “a search query” is not further defined in any of the claims. Accordingly, it is not required to be the search query that discovered the content item, which is what the Amendment argues. The device of Gray receives many search queries, its purpose is to answer search queries. Thus, the summarization described by Gray would occur before “obtaining a search query.”
The amendment further argues that Gray does not describe “the text summary representing the entirety of the content item by removing at least a portion of the text of the content item while retaining key points associated with the content item.” However, Gray clearly describes that the text generated is a “summary,” (paragraph 4) which is a subset of the original content while retaining the key points. Thus, Gray describes “the text summary representing the entirety of the content item by removing at least a portion of the text of the content item while retaining key points associated with the content item.”
Finally with respect to Claim 1, the Amendment argues that Berg does not describe “generating, via a text embedding model, a text embedding representing the text summary of the content item.” However, Berg is only cited as describing generating an embedding from a text portion (summary). Gray is cited as describing that the text portion is a summary of a content item. Thus, the combination of Gray and Berg describes or suggests “generating, via a text embedding model, a text embedding representing the text summary of the content item.”
With respect to Claim 10, the rejection is maintained. The Amendment argues that none of the references describes both a prefix and a semantic search, or concurrently displaying the results of both searches. However, the combination of Mart (semantic searching) and Liao (prefix searching) together describes or suggests these features.
With respect to Claim 15, the Amendment argues that Gray does not describe an “image-to-text model.” However, paragraph 65 of Gray describes a model that can automatically generate image captions given an image. Any model that can go from an image to text is considered “an image-to-text model,” according to the broadest reasonable interpretation of this phrase.
Claim Rejections - 35 USC § 103
5 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1 and 3-9 are rejected under 35 U.S.C. 103 as unpatentable over U.S. Pat. App. Pub. No. 20240403341 (Berglund et al., hereinafter “Berg”) in view of U.S. Pat. App. Pub. No. 20240273105 (Martigny et al., hereinafter “Mart”) and U.S. Pat. App. Pub. No. 20250005303 (Gray et al., hereinafter “Gray”).
With regard to Claim 1, Berg describes:
“A computer-implemented method for including:
obtaining a content item having text and an image; (Paragraph 14 describes that a content repository stores a plurality of content items that can be searched with a query.)
generating, via a text embedding model, a text embedding representing the text summary of the content item; and (Paragraph 14 describes that the text chunks are turned into text embeddings.)
storing the text embedding representing the text summary of the content item, the text embedding stored for subsequently performing a [[semantic]] search to determine that the content item is relevant to a search query.” (Paragraph 14 describes that the text embeddings are compared to query embeddings to determine a similarity between the text embedding and the query embedding.)
Berg does not explicitly describe “generating, prior to obtaining a search query, a text summary of the content item that is constructed to represent the entirety of the content item by providing to a large language model (LLM) a model prompt including the text of the content item and text associated with the image, the text summary representing an entirety of the content item by removing at least a portion of the text of the content item while retaining key points associated with the content item” or comparing the summary to the query using a semantic search.
However, paragraph 40 of Mart describes that a semantic search can be used to find content similar to a query.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the semantic search as described by Mart into the system of Bert to determine the most relevant content to a query, as described in paragraph 40 of Mart.
Bert in view of Mart does not explicitly describe “generating, prior to obtaining a search query, a text summary of the content item that is constructed to represent the entirety of the content item by providing to a large language model (LLM) a model prompt including the text of the content item and text associated with the image, the text summary representing an entirety of the content item by removing at least a portion of the text of the content item while retaining key points associated with the content item.”
However, paragraph 65 of Gray describes generating text summaries by inputting content into a model. The content may include text, images, and captions of the images. Gray describes a device that receives many queries, as it is designed to answer search queries. Thus, the described summarization would occur prior to obtaining a subsequent search query. Finally, Gray clearly describes that the text generated is a “summary,” (paragraph 4) which is a subset of the original content while retaining the key points.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the summarization as described by Gray into the system of Bert in view of Mart to efficiently generate text summaries for search results, as described in paragraph 65 of Gray.
With regard to Claim 3, Bert describes “the content item summarized by the text summary comprises a set of one or more pages, a set of one or more sections, or a set of one or more paragraphs.” Paragraph 27 of Berg describes that the content is broken into sentence length sections, and paragraph 29 describes that these sections are used to determine the corresponding text chunks.
With regard to Claim 4, Bert describes “the text of the content item includes an image caption generated for an image of the content item.” Paragraph 27 describes that video content can be summarized by text transcripts, which are cited as an “image caption.”
With regard to Claim 5, Bert describes “generating an image caption for an image of the content item and incorporating the image caption in the content item such that the content item having the image caption is summarized in the text summary.” Paragraph 27 describes that video content can be summarized by text transcripts, which are cited as an “image caption.” The transcripts are divided into sentence length sections, which are used to create the text chunk summaries.
With regard to Claim 6, Bert describes:
“obtaining the search query; (Paragraph 14 describes that a search query is received.)
generating a query text embedding, via the text embedding model, that represents the search query; (Paragraph 14 describes that the query is converted to a query embedding.)
performing the [[semantic]] search to determine that the content item is relevant to the search query by comparing the query text embedding to the text embedding representing the text summary to analyze similarity between the query text embedding and the text embedding representing the text summary; and (Paragraph 14 describes that a similarity analysis is done between the query embedding and the text chunk embedding to determine relevant content.)
providing a search result corresponding with the content item for presentation in response to the search query.” (Paragraph 14 describes that an answer is returned to the user, which may include the relevant content.)
Berg does not explicitly describe comparing the summary to the query using a semantic search.
However, paragraph 40 of Mart describes that a semantic search can be used to find content similar to a query.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the semantic search as described by Mart into the system of Bert to determine the most relevant content to a query, as described in paragraph 40 of Mart.
With regard to Claim 7, Berg describes “the search result includes a result context that indicates at least a portion of the text summary that corresponds with the search query.” Paragraph 14 describes that an answer is returned to the user, which may include the content which contained the relevant text chunks.
With regard to Claim 8, Berg describes:
“obtaining a user feedback modifying the at least the portion of the text summary that corresponds with the search query; (Paragraph 53 describes that the user can provide feedback. Paragraph 18 describes that the user can modify the content included in the content store.)
updating the text summary to incorporate the user feedback; and (Paragraphs 27-29 describe how the device will create the text chunk summary for modified content.)
generating a new text embedding for the updated text summary.” (Paragraph 14 describes that the text chunks are used to create the text embeddings.)
With regard to Claim 9, Berg does not explicitly describe this subject matter. However, Mart describes:
“generating, via a lexical data model, lexical search data based on the content item or the text summary; and (Paragraph 87 describes that a lexical search is done of stored content.)
storing the lexical search data for subsequently performing a lexical search to determine that the content item is relevant to a particular search query.” (Paragraph 87 describes that a ranking of lexical similarity is created.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the lexical search as described by Mart into the system of Bert to determine the most relevant lexical content to a query, as described in paragraph 87 of Mart.
7. Claims 10-13 are rejected under 35 U.S.C. 103 as unpatentable over Berg in view of Mart and U.S. Pat. App. Pub. No. 20100312764 (Liao et al., hereinafter “Liao”).
With regard to Claim 10, Berg describes:
“A computer-implemented method comprising:
obtaining a search query; (Paragraph 14 describes that a search query is received.)
using the search query to perform a [[semantic] search to identify a first content item [[semantically]] similar to the search query [[and to perform a prefix search to identify a second content item lexically similar to the search query]], wherein the [[semantic]] search is performed by:
generating a query text embedding to represent the search query; (Paragraph 14 describes generating a query embedding.)
comparing the query text embedding to a set of text embeddings representing text summaries generated for corresponding content items having text; (Paragraph 14 describes comparing the query embedding to text chunk embeddings.)
based on the comparing, identifying the first content item, of the content items, as [[semantically]] similar to the search query; and (Paragraph 14 describes determining similar content items based on the comparison.)
providing, for [[concurrent]] display, a search result set comprising [[both]] an indication of the first content item identified as [[semantically]] similar to the search query.” (Paragraph 14 describes that an answer is returned to the user, which may include the relevant content.)
Berg does not explicitly describe “wherein the prefix search is performed by using the search query to match one or more terms of the search query to one or more terms of the second content item, of the content items, to identify the second content item as lexically similar to the search query,” concurrently displaying the second content item, or comparing the summary to the query using a semantic search.
However, paragraph 40 of Mart describes that a semantic search can be used to find content similar to a query.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the semantic search as described by Mart into the system of Bert to determine the most relevant content to a query, as described in paragraph 40 of Mart.
Bert in view of Mart does not explicitly describe “wherein the prefix search is performed by using the search query to match one or more terms of the search query to one or more terms of the second content item, of the content items, to identify the second content item as lexically similar to the search query,” and displaying the second content item.
However, paragraph 14 of Liao describes performing a search on a prompt to find lexically similar content. Further, the search results are displayed in 1382 as described in paragraph 47. Further, it would have been obvious to concurrently display this search result with the first search result. The combination of the references describes displaying the search results to the user, which would be all the search results at once.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the lexical search as described by Liao into the system of Bert in view of Mart to determine similar content to a query, as described in paragraph 83 of Liao.
With regard to Claim 11, Berg describes “the content item includes an image, and wherein a text summary generated for the content item is based on an image caption generated for the image of the content item.” Paragraph 27 describes that video content can be summarized by text transcripts, which are cited as an “image caption.”
With regard to Claim 12, Berg describes “the content item is identified as [[semantically]] similar to the search query based on a similarity distance between the query text embedding and a text embedding representing a text summary generated for the content item.” Paragraph 51 describes that a similarity distance is computed between the text embedding and the query embedding.
Berg does not explicitly describe comparing the summary to the query using a semantic search.
However, paragraph 40 of Mart describes that a semantic search can be used to find content similar to a query.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the semantic search as described by Mart into the system of Bert to determine the most relevant content to a query, as described in paragraph 40 of Mart.
With regard to Claim 13, Berg describes “the search result set includes a result context that indicates at least a portion of a text summary, generated for the content item, that corresponds with the search query.” Paragraph 14 describes that an answer is returned to the user, which may include the content which contained the relevant text chunks.
8. Claims 15 and 18-20 are rejected under 35 U.S.C. 103 as unpatentable over Berg in view of Gray.
With regard to Claim 15, Berg describes “One or more computer storage media having computer-executable instructions embodied thereon that, when executed by one or more processors, cause the one or more processors to perform a method, (Paragraph 88) the method comprising:
obtaining a content item including an image; (Paragraph 14 describes that a content repository stores a plurality of content items. Paragraph 17 describes that some of the content may be images.)
for a search query, performing a search in association with the text summary that summarizes the content item to determine that the content item is relevant to the search query; and (Paragraph 14 describes that the text embeddings are compared to query embeddings to determine a similarity between the text embedding and the query embedding.)
providing, for display, a search result indicating the content item determined to be relevant to the search query. (Paragraph 53 describes that search results are provided to a user on a display.)
Bert does not explicitly describe:
“generating, via a machine learning model, a text summary that summarizes the content item, wherein generating the text summary comprises inputting the image caption into the machine learning model and obtaining, in response, the text summary that summarizes at least the image caption;
generating, via an image-to-text model, an image caption providing a text description of the image.”
However, Gray describes “generating, via a machine learning model, a text summary that summarizes the content item, wherein generating the text summary comprises inputting the image caption into the machine learning model and obtaining, in response, the text summary that summarizes at least the image caption; (Paragraph 65 of Gray describes generating text summaries by inputting content into a model. The content may include text, images, and captions of the images.)
generating, via an image-to-text model, an image caption providing a text description of the image.” (Paragraph 65 describes that the model can automatically generate image captions. Any model that can go from an image to text is considered “an image-to-text model,” according to the broadest reasonable interpretation of this phrase.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the text generation as described by Gray into the system of Bert to efficiently generate text summaries for search results, as described in paragraph 65 of Gray.
With regard to Claim 18, Berg describes “the search result includes an indication of the content item and a result context that indicates at least a portion of the text summary, generated for the content item, that corresponds with the search query.” Paragraph 14 describes that an answer is returned to the user, which may include the content which contained the relevant text chunks.
With regard to Claim 19, Berg describes “the search result includes an indication of the content item and a result context that indicates at least a portion of the image caption that corresponds with the search query.” Paragraph 14 describes that an answer is returned to the user, which may include the answer description which is based on the relevant text chunks.
With regard to Claim 20, Berg describes:
“obtaining a user feedback modifying the at least the portion of the image caption that corresponds with the search query; (Paragraph 53 describes that the user can provide feedback. Paragraph 18 describes that the user can modify the content included in the content store. Modifying video would change the corresponding transcript for the video.)
updating the image caption to incorporate the user feedback; and (Paragraphs 27-29 describe how the device will create the text chunk summary for modified content based on the new transcript.)
generating a new text embedding for the updated image caption.” (Paragraph 14 describes that the text chunks are used to create the text embeddings.)
10. Claim 16 is are rejected under 35 U.S.C. 103 as unpatentable over Berg in view of Gray and further in view of Mart.
With regard to Claim 16, Berg describes “the search comprises a [[semantic]] search performed by:
generating a query text embedding to represent the search query; (Paragraph 14 describes generating a query embedding.)
generating a content text embedding to represent the text summary of the content item; and (Paragraph 14 describes generating a text chunk embedding.)
performing similarity analysis of the query text embedding and the content text embedding to determine [[semantic]] similarity between the search query and the content item. (Paragraph 14 describes comparing the query embedding to text chunk embeddings to determine similarity.)
Berg in view of Gray does not explicitly describe comparing the summary to the query using a semantic search.
However, paragraph 40 of Mart describes that a semantic search can be used to find content similar to a query.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the semantic search as described by Mart into the system of Bert in view of Gray to determine the most relevant content to a query, as described in paragraph 40 of Mart.
9. Claim 17 is rejected under 35 U.S.C. 103 as unpatentable over Berg in view of Gray and further in view of U.S. Pat. App. Pub. No. 20150058720 (Smadja et al., hereinafter “Sma”).
With regard to Claim 17, Berg in view of Gray does not explicitly describe this subject matter. However, Sma describes “for the search query, performing a prefix search to determine a second content item relevant to the search query; and (Paragraph 69 describes performing a lexical prefix search for content.)
providing, for display, a second search result indicating the second content item determined to be relevant to the search query. (Paragraph 70 describes that the search results are displayed to the user.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the lexical prefix search as described by Sma into the system of Bert in view of Gray to more accurately search content with lexically similar words, as described in paragraphs 68 and 69 of Sma.
Conclusion
10. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pat. No. 11144594 (Liu et al.) also describes merging the results of several search operations.
11. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD TRACY whose telephone number is (571)272-8332. The examiner can normally be reached Monday-Friday 9 AM- 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD TRACY JR./Examiner, Art Unit 2656
/BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656
Prosecution Insights