DETAILED ACTION
Case Status
This office action is in response to remarks and amendments of 18 February 2026. Claims 1-20 have been examined.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
20050015713
Pars. 21, 22, 28, 40-46
Display of metadata aggregated from multiple sources
11216697
col. 4, lines 53-54, col. 12 lines 42-44
Back-filling / re-indexing using a new, user selected embedding model for image search
8713007
Abstract
Classifying documents using multiple classifiers that generate a score associated with probabilities
20210264195
Pars. 75-77, 105, fig. 17
User training and selecting multiple classifiers to classify items for subsequent querying
20230409647
Fig.’s 1, 19, 20
Building/training classification models and user selecting one or more model for searching
20220019860
Abstract, par. 49
User directed re-training of image data ML classifier using user-verified labels
20190171915
Pars. 82-85
Users manually modify image classifications to provide training data for ML systems
20180300576
Pars. 108-110
Re-labelling images to reprocess them using an ML model
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 5-11, and 13-19 are rejected under 35 U.S.C. 103 as being unpatentable over Shaji et al., US 20180039887 A1, hereinafter Shaji in view of Lemmons et al, US 20220138805 A1, hereinafter Lemmons and further in view of Israel et al., Pub. No.: US 20050182792 A1, hereinafter Israel.
As per claim 1, Shaji discloses A computer-implemented method comprising:
identifying a user selection of a […] component […] within a content search platform of a searchable content management system comprising a set of content items, wherein the content search platform comprises a feature extraction component corresponding to a feature labeling task (pars. 22, 23, 53, 57, 77, 80 disclose that the chosen/selected classifier (i.e. component) is based on a user’s decision/acceptance (i.e. a user selection), and a separate feature extraction step that results in identification and determination of features that are extracted for particular content items (i.e. feature labeling task); additionally, and/or alternatively, par. 81 and fig. 7 disclose that a user selects an updated classifier/component by means of the user manually inputting whether or not he/she is satisfied with it);
Shaji does not explicitly disclose identifying a user selection of a classification model component trained for a classification task to integrate within an existing content search platform. Note, however, that Shaji does at least mention that information from multiple modalities is used to augment the feature representation from the base model in pars. 48-49; in other words, Shaji’s system at least integrates the results of multiple feature extraction components, at least one of which is a user selected component.
Lemmons in the related field of endeavor of information retrieval discloses identifying a user selection of a classification model component trained for a classification task to integrate within an existing content search platform (Lemmons, par. 130, 133, 136-138 disclose user selection of trained classifications models that are applied to data in order to label/re-label and index the data).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Lemmons would have allowed Shaji’s image classification system to enable a user to utilize additional trained ML models that assign one or more additional labels to images which will “allow the user to converge on a subset of images that rank highly in all models” (Lemmons, par. 130). integrating the classification model component within the content search platform (see Lemmons and Shaji as cited above);
based on integration of the classification model component, utilizing the classification model component with the set of content items to automatically generate one or more labels for the set of content items by: generating, utilizing the classification model component, a predicted classification label associated with the classification task for a content item (see Lemmons and Shaji as cited above); and generating modified metadata for the content item by backfilling the predicted classification label within metadata of the content item; (see Lemmons, par. 130, 133, 136-138 which disclose user selection of trained classifications models that are applied to data in order to label/re-label and index the data repeatedly and over time (i.e. backfilling); also, note that Shahji’s disclosed augmenting, auto-tagging and generating/storing new tags in pars. 48, 57-59 are examples of backfilling as claimed);
based on receiving a search query prompt within the content search platform of the searchable content management system, providing, for display within a graphical user interface of a client device, one or more content items from the set of content items utilizing a match between the one or more labels and the search query prompt (as per Shaji par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries); and
Shaji as modified does not expressly disclose, however Israel in the related field of endeavor of information retrieval discloses providing, for display within the graphical user interface of the client device, a consolidated metadata display that indicates the modified metadata for the content item that comprises the predicted classification label from the classification model component (Israel, pars. 9, 29, 40, 45-46 disclose a GUI metadata display that displays metadata consolidated from multiple sources including modified metadata and see Shaji and Lemmons as cited above for the predicted classification label from the classification model component).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Israel would have allowed the combination to obtain metadata from multiple sources, validate metadata, synthesize new metadata, automate the adjustment of existing metadata as part of an augmentation process, and use customizable rules to automatically update and modify metadata thereby providing metadata brokering servers and methods which enable the capturing, processing, synthesizing and forwarding of metadata suitable to enable advanced user interfaces to be generated (Israel, par. 8, 29, 40, 45-46).
As per claim 11, Shaji discloses A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
utilizing a classification model component integrated in a searchable content management system, the classification model component enabling a first classification model to output classification labels for a set of content items within the searchable content management system to automatically map one or more classification labels from the first classification model to the set of content items (pars. 53, 57 disclose that the chosen/selected classifier (i.e., classification model component) is based on a user’s decision/acceptance; additionally, and/or alternatively, par. 81 and fig. 7 disclose that a user selects an updated classifier by means of the user manually inputting whether or not he/she is satisfied with it. Pars. 5, 33, 57-58, Table 1 disclose that the new classifier re-classifies all of the images (and/or text) in the database and augments the search index with labels/keywords associated with the new classifier; Table 1 in par. 58 says “deploy a classifier that will auto-tag” using a new/custom classifier trained on a keyword);
Shaji does not explicitly disclose, however Lemmons, in the related field of endeavor of information retrieval discloses utilizing an additional classification model component integrated in a searchable content management system (Lemmons, par. 130, 133, 136-138 disclose user selection of trained classifications models that are applied to data in order to label/re-label and index the data).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Lemmons would have allowed Shaji’s image classification system to enable a user to utilize additional trained ML models that assign one or more additional labels to images which will “allow the user to converge on a subset of images that rank highly in all models” (Lemmons, par. 130)
the additional classification model component enabling a second classification model to output additional classification labels for a set of content items within the searchable content management system to automatically map one or more additional classification labels from the second classification model to the set of content items (See Shaji and Lemmons as cited above and note that Shaji mentions that information from multiple modalities is used to augment the feature representation from the base model in pars. 48-49; additionally, and/or alternatively, note that Lemmons, par. 130, 133, 136-138 disclose that the multiple models are used to relabel or reclassify images);
based on receiving a search query prompt within a content search platform of the searchable content management system, provide for display, within a graphical user interface, one or more content items from the set of content items utilizing a match between the search query prompt and the one or more classification labels or the one or more additional classification labels (Shaji, par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries. See also, pars. 80, 83-85, and Lemmons, par. 130, 133, 136-138). and
Shaji as modified does not expressly disclose, however Israel in the related field of endeavor of information retrieval discloses provide, for display within the graphical user interface, a consolidated metadata display that indicates modified metadata for the one or more content items that comprises the one or more classification labels from the first classification model and the one or more additional classification labels from the second classification model (Israel, pars. 9, 29, 40, 45-46 disclose a GUI metadata display that displays metadata consolidated from multiple sources including modified metadata and see Shaji and Lemmons as cited above for the predicted classification label from the classification model component).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Israel would have allowed the combination to obtain metadata from multiple sources, validate metadata, synthesize new metadata, automate the adjustment of existing metadata as part of an augmentation process, and use customizable rules to automatically update and modify metadata thereby providing metadata brokering servers and methods which enable the capturing, processing, synthesizing and forwarding of metadata suitable to enable advanced user interfaces to be generated (Israel, par. 8, 29, 40, 45-46).
As per claim 16, Shaji discloses A system comprising: at least one processor; and at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to: utilize […a…] classification model component […] integrated within a searchable content management system comprising a set of content items to automatically assign classification model label data to the set of content items (pars. 53, 57 disclose that the chosen/selected classifier (i.e., classification model component) is based on a user’s decision/acceptance; additionally, and/or alternatively, par. 81 and fig. 7 disclose that a user selects an updated classifier by means of the user manually inputting whether or not he/she is satisfied with it. Pars. 5, 33, 57-58, Table 1 disclose that the new classifier re-classifies all of the images (and/or text) in the database and augments the search index with labels/keywords associated with the new classifier; Table 1 in par. 58 says “deploy a classifier that will auto-tag” using a new/custom classifier trained on a keyword);
Shaji does not explicitly disclose, however Lemmons, in the related field of endeavor of information retrieval discloses: utilize multiple classification model components integrated within a searchable content management system comprising a set of content items (Lemmons, par. 130, 133, 136-138 disclose user selection of trained classifications models that are applied to data in order to label/re-label and index the data; Note that Shaji does at least mention that information from multiple modalities is used to augment the feature representation from the base model in pars. 48-49; in other words, Shaji’s system at least integrates the results of multiple feature extraction components, at least one of which is a user selected feature extraction component).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Lemmons would have allowed Shaji’s image classification system to enable a user to utilize additional trained ML models that assign one or more additional labels to images which will “allow the user to converge on a subset of images that rank highly in all models” (Lemmons, par. 130)
based on receiving a search query prompt within a content search platform of the searchable content management system, provide for display, within a graphical user interface, one or more content items from the set of content items and one or more classification model label data corresponding to the one or more content items utilizing the assigned classification model label data and the search query prompt (Shaji, as per par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries; see Shaji also, pars. 80, 83-85, and Lemmons, par. 130, 133, 136-138); and
Shaji as modified does not expressly disclose, however Israel in the related field of endeavor of information retrieval discloses provide, for display within the graphical user interface, a consolidated metadata display that indicates modified metadata for the one or more content items that comprises the classification model label data (Israel, pars. 9, 29, 40, 45-46 disclose a GUI metadata display that displays metadata consolidated from multiple sources including modified metadata and see Shaji and Lemmons as cited above for the predicted classification model label data).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Israel would have allowed the combination to obtain metadata from multiple sources, validate metadata, synthesize new metadata, automate the adjustment of existing metadata as part of an augmentation process, and use customizable rules to automatically update and modify metadata thereby providing metadata brokering servers and methods which enable the capturing, processing, synthesizing and forwarding of metadata suitable to enable advanced user interfaces to be generated (Israel, par. 8, 29, 40, 45-46).
As per claim 3, Shaji as modified discloses The computer-implemented method of claim 1, further comprising providing, for display within an additional a graphical user interface of an additional client device, a first set of content item results from the content search platform of the searchable content management system utilizing the feature extraction component and a first search query prompt (Shaji, par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries; see also, Shaji pars. 80, 83-85, and and Lemmons, par. 130, 133, 136-138 with rationale to combine as provided in the rejection of claim 1).
As per claim 5, Shaji as modified discloses The computer-implemented method of claim 1, further comprising: integrating the classification model component on a live searchable content management system (Lemmons, par. 130, 133, 136-138); and based on receiving the search query prompt within the content search platform of the live searchable content management system, providing, for display within the graphical user interface that comprises a developer graphical user interface in a system testing environment, the consolidated metadata display and the one or more content items from the set of content items utilizing the match between the one or more labels and the search query prompt (Shaji, par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries; see also, Israel as cited above for consolidated metadata display and Shaji pars. 80, 83-85, and Lemmons, par. 130, 133, 136-138 with rationale to combine as provided in the rejection of claim 1. Note that the claim does not recite steps directed to development or system testing of the claimed displayed content item; in other words, nothing is recited as being “developed” or “system tested” by a mere display of a content item).
As per claim 6, Shaji as modified discloses the computer-implemented method of claim 1, wherein the classification model component comprises a classification model trained for the classification task separately from the feature extraction component (see rejection of first limitation of claim 1).
As per claim 7, Shaji as modified discloses The computer-implemented method of claim 6, further comprising utilizing the classification model to: generate image classification labels for image-based content items; or generate text classification labels for text-based content items (see rejection of claim 1).
As per claim 8, Shaji as modified discloses The computer-implemented method of claim 1, further comprising providing, for display within the graphical user interface, the content item from the set of content items and the consolidated metadata display for the content item, wherein the consolidated metadata display comprises the predicted classification label, a feature label from the feature extraction component, and additional file information from an additional data source (see rejection of claim 1 including at least Shaji, fig.’s 6-7 and Lemmons, fig.’s 4-12 and Israel pars. 9, 29, 40, 45-46).
As per claim 9, Shaji as modified discloses The computer-implemented method of claim 1, further comprising identifying a content item function within the content search platform based on the one or more labels assigned to the set of content items (see rejection of claim 1 and note that at least the Value Add column of the Table in Shaji par. 58 identifies label based content item functions as claimed; see Lemmons as cited in the rejection of claim 1).
As per claim 10, Shaji as modified discloses The computer-implemented method of claim 1, further comprising: providing, for display within the graphical user interface, a selectable option to select the classification model component (Lemmons, par. 130, 133, 136-138); and in response to a selection of the selectable option, integrating the classification model component within the content search platform (Lemmons, par. 130, 133, 136-138).
As per claim 13, Shaji as modified discloses The non-transitory computer-readable medium of claim 11, further comprising instructions that, when executed by the at least one processor, cause the computing device to: integrate the classification model component in a live searchable content management system (Lemmons, par. 130, 133, 136-138); and
based on receiving the search query prompt within the content search platform of the live searchable content management system, provide, for display within the graphical user interface that comprises a developer graphical user interface in a system testing environment, the one or more content items from the set of content items utilizing a mapping between the one or more classification labels from the first classification model to the set of content items (Shaji, par. 5 and Table 1 in par. 58, the purpose of the invention is for searchers to view more relevant image and/or text search results by means of the new classifier-based augmenting of a search index that is re-indexed with keywords that correspond to keywords in incoming search queries; see also, Shaji pars. 80, 83-85, and Lemmons, par. 130, 133, 136-138, with rationale to combine as provided in the rejection of claim 1. Note that the claim does not recite steps directed to development or system testing of the claimed displayed content item; in other words, nothing is recited as being “developed” or “system tested” by a mere display of a content item).
As per claims 14, 18 and 19, they are analogous/similar to claims 5 and 7 and therefore likewise rejected.
As per claim 15, Shaji as modified discloses The non-transitory computer-readable medium of claim 11, further comprising instructions that, when executed by the at least one processor, cause the computing device to provide, for display within the graphical user interface, a content item from the one or more content items and metadata for the content item, wherein the metadata comprises a first classification label from the classification model component and a second classification label from the additional classification model component (see rejection of claim 1, and also at least Shaji, fig.’s 6-7 and Lemmons, fig.’s 4-12).
As per claim 17, Shaji as modified discloses The system of claim 16, further comprising instructions that, when executed by the at least one processor, cause the system to modify metadata associated with a content item to assign particular classification model label data to the content item (see Lemmons as cited in the rejection of claim 16; also, Shaji pars. 5, 33, 57-58, Table 1 disclose that the new classifier re-classifies all of the images in the database and augments the search index with labels/keywords associated with the new classifier; Table 1 in par. 58 says “deploy a classifier that will auto-tag” using a new/custom classifier trained on a keyword).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Shaji in view of Lemmons and Israel and further in view of Sigurjonsson et al., Pub. No.: US 20020107829 A1, hereinafter Sigurjonsson.
As per claim 2, Shaji as modified discloses the computer-implemented method of claim 1, further comprising: identifying additional metadata corresponding to the one or more content items (pars. 7, 44), Shaji and Lemmons does not explicitly disclose however Sigurjonsson in the related field of endeavor of information search and retrieval discloses wherein the additional metadata comprises an identifier for a content item, a file name, and the one or more labels (Sigurjonsson pars. 17, 22, 32, 46, 96-98); and providing, for display within the consolidated metadata display, the additional metadata in relation to the one or more content items (Sigurjonsson pars. 22, 29, 32, 46, 96-98 and see Israel pars. 9, 29, 40, 45-46 for the consolidated metadata display).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Sigurjonsson’s teaching would have allowed the combination to organize attributes of content as a hierarchy of tag and sub-tag attributes which can be displayed as part of a system that is intended for “capturing content from many sources and organizing the content for quick and easy retrieval.” (Sigurjonsson, pars. 7, 29). “The search of the present invention is very powerful as the searcher allows looking for content from the intersection point of an content in the album can be found. The user can choose how much data will become visible when searching. Thus, for example, it can be easy to find all holiday photos of certain family members and photos of special occasions.” (Sigurjonsson, pars. 52, 53).
Claims 4, 12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shaji in view of Lemmons and Israel and further in view of Wang et al., US 10963734 B1, hereinafter Wang.
As per claim 4, Shaji as modified discloses The computer-implemented method of claim 1, further comprising, Shaji and Lemmons does not explicitly disclose however Wang in the related field of endeavor of information search and retrieval discloses based on receiving the search query prompt within the content search platform of the searchable content management system, providing, for display within the consolidated metadata display, classification model component data object in relation to the content item to indicate the predicted classification label and a classification label probability corresponding to the predicted classification label for the content item (Wang fig.’s 7A, 9C, 11B, col. 22, lines 46-50, col. 23, lines 57-60, col. 33, lines 12-15 and see Israel pars. 9, 29, 40, 45-46 for the consolidated metadata display).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the cited references because Wang’s teaching would have allowed the combined teaching to provide a more efficient search experience in which users are provided with more context, helping them quickly assess the relevance of search results. The classification information that is displayed in connection with search results improves transparency by showing the confidence level of each classification which enables users to prioritize results with higher relevance scores.
As per claims 12 and 20, they are analogous and therefore likewise rejected.
Response to Arguments
Applicant's arguments filed 18 February 2026 have been fully considered; the Israel reference has been introduced in response to the change in claim scope due to claim amendments.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED HASAN whose telephone number is (571)270-5008. The examiner can normally be reached M-F 8am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571)270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED H HASAN/Primary Examiner, Art Unit 2154
Prosecution Insights