DETAIL OFFICE ACTIONS
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 4/16/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Amendment
Applicant submitted amendments on 4/16/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Applicant Arguments:
In regards to Argument 1, Applicant/s state/s Omote et al (U.S. Patent Pub. No. 2019/0104453, hereafter referred to as Omote) in view of Yuan et al (U.S. Patent No. 12,518,512, hereafter referred to as Yuan), does not teach on the amended claims, therefore, the rejection of 35 U.S.C. for Claims 1 & 8 should be removed.
In regards to Argument 2, Applicant/s state/s that the amendment to the claims overcomes the rejection of 35 U.S.C. 101, therefore, the rejection of 35 U.S.C. 101 should be removed.
Examiner’s Responses:
In response to Argument 1, Applicant’s arguments, see Remarks, filed 5/30/2026, with respect to the rejection(s) of claim(s) 1-4 and 8-11 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration and amendments to the claims, a new ground(s) of rejection is made in view of Brandt et al (U.S. Patent Pub. No. 2024/0169624, hereafter referred to as Brandt) in view of Li (WIPO Patent Pub. 2024/005784, hereafter referred to as Li).
Specifically, Brandt teaches a scene-based image editing system that determines object attributes for objects portrayed in a digital image utilizing a multi-attribute contrastive classification neural network. Brandt discloses generating feature vectors—including a localized image-object feature vector (Z_rel), a localized low-level attribute feature vector (Z_low), and a multi-attention feature vector (Z_att)—through the cooperative operation of an embedding neural network, a localizer neural network, a multi-attention neural network, and a classifier neural network. Brandt further teaches utilizing attention mechanisms to focus on different spatial locations relative to a portrayed object to predict attribute labels (positive, negative, and unknown) for that object. Additionally, Brandt discloses facilitating user interactivity with the displayed set of object attributes, enabling user interactions that modify the digital image by changing object attributes. Next, Li teaches a system and method for retrieving video content in response to a textual query. Li discloses an electronic device that includes an image encoder and a text encoder—specifically a contrastive language-image pre-training (CLIP) network—that generates a textual feature vector from a textual query and first visual feature vectors from a subset of image frames of an input video clip within a shared semantic space. Li further teaches iteratively correlating the plurality of first visual feature vectors based on at least one shifted window scheme to generate second visual feature vectors, generating a video feature vector therefrom, and retrieving a video clip based on a video-query similarity level between the textual feature vector and the video feature vector. The Examiner finds that both references can be combined to read on the claimed invention because both Brandt and Li are directed to systems that employ neural network architectures to process and relate visual content to semantic/textual representations. The Examiner reasons that one of ordinary skill in the art would have been motivated to combine Brandt’s attribute-based object recognition and multi-attention feature extraction framework with Li’s contrastive language-image encoding and text-to-visual retrieval mechanism to achieve a predictable result of enabling enhanced semantic understanding and retrieval of visual content based on textual descriptions or attribute-based queries.
In response to Argument 2, Applicant’s arguments, see Remarks, filed 5/30/2026, with respect to the rejection(s) of claim(s) 1-4 and 8-11 under 35 U.S.C. 101 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 4, 6, 8, 10, 11, 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Brandt et al (U.S. Patent Pub. No. 2024/0169624, hereafter referred to as Brandt) in view of Li (WIPO Patent Pub. 2024/005784, hereafter referred to as Li).
Regarding Claim 1, Brandt teaches a visual-linguistic feature fusion method comprising:
generating a linguistic feature using a text encoder based on text (paragraph 110, paragraph 111, Figure 3 item 316, Brandt teaches capturing an image and generating visual text features);
PNG
media_image1.png
422
671
media_image1.png
Greyscale
generating a visual feature using a video encoder based on a video frame (paragraph 115, paragraph 116, Brandt teaches feature detection and text around the features.); and
PNG
media_image2.png
367
638
media_image2.png
Greyscale
Brandt does not explicitly disclose generating a fused feature of the linguistic feature and the visual feature using an attention technique based on the linguistic feature and the visual feature, wherein the attention technique includes cross-attention, wherein the fused feature includes the linguistic feature generated by propagating the visual feature to the linguistic feature and the visual feature generated by propagating the linguistic feature to the visual feature.
Li is in the same field of art of neural network feature detection . Further, Li teaches generating a fused feature of the linguistic feature and the visual feature using an attention technique based on the linguistic feature and the visual feature (the Examiner interprets that the linguistic features can be text input data, since the claim does not define what “linguistic features is”, page 11, paragraph 36, page 14, paragraph 42, Li teaches using textual query and video image frames as input to search for images.),
wherein the attention technique includes cross-attention (page 17, paragraph 50, Li teaches using a local attention and self-attention for the different layers of the image data.), wherein the fused feature includes the linguistic feature generated by propagating the visual feature to the linguistic feature and the visual feature generated by propagating the linguistic feature to the visual feature (paragraph 58, page 20-paragraph 61, page 22, Li teaches using both the text features and the frame attention of the image to determine the similar image data.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Brandt by incorporating the attention maps for the features in the image space using a neural network that is taught by Li, to make the invention that captures image feature and generate text about the features and other areas image (Brandt) and uses the attention map and other maps and combining the images and text features for image or video data; thus, one of ordinary skilled in the art would be motivated to combine the references since there is a need for an efficient and accurate mechanism for
identifying one or more video clips in a dataset ( e.g., a photo album of a mobile device) in
response to a textual query. In various embodiments of this application, the textual query and
each image frame of a video clip is converted to a text feature vector and a respective visual
feature vector using encoders of a contrastive language-image pre-training (CLIP) network in
a semantic text-image space (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 3, Brandt in view of Li discloses wherein the attention technique includes cross-attention and self-attention (paragraph 61, page 22. Li).
In regards to Claim 4, Brandt in view of Li discloses wherein the generating of the fused feature includes generating a new fused feature using the attention technique based on the fused feature (page 22, paragraph 61-page 23, paragraph 65, Li teaches textual query and feature attention maps.).
In regards to Claim 6, Brandt in view of Li discloses in the generating of the fused feature, the cross-attention is performed after setting one of the linguistic feature and the visual feature as a giving feature (page 23, paragraph 63-paragraph 65, Li teaches using the text query and well as frame attention block for local attention feature vectors.), setting the other feature as a receiving feature, setting the receiving feature as a query of the cross-attention, and setting the giving feature as a key and a value of the cross-attention (page 23, paragraph 63-paragraph 65, Li teaches query for the same semantic space for based on feature vectors.).
Regarding Claim 8, Brandt teaches a visual-linguistic feature fusion system comprising:
a memory configured to store computer-readable instructions (paragraph 522, Brandt); and
at least one processor configured to execute the instructions, wherein the at least one processor (paragraph 522, Brandt) is configured to execute the instructions to generate a linguistic feature using a text encoder based on text (paragraph 111, Figure 3 item 302),
PNG
media_image1.png
422
671
media_image1.png
Greyscale
generate a visual feature using a video encoder based on a video frame (paragraph 115, paragraph 116, Brandt teaches feature detection and text around the features.).
Brandt does not explicitly discloses generate a fused feature of the linguistic feature and the visual feature using an attention technique based on the linguistic feature and the visual feature,
wherein the attention technique includes cross-attention, wherein the fused feature includes the linguistic feature generated by propagating the visual feature to the linguistic feature and the visual feature generated by propagating the linguistic feature to the visual feature.
Li is in the same field of art of neural network feature detection . Further, Li teaches generate a fused feature of the linguistic feature and the visual feature using an attention technique based on the linguistic feature and the visual feature (the Examiner interprets that the linguistic features can be text input data, since the claim does not define what “linguistic features is”, page 11, paragraph 36, page 14, paragraph 42, Li teaches using textual query and video image frames as input to search for images.),
wherein the attention technique includes cross-attention (page 17, paragraph 50, Li teaches using a local attention and self-attention for the different layers of the image data.), wherein the fused feature includes the linguistic feature generated by propagating the visual feature to the linguistic feature and the visual feature generated by propagating the linguistic feature to the visual feature (paragraph 58, page 20-paragraph 61, page 22, Li teaches using both the text features and the frame attention of the image to determine the similar image data.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Brandt by incorporating the attention maps for the features in the image space using a neural network that is taught by Li, to make the invention that captures image feature and generate text about the features and other areas image (Brandt) and uses the attention map and other maps and combining the images and text features for image or video data; thus, one of ordinary skilled in the art would be motivated to combine the references since there is a need for an efficient and accurate mechanism for
identifying one or more video clips in a dataset ( e.g., a photo album of a mobile device) in
response to a textual query. In various embodiments of this application, the textual query and
each image frame of a video clip is converted to a text feature vector and a respective visual
feature vector using encoders of a contrastive language-image pre-training (CLIP) network in
a semantic text-image space (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 10, Brandt in view of Li discloses wherein the attention technique includes cross-attention and self-attention (paragraph 61, page 22.).
In regards to Claim 11, Brandt in view of Li discloses wherein the at least one processor is configured to additionally perform an operation of generating a new fused feature using an attention technique based on the fused feature (page 22, paragraph 61-page 23, paragraph 65, Li teaches textual query and feature attention maps.).
In regards to Claim 13, Brandt in view of Li discloses wherein the at least one processor is configured to perform the cross-attention after setting one of the linguistic feature and the visual feature as a giving feature (page 23, paragraph 63-paragraph 65, Li teaches using the text query and well as frame attention block for local attention feature vectors.), setting the other feature as a receiving feature, setting the receiving feature as a query of the cross-attention, and setting the giving feature as a key and a value of the cross-attention (page 23, paragraph 63-paragraph 65, Li teaches query for the same semantic space for based on feature vectors.).
Allowable Subject Matter
Claims 7 and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ONEAL R MISTRY whose telephone number is (313)446-4912. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights