DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/12/2025 has been made record of and considered by the examiner.
Claim Objections
Claim 2 is objected to because of the following informalities: minor informality regarding “at least one of a key and/or a query”. “At least one of” and “and/or” is redundant. The claim could be clarified by amending to “at least one of a key or a query.” Appropriate correction is respectfully suggested.
Claim 5 is objected to because of the following informalities: regarding “a subspace of the key.” Claim 2 discloses “a key”, but “a subspace of the key” is ambiguous. The examiner recommends clarifying claim 5 by including something along the lines of “the principal component analysis to the object region corresponding keys to thereby generate a subspace, and then extracts the information of the object region corresponding keys originally included in the original query by projecting the corresponding original query onto the generated subspace,” see [0069]. Appropriate correction is respectfully suggested.
Claim 8 is objected to because of the following informalities: minor informality regarding “acquire the object region information associated which the metadata related to the input information”. The examiner believes “which” to be a typo and should be corrected to: “acquire the object region information associated with the metadata related to the input information.” Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-7, and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (CN116737979A), and further in view of Gordo (US 2018/0260415 A1).
Consider claims 1, 9, and 10, Liu discloses an image retrieval device [claim 9: method] comprising (¶1, 155; “Image and Text Retrieval Method and System Based on Context-Guided Multimodal Association”):
(¶155); and
at least one processor configured to execute the instructions to (¶155; “computer program instructions may be provided to a processor of a general-purpose computer…”):
acquire input information regarding a retrieval (¶149; “Figure 6(B) contains three sets of T→I retrieval examples”);
acquire object region information regarding a region of an object included in images in an image database where the retrieval is performed (¶60-64; “extract salient regions of the image modality in the Faster R-CNN object detection model with a top-down attention mechanism, extract the features of the salient regions, map them to the common embedding space using a fully connected layer, obtain the position vectors of the salient regions… features of these regions are extracted using ResNet-101”);
calculate, as features of the images, image features obtained by integrating the object region information and local features which are extracted from the images (¶60-64; “learn the absolute position features of the regions using the position vectors, integrate the features of each salient region and the position features corresponding to the salient regions, and then use the context-aware unit to capture the context information of the visual regions”; ¶66-69; “we first integrate the features of each region and its corresponding location features, and then use a context-aware unit to capture the contextual information of the visual region”); and
retrieve an image related to the input information from the image database, based on a degree of similarity between the image features and features of the input information (¶139-146; “F(Ii, Si) represents the similarity between the i-th image and the i-th text”; ¶149; “Figure 6(B) contains three sets of T→I retrieval examples”).
Liu fails to explicitly disclose a non-transitory computer readable storage medium storing a program executed by a computer, the program causing the computer to:]
at least one memory configured to store instructions.
In related art, Gordo explicitly discloses an image retrieval device [claim 9: method] comprising (Gordo ¶17; “image retrieval device 10”):
[claim 10: A non-transitory computer readable storage medium storing a program executed by a computer, the program causing the computer to (Gordo ¶16; “non-transitory storage medium (not shown) which stores instructions readable and executable by the electronic processor 12”):]
at least one memory configured to store instructions (Gordo ¶16); and
at least one processor configured to execute the instructions to:
acquire input information regarding a retrieval (Gordo ¶15-17; “input image 20 is drawn from a repository 40 of images in a database or video stream (in which case the images are video frames) which is to be searched for instances of the object of interest”);
acquire object region information regarding a region of an object included in images in an image database where the retrieval is performed (Gordo ¶17-19, 26-27; “Region Proposal Network (RPN) 52 which is trained to localize regions of interest in images… The RPN 52 predicts , for a set of candidate boxes of various sizes and aspects ratio, and at all possible image locations, a score describing how likely each box contains an object of interest. Simultaneously, for each candidate box it performs regression to improve its location”);
retrieve an image related to the input information from the image database, based on a degree of similarity between the image features and features of the input information (Gordo ¶15; “Representations of different images can be then compared using a dot-product or other similarity metric between the vectors representing the images”; ¶16-27; “image retrieval device 10 processes successive input images in turn to identify a collection 42 of images (or video frames) depicting the same object as is shown in the reference image 22”).
Both Liu and Gordo disclose methods/systems for image retrieval. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the conventional computing architecture of Gordo into the image retrieval method/system of Liu to provide the ordinary infrastructure for storing the images.
Consider claim 2, Liu, as modified by Gordo, discloses the claimed invention wherein the at least one processor is configured to determine, based on the object region information, at least one of a key and/or a query used in integrating the local features based on an attention mechanism (Liu ¶39-45, 60-69; Gordo ¶15-17).
Consider claim 3, Liu, as modified by Gordo, discloses the claimed invention wherein the at least one processor is configured to determine the query based on a position of the region indicated by the object region information (Liu ¶39-45, 60-69, Gordo ¶15-19).
Consider claim 4, Liu, as modified by Gordo, discloses the claimed invention wherein the at least one processor is configured to determine, based on a position of a local region corresponding to the local features, the key corresponding to the local features (Liu ¶39-45, 60-69, 148-149, Gordo ¶15-19, 26-27).
Consider claim 6, Liu, as modified by Gordo, discloses the claimed invention wherein the at least one processor is configured to acquire the object region information regarding the region of the object selected based on the input information (Liu ¶39-45, 60-69, 148-149, Gordo ¶15-19, 26-27).
Consider claim 7, Liu, as modified by Gordo, discloses the claimed invention wherein the at least one processor is configured to acquire the object region information corresponding to the image from a database which stores the object region information (Liu ¶360-69, 148-149, Gordo ¶13-19, 26-27, 34).
Allowable Subject Matter
Claims 5 and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Relevant Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
JP7366204B1 (from IDS) discloses text-based image search and extraction methods.
CN112966137A discloses an image retrieval method and system based on global and local feature rearrangement.
CN114860974A discloses a remote sensing image retrieval locating method.
Tran (‘KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All’).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHLEY HYTREK whose telephone number is (703)756-4562. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steve Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASHLEY HYTREK/Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights