Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: Step 308 in paragraph 0057, Step 302 in paragraph 0060. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2, 7-9, 11-12, 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over BIAN (US 2023/0133981) in view of Zhang (US 2024/0386621).
As to claim 1, BIAN discloses a computer-implemented method for content understanding model and content generation model training, comprising:
separately training a content understanding model and a content generation model with an image-text pair formed by an image and a text in a target training set, wherein the content understanding model is used to generate an image description text based on an input image, and wherein the content generation model is used to generate a corresponding image based on an input description text (para. 0066, 0090);
performing sample processing on sample set (para. 0026, 0049), and the sample processing comprises:
inputting a first image in any first image-text pair in the sample set into the content understanding model to obtain several candidate texts (para. para. 0027, 0030, 0035, 0041, 0042);
separately inputting a first text in the first image-text pair and the several candidate texts into the content generation model to obtain multiple candidate images (0048, 0049, 0052);
performing similarity matching between the multiple candidate images and the first image, and determining a target text based on a matching result (para. 0060,-0071); and
forming a second image-text pair by using the first image and the target text and adding the second image-text pair to the target training set, as an updated target training set, to continue to train the content understanding model and the content generation model (para. 0072-0077).
BIAN is silent regarding performing sample processing on a noise-containing sample set, wherein an image-text matching degree of an image-text pair in the noise-containing sample set is less than that in the target training set.
Zhang teaches performing sample processing on a noise-containing sample set, wherein an image-text matching degree of an image-text pair in the noise-containing sample set is less than that in the target training set (para. 0034, 0038, 0048, 0051, e.g., “the pre-trained multimodal model 120 to produce generated text-image pairs. The generated text-image pairs are then provided to the text-to-image generation model 112 to train and/or implement the model 112. In this way, the text-to-image generation model 112 can be trained and implemented using few or even no manually created text-image pairs”; the generated text-image pairs correspond to “a noise-containing sample set”; an image-text matching degree of the generated text-image pairs is less than that in the target training set of the pre-trained multimodal model, such as the contrastive language-image pre-training (CLIP) model).
As to claim 2, the combination of BIAN and Zhang discloses the computer-implemented method of claim 1, comprising: continuing, using the updated target training set, to train the content understanding model and the content generation model; or continuing to train the content understanding model and the content generation model by using an image-text pair newly added to the target training set (BIAN, para. 0072-0077; Zhang, para. 0034, 0038, 0048, 0051).
As to claim 7, the combination of BIAN and Zhang discloses the computer-implemented method of claim 1, wherein performing similarity matching between the multiple candidate images and the first image, comprises: separately inputting the multiple candidate images and the first image into an image encoder to obtain multiple candidate image representations and a first image representation (BIAN, para. 0063, 0066, 0090; Zhang, para. 0034, 0044, 0051); and separately performing similarity matching between the multiple candidate image representations and the first image representation ((BIAN, para. 0063, 0066, 0090; Zhang, para. 0034, 0044, 0051).
As to claim 8, the combination of BIAN and Zhang discloses the computer-implemented method of claim 7, wherein the image encoder is an image encoder of a contrastive language-image pre-training (CLIP) mode (BIAN, para. 0066; Zhang, para. 0034, 0051).
As to claim 9, the combination of BIAN and Zhang discloses the computer-implemented method of claim 1, wherein determining a target text based on a matching result, comprises: determining a candidate image having a highest similarity with the first image as a first target image (BIAN, para. 0063, 0066, 0090; Zhang, para. 0034, 0044, 0051); and determining, as the target text, a text used when the first target image is generated (BIAN, para. 0063, 0066, 0090; Zhang, para. 0034, 0044, 0051).
As to claims 11-12, 17-20, these claims recite features similar to those discussed above. Therefore, they are rejected for reasons similar to those discussed above.
Allowable Subject Matter
Claims 3-6, 10, 13-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art discloses the claim limitations discussed above, but fails to disclose the combined features required by each of dependent claims 3, 10, 13.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
LI et al. disclose a method for generating a target object. The method includes: generating a first discrete encoded sequence corresponding to an original object by performing discrete encoding on the original object, in which the original object is of an image type, a text type, or a text-image-combined type; obtaining a second discrete encoded sequence by inputting the first discrete encoded sequence into a generative model; generating a target object based on the second discrete encoded sequence, in which the target object is of an image type or a text type, and when the original object is of an image type, the target object is of a text type, and when the original object is of a text type, the target object is of an image type.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUOC TRAN whose telephone number is (571)272-7399. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUOC TRAN/Primary Examiner, Art Unit 2668
Prosecution Insights