DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/23/2025 has been entered.
Response to Arguments
Applicant has amended claims 1, 5, 11-12, and 15; and canceled claims 3, 6, and 14, leaving claims 1,4-5, 7-9, 11-12, 15, and 17-19 currently pending.
Applicant's arguments filed 12/23/2025 have been fully considered. Applicant’s arguments with respect to Obviousness Double Patenting are not persuasive and the rejection is made in view of Rymkowski (Pub. No. US20180082407A1) and Ruixing (Pub. No. CN108629747).
Applicant’s arguments, with respect to amendments overcoming claim objections have been fully considered and are persuasive. Therefore objections of claims 1, 5, 11, 12, and 15 have been withdrawn. However, in light of the amendment new objections are made to claims 1, 4, 8, 11, 12, and 18 as is detailed below.
Applicant’s arguments, with respect to 35 USC § 112(b) rejection have been fully considered and are persuasive. The previous 35 USC § 112(b) rejections of claims 1, 4-5, 7-9, 11-12, 15, and 17-19 have been withdrawn. However, upon further review, a new rejection is made as is detailed below.
Applicant's arguments with respect to 35 USC § 103 rejection have been considered have been fully considered. Applicant’s argument that in Rymkowski “there is not mention of changing internal parameters of the recognition model based on the obtained internal parameter information” is persuasive. Therefore the rejection is withdrawn.
Examiner would like to note that in order to facilitate compact prosecution, an attempt was made to contact applicant’s representative and propose an examiner’s amendment. However, applicant’s representative indicated a preference for a written rejection.
As such, this action is made NON-FINAL.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 12/8/2025 and 6/5/2026 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) has/have been considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1,4-5, 7-9, 11-12 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1,6, 8-13 of U.S. Patent No. 11887215 B2 in view of Rymkowski (Pub. No. US20180082407A1) and Ruixing (Pub. No. CN108629747 see attached translation for line numbers). Although the claims at issue are not identical, they are not patentably distinct from each other because the claimed invention of US Patent No. 11887215 B2 obviously encompasses the present claimed invention and differ only in the terminology. The chart below highlights differences between method and apparatus claims 1, 4-5, and 7-9, and 11-12 of the instant application (bold and underlined represent differences) and method and apparatus claims of US Patent No. 11887215 B2 1, 6, 8-13 (bolded represent similarities).
Claims of Instant Application 18/524905
Claims of US Patent 11887215B2
Claim 1
An image processing method comprising:
displaying, on a display, a plurality of reference image groups;
based on a receiving a user selection of at least one reference image group from among the plurality of reference image groups, obtaining data related to the at least one reference image group, wherein the data related to the at least one reference image group includes identification information of a creator of the at least one reference group and reference style data of each reference image of the at least one reference group, visual sentiment labels assigned to each reference image in the plurality of reference image groups, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference image;
changing internal parameters of the recognition model based on the obtained internal parameter information included in the data related to the at least one reference image group;
obtaining a first label of a first image by inputting the first image to recognition model, that has changed internal parameters;
Selecting reference style data for a target reference image from among the obtained reference style data of each reference image, wherein the target reference image represents a reference image to which a visual sentiment label related to the first label of the first image is assigned, from among the obtained visual sentiment labels assigned to each reference image;
obtaining first style data for the first image by inputting the first image to a feature extraction model wherein the feature extraction model comprises at least one convolution layer for extracting a feature data of the first image;
generating second style data by inputting the first style data for the first image and the selected reference style data to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution laver for obtaining a style-transformed feature data corresponding to the second style data based on the feature data of the first image and the reference style data;
and generating a second image based on the second style data.
Claim 1 + Claim 8 (italicized for clarity)
An image processing method comprising:
changing internal parameters of the recognition model to internal parameters corresponding to a reference image group selected by a user from among a plurality of reference image groups; and inputting the first image to the recognition model that has the changed internal parameters.
obtaining a plurality of labels of a first image and probability values respectively corresponding to the plurality of labels by inputting the first image to a recognition model; for each of at least one candidate reference image, to which a same one or more visual sentiment label as one or more of the plurality of labels of the first image are assigned calculating a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the one or more of the plurality of labels as the degree of relevance;
obtaining, based on the calculated degree of relevance, reference style data for a target reference image among the at least one candidate reference image;
generating second style data based on first style data for the first image and the obtained reference style data;
and generating a second image based on the generated second style data.
Claim 4
wherein the plurality of reference image groups are classified according to a creator of reference images.
Claim 9
wherein the plurality of reference image groups are classified according to a creator of reference images.
Claim 5
receiving a reference image group list representing the plurality of the reference image groups from a server; receiving a selection, from a user, of the at least one reference image group in the reference image group list; and receiving, from the server, the data related to the at least one reference image group.
Claim 10
connecting to a server and receiving a reference image group list from the server; receiving a selection, from a user, of at least one reference image group in the reference image group list; and receiving, from the server, internal parameters of the recognition model trained based on reference images included in the selected at least one reference image group, reference style data for the reference images included in the at least one reference image group, and visual sentiment labels assigned to the reference images included in the at least one reference image group.
Claim 7
wherein the obtaining of the reference style data for the target reference image comprises: selecting the target reference image assigned a same visual sentiment label as the first label among visual sentiment labels pre-assigned to the reference images included in the selected at least one reference image group.
Claim 6
wherein the obtaining of the reference style data for the target reference image comprises: identifying the at least one candidate reference image to which the same visual sentiment label as the label of the region of interest is assigned; calculating a the degree of relevance to the first image for each of the at least one candidate reference image; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 8
wherein the obtaining of the reference style data for the target reference image comprises: calculating a degree of relevance to the first image for each of at least one candidate reference image to which a same visual sentiment label as the first label is assigned; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 6
wherein the obtaining of the reference style data for the target reference image comprises: identifying the at least one candidate reference image to which the same visual sentiment label as the label of the region of interest is assigned; calculating a the degree of relevance to the first image for each of the at least one candidate reference image; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 9
wherein the obtaining of the first label of the first image comprises: obtaining a plurality of labels of the first image and probability values respectively corresponding to the plurality of labels, and wherein the calculating of the degree of relevance comprises: for each of the at least one candidate reference image to which same one or more visual sentiment labels as one or more of the plurality of labels of the first image are assigned, calculating an average of probability values respectively corresponding to the one or more labels as the degree of relevance.
Claim 1
An image processing method comprising: obtaining a plurality of labels of a first image and probability values respectively corresponding to the plurality of labels by inputting the first image to a recognition model; for each of at least one candidate reference image, to which a same one or more visual sentiment label as one or more of the plurality of labels of the first image are assigned calculating a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the one or more of the plurality of labels as the degree of relevance; obtaining, based on the calculated degree of relevance, reference style data for a target reference image among the at least one candidate reference image; generating second style data based on first style data for the first image and the obtained reference style data; and generating a second image based on the generated second style data.
Claim 11
One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions that, when executed by one or more processors of an electronic device individually or collectively, cause the electronic device to perform operations, the operations comprising:
(additional limitations correspond to claim 1, see chart above)
Claim 12
A program stored in a non-transitory computer readable medium to perform the image processing method of claim 1.
Claim 12
An image processing apparatus comprising: at least one processor; and a memory storing a recognition model and at least one program, wherein the at least one processor is configured to execute the at least one program to:
(additional limitations correspond to claim 1, see chart above)
Claim 13
An image processing apparatus comprising: a processor; and a memory storing a recognition model and at least one program, wherein the processor is configured to execute the at least one program to:
(additional limitations correspond to claim 1, see chart above)
Claim 1 + Claim 8 of US Patent 11887215B2 does not explicitly disclose or fairly suggest “displaying, on a display, a plurality of reference image groups; based on a receiving a user selection of at least one reference image group from among the plurality of reference image groups, obtaining data related to the at least one reference image group, wherein the data related to the at least one reference image group includes identification information of a creator of the at least one reference group and reference style data of each reference image of the at least one reference group, visual sentiment labels assigned to each reference image in the plurality of reference image groups, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference image;”
Rymkowski, however, discloses displaying, on a display, a plurality of reference image groups; (Rymkowski ¶52-53; a display is used to interact with a user to obtain their input; ¶42 an artistic style can be chosen from a list of artists. Therefore a list would be displayed.) based on a receiving a user selection of at least one reference image group from among the plurality of reference image groups, obtaining data related to the at least one reference image group, (Rymkowski ¶42; an artistic style can be chosen, from a list of artists (a reference group).) wherein the data related to the at least one reference image group includes identification information of a creator of the at least one reference group and reference style data of each reference image of the at least one reference group, (Rymkowski ¶42; artistic style (reference style) from a list of artists (creator associated) is chosen.) visual sentiment labels assigned to each reference image in the plurality of reference image groups, (Rymkowski ¶42; classifying a scene is disclosed. The example used is a sunset. Therefore labels based on content are known.) and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference image (Rymkowski ¶42-43; classification of the scene can lead to better style transfer results. Utilizing pretrained style-specific neural networks are disclosed. The pretrained network would include internal parameters specific to the style/ visual sentiment of the reference images.)
Therefore it would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify claim 1+ claim 8 of US Patent 11887215B2 by including the artist of the image style as part of the style data, as in Rymkowski, in order to provide the user straightforward selection options for the desired style they want for their images.
Claim 1 + Claim 8 of US Patent 11887215B2 in view of Rymkowski does not explicitly disclose or fairly suggest “obtaining first style data for the first image by inputting the first image to a feature extraction model wherein the feature extraction model comprises at least one convolution layer for extracting a feature data of the first image;… to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution laver for obtaining a style-transformed feature data corresponding to the second style data based on the feature data of the first image and the reference style data;”
Ruixing, however, discloses obtaining first style data for the first image by inputting the first image to a feature extraction model (Ruixing p. 4 lines 52-55; the target image (first image) is semantically segmented (i.e. labeled).)
wherein the feature extraction model comprises at least one convolution layer for extracting a feature data of the first image; (Ruixing p. 6 liens 5-9 and Fig. 5; inputting the target image and reference image into the style migration enhancement network is disclosed. Getting style data can be seen in Fig. 5 where images are input into the network and feature maps are created in the convolutional layers (see right side of figure).) to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution laver for obtaining a style-transformed feature data corresponding to the second style data based on the feature data of the first image and the reference style data; (Ruixing p. 5 lines 48-57 and p. 4 lines 57-59; style data from the refence image is transferred to the style of matching semantic regions of the target image (first image). The style migration network includes convolution layers. Meaning a new image (second image is created) by synthesizing the style of the reference image with the content of the input image. Fig. 5 also shows the generated image feature map being deconvolved (see arrow pointing left on top right side of image).)
Therefore it would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify claim 1+ claim 8 of US Patent 11887215B2 in view of Rymkowski by including convolutional layers in style transfer, as in Ruixing, in order to a low processing power way to obtain style transferred images.
Claim 11 of the instant application is the corresponding non-transitory computer readable medium claim to claim 1 and is rejected for similar reasons for non-statutory double patenting as not being patentably distinct from claim 12 of US Patent 11887215B2.
Claim 12 of the instant application is the corresponding image processing apparatus claim to claim 1 and is rejected for similar reasons for non-statutory double patenting as not being patentably distinct from claim 13 of US Patent 11887215B2.
Claim Objections
Claim 1, 4, 8, 11, 12, and 18 are objected to because of the following informalities:
Claim 1 line 3 “a receiving a user selection” should be “receiving a user selection”
Claim 4 line 2 “a creator” should be “the creator”
Claim 8 line 5 “a same visual sentiment label” should be “the same visual sentiment label”
Claim 11 line 7 “a receiving a user selection” should be “receiving a user selection”
Claim 12 line 3-4 “at least one program one or more computer programs;” should be “one or more computer programs;”
Line 10 “a receiving a user selection” should be “receiving a user selection”
Claim 18 line 6 “a same visual sentiment label” should be “the same visual sentiment label”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 12 recites the following limitations which make the claim unclear:
Lines 10-11 recite the limitation “a recognition model for outputting the visual sentiment labels from the reference image;” it is unclear what reference image is being referred to, it is additionally unclear if the reference image is being used as an input to the recognition model or if labels from the reference image are being output or if it is being used by the model in some other way. Examiner suggests “a recognition model for outputting , trained using the reference images of the reference image group
Independent claims 1 and 11 have similar issues and are rejected for similar reasons.
Claims 4-5, 7-9, 15 and 17-19 are rejected for being dependent on a rejected claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5. Examiner’s email is Meredith.Taylor@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached on (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights