DETAILED ACTION
Response to Arguments
Applicant has amended claims 1, 7-8, and 11-12; and canceled claims 2 and 13, leaving claims 1,3-9, 11-12, 14-15, and 17-19 currently pending.
Applicant's arguments filed 09/15/2025 have been fully considered. Applicant’s arguments with respect to Obviousness Double Patenting are not persuasive and the rejection is made in view of Rymkowski (Pub. No. US20180082407A1) and Ruixing (Pub. No. CN108629747).
Applicant’s arguments, filed 09/15/2025, with respect to 35 USC § 101 rejection have been fully considered and are persuasive. The 35 USC § 101 rejection of 1,3-9, 11-12, 14-15, and 17-19 has been withdrawn.
Applicant's arguments with respect to 35 USC § 103 rejection have been considered have been fully considered but they are not persuasive. Specifically applicant argues that previously applied references do not disclose “wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference image group, visual sentiment labels assigned to each reference images, reference style data of each reference images of the at least one reference image group, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images.” Examiner respectfully disagrees. Rymkowski discloses utilizing artistic style and classified scenes (semantic labels) for reference images (Rymkowski ¶42). Rymkowski further discloses that pretrained neural networks for reference style can be utilized rather than an optimization technique (Rymkowski ¶43). Therefore the internal neural network parameters are used based on which reference image is selected by the user. Therefore the broadest reasonable interpretation of the claim limitations are considered to be taught by the previously cited prior art. As such, this action is made FINAL.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1,3-5, 7-9, 11-12 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1,6, 8-13 of U.S. Patent No. 11887215 B2 in view of Rymkowski (Pub. No. US20180082407A1) and Ruixing (Pub. No. CN108629747 see attached translation for line numbers). Although the claims at issue are not identical, they are not patentably distinct from each other because the claimed invention of US Patent No. 11887215 B2 obviously encompasses the present claimed invention and differ only in the terminology. The chart below highlights differences between method and apparatus claims 1-5, and 7-12 of the instant application and method and apparatus claims of US Patent No. 11887215 B2 1, 6, 8-13.
Claims of Instant Application 18/524905
Claims of US Patent 11887215B2
Claim 1
An image processing method comprising:
obtaining data related to at least one reference image group selected by a user from a plurality of reference image group, wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference group and reference style data of each reference images of the at least one reference group, visual sentiment labels assigned to each reference images, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images;
obtaining a first label of a first image by inputting the first image to a recognition model, wherein an internal parameter of recognition model is updated based on the internal parameter information of the selected at least one reference group;
Selecting as reference style data for a target reference image reference style data of at least one reference image, the at least one reference image being assigned a visual sentiment label corresponding to the first label from among visual sentiment labels assigned to the reference images included in the selected at least one reference image group;
obtaining first style data for the first image by inputting the first image to a feature extraction model wherein the feature extraction model comprises at least one convolution layer for extracting a feature map of the first image
generating second style data by inputting the first style data for the first image and the obtained reference style data to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution laver for inputting the feature map of the first image and reference feature map corresponding to the reference style data to obtain a style-transformed feature map corresponding to the second style data;
and generating a second image based on the second style data.
Claim 1 + Claim 11 (italicized for clarity)
An image processing method comprising:
obtaining a plurality of labels of a first image and probability values respectively corresponding to the plurality of labels by inputting the first image to a recognition model; for each of at least one candidate reference image, to which a same one or more visual sentiment label as one or more of the plurality of labels of the first image are assigned calculating a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the one or more of the plurality of labels as the degree of relevance;
obtaining, based on the calculated degree of relevance, reference style data for a target reference image among the at least one candidate reference image;
obtaining the first style data for the first image by inputting the first image to a feature extraction model;
generating second style data based on first style data for the first image and the obtained reference style data;
and generating a second image based on the generated second style data.
generating a second image by inputting the generated second style data to a feature synthesis model.
Claim 3
changing internal parameters of the recognition model to internal parameters corresponding to the selected at least one reference image group; and inputting the first image to the recognition model that has the changed internal parameters.
Claim 8
further comprising: changing internal parameters of the recognition model to internal parameters corresponding to a reference image group selected by a user from among a plurality of reference image groups; and inputting the first image to the recognition model that has the changed internal parameters.
Claim 4
wherein the plurality of reference image groups are classified according to a creator of reference images.
Claim 9
wherein the plurality of reference image groups are classified according to a creator of reference images.
Claim 5
receiving a reference image group list representing the plurality of the reference image group from a server; receiving a selection, from a user, of the at least one reference image group in the reference image group list; and receiving, from the server, the data related to the at least one reference image group.
Claim 10
connecting to a server and receiving a reference image group list from the server; receiving a selection, from a user, of at least one reference image group in the reference image group list; and receiving, from the server, internal parameters of the recognition model trained based on reference images included in the selected at least one reference image group, reference style data for the reference images included in the at least one reference image group, and visual sentiment labels assigned to the reference images included in the at least one reference image group.
Claim 7
wherein the obtaining of the reference style data for the target reference image comprises: selecting the target reference image assigned a same visual sentiment label as the first label among visual sentiment labels pre-assigned to the reference images included in the selected at least one reference image group.
Claim 6
wherein the obtaining of the reference style data for the target reference image comprises: identifying the at least one candidate reference image to which the same visual sentiment label as the label of the region of interest is assigned; calculating a the degree of relevance to the first image for each of the at least one candidate reference image; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 8
wherein the obtaining of the reference style data for the target reference image comprises: calculating a degree of relevance to the first image for each of at least one candidate reference image to which a same visual sentiment label as the first label is assigned; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 6
wherein the obtaining of the reference style data for the target reference image comprises: identifying the at least one candidate reference image to which the same visual sentiment label as the label of the region of interest is assigned; calculating a the degree of relevance to the first image for each of the at least one candidate reference image; and selecting, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image.
Claim 9
wherein the obtaining of the first label of the first image comprises: obtaining a plurality of labels of the first image and probability values respectively corresponding to the plurality of labels, and wherein the calculating of the degree of relevance comprises: for each of the at least one candidate reference image to which same one or more visual sentiment labels as one or more of the plurality of labels of the first image are assigned, calculating an average of probability values respectively corresponding to the one or more labels as the degree of relevance.
Claim 1
An image processing method comprising: obtaining a plurality of labels of a first image and probability values respectively corresponding to the plurality of labels by inputting the first image to a recognition model; for each of at least one candidate reference image, to which a same one or more visual sentiment label as one or more of the plurality of labels of the first image are assigned calculating a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the one or more of the plurality of labels as the degree of relevance; obtaining, based on the calculated degree of relevance, reference style data for a target reference image among the at least one candidate reference image; generating second style data based on first style data for the first image and the obtained reference style data; and generating a second image based on the generated second style data.
Claim 11
A program stored in a medium to perform the image processing method of claim 1.
Claim 12
A program stored in a medium to perform the image processing method of claim 1.
Claim 12
An image processing apparatus comprising: at least one processor; and a memory storing a recognition model and at least one program, wherein the at least one processor is configured to execute the at least one program to:
obtain data related to at least one reference image group selected by a user from a plurality of reference image group,
obtain a first label of a first image by inputting the first image to a recognition model,
obtain reference style data for a target reference image related to the first label among reference images included in the selected at least one reference image group, and generate a second image based on first style data for the first image and the obtained reference style data,
wherein the data related to at least one reference image group includes identification of a creator of the at least one reference group.
Claim 13
An image processing apparatus comprising: a processor; and a memory storing a recognition model and at least one program, wherein the processor is configured to execute the at least one program to:
obtain a plurality of labels of a first image and probability values respectively corresponding to the plurality of labels by inputting the first image to the recognition model, for each of at least one candidate reference image, to which a same one or more visual sentiment label as one or more of the plurality of labels of the first image are assigned, calculate a degree of relevance to the first image by calculating an average of probability values respectively corresponding to the one or more of the plurality of labels as the degree of relevance, obtain, based on the calculated degree of relevance, reference style data for a target reference image visual sentiment labels pre-assigned to reference images among the at least one candidate reference image, generate second style data based on first style data for the first image and the obtained reference style data, and generate a second image based on the generated second style data.
Claim 1 + Claim 11 of US Patent 11887215B2 does not explicitly disclose or fairly suggest “obtaining data related to at least one reference image group selected by a user from a plurality of reference image group, wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference group and reference style data of each reference images of the at least one reference group, visual sentiment labels assigned to each reference images, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images…wherein an internal parameter of recognition model is updated based on the internal parameter information of the selected at least one reference group.”
Rymkowski, however, discloses obtain data related to at least one reference image group selected by a user from a plurality of reference image group, (Rymkowski ¶42; an artistic style can be chosen, from a list of artists (a reference group).)wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference image group, reference style data of each reference images of the at least one reference image group, (Rymkowski ¶42; artistic style (reference style) from a list of artists (creator associated) is chosen.) visual sentiment labels assigned to each reference images, (Rymkowski ¶42; classifying a scene is disclosed. The example used is a sunset. Therefore labels based on content are known.) and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images, (Rymkowski ¶42-43; classification of the scene can lead to better style transfer results. Utilizing pretrained style-specific neural networks are disclosed. The pretrained network would include internal parameters specific to the style/ visual sentiment of the reference images.) wherein an internal parameter of recognition model is updated based on the internal parameter information of the selected at least one reference group, (Rymkowski ¶42-43; classification of the scene can lead to better style transfer results. Utilizing pretrained style-specific neural networks are disclosed. The pretrained network would include internal parameters specific to the style/ visual sentiment of the reference images.)
Therefore it would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify claim 1+ claim 11 of US Patent 11887215B2 by including the artist of the image style as part of the style data, as in Rymkowski, in order to provide the user straightforward selection options for the desired style they want for their images.
Claim 1 + Claim 11 of US Patent 11887215B2 in view of Rymkowski does not explicitly disclose or fairly suggest” wherein the feature extraction model comprises at least one convolution layer for extracting a feature map of the first image…to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution laver for inputting the feature map of the first image and reference feature map corresponding to the reference style data to obtain a style-transformed feature map corresponding to the second style data”
Ruixing, however, discloses wherein the feature extraction model comprises at least one convolution layer for extracting a feature map of the first image, generate second style data by inputting the first style data for the first image and the obtained reference style data (Ruixing p. 6 liens 5-9 and Fig. 5; inputting the target image and reference image into the style migration enhancement network is disclosed. Getting style data can be seen in Fig. 5 where images are input into the network and feature maps are created in the convolutional layers (see right side of figure).) to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution layer for inputting the feature map of the first image and reference feature map corresponding to the reference style data to obtain a style-transformed feature map corresponding to the second style data, and generate a second image based on the second style data. (Ruixing p. 5 lines 48-57 and p. 4 lines 57-59; style data from the refence image is transferred to the style of matching semantic regions of the target image (first image). The style migration network includes convolution layers. Meaning a new image (second image is created) by synthesizing the style of the reference image with the content of the input image. Fig. 5 also shows the generated image feature map being deconvolved (see arrow pointing left on top right side of image).)
Therefore it would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify claim 1+ claim 11 of US Patent 11887215B2 in view of Rymkowski by including convolutional layers in style transfer, as in Ruixing, in order to a low processing power way to obtain style transferred images.
Claim 12 of the instant application is the corresponding image processing apparatus claim to claim 1 and is rejected for similar reasons for non-statutory double patenting as not being patentably distinct from claim 13 of US Patent 11887215B2.
Claim Objections
Claim 1, 5, 11, 12 and 15 are objected to because of the following informalities:
Claim 12
Line 9 “a plurality of reference image group” should be “a plurality of reference image groups”. Appropriate correction is required.
Line 32-33 “reference feature map corresponding to the reference style data” should be “a reference feature map corresponding to the reference style data”
Independent claims 1 and 11 are objected to for similar reasons as claim 12.
Claim 15 Lines 5-6 recites “the plurality of the reference image group” should be “the plurality of the reference image groups”
Claim 5 is objected to for similar reasons to claim 15.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 12 recites the following limitations which make the claim unclear:
Lines 11-12 and 13 recite the limitation “each reference images” this limitation appears to lack antecedent basis.
Lines 14, 16 recites “a recognition model” for proper antecedent basis these should recite “the recognition model”
Line 17 recites “recognition model” should be “the recognition model”
Line 22 “reference style data” should be “the reference style data”
Line 24 the limitation “the first label from among visual sentiment labels assigned to the reference images” is unclear if it is referring to the same “a first label” in line 16 or not. For purposes of examination it is taken to mean the same reference label in line 16.
Line 30 recites “the obtained reference style data” should be “the selected reference style data.”
Independent claims 1 and 11 have similar issues and are rejected for similar reasons.
Claims 3-9 and 14-15, 17-19 are rejected for being dependent on a rejected claim.
Claim 14 recites the following limitation which makes the claim unclear: Line 5 recites “change internal parameters” it is unclear if the same internal parameters are meant that were already claimed in claim 12. For purposes of examination they are assumed to the be the same parameters as claimed in 12.
Claim 3 is rejected for similar reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 6-8, 11-12, 14, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ruixing (Pub. No. CN108629747 see attached translation for line numbers) in view of Rymkowski (Pub. No. US20180082407A1).
Regarding claim 12, Ruixing discloses An image processing apparatus comprising: : a memory storing a recognition model and at least one program one or more computer programs; and one or more processors communicatively coupled to the memory, wherein one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the image processing apparatus to (Ruixing p. 2 lines 9-11; a device with a processor and memory with instructions stored to be implemented by the processor is disclosed.) obtain a first label of a first image by inputting the first image to a recognition model, (Ruixing p. 4 lines 52-55; the target image (first image) is semantically segmented (i.e. labeled).) select, as reference style data for a target reference image, reference style data of at least one reference image, the at least one reference image being assigned a visual sentiment label corresponding to the first label (Ruixing p. 4 lines 47-50; the reference image is selected by having as similar as possible semantic information. The hue/style is received as well.) obtain first style data for the first image by inputting the first image to a feature extraction model, wherein the feature extraction model comprises at least one convolution layer for extracting a feature map of the first image, generate second style data by inputting the first style data for the first image and the obtained reference style data (Ruixing p. 6 liens 5-9 and Fig. 5; inputting the target image and reference image into the style migration enhancement network is disclosed. Getting style data can be seen in Fig. 5 where images are input into the network and feature maps are created in the convolutional layers (see right side of figure).) to a feature synthesis model, wherein the feature synthesis model comprises at least one deconvolution layer for inputting the feature map of the first image and reference feature map corresponding to the reference style data to obtain a style-transformed feature map corresponding to the second style data, and generate a second image based on the second style data. (Ruixing p. 5 lines 48-57 and p. 4 lines 57-59; style data from the refence image is transferred to the style of matching semantic regions of the target image (first image). The style migration network includes convolution layers. Meaning a new image (second image is created) by synthesizing the style of the reference image with the content of the input image. Fig. 5 also shows the generated image feature map being deconvolved (see arrow pointing left on top right side of image).)
Although Ruixing describes enhancing images with night scene effects, where a user can manually select a style template (Ruixing p. 1 lines 23-28), Ruixing does not disclose obtain data related to at least one reference image group selected by a user from a plurality of reference image group, wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference image group, reference style data of each reference images of the at least one reference image group, visual sentiment labels assigned to each reference images, and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images, … wherein an internal parameter of recognition model is updated based on the internal parameter information of the selected at least one reference group, …from among visual sentiment labels assigned to the reference images included in the selected at least one reference image group
Rymkowski, however, discloses obtain data related to at least one reference image group selected by a user from a plurality of reference image group, (Rymkowski ¶42; an artistic style can be chosen, from a list of artists (a reference group).)wherein the data related to at least one reference image group includes identification information of a creator of the at least one reference image group, reference style data of each reference images of the at least one reference image group, (Rymkowski ¶42; artistic style (reference style) from a list of artists (creator associated) is chosen.) visual sentiment labels assigned to each reference images, (Rymkowski ¶42; classifying a scene is disclosed. The example used is a sunset. Therefore labels based on content are known.) and internal parameter information of a recognition model for outputting the visual sentiment labels from the reference images, (Rymkowski ¶42-43; classification of the scene can lead to better style transfer results. Utilizing pretrained style-specific neural networks are disclosed. The pretrained network would include internal parameters specific to the style/ visual sentiment of the reference images.) wherein an internal parameter of recognition model is updated based on the internal parameter information of the selected at least one reference group, (Rymkowski ¶42-43; classification of the scene can lead to better style transfer results. Utilizing pretrained style-specific neural networks are disclosed. The pretrained network would include internal parameters specific to the style/ visual sentiment of the reference image.) from among visual sentiment labels assigned to the reference images included in the selected at least one reference image group, (Rymkowski ¶42; styles can be developed based on paintings of artists with related content to the input image.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the image processing apparatus of Ruixing with the teachings of Rymkowski by including image style group selection with associated creator and reference style incorporation with similar semantic labels and associated pretrained neural network as disclosed in Rymkowski in order to provide the user with multiple style types to choose from while keeping the advantage of more visually pleasing results when using a reference image with similar content and lower processing power (Rymkowski ¶42 and ¶33).
Regarding claim 14, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 12, as described above. They further disclose wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the image processing apparatus to: change internal parameters of the recognition model to internal parameters corresponding to the selected at least one reference image group, and (Rymkowski ¶33, 39, and 43; style specific neural networks are pre-trained (i.e. the reference group are input and the style is learned (internal parameters are changed) to allow style transfer for the selected reference video). Wherein it would have been obvious to include image style groups and further train the model using the style images in order to provide the user with multiple style types to choose from, and utilize the users selection.) input the first image to the recognition model that has the changed internal parameters. (Ruixing p. 4 lines 57-59; the style of the reference image is transferred to the target image (first image). Meaning a second image is generated with the content of the target image and style of the reference image.)
Regarding claim 17, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 12, as described above. They further disclose wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the image processing apparatus to: select the target reference image assigned a same visual sentiment label as the first label among visual sentiment labels pre-assigned to the reference images included in the selected at least one reference image group. (Ruixing p. 5 lines 22-32; a reference image is chosen that has the closest semantic similarity to the target image (first image). Meaning the reference image is chosen based on the closest matching content image from the stored references.)
Regarding claims 18, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 12, as described above. They further disclose wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the image processing apparatus to: calculate a degree of relevance to the first image for each of at least one candidate reference image to which a same visual sentiment label as the first label is assigned, and select, based on the calculated degree of relevance, the target reference image from among the at least one candidate reference image. (Ruixing p. 5 lines 22-27; a semantic/content similarity ranking is performed to select the reference image. Meaning semantic labels are matched between the target image and the reference image.)
Regarding claims 1,3, and 7-8; they are the corresponding method claims to 12,14, 17, and 16/18 respectively and are rejected for similar reasons.
Regarding claim 4, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 1, as described above. They further disclose wherein the plurality of reference image groups are classified according to a creator of reference images. , (Rymkowski ¶42; artistic style (reference style) from a list of artists (creator associated) is chosen. Wherein it would have been obvious to include image style groups in order to provide the user with multiple style types to choose from.)
Regarding claim 6, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 1, as described above. They further disclose displaying, on a display, a reference image group list representing the plurality of the reference image group; and receiving a selection, from a user, of the at least one reference image group in the reference image group list. (Ruixing p.13 lines 12-22; discloses user interaction using a display.)
Regarding claim 11, it is the corresponding one or more non-transitory computer-readable storage media claim to claim 12 and is rejected for similar reasons. (Ruixing p. 2 lines 9-11; a memory with computer executable instructions is disclosed.)
Claim(s) 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ruixing (Pub. No. CN108629747 see attached translation for line numbers) in view of Rymkowski (Pub. No. US20180082407A1), and Newman (Pub. No. US20190208124A1).
Regarding claim 15, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 12, as described above.
The combination of Ruixing and Rymkowski do not explicitly disclose
Newman, however, discloses wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the image processing apparatus to: receive a reference image group list representing the plurality of the reference image group from a server, (Newman ¶[0081] a server is disclosed which can operate in conjunction with a computing system to perform the disclosed methods.) receive a selection, from a user, of the at least one reference image group in the reference image group list, and(Newman ¶ [0078]; user selected cinematic styles are disclosed. This is considered a group, because a video would contain multiple images to base the style off of.) receive, from the server, the data related to the at least one reference image group. (Newman ¶[0023]; the program is trained using cinematic input to learn the cinematic style ( i.e. videos (groups) are input and the style is learned (internal parameters are changed) to allow style transfer for the selected reference video). Wherein it would have been obvious to include image style groups received from a server and further train the model using the style images in order to provide the user with multiple style types to choose from, and utilize the users selection.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the image processing apparatus of Ruixing and Rymkowski with the teachings of Newman by including style selection from a server as disclosed in Newman in order to provide the user with a greater variety of styles than could be stored on a local device.
Regarding claim 5, it is the corresponding method claim to claim 15 and is rejected for similar reasons.
Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ruixing (Pub. No. CN108629747 see attached translation for line numbers) in view of Rymkowski (Pub. No. US20180082407A1), and in further view of Lennon (Patent No. US6718063B1).
Regarding claim 19, the combination of Ruixing and Rymkowski discloses the claim limitations with regards to claim 18, as described above. wherein the data related to at least one reference image group comprises at least one of internal parameters of the recognition model trained based on reference images included in the selected at least one reference image group, the reference style data for the reference images included in the at least one reference image group, and the labels assigned to the reference images included in the at least one reference image group based on the selection of the at least one reference image group. (Rymkowski ¶33, 39, and 43; style specific neural networks are pre-trained (i.e. the reference group are input and the style is learned (internal parameters are changed) to allow style transfer for the selected reference video). Wherein it would have been obvious to include image style groups and further train the model using the style images in order to provide the user with multiple style types to choose from, and utilize the users selection.)
The combination of Ruixing and Rymkowski does not explicitly disclose wherein the at least one processor is further configured to execute the at least one program to: obtain a plurality of labels of the first image and probability values respectively corresponding to the plurality of labels, and for each of the at least one candidate reference image to which same one or more visual sentiment labels as one or more of the plurality of labels of the first image are assigned, calculate an average of probability values respectively corresponding to the one or more labels as the degree of relevance.
Lennon, however, discloses wherein the at least one processor is further configured to execute the at least one program to: obtain a plurality of labels of the first image and probability values respectively corresponding to the plurality of labels, and (Lennon abstract; semantic labels are given probabilities.) for each of the at least one candidate reference image to which same one or more visual sentiment labels as one or more of the plurality of labels of the first image are assigned, calculate an average of probability values respectively corresponding to the one or more labels as the degree of relevance. (Lennon abstract; an average difference between semantic labels of two images is disclosed as a distance metric between two images (degree of relevance).)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the image processing apparatus of the combination of Ruixing and Rymkowski with the teachings of Lennon by including probability values for the semantic labels and the degree of relevance between images as disclosed in Lennon in order to provide similarity measures between images in more than just color and texture, but also specific content (Lennon col.2 lines 9-33).
Regarding claim 9, it is the corresponding method claim to claim 19 and is rejected for similar reasons.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5. Examiner’s email is Meredith.Taylor@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached on (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights