DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Application claims priority to foreign application with application number CN 202111641156 dated 12/29/2021. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The information disclosure statements dated 07/02/2024 and 06/16/2025 have been considered and placed in the application file.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11, 12, and 13 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 11 and 12 each recite the limitation "the method", followed by a list of steps. There is insufficient antecedent basis for this limitation in the claims. For purposes of examination, this limitation will be interpreted as “a method” instead of referring to any specific or previously referenced method.
Claim 13 recites both an apparatus and a process of using the apparatus. When both an apparatus and a method are claimed in the same claim, it is unclear whether infringement occurs when the apparatus is constructed or when the apparatus is used. Therefore, the scope of the claim is indefinite. See MPEP 2173.05(p).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-9 and 11-21 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 8 of copending Application No. 18/856,038 (reference application), including its parent claims 1, 4, and 7. Although the claims at issue are not identical, they are not patentably distinct from each other because the instant application claims are broader than the patent claims and are therefore an obvious variant thereof. Since the applications pertain to Chinese text, the term “word” under its broadest reasonable interpretation includes single-character representations as found in Chinese writing.
Current application (18/725,679)
Claim 1
Reference application (18/856,038)
Claims 1, 4, 7, and 8
A method for generating a word, comprising:
(Claim 1) A character processing method, comprising:
(Claim 7) …to determine a target display character image based on a triggering operation.
obtaining images to be processed corresponding to a word to be processed and a reference word respectively;
(Claim 7) receiving a target reference style character image and a target style conversion character image;
and inputting the images to be processed into a target font style fusion model to obtain a target word of the word to be processed in a target font style,
(Claim 4) …obtaining a target style feature fusion model by training a to-be-trained style feature fusion model…
(Claim 7) receiving a target reference style character image and a target style conversion character image
(Claim 8) …based on the target style feature fusion model corresponding to the target display character image.
wherein the target font style is determined based on the target font style fusion model fusing a reference font style of the reference word and a font style to be processed of the word to be processed.
(Claim 4) wherein the target style feature fusion model is configured to fuse at least two font styles.
(Claim 8) …based on the target style feature fusion model corresponding to the target display character image.
18/725,679
1
3
4
5
11
12
13
15
16
17
18/856,038
6
6
6
8
16
16
16
16
16
18
The dependent claims not listed in the above chart are also provisionally rejected on the grounds of nonstatutory double patenting due to their dependence on provisionally rejected claims 1 or 11.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 5, 11-13, 15, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("Chinese Character Style Transfer Based on the Fusion of Multi-scale Content and Style Features". 2021 40th Chinese Control Conference (CCC) (26-28 Jul 2021). https://doi.org/10.23919/CCC52363.2021.9550311, hereinafter "Zhou") in view of Reddy et al. (US 20230110114 A1, hereinafter "Reddy").
Regarding claim 1, Zhou teaches: A method for generating a word, comprising:
obtaining images to be processed corresponding to a word to be processed and a reference word respectively (figs. 1 and 2, content input xc corresponds to “word to be processed”, style input xs corresponds to “reference word”; pg. 8249 col. 1 “One input is called the content input (xc) and the other is called the style input (xs). xc is the source font Chinese characters, and xs is the character with the target style. The output of the model is the Chinese character with the style of xs and the same semantics as xc. The network is mainly divided into four parts, the content image encoding module, the style image encoding module, feature fusion module and the image decoding module.”);
and inputting the images to be processed into a target font style fusion model to obtain a target word of the word to be processed in a target font style (fig. 2 network performs “feature fusion”, producing output image G(xc, xs) with the text content of content image xc and the font style of style image xs).
Zhou does not explicitly teach: wherein the target font style is determined based on the target font style fusion model fusing a reference font style of the reference word and a font style to be processed of the word to be processed.
Reddy teaches: wherein the target font style is determined based on the target font style fusion model fusing a reference font style of the reference word and a font style to be processed of the word to be processed (fig. 8; [0155] “As mentioned above, the glyph generation system 106 can interpolate between two or more font styles to generate glyphs having a new and unknown font style… For example, the glyph generation system 106 interpolates between two known font styles to generate a font style code that has some visual traits of one font style and other visual traits of another font style (e.g., for a font that is halfway between the two font styles).”).
Zhou and Reddy are analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou with the teachings of Reddy to include Reddy’s feature of combining the font styles of both text inputs to produce output with a fused font style. The motivation would have been to be “able to both recreate font having known styles as well as create new fonts having new font styles” (Reddy [0028]), increasing versatility and user control over the output.
Regarding claim 3, the combination of Zhou in view of Reddy teaches: The method of claim 1, wherein the target font style fusion model comprises a font style extraction sub-model (Zhou fig. 2, Content encoder), an image feature extraction sub-model (Zhou fig. 2, Style encoder), and an encoding sub-model (Zhou fig. 2, Decoder), and inputting the images to be processed into a target font style fusion model to obtain a target word of the word to be processed in a target font style comprises:
extracting the reference font style of the reference word based on the font style extraction sub-model (Zhou pg. 8247 “The content encoder and the style encoder can extract the corresponding features of input characters.”);
extracting image features corresponding to the word to be processed based on the image feature extraction sub-model (Zhou pg. 8247 “The content encoder and the style encoder can extract the corresponding features of input characters.”), wherein the image features comprise a content feature and a font style feature to be processed (Reddy teaches training a neural network to recognize both content and style features of a text input:
[0065] “In various implementations, the glyph generation system 106 also learns a font style embedding space (i.e., latent embedding space) as part of the implicit network. For example, the glyph generation system 106 learns the font styles of input glyphs and maps font style codes to the font style embedding space.”
[0075] Further, as described above, in some implementations, the glyph generation system 106 trains the implicit network 310 or uses another model (e.g., a machine-learning text recognition model) to determine glyph labels of the input glyphs 304. In some cases, the glyph generation system 106 generates the glyph label in the form of a one-hot vector indicating the character that is depicted (e.g., the lowercase “a”) in connection with providing the glyph label to the implicit network 310.); and
obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style and the image features (Zhou fig. 2, decoder generates output image G(xc, xs) based on input from content encoder and style encoder; pg. 8249 “By building a feature fusion module on multiple scales, the decoder can obtain effective information from the encoding result.”).
Zhou and Reddy are analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the invention of Zhou in view of Reddy with the additional teachings of Reddy to modify Zhou’s content sub-model to learn both content and style features to facilitate combining style features of both inputs. The motivation would have been to be “able to both recreate font having known styles as well as create new fonts having new font styles” (Reddy [0028]), increasing versatility and user control over the output.
Regarding claim 5, the combination of Zhou in view of Reddy teaches: The method of claim 1, further comprising:
generating words to be used for different words in the target font style based on the target font style fusion model, and generating a word package based on the words to be used (Reddy [0144] “In some implementations, the glyph generation system 106 and/or a user provides multiple glyph labels 604 to the implicit network 610. For example, the glyph generation system 106 detects user input specifying glyph labels for characters in a word or sentence. In another example, the glyph generation system 106 detects user input specifying a subset of glyphs to include in a glyph set (e.g., 52 glyph labels for a complete English alphabet glyph set and/or additional glyph labels for numbers). In some implementations, the implicit network 610 maintains one or more lists of characters to include in a glyph set, ranging from commonly used glyphs to all possible glyphs and/or foreign language glyphs.”).
Zhou and Reddy are analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the additional glyph selection implementation(s) of Reddy described in the above citation to apply them to the Chinese characters of Zhou, which may each comprise full words. The motivation would have been for computational efficiency, generating only the characters that need to be used according to the input image or the user’s selection.
Regarding claim 11, it is rejected using the same references, rationale, and motivation to combine as claim 1 because its limitations substantially correspond to the limitations of claim 1, with the additional limitation of:
An electronic device, comprising:
at least one processor; and
a memory configured to store at least one program, wherein when the at least one program is executed by the at least one processor (Zhou pg. 8251 section 5 “Experiment”: “All experiments shown in this paper are launched on a workstation with Intel Xeon E5-2600 CPU, RAM of 32GB, and a NVIDIA GTX1080Ti GPU with memory of 11GB.”).
Regarding claim 12, it is rejected using the same references, rationale, and motivation to combine as claim 1 because its limitations substantially correspond to the limitations of claim 1, with the additional limitation of:
A storage medium comprising a computer-executable instruction, wherein the computer-executable instruction, when being executed by a processor of a computer, is configured to execute the method (Reddy [0187] “For example, the components of the glyph generation system 106 can include one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices (e.g., the computing device 1300). When executed by one or more processors, the computer-executable instructions of the glyph generation system 106 can cause the computing device 1300 to perform the methods described herein.”)
Zhou and Reddy are analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy to implement it using the standard storage and processing hardware described by Reddy in order to gain all of the advantages of using standard computational hardware including convenience, compatibility, ease of editing and running programs, etc.
Regarding claim 13, it is rejected using the same references, rationale, and motivation to combine as claim 1 because its limitations substantially correspond to the limitations of claim 1, with the additional limitation of:
a computer program product, comprising a computer program stored in a non-transitory computer-readable medium, wherein the computer program comprises a program code configured to execute the method (Reddy [0187] “For example, the components of the glyph generation system 106 can include one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices (e.g., the computing device 1300). When executed by one or more processors, the computer-executable instructions of the glyph generation system 106 can cause the computing device 1300 to perform the methods described herein.”).
The motivation to combine Zhou in view of Reddy with the additional teachings of Reddy would have been the same as described for claim 12.
Regarding claims 15 and 17, they are rejected using the same references, rationale, and motivation to combine as claims 3 and 5 respectively because their limitations substantially correspond to the limitations of claims 3 and 5 respectively.
Claim(s) 2 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("Chinese Character Style Transfer Based on the Fusion of Multi-scale Content and Style Features") in view of Reddy (US 20230110114 A1) as applied to claims 1 and 11 above, and further in view of Kumawat et al. (US 20210118207 A1, hereinafter "Kumawat").
Regarding claim 2, the combination of Zhou in view of Reddy teaches: The method of claim 1, but does not explicitly teach: wherein obtaining images to be processed corresponding to a word to be processed and a reference word comprises:
generating the images to be processed corresponding to the word to be processed and the reference word respectively based on the word to be processed and the reference word that are edited in an editing control.
Kumawat teaches: wherein obtaining images to be processed corresponding to a word to be processed and a reference word comprises:
generating the images to be processed corresponding to the word to be processed and the reference word respectively based on the word to be processed and the reference word that are edited in an editing control ([0081] “In some implementations, a preview of the synthesized font 208 may be displayed by the rendering module 124 together with one or more user interface controls that enable a user of the computing device implementing the font modification system 104 to fine-tune various glyph modifications used to generate the synthesized font 208.”).
Kumawat and the combination of Zhou in view of Reddy are both analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the teachings of Kumawat to provide a user interface for modifying the input images. The motivation would have been to give a user more control over the output.
Regarding claim 14, it is rejected using the same references, rationale, and motivation to combine as claim 2 because its limitations substantially correspond to the limitations of claim 2.
Claim(s) 4, 7, 8, 16, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("Chinese Character Style Transfer Based on the Fusion of Multi-scale Content and Style Features") in view of Reddy (US 20230110114 A1) as applied to claims 3 and 15 above, and further in view of Liu (CN 110956678 A).
Regarding claim 4, the combination of Zhou in view of Reddy teaches: The method of claim 3,
wherein the obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style and the image features comprises:
obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style and the image features (Zhou, see claim 3 rejection).
The combination of Zhou in view of Reddy does not explicitly teach: wherein the target font style fusion model further comprises a stroke feature extraction sub-model; and inputting the images to be processed into a target font style fusion model to obtain a target word of the word to be processed in a target font style comprises:
extracting a stroke feature of the word to be processed based on the stroke feature extraction sub-model; and
wherein the obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style and the image features comprises:
obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style, the stroke feature and the image features.
Liu teaches: wherein the target font style fusion model further comprises a stroke feature extraction sub-model ([0019] “…the second model is used to extract the skeleton line image from the character image.”); and inputting the images to be processed into a target font style fusion model to obtain a target word of the word to be processed in a target font style comprises:
extracting a stroke feature of the word to be processed based on the stroke feature extraction sub-model ([0067] “Specifically, in this embodiment, the skeleton line image corresponding to the glyph of the target text of the first font type is obtained by inputting the glyph of the target text of the first font type into the skeleton line image extraction network, that is, the structural information of the glyph of the target text of the first font type is obtained.”); and
wherein the obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style and the image features comprises:
obtaining the target word of the word to be processed in the target font style based on the encoding sub-model processing the reference font style, the stroke feature and the image features ([0012] “The skeleton line image is input into the first model to obtain the character image corresponding to the glyph of the target text”).
Liu and the combination of Zhou in view of Reddy are both analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the teachings of Liu to include an additional sub-model to extract stroke information and include it while generating the output images. The motivation would have been “improving the efficiency of new font development” (Liu [0007]), as well as obtaining more detailed style and content information while merging fonts.
Regarding claim 7, the combination of Zhou in view of Reddy and further in view of Liu teaches: The method of claim 4, further comprising:
obtaining the stroke feature extraction sub-model in the target font style fusion model by training (Liu [0081] “The architecture of the skeleton line extraction network model obtained through iterative training can be seen in Figure 4.”),
wherein obtaining the stroke feature extraction sub-model in the target font style fusion model by training comprises:
obtaining a first training sample set, wherein the first training sample set comprises a plurality of first training samples, and the first training sample comprises a first image and a first stroke vector corresponding to a first training word; and
using, for the plurality of first training samples, a first image of a current first training sample as an input parameter of a stroke feature extraction sub-model to be trained, using a corresponding first stroke vector as an output parameter of the stroke feature extraction sub-model to be trained to train the stroke feature extraction sub-model to be trained to obtain the stroke feature extraction sub-model (Liu [0080] “Specifically, in this embodiment, the character images corresponding to the glyphs of multiple second historical texts of different font types and the corresponding skeleton line images are input into the second initial model, i.e., the untrained generative adversarial network, and the network is iteratively trained until the output result of the trained network is the same as or similar to the preset result, thereby obtaining a stable generative adversarial network, i.e., the second model, also called the skeleton line extraction network model.”;
[0081] “The skeleton line extraction network model ModelA includes a generator and a discriminator. The generator is used to extract skeleton line images, and the discriminator is used to determine the authenticity of the generated skeleton line images. After iterative training, the generator serves as the skeleton line extraction model, so that all subsequent fusion work can directly call the skeleton line extraction model to extract skeleton lines.”.
The “character images corresponding to the glyphs of multiple second historical texts of different font types and the corresponding skeleton line images” correspond to the claimed “first image” and “first stroke vector” respectively; where the “character image” is used as the training input and the “skeleton line image” is the output being evaluated by the discriminator).
Liu and the combination of Zhou in view of Reddy are both analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the teachings of Liu to train an additional sub-model to extract stroke information and include it while generating the output images. The motivation would have been “improving the efficiency of new font development” (Liu [0007]), obtaining more detailed style and content information while merging fonts, and (for the training in particular) being able to tune the model to better suit the situation.
Regarding claim 8, the combination of Zhou in view of Reddy and further in view of Liu teaches: The method of claim 7, further comprising:
obtaining the target font style fusion model by training (Zhou pg. 8249 “Different from others’ work [3, 4, 6, 14, 15], the content encoding module and the style encoding module can be trained in our method, rather than using the pretrained VGG-16 network.”), wherein obtaining the target font style fusion model by training comprises:
obtaining a second training sample set (Zhou pg. 8250 section 4 “Configuration of Data Set”), wherein the second training sample set comprises a plurality of second training samples (Zhou pg. 8250 section 4 “Configuration of Data Set”: “In this experiment, we use 10 fonts”), and the second training sample comprises a second training image of a second training word, a third training image of a third training word (Zhou pg. 8250 section 4 “Configuration of Data Set”: “a total of 4500 characters… each Chinese character will appear 5 times in different font styles”), and a font style label of the third training word (Reddy teaches the inclusion of font style labels in the training data: [0073] “the training data 302 includes input glyphs 304, glyph labels 305, and font style codes 306.”), and wherein the second training word and the third training word have the same or different font styles (inherent as this limitation covers all possibilities);
inputting, for the plurality of second training samples, a current second training sample into a font style fusion model to be trained to:
obtain a font style to be fused based on the font style extraction sub-model to be trained processing the font style label and the third training image of the third training word (Zhou pg. 8250 section 4 “Configuration of Data Set”: “In this experiment, we use 10 fonts, and then randomly select 2 different fonts in these 10 fonts as content and style images respectively.”; pg. 8249 “the content encoding module and the style encoding module can be trained”; fig. 2 shows the use of the trained style encoder to generate style data);
obtain the content feature to be fused based on the image feature extraction sub-model to be trained performing content feature extraction on the second training image (Zhou pg. 8250 section 4 “Configuration of Data Set”: “In this experiment, we use 10 fonts, and then randomly select 2 different fonts in these 10 fonts as content and style images respectively.”; pg. 8249 “the content encoding module and the style encoding module can be trained”; fig. 2 shows the use of the trained content encoder to generate content data; Reddy [0073] teaches the use of font style labels in training data),
obtain the stroke feature based on the stroke feature extraction sub-model performing stroke feature extraction on the second training word in the second training image (Liu teaches the training of the stroke feature extraction sub-model as discussed for claim 7; it would have been obvious to one of ordinary skill in the art to train this model alongside the content feature sub-model, since the stroke skeleton comprises content of the input image), and
obtain an actual output image based on an encoding sub-model to be trained processing the font style to be fused, the content feature to be fused and the stroke feature, wherein the font style fusion model to be trained comprises the font style extraction sub-model to be trained, the image feature extraction sub-model to be trained, and the encoding sub-model to be trained (Zhou fig. 2 shows the generation of an output image based on the trained content encoder and style encoder; Liu teaches the addition and training of the font style extraction sub-model, see claims 4 and 7 respectively);
performing, based on at least one loss function, loss processing on the actual output image and a corresponding theoretical output image to determine a loss value to correct at least one model parameter in the font style fusion model to be trained based on the loss value (fig. 1 shows the use of a discriminator to judge a real image and generated image based on a loss function; pg. 8250 col. 1 “…where α can be adjusted according to the training result.” – α is a parameter included in the loss function); and
setting convergence of the at least one loss function as a training target to obtain the target font style fusion model (Zhou pg. 8248 section 3.1 “Multi-Scale Content and Style Feature Fusion Network”: “Adversarial loss helps the generator to converge.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the invention of Zhou in view of Reddy and Liu with the additional teachings of Reddy to include font style labels in the style training data. The motivation would have been to obtain more consistent or reliable results. Furthermore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the invention of Zhou in view of Reddy and Liu with the additional teachings of Liu to incorporate the skeleton line image sub-model, including its training process, into the invention. The motivation would have been “improving the efficiency of new font development” (Liu [0007]), obtaining more detailed style and content information while merging fonts, and (for the training in particular) being able to tune the model to better suit the situation.
Regarding claims 16, 19, and 20, they are rejected using the same references, rationale, and motivation to combine as claims 4, 7, and 8 respectively because their limitations substantially correspond to the limitations of claims 4, 7, and 8 respectively.
Claim(s) 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("Chinese Character Style Transfer Based on the Fusion of Multi-scale Content and Style Features") in view of Reddy (US 20230110114 A1) as applied to claims 5 and 17 above, and further in view of Singh et al. (US 20200151442 A1, hereinafter "Singh").
Regarding claim 6, the combination of Zhou in view of Reddy teaches: The method of claim 5, but does not explicitly teach: further comprising:
obtaining, in response to detecting that a font style selected from a font style list is the target font style and that the word to be processed is edited, the target word corresponding to the word to be processed from the word package.
Singh teaches: obtaining, in response to detecting that a font style selected from a font style list is the target font style and that the word to be processed is edited, the target word corresponding to the word to be processed from the word package (fig. 2C shows user selecting generated font style for the selected heading;
[0057] As mentioned, FIG. 2C illustrates advantages of the font matching system 102 over the conventional systems of FIG. 2B. More particularly, the font matching system 102 generates a font that matches “Font 1” and that includes the Cyrillic glyphs of “Font 1” required for the heading 208. Indeed, as shown the font matching system 102 identifies a number of fonts that include the required glyphs within the font menu 206 to provide for display on the client device 108. For instance, the font matching system 102 generates “Font 2,” “Font 3,” “Font 4” and so on that each include the required glyphs for the heading 208. In some embodiments, the font matching system 102 provides the fonts within the font menu 206 in order of similarity to “Font 1” (the font used within the digital document 202), where fonts higher in the list are more similar than fonts lower in the list.
[0058] The font matching system 102 therefore provides one or more matching fonts for display within the font menu 206, whereby the font matching system 102 can receive user input to select a font to change the selected heading 208. As illustrated, the font matching system 102 receives a user interaction to select “Font 3” from the font menu 206, and the font matching system 102 thereby changes the selected heading 208 from “Font 1” to “Font 3.” As also illustrated, the font generated by the font matching system 102 (“Font 3”) not only complements the visual appearance of “Font 1,” but also includes the required glyphs to present legible, comprehensible glyphs within the heading 208.).
Singh and the combination of Zhou in view of Reddy are both analogous to the claimed invention because they are in the same field of font/character style transfer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the teachings of Singh to include a user interface for selecting a desired style input for merging. The motivation would have been to give a user more control over the output.
Regarding claim 18, it is rejected using the same references, rationale, and motivation to combine as claim 6 because its limitations substantially correspond to the limitations of claim 6.
Claim(s) 9 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("Chinese Character Style Transfer Based on the Fusion of Multi-scale Content and Style Features") in view of Reddy (US 20230110114 A1) as applied to claims 8 and 20 above, and further in view of Shao et al. ("DMDIT: Diverse multi-domain image-to-image translation". Knowledge-Based Systems, vol. 229 (11 Oct 2021). https://doi.org/10.1016/j.knosys.2021.107311, hereinafter "Shao") and Li et al. ("Font Generation and Keypoint Ranking for Stroke Order of Chinese Characters by Deep Neural Networks". SN Computer Science, vol. 2, article 324 (05 Jun 2021). https://doi.org/10.1007/s42979-021-00717-2, hereinafter "Li").
Regarding claim 9, the combination of Zhou in view of Reddy teaches: The method of claim 8, wherein the at least one loss function comprises an adversarial loss function (Zhou pg. 8249 col. 2, section 3.2 “Loss Function”: “The loss function includes two parts, adversarial loss and content loss.”) and a style encoding loss function (Reddy [0101] “the glyph generation system 106 utilizes the global shape loss 324 to capture glyph shapes globally.”)
The combination of Zhou in view of Reddy does not explicitly teach: wherein the at least one loss function comprises a reconstruction loss function, a stroke order loss function, and a style regularization loss function.
Shao teaches: a reconstruction loss function (pg. 5 section 3.3.3. “Reconstruction loss”), an adversarial loss function (pg. 4 section 3.3.1. “Adversarial loss”), and a style regularization loss function (pg. 5 section 3.3.6. “Style noise regularization loss”).
Shao and the combination of Zhou in view of Reddy are both analogous to the claimed invention because they are in the same field of image-to-image feature transfer using neural networks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy with the teachings of Shao to incorporate additional loss functions including a reconstruction loss function (to “ensure the generated image can retain the underlying characters of the source image”, Shao pg. 4 col. 2) and a style regularization loss function (to “increase the variety of outputs”, Shao pg. 2 col. 1”).
The combination of Zhou in view of Reddy and further in view of Shao does not explicitly teach: wherein the at least one loss function comprises a stroke order loss function.
Li teaches a stroke order loss function (pg. 2 col. 1 “Inspired by [1], we use Cross Entropy Loss as the loss function to train the ranking model.”, where the “ranking model” is used to order the strokes: pg. 3 col. 2 “Overall, the stroke ordering method consists of three parts: (1) splitting character into strokes (see “Stroke Extraction”), (2) feature extraction from strokes, and (3) keypoint ranking model.”).
Li and the combination of Zhou in view of Reddy and further in view of Shao are analogous to the claimed invention because they are in the same field of font transfer and generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Zhou in view of Reddy and further in view of Shao with the teachings of Li to incorporate another loss function for stroke order in order to provide the font fusion model with additional information.
Regarding claim 21, it is rejected using the same references, rationale, and motivation to combine as claim 9 because its limitations substantially correspond to the limitations of claim 9.
References Cited
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xie et al. ("DG-Font: Deformable Generative Networks for Unsupervised Font Generation". arXiv preprint (8 Apr 2021)) teaches a method of font style transfer that uses separate style and content sub-models, takes strokes into account, and has adversarial and image reconstruction loss functions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN STATZ whose telephone number is (571)272-6654. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENJAMIN TOM STATZ/Examiner, Art Unit 2611
/TAMMY PAIGE GODDARD/Supervisory Patent Examiner, Art Unit 2611