DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Received 03/12/2026
Claim(s) 1-7 and 9-20 is/are pending.
Claim(s) 1, 15, and 20 has/have been amended.
Claim(s) 8 has/have been cancelled.
The 35 U.S.C § 103 rejection to claim(s) 1-7 and 9-20 have been fully considered in view of the amendments received on 03/12/2026 and are fully addressed in the prior art rejection below.
Response to Arguments
Received 03/12/2026
Regarding independent claim(s) 1, 15, and 20:
Applicant’s arguments (Remarks, Page 7: ¶ 4-5), filed 03/12/2026, with respect to the rejection(s) of claim(s) 1, 21, and 24 under 35 U.S.C § 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of Karpman et al. (US Patent No. 11995803 B1), in view of Geng et al. (US PGPUB No. 20240212327 A1), in view of Ghosh et al. (US PGPUB No. 20230106474 A1), and further in view of Ren et al. (US Patent No. 11341177 B1).
Applicant’s arguments (Remarks, Page 8: ¶ 1), filed 03/12/2026, with respect to the rejection(s) of claim(s) 15 and 20 under 35 U.S.C § 103 have been fully considered and are persuasive due claim 15's and claim 20's similarity to claim 1. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of the prior art as mentioned above.
Regarding dependent claim(s) 2-7, 9-14, and 16-19:
Applicant’s arguments (Remarks, Page 8: ¶ 2), filed 03/12/2026, with respect to the rejection(s) of claim(s) 2-7, 9-14, and 16-19 under 35 U.S.C § 103 have been fully considered and are persuasive due the dependency upon claims 1, 15, and 20 respectively. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of the prior art as mentioned above.
Applicant's arguments filed 03/21/2026 have been fully considered but they are not persuasive; as expressed below.
Regarding independent claim(s) 1, 15, and 20:
Applicant argues (Remarks, Page 7, ¶ 5), that “Applicant notes that the content of claims 3-6 is not incorporated into claim 1 and submits that this subject matter is not necessary for allowance of claim 1.”
The Examiner disagrees. Wherein, one or more limitations within former claim 8 are further narrowed/limited by the subject matter of the former claim’s previous parent claims 6, 5, 4, and 3. Wherein, former claim 8 incorporates the subject matter regarding “… the accumulated reward obtained in the generation sequence” and “… a product of the reward of the last stage of the generation sequence” that are linked to the limitations of “… the generation sequence” and more so of “… for each stage of the generation sequence” of claim 5, and by extension the limitations of “… the reward of the current stage” of claim 6. Such that, the generation sequence of claim 5 continued by claim 6 gives the subject matter of “the generations sequence” of claim 8 further distinction over the prior art. Even further, the accumulated reward is an accumulation based on the calculated reward from the stages associated with the generation sequence (see claim 1, “… the accumulated reward is obtained based on a reward of each stage of the generation sequence”), thus without the subject matter of claims 6 and 5 the accumulated reward is broader than the indicated allowable subject matter (by the Office Action of 01/09/2026) within the former claim 8.
Additionally, the limitations of claims 3 and 4 further narrow the “reinforcement learning policy” and by extinction “the accumulated reward” of independent claim 1. Wherein, claim 1 recites “… wherein an accumulated reward (is) obtained by the second Text-to-Image model in a generation sequence for implementing Text-to-Image satisfies a preset condition” and “… a reinforcement learning policy (is used) to obtain a second Text-to-Image model”. Thus, without the subject matter of claims 3 and 4 the accumulated reward is broader than the indicated allowable subject matter (by the Office Action of 01/09/2026) within the former claim 8.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 9, and 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karpman et al., US Patent No. 11995803 B1, hereinafter Karpman, in view of Geng et al., US PGPUB No. 20240212327 A1, hereinafter Geng, in view of Ghosh et al., US PGPUB No. 20230106474 A1, hereinafter Ghosh, and further in view of Ren et al., US Patent No. 11341177 B1, hereinafter Ren.
Regarding claim 15, Karpman discloses an electronic device (Karpman; an electronic device [Col. 2, lines 23-32]; moreover, computer device [Col. 24, lines 8-48]), comprising:
a processor (Karpman; the device [as addressed above] comprises a processor [Col. 2, lines 23-32]; moreover, computer device processor [Col. 24, lines 8-48]); and
a memory communicatively connected to the processor (Karpman; the device [as addressed above] comprises a memory communicatively connected to the processor [Col. 2, lines 23-32 and Col. 24, lines 8-37]), wherein the memory stores instructions executable by the processor (Karpman; the memory stores instructions executable by the processor [Col. 2, lines 23-32, Col. 24, lines 8-33, and Col. 24, line 49 to Col. 25, line 3]), and the instructions, when executed by the processor, cause the processor to perform operations (Karpman; the instructions cause the processor to perform operations when executed by the processor [Col. 2, lines 23-32, Col. 24, lines 8-33, and Col. 24, line 49 to Col. 25, line 3]) comprising:
obtaining a first Text-to-Image model and a pre-trained reward model (Karpman; the processor to perform operations [as addressed above] comprises obtaining a 1st Text-to-Image model and a pre-trained reward model [Col. 2, line 51 to Col. 3, line 36]; moreover, reward model [Col. 8, lines 25-64]), wherein the first Text-to-Image model is used to generate a corresponding image based on an input text (Karpman; the 1st Text-to-Image model is used to generate a corresponding image based on an input text [Col. 2, line 51 to Col. 3, line 36]), and the pre-trained reward model is used to score a data pair composed of the input text and the corresponding image (Karpman; the pre-trained reward model is used to score a data pair composed of the input text and the corresponding image [Col. 8, lines 25-64]);
adjusting parameters of the first Text-to-Image model based on the pre-trained reward model and a reinforcement learning policy to obtain a second Text-to-Image model (Karpman; the processor to perform operations [as addressed above] comprises adjusting parameters of the 1st Text-to-Image model based on the pre-trained reward model and a reinforcement learning policy to obtain an implicit 2nd Text-to-Image model (given multiple models) [Col. 3, lines 15-36 and Col. 19, lines 1-38]; wherein, the image diffusion model is trained to generate a quality score [Col. 6, line 42 to Col. 7, line 32] in relation with modifying and/or optimizing parameters of the text-to-image diffusion model using a reward model [Col. 8, liens 25-38 and Col. 13, lines 21-50] and associated with fine-tuning image generation parameters [Col. 15, line 14 to Col. 16, line 3]; wherein, reinforcement learning algorithm are executed [Col. 15, lines 31-54]), wherein an accumulated reward obtained by the second Text-to-Image model in a generation sequence for implementing Text-to-Image satisfies a preset condition (Karpman; an accumulated reward (i.e. cumulative reward) obtained by the implicit 2nd Text-to-Image model (given multiple models) in a generation sequence for implementing Text-to-Image satisfies a preset condition [Col. 15, lines 14-54]), and the accumulated reward is obtained based on a reward of each stage of the generation sequence (Karpman; the accumulated reward (i.e. cumulative reward) is obtained based on a reward of each stage of the generation sequence [Col. 15, lines 14-54]); and
wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item (Karpman; the accumulated reward [as addressed above] obtained in the generation sequence comprises a total score [Col. 15, line 14 to Col. 16, line 3]; wherein, a reward model involves Kullback-Leibler (KL) divergence (otherwise known as relative entropy) in relation with optimization [id.]; wherein, parameters and other metrics are modifiable within the training of the reward model [Col. 8, lines 25-64]), wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence (Karpman; the total score [as addressed above] is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence [Col. 8, lines 39-64, Col. 9, lines 14-35, and Col. 15, lines 14-54]; wherein, the system includes a reward model trained to generate a reward value (e.g., a numerical score) given an image generated by the text-to-image diffusion model (e.g., after the pre-training stage) [id.]; moreover, a reinforcement learning algorithm involves inputting the text prompt to the (pre-trained) text-to-image diffusion model, inputting the text prompt to the copy of the text-to-image diffusion model, inputting the diffusion model outputs to the reward model, and computing a preferability score based reward values outputted by the reward model(s)( in relation with parameter updating and increasing and/or maximizing a cumulative reward (e.g., a sum of preferability scores) output by the reward model [id.]).
Karpman fails to explicitly disclose a second Text-to-Image model;
an accumulated reward obtained by the second Text-to-Image model; and
wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item, wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient.
However, Geng teaches a second Text-to-Image model (Geng; a 2nd Text-to-image model corresponding to model reprogramming (i.e. generation of an updated model) [¶ 0025-0026, ¶ 0037, and 0039]; moreover, reinforcement learning involving joint text-image encoders [¶ 0013 and ¶ 0033]; moreover, backpropagation (in relation with a 2nd or updated function) [¶ 0022 and ¶ 0054-0055], and similarity loss function [¶ 0046-0049]).
Karpman and Geng are considered to be analogous art because both pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman, to incorporate a second Text-to-Image model (as taught by Geng), in order to provide mathematical modeling that effectively aids in predictions or decisions without requiring explicit programming to perform tasks (Geng; [¶ 0005-0007 and ¶ 0013]).
Karpman as modified by Geng fails to disclose an accumulated reward obtained by the second model; and
wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item, wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient.
However, Ghosh teaches an accumulated reward obtained by the second model in a generation sequence for implementation that satisfies a preset condition (Ghosh; an accumulated reward obtained by the 2nd model in a generation sequence for implementation that satisfies a preset condition [¶ 0019, ¶ 0029, and ¶ 0092]; moreover, determining highest cardinality from different cardinalities associated with a threshold condition [¶ 0045-0046] associating each combination of actions with a cumulative reward [¶ 0049, ¶ 0051-0053, and ¶ 0063-0065]; wherein, maximize a cumulative reward or minimize a cumulative cost [¶ 0003-0004]).
Karpman in view of Geng and Ghosh are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, to incorporate an accumulated reward obtained by the second model in a generation sequence for implementation that satisfies a preset condition (as taught by Ghosh), in order to provide faster machine learning modeling that utilizes fewer computational resources (Ghosh; [¶ 0003-0005 and ¶ 0008]).
Geng as modified by Ghosh wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item, wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient.
However, Ren teaches wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item (Ren; the accumulated reward (i.e. total reward) [Col. 7, line 48 to Col. 8, line 15] obtained in the generation sequence comprises a total score and a loss item [Col. 8, line 22 to Col. 9, line 24]; additionally, reinforcement learning parameters for an agent using a total reward the agent can expect [Col. 9, line 25 to Col. 10, line 6]; moreover, curriculum learning to train the actor-critic model and iteratively leaning [id.]), wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient (Ren; the total score [as addressed above] is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient [Col. 9, line 25 to Col. 10, line 6]; wherein, the learning model calculations (associated with the CNN, RNN) correspond to an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient [Col. 8, line 16 to Col. 9, line 24]; and wherein , [Col. 5, line 59 to Col. 6, line 8]; moreover, feeding into a model training and forming a trained model [Col. 3, lines 34-24] in relation with a trained policy network and value network [Col. 3, lines 25-45 and Col. 11, lines 13-20] associated with embedding reward and reward prediction [Col. 2, lines 20-47 and Col. 3, line 46 to Col. 4, line 5]).
Karpman in view of Geng and Ghosh and Ren are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng and Ghosh, to incorporate wherein the accumulated reward obtained in the generation sequence comprises a total score and a loss item, wherein the total score is obtained by the pre-trained reward model based on an initial input and a final output of the generation sequence, and the loss item is a product of the reward of the last stage of the generation sequence and a loss coefficient (as taught by Ren), in order to provide improved word comprehension within a machine learning model while reducing errors (Ren; [Col. 2, lines 6-32 and Col. 2, line 48 to Col. 3, line 3]).
Regarding claim 16, Karpman in view of Geng, Ghosh, and Ren further discloses the electronic device of claim 15, wherein the preset condition comprises that the accumulated reward obtained by the second Text-to-Image model in the generation sequence for implementing Text-to-Image is higher than an accumulated reward obtained by the first Text-to-Image model in a generation sequence for implementing Text-to-Image (Karpman; the preset condition [as addressed within the parent claim(s)] comprises that the accumulated reward (i.e. cumulative reward) obtained by the 2nd Text-to-Image model in the generation sequence for implementing Text-to-Image is implicitly higher (given preferability score) than an accumulated reward (i.e. cumulative reward) obtained by the 1st Text-to-Image model in a generation sequence for implementing Text-to-Image [Col. 15, line 31 to Col. 16, line 3]).
Regarding claim 17, Karpman in view of Geng, Ghosh, and Ren further discloses the electronic device of claim 15, wherein the reinforcement learning policy comprises a proximal policy optimization algorithm (Karpman; the reinforcement learning policy [as addressed within the parent claim(s)] comprises a proximal policy optimization algorithm [Col. 8, line 25 to Col. 9, line 35]; wherein, discourage and/or prevent over-optimizing [Col. 15, line 31 to Col. 16, line 3]).
Regarding claim 18, Karpman in view of Geng, Ghosh, and Ren further discloses the electronic device of claim 17, wherein the proximal policy optimization algorithm uses a behavior sub-model and an evaluation sub-model (Karpman; the proximal policy optimization algorithm uses a behavior/human sub-model and an evaluation sub-model [Col. 8, line 25 to Col. 9, line 35]), wherein the behavior sub-model is obtained based on initialization of the first Text-to-Image model (Karpman; the behavior/human sub-model is obtained based on initialization of the 1st Text-to-Image model [Col. 8, line 25 to Col. 9, line 35]), and the evaluation sub-model is obtained based on initialization of the pre-trained reward model (Karpman; the evaluation sub-model is obtained based on initialization of the pre-trained reward model [Col. 8, line 25 to Col. 9, line 35]).
Regarding claim 19, Karpman in view of Geng, Ghosh, and Ren further discloses the electronic device of claim 18, wherein the generation sequence comprises at least one stage (Karpman; the generation sequence [as addressed within the parent claim(s)] comprises at least one stage [Col. 9, line 36 to Col. 10, line 7 and Col. 16, line 21-44], as illustrated within Figs. 2 and 3), and wherein the operations further comprise:
for each stage of the generation sequence (Karpman; the operations [as addressed within the parent claim(s)] further comprises generation and output for each stage of the generation sequence [Col. 9, line 36 to Col. 10, line 7 and Col. 16, line 21-44], as illustrated within Figs. 2 and 3):
generating, by the behavior sub-model, a corresponding output noisy image based on the input text provided (Karpman; generating a corresponding output noisy image based on the input text provided by the behavior/human sub-model [Col. 2, line 51 to Col. 3, line 36 and Col. 8, line 25 to Col. 9, line 35]; moreover, image diffusion model in relation with generating images from random noise [Col. 5, lines 15-62 and Col. 13, line 51 to Col. 14, line 31]); and
outputting, by the evaluation sub-model, the reward of a current stage based on the input text and the output noisy image of the current stage (Karpman; outputting the reward of a current stage based on the input text and the output noisy image of the current stage by the evaluation sub-model [Col. 8, line 25 to Col. 9, line 35]).
Regarding claim 20, the rejection of claim 20 is addressed within the rejection of claim 15, due to the similarities claim 20 and claim 15 share, therefore refer to the rejection of claim 15 regarding the rejection of claim 20; however, the subject matter/limitations not addressed by claim 15 is/are addressed below.
Karpman discloses a non-transitory computer readable storage medium storing computer instructions (Karpman; a non-transitory computer readable storage medium storing computer instructions [Col. 24, lines 8-33 and Col. 24, lines 49-63]).
(further refer to the rejection of claim 15)
Regarding claim 1, the rejection of claim 1 is addressed within the rejection of claim 15, due to the similarities claim 1 and claim 15 share, therefore refer to the rejection of claim 15 regarding the rejection of claim 1. Although, claim 1 and claim 15 may not be identical, they are considerably comparable or substantially equivalent given their overlapping subject matter. Thus, it is reasonable to reject claim 1 based on the teachings and rational in relation with the prior art within the rejection of claim 15.
Regarding claim 2, the rejection of claim 2 is addressed within the rejection of claim 16, due to the similarities claim 2 and claim 16 share, therefore refer to the rejection of claim 16 regarding the rejection of claim 2.
Regarding claim 3, the rejection of claim 3 is addressed within the rejection of claim 17, due to the similarities claim 3 and claim 17 share, therefore refer to the rejection of claim 17 regarding the rejection of claim 3.
Regarding claim 4, the rejection of claim 4 is addressed within the rejection of claim 18, due to the similarities claim 4 and claim 18 share, therefore refer to the rejection of claim 18 regarding the rejection of claim 4.
Regarding claim 5, the rejection of claim 5 is addressed within the rejection of claim 19, due to the similarities claim 5 and claim 19 share, therefore refer to the rejection of claim 19 regarding the rejection of claim 5.
Regarding claim 6, Karpman in view of Geng and Ghosh further discloses the method of claim 5, wherein the reward of the current stage comprises a relative entropy between an output of the behavior sub-model in a previous stage prior to the current stage and an output of the behavior sub-model in the current stage (Karpman; the reward of the current stage comprises a relative entropy (i.e. score, diffusion) between an output of the behavior/human sub-model in a previous stage prior to the current stage and an output of the behavior/human sub-model in the current stage [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 to Col. 16, line 3]).
Regarding claim 7, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 5, wherein the reward of the current stage comprises a difference between an evaluation value of a previous stage prior to the current stage and an evaluation value of the current stage (Karpman; the reward of the current stage comprises a difference between an evaluation value and an evaluation value [Col. 8, line 25 to Col. 9, line 35]; wherein, multi-modal encoder-decoder [Col. 3, lines 15-36 and Col. 4, lines 46 to Col. 5, line 6]), wherein the evaluation value is scored by the pre-trained reward model based on the input text provided and the corresponding output noisy image (Karpman; the evaluation value is scored (i.e. rank, score) by the pre-trained reward model based on the input text provided and the corresponding output noisy image [Col. 2, line 51 to Col. 3, line 36 and Col. 8, line 25 to Col. 9, line 35]).
Geng further teaches wherein the reward of the current stage comprises a difference between an evaluation value of a previous stage prior to the current stage and an evaluation value of the current stage (Geng; the reward of the current stage comprises a difference between an evaluation value of a previous stage prior to the current stage and an evaluation value of the current stage (corresponding to a backprogation with loss function) [¶ 0046-0049]), wherein the evaluation value is scored by the pre-trained reward model based on the input text provided and the corresponding output noisy image (Geng; the evaluation value is scored by the pre-trained implicit reward model (given RL) based on the input text provided and the corresponding output noisy image [¶ 0046-0049], as illustrated within Fig. 3; wherein, image reprogramming function [¶ 0040-0042]; moreover, RL associated with determining a reward [¶ 0013]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, Ghosh, and Ren, to incorporate wherein the reward of the current stage comprises a difference between an evaluation value of a previous stage prior to the current stage and an evaluation value of the current stage, wherein the evaluation value is scored by the pre-trained reward model based on the input text provided and the corresponding output noisy image (as taught by Geng), in order to provide mathematical modeling that effectively aids in predictions or decisions without requiring explicit programming to perform tasks (Geng; [¶ 0005-0007 and ¶ 0013]).
Regarding claim 9, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 1, wherein the parameters of the second Text-to-Image model are obtained by a machine-learning algorithm based on the accumulated reward in the generation sequence of the second Text-to-Image model (Karpman; the parameters of the implicit 2nd Text-to-Image model (given multiple models) are obtained by a machine-learning algorithm based on the accumulated reward in the generation sequence of the 2nd Text-to-Image model [Col. 3, lines 15-36, Col. 8, liens 25-38, and Col. 19, lines 1-38]; moreover, reinforcement learning algorithm are executed [Col. 15, lines 31-54]).
Geng further teaches wherein the parameters of the second Text-to-Image model are obtained by a back-propagation algorithm based on the accumulated reward in the generation sequence of the second Text-to-Image model (Geng; the parameters of the 2nd Text-to-Image model (i.e. model reprogramming, generation of an updated model) are obtained by a back-propagation algorithm [¶ 0022-0023 and ¶ 0047] based on the implicit accumulated/cumulative reward (given RL) in the generation sequence of the 2nd Text-to-Image model (i.e. model reprogramming, generation of an updated model) [¶ 0054-0055]; moreover, RL in relation with a cumulative reward [¶ 0013]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, Ghosh, and Ren, to incorporate the parameters of the second Text-to-Image model are obtained by a back-propagation algorithm based on the accumulated reward in the generation sequence of the second Text-to-Image model (as taught by Geng), in order to provide mathematical modeling that effectively aids in predictions or decisions without requiring explicit programming to perform tasks (Geng; [¶ 0005-0007 and ¶ 0013]).
Regarding claim 14, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 1, wherein obtaining the first Text-to-Image model (Karpman; obtaining the 1st Text-to-Image model [as addressed within the parent claim(s)]) comprises:
obtaining a manually labeled image-text pair as a training sample of the first Text-to-Image model to be trained (Karpman; obtaining a manually labeled image-text pair as a training sample of the 1st Text-to-Image model to be trained [Col. 2, line 33 to Col. 3, line 36]; moreover, training using human-labeling and human-annotating [Col. 6, line 42 to Col. 7, line 3 and Col. 8, line 39 to Col. 9, line 13]); and
updating parameters of the first Text-to-Image model to be trained based on a back propagation algorithm to obtain the first Text-to-Image model that has gone through supervised training (Karpman; updating parameters of the 1st Text-to-Image model to be trained based on a machine-learning algorithm to obtain the 1st Text-to-Image model that has gone through supervised training [Col. 9, lines 14-35 and Col. 14, lines 4-38]; wherein, during pre-training and/or training stages executing multi-modal encoder-decoder [Col. 4, line 46 to Col. 5, line 6]).
Geng further teaches updating parameters of the first Text-to-Image model to be trained based on a back propagation algorithm to obtain the first Text-to-Image model that has gone through supervised training (Geng; updating parameters of the 1st Text-to-Image model to be trained based on a back propagation algorithm to obtain the 1st Text-to-Image model that has gone through supervised training [¶ 0022-0023 and ¶ 0047]; wherein, each backpropagated parameter can be updated and/or optimized [¶ 0055]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, Ghosh, and Ren, to incorporate updating parameters of the first Text-to-Image model to be trained based on a back propagation algorithm to obtain the first Text-to-Image model that has gone through supervised training (as taught by Geng), in order to provide mathematical modeling that effectively aids in predictions or decisions without requiring explicit programming to perform tasks (Geng; [¶ 0005-0007 and ¶ 0013]).
Claim(s) 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karpman in view of Geng, Ghosh, and Ren as applied to claim(s) 1 above, and further in view of Baughman et al., US Patent No. 11645498 B2, hereinafter Baughman.
Regarding claim 10, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 1, wherein the pre-trained reward model is obtained by training based on a feedback dataset (Karpman; the pre-trained reward model is obtained by training based on a feedback dataset [Col. 8, line 25 to Col. 9, line 35 and Col. 17, line 53 to Col. 18, line 30]), wherein the feedback dataset comprises a plurality of feedback data (Karpman; the feedback dataset [as addressed above] comprises a plurality of feedback data [Col. 17, line 53 to Col. 18, line 30]), and the plurality of feedback data comprise a data pair composed of the input text and the corresponding generated image and a feedback state corresponding to the data pair (Karpman; the plurality of feedback data [as addressed above] comprises a data pair composed of the input text and the corresponding generated image and a feedback state corresponding to the data pair [Col. 2, line 33 to Col. 3, line 36 and Col. 8, line 25 to Col. 9, line 35]; moreover, generating text captions describing each image in a set of training images [Col. 11, line 62 to Col. 12, line 65]), wherein the feedback state is used to represent that the corresponding generated image which is generated relative to a same input text belongs to a positive feedback or a negative feedback (Karpman; the feedback state [as addressed above] is used to represent that the corresponding generated image which is generated relative to a same input text belongs to an implicit positive feedback or a negative feedback (given a numeric value/score) [Col. 8, line 25 to Col. 9, line 35]; additionally, generating a quality score [Col. 6, line 30 to Col. 7, line 3]).
Karpman as modified by Geng and Ghosh fails to explicitly disclose a positive feedback or a negative feedback.
However, Baughman teaches a positive feedback or a negative feedback (Baughman; a positive feedback or a negative feedback [Col. 1, line 33 to Col. 2, line 6]; moreover, RL in relation with positive and negative rewards [Col. 3, lines 3-16]).
Karpman in view of Geng, Ghosh, and Ren and Baughman are considered to be analogous art because they pertain to generating and/or managing data in relation with providing learning model, wherein one or more computerized units are utilized in order to produce decisions that are not explicitly implemented.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, Ghosh, and Ren, to incorporate a positive feedback or a negative feedback (as taught by Baughman), in order to provide improved decision processing for learning models (Baughman; [Col. 1, lines 6-49]).
Regarding claim 11, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 10, further comprising training the reward model (Karpman; training the reward model [Col. 8, line 25 to Col. 9, line 35]), wherein training the reward model comprises:
training the reward model in a comparative learning manner based on the plurality of feedback data (Karpman; training the reward model [as addressed above] comprises training the reward model in a comparative learning manner based on the plurality of feedback data dataset [Col. 8, line 25 to Col. 9, line 35 and Col. 17, line 53 to Col. 18, line 30]), such that the reward model outputs a first reward score for the data pair having a feedback state of positive feedback (Karpman; the reward model outputs a 1st reward score for the data pair having a feedback state of implicit positive feedback (given a numeric value/score) [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 to Col. 16, line 3]), and outputs a second reward score for the data pair having a feedback state of feedback (Karpman; outputs a 2nd reward score for the data pair having a feedback state of feedback [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 to Col. 16, line 3]), wherein to represent the quality difference of the corresponding generated images (Karpman; representing the quality difference of the corresponding generated images [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 to Col. 16, line 3]; moreover, generated quality score based on visual features [Col. 6, line 42 to Col. 7, line 3]).
Ghosh further teaches a difference between the first reward score and the second reward score is used to represent the quality difference (Ghosh; a difference between the first reward score and the second reward score is used to represent the quality difference [¶ 0018-0019 and ¶ 0029-0030]; moreover, to maximize a cumulative reward or minimize a cumulative cost [Id.]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng and Ghosh, to incorporate a difference between the first reward score and the second reward score is used to represent the quality difference (as taught by Ghosh), in order to provide faster machine learning modeling that utilizes fewer computational resources (Ghosh; [¶ 0003-0005 and ¶ 0008]).
Karpman as modified by Geng, Ghosh, and Ren fails to disclose a feedback state of positive feedback;
a feedback state of negative feedback; and
a difference between the first reward score and the second reward score is used.
However, Baughman teaches a feedback state of positive feedback (Baughman; a feedback state of positive feedback [Col. 3, lines 3-16]);
a feedback state of negative feedback (Baughman; a feedback state of negative feedback [Col. 3, lines 3-16]); and
a difference between the first reward score and the second reward score is used (Baughman; a difference between the 1st reward score and the 2nd reward score is used [Col. 1, line 32 to Col. 2, line 6]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Karpman as modified by Geng, Ghosh, and Ren, to incorporate a feedback state of positive feedback; a feedback state of negative feedback; and a difference between the first reward score and the second reward score is used (as taught by Baughman), in order to provide improved decision processing for learning models (Baughman; [Col. 1, lines 6-49]).
Regarding claim 12, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 10, wherein the feedback dataset comprises the plurality of feedback data from at least two different sources (Karpman; the feedback dataset comprises the plurality of feedback data from at least two different sources (i.e. one source corresponding to a human means and another source corresponding to a reward means) [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 and Col. 16, line 3]).
Regarding claim 13, Karpman in view of Geng, Ghosh, and Ren further discloses the method of claim 12, wherein the plurality of feedback data comprises at least two of data fed back by a user, manually labeled data, or manually compared data (Karpman; the plurality of feedback data comprises at least two of data fed back by a user, manually labeled data, or manually compared data [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 and Col. 16, line 3]; moreover, human labeled images and human annotators further linked with human preferences and/or feedback [Col. 8, line 25 to Col. 9, line 35]), wherein:
the data fed back by the user includes the feedback state based on user behavior (Karpman; the data fed back by the user includes the feedback state based on user behavior [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 and Col. 16, line 3]);
the manually labeled data includes the feedback state based on a result of manual labeling (Karpman; the manually labeled data includes the feedback state based on a result of manual labeling [Col. 8, line 25 to Col. 9, line 35 and Col. 15, line 14 and Col. 16, line 3]);
the manually compared data includes the feedback state based on different versions of the generated images (Karpman; the manually compared data includes the feedback state based on different versions of the generated images [Col. 19, line 40 to Col. 20, line 18 and Col. 21, line 57 to Col. 22, line 19]; moreover, interactive text field that enables user input [Col. 20, lines 19-55]; and moreover, image generated requests [Col. 16, line 21 to Col. 17, line 33]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Charles Lloyd Beard whose telephone number is (571)272-5735. The examiner can normally be reached Monday - Friday, 8:00 AM - 5: 00 PM, alternate Fridays EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHARLES LLOYD. BEARD
Primary Examiner
Art Unit 2611
/CHARLES L BEARD/Primary Examiner, Art Unit 2611
Prosecution Insights