DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on December 5, 2025 has been entered.
Status of Claims
Claims 1-10 and 12-20 are pending. Claim 11 has been canceled.
Response to Arguments
Applicant's arguments filed December 5, 2025 have been fully considered but they are moot because of the new grounds of rejection presented in the sections below. Applicant argues that none of the previously presented references teach the newly added limitation “wherein the reward value is computed based on a range of values for features of the modified media object”. However, in an analogous field of endeavor, Takami teaches a reward function that varies a reward value according to a degree that a value of a parameter (i.e., feature) deviates from a target range of the target setting data (i.e., range of values for features). Examiner asserts that in combination with Eggenberger’s reward value based on the context data of the modified media object, Yang’s teaching of a reinforcement model to optimize model parameters, and Chai’s teaching of presenting virtual content within scene context, one having ordinary skill in the art would have found claim 1 obvious, as described in the 35 USC 103 rejections below. Similarly, independent claims 10 and 17 are obviated by the Takami reference. Therefore, the 35 USC 103 rejections are upheld.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 and 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) and Chai et al. (US 2023/0419599 A1, filed June 22, 2022).
Regarding claim 1, Eggenberger teaches a method for media generation, comprising:
obtaining a media object and context data describing a context of the media object (Eggenberger, Para. [0050], the method comprises receiving an object as input for the GAN. In a next process step, a set of features, in particular contextual attributes or real attributes of the input, are determined. Para. [0044], the object may be an image (in particular, in a digital representation)), wherein the media object comprises one or more modification parameters (Eggenberger, Para. [0044], the set of features may comprise at least one of the group comprising a geometrical form in the image an orientation of the geometrical form, a color value of the geometrical form, a size of the geometrical form, a position of the geometrical form, pixel density, and a brightness value. Para. [0025], the term ‘modification to one feature’ may denote that at least one of the attributes describing the feature may be altered. It may relate to an orientation of the feature, its rotation angle, a change in the underlying color space (varying values of the RGB color space), and application of image filters, or any other observable modification of an appearance of the feature);
Although Eggenberger teaches a reward value computed based on the context data (Eggenberger, Para. [0034]), Eggenberger does not explicitly teach “generating, using a reinforcement learning model, a modified media object by iteratively optimizing the one or more modification parameters based on the context data and a reward value”. However, in an analogous field of endeavor, Yang teaches a reward may be used to back-propagate the neural network (e.g., reinforcement learning model) to update the parameters of the layers which may refine the reinforcement model’s behaviors (Yang, Para. [0052]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger with the teachings of Yang by including updating the parameters of the reinforcement model based on the reward value of Eggenberger. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for promoting better decisions by the reinforcement learning model, as recognized by Yang. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Although Eggenberger in view of Yang teaches updating the parameters of a reinforcement model using a reward value (Yang, Para. [0052]), they do not explicitly teach “wherein the reward value is computed based on a range of values for features of the modified media object”. However, in an analogous field of endeavor, Takami teaches a reward function may be a function that varies a reward value, according to a degree that a value of the state parameter relating to the piece of equipment deviates from a target range of the target setting data (i.e.., range of values for features of the object) (Takami, Para. [0057]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang with the teachings of Takami by including computing a reward value based on a range of values for features of the object (i.e., the modified media object of Eggenberger and Yang). One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for optimizing a model using a reward function, as recognized by Takami.
Although Eggenberger in view of Yang further in view of Takami teaches generating an outputting a modified object (Eggenberger, [0052]), they do not explicitly teach “providing the modified media object within the context”. However, in an analogous field of endeavor, Chai teaches the system renders the virtual content, applies a shading based on the existing lighting conditions of the scene, and presents the shaded virtual content in a display of the AR device relative to a frame of reference (external to the display device) so that the virtual content appears correctly in the display (Chai, Para. [0029]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang further in view of Takami with the teachings of Chai by including after generating the modified media object, providing it within the context (i.e., presenting the modified media object in the display). One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for virtual content to appear in context and consistent with the texture/brightness and lighting conditions of the context, as recognized by Chai. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 2, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, and further teaches wherein:
the context comprises a graphical user interface (Chai, Para. [0053], a modified image or video stream may be presented in a graphical user interface displayed on the ART device as soon as the image or video stream is captured, and a specified modification is selected).
The proposed combination as well as the motivation for combining the Eggenberger, Yang, Takami and Chai references presented in the rejection of Claim 1, apply to Claim 2 and are incorporated herein by reference. Thus, the method recited in Claim 2 is met by Eggenberger in view of Yang further in view of Takami and Chai.
Regarding claim 5, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, further comprising:
receiving feedback based on the modified media object (Eggenberger, Para. [0034], the reaction based on the image may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object);
computing a reward value based on the feedback (Eggenberger, Para. [0043], the feedback loop may be instantiated as a reinforcement learning (RL) model or RL system comprising a reward function (controlling the behavior of a software agent)); and
updating the reinforcement learning model based on the reward value (Eggenberger, Para. [0030], reinforcement learning uses rewards and punishments as signals for positive and negative behavior. In reinforcement learning the goal is to find a suitable action model that would maximize the total cumulative reward of the agent).
Regarding claim 6, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 5, further comprising:
generating a subsequent modified media object using the updated reinforcement learning model (Eggenberger, Para. [0034], the reaction based on the image may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object).
Regarding claim 7, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, further comprising:
generating state information for the media object based on features of the media object, wherein the modified media object is generated based on the state information (Eggenberger, Para. [0043], the feedback signal may be used as a state parameter for the reinforcement learning model or loop. Para. [0034], the reaction based on the image may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) and Chai et al. (US 2023/0419599 A1, filed June 22, 2022), as applied to claims 1-2 and 5-7 above, and further in view of Michael Oren Belkin (US 10,817,981 B1).
Regarding claim 3, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 2, as described above.
Although Eggenberger in view of Yang further in view of Takami and Chai teaches determining contextual or real attributes of the input (Eggenberger, Para. [0050]), they do not explicitly teach “the context data includes at least one of a background color, a font style, or a font color of the graphical user interface”. However, in an analogous field of endeavor, Belkin teaches the image processor further receives context features associated with the content item, which relate to how the content item is to be displayed. The context features may indicate a type of the content item (e.g., a link, a video, etc.), how the content item is to be displayed (e.g., as a newsfeed post, as a sponsored post, etc.), a type of interface element the accent color is to be used for (e.g., a button, a banner, a navigation bar, etc.), a platform or social network the content item is to be displayed on, a type of interface on which the content item is to be displayed to the user (e.g., mobile or desktop interface), one or more characteristics of the interface currently being displayed to the viewing user (e.g., background color of the interface), etc. (Belkin, Col. 10, lines 40-58).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the method of Eggenberger in view of Yang further in view of Takami and Chai with the teachings of Belkin by including that the context data includes a background color of the interface. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for the displayed content item to have a more eye-catching look and feel, as recognized by Belkin. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 4 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) and Chai et al. (US 2023/0419599 A1, filed June 22, 2022), as applied to claims 1-2 and 5-7 above, and further in view of Wilensky et al. (US 2020/0394773 A1).
Regarding claim 4, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, as described above.
Although Eggenberger in view of Yang further in view of Takami and Chai teaches the set of features may include a color value and brightness value (Eggenberger, Para. [0044]), they do not explicitly teach “the one or more modification parameters includes a contrast, a hue, and a brightness of the media object”. However, in an analogous field of endeavor, Wilensky teaches a parameter of a digital image includes an adjustable characteristic or setting that impacts the appearance of a digital image, including hue (e.g. color on a color wheel), saturation, brightness (e.g., relative lightness or darkness of a color from black to white_, contrast (e.g., a measure of difference between light and dark pixels in a digital image), or exposure (Wilensky, Para. [0034]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang further in view of Takami and Chai with the teachings of Wilensky by including modification parameters including hue, brightness and contrast. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for modifying parameter values to generate an enhanced digital image, as recognized by Wilensky. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 9, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, as described above.
Although Eggenberger in view of Yang further in view of Takami and Chai teaches performing at least one modification to at least one feature of the set of features of the object (Eggenberger, Para. [0052]), they do not explicitly teach “selecting an action from an action set corresponding to potential values of the one or more modification parameters using the reinforcement learning model, wherein the modified media object is generated by applying the action to the media object using a media editing application”. However, in an analogous field of endeavor, Wilensky teaches a digital image editing engine that can create, generate, edit, modify, render and/or display an enhanced digital image. For example, the digital image enhancement system can identify modifications and make modifications in relation to a digital image (Wilensky, Para. [0142]).
The proposed combination as well as the motivation for combining the Eggenberger, Yang, Takami, Chai and Wilensky references presented in the rejection of Claim 4, apply to Claim 9 and are incorporated herein by reference. Thus, the method recited in Claim 9 is met by Eggenberger in view of Yang further in view of Takami, Chai and Wilensky.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) and Chai et al. (US 2023/0419599 A1, filed June 22, 2022), as applied to claims 1-2 and 5-7 above, and further in view of Kunde et al. (US 2023/0305544 A1, filed February 13, 2023).
Regarding claim 8, Eggenberger in view of Yang further in view of Takami and Chai teaches the method of claim 1, and further teaches wherein the reward network comprises a neural network (Yang, Para. [0047], the reinforcement learning model (i.e., reward network) may include a neural network),
the (Eggenberger, Para. [0034], if the object is a digital image and an observer, in particular an evaluating entity in form of a user, looks at the image he may produce a certain reaction. This reaction may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object),
Although Eggenberger in view of Yang further in view of Takami and Chai teaches a reward value computed based on the context data of the modified media object (Eggenberger, Para. [0034]) and that the reinforcement learning model (i.e., reward network) may include a neural network (Yang, Para. [0047]), they do not explicitly teach “computing a dynamic reward using a reward network” and “the reward value includes the dynamic reward”. However, in an analogous field of endeavor, Kunde teaches determining an optimal set of features using a dynamic reward function that changes across episodes (Kunde, Para. [0062]). The reward of choosing an action depends on the current state (i.e., whether the modified media object is appropriate for the context). In this scenario, the meta features selected form the state and the reward for selecting the next meta feature depends on the meta features that have already been selected (Kunde, Para. [0069]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang further in view of Takami and Chai with the teachings of Kunde by including that the reward function computed based on features of the media object is a dynamic reward. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for performing best feature selection using reinforcement learning, as recognized by Kunde. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022).
Regarding claim 10, Eggenberger teaches a method for media generation, comprising:
obtaining a media object and context data describing a context of the media object (Eggenberger, Para. [0050], the method comprises receiving an object as input for the GAN. In a next process step, a set of features, in particular contextual attributes or real attributes of the input, are determined. Para. [0044], the object may be an image (in particular, in a digital representation)), wherein the media object comprises one or more modification parameters (Eggenberger, Para. [0044], the set of features may comprise at least one of the group comprising a geometrical form in the image an orientation of the geometrical form, a color value of the geometrical form, a size of the geometrical form, a position of the geometrical form, pixel density, and a brightness value. Para. [0025], the term ‘modification to one feature’ may denote that at least one of the attributes describing the feature may be altered. It may relate to an orientation of the feature, its rotation angle, a change in the underlying color space (varying values of the RGB color space), and application of image filters, or any other observable modification of an appearance of the feature);
generating a modified media object by adjusting the one or more modification parameters using a reinforcement learning model based on the context data (Eggenberger, Para. [0034], if the object is a digital image and an observer, in particular an evaluating entity in form of a user, looks at the image he may produce a certain reaction. This reaction may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object. This process may be repeated until the user, observing the constantly modified object, may be satisfied. The satisfaction may be measurable and may, at the same time, produce the highest determinable reward for the software agent in the feedback loop);
Although Eggenberger teaches reinforcement learning uses rewards and punishments as signals for positive and negative behavior (Eggenberger, Para. [0030]), Eggenberger does not explicitly teach “updating parameters of the reinforcement learning model based on the reward value”. However, in an analogous field of endeavor, Yang teaches a reward may be used to back-propagate the neural network (e.g., reinforcement learning model) to update the parameters of the layers which may refine the reinforcement model’s behaviors (Yang, Para. [0052]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger with the teachings of Yang by including updating the parameters of the reinforcement model based on the reward value. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for promoting better decisions by the reinforcement learning model, as recognized by Yang.
Although Eggenberger in view of Yang teaches a reward value computed based on the context data (Eggenberger, Para. [0034]), they do not explicitly teach “computing a reward value based on a range of values for features of the modified media object”. However, in an analogous field of endeavor, Takami teaches a reward function may be a function that varies a reward value, according to a degree that a value of the state parameter relating to the piece of equipment deviates from a target range of the target setting data (i.e.., range of values for features of the object) (Takami, Para. [0057]).
The proposed combination as well as the motivation for combining the Eggenberger, Yang, and Takami references presented in the rejection of Claim 1, apply to Claim 10 and are incorporated herein by reference. Thus, the method recited in Claim 10 is met by Eggenberger in view of Yang further in view of Takami.
Regarding claim 12, Eggenberger in view of Yang further in view of Takami teaches the method of claim 10, and further teaches determining whether the adjusted modification parameter is within the range of values for the features of the modified media object, wherein the reward value is based on the determination (Takami, Para. [0057], the reward function may be a function that sets a reward value to 1 when a value of the state parameter relating to the piece of equipment operated using the control parameter output from the operation model satisfies the content of the target setting data, and sets a reward value to 0 when a value of the state parameter does not satisfy the content of the target setting data. The reward function may be a function that varies a reward value, according to a degree that a value of the state parameter relating to the piece of equipment 2 operated using the control parameter output from the operation model 401 deviates from a target range of the target setting data (i.e., reward value is based on determining if adjusted modification parameter is within the range of values)).
The proposed combination as well as the motivation for combining the Eggenberger, Yang, and Takami references presented in the rejection of Claim 1, apply to Claim 12 and are incorporated herein by reference. Thus, the method recited in Claim 12 is met by Eggenberger in view of Yang further in view of Takami.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022), as applied to claims 10 and 12 above, and further in view of Kunde et al. (US 2023/0305544 A1, filed February 13, 2023).
Regarding claim 13, Eggenberger in view of Yang further in view of Takami teaches the method of claim 10, and further teaches computing a (Eggenberger, Para. [0034], if the object is a digital image and an observer, in particular an evaluating entity in form of a user, looks at the image he may produce a certain reaction. This reaction may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object),
wherein the reward network comprises a neural network (Yang, Para. [0047], the reinforcement learning model (i.e., reward network) may include a neural network)
Although Eggenberger in view of Yang further in view of Takami teaches a reward value computed based on the context data (Eggenberger, Para. [0034]), they do not explicitly teach “the reward value includes the dynamic reward” However, in an analogous field of endeavor, Kunde teaches determining an optimal set of features using a dynamic reward function that changes across episodes (Kunde, Para. [0062]). The reward of choosing an action depends on the current state. In this scenario, the meta features selected form the state and the reward for selecting the next meta feature depends on the meta features that have already been selected (Kunde, Para. [0069]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang further in view of Takami with the teachings of Kunde by including that the reward function computed based on features of the media object is a dynamic reward. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for performing best feature selection using reinforcement learning, as recognized by Kunde. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Yang et al. (US 2020/0349119 A1) further in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) and Kunde et al. (US 2023/0305544 A1, filed February 13, 2023), as applied to claim 13 above, and further in view of Singhal et al. (US 2020/0293586 A1).
Regarding claim 14, Eggenberger in view of Yang further in view of Takami and Kunde teaches the method of claim 13, as described above.
Although Eggenberger in view of Yang further in view of Takami and Kunde teaches reinforcement learning using a dynamic reward function (Kunde, Para. [0062]), they do not explicitly teach “receiving instructor feedback based on the modified media object”, “computing a dynamic reward loss based on the instructor feedback” and “updating parameters of the reward network based on the dynamic reward loss”. However, in an analogous field of endeavor, Singhal teaches instead or in addition to using a loss function with regard to a classification task, the model may be trained via reinforcement learning with regard to reinforcement signals, e.g., with regard to user satisfaction or user-provided feedback regarding relevance of selected rich segment experiences. In either case, whether loss function or reinforcement learning is used, the system may be adjusted over time to “penalize” incorrect actions by adjusting parameters of the system so that the incorrect actions are less likely in the future given similar inputs, and to “reward” correct actions by adjusting parameters of the system so that the correct actions are more likely in the future. The loss function and/or reinforcement may be configured to more heavily reward and/or penalize answers in proportion to the confidence value (Singhal, Para. [0022]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Yang further in view of Takami and Kunde with the teachings of Singhal by including receiving user-provided feedback on the modified media object, determining a loss based on the feedback, and updating the parameters of the system based on the loss. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a model trained via reinforcement learning that rewards confident, correct answers, as recognized by Singhal. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 15, Eggenberger in view of Yang further in view of Takami, Kunde and Singhal teaches the method of claim 14, further comprising:
generating an additional modified media object, wherein the instructor feedback is based on the additional modified media object (Eggenberger, Para. [0034], This reaction may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object. This process may be repeated until the user, observing the constantly modified object, may be satisfied. The satisfaction may be measurable and may, at the same time, produce the highest determinable reward for the software agent in the feedback loop).
Regarding claim 16, Eggenberger in view of Yang further in view of Takami, Kunde and Singhal teaches the method of claim 15, further comprising:
including in a dataset a first trajectory corresponding to the modified media object, a second trajectory corresponding to the additional modified media object, and the instructor feedback (Kunde, Para. [0033], the memory includes a database that stores a) a plurality of datasets, also referred as datasets, which define the problem space, b) a plurality of regression algorithms, also referred to as candidate algorithms, c) a meta data repository, d) a knowledgebase storing the set of hyperparameters identified for each regression algorithm, e) a learner dataset and the like. This meta data repository contains a) the classification meta features, b) the regression meta features, and c) the domain meta features Further, the memory may comprise information pertaining to input(s)/output(s) of each step performed by the processor(s) of the system and methods of the present disclosure).
The proposed combination as well as the motivation for combining the Eggenberger, Yang, Takami, Kunde, and Singhal references presented in the rejection of Claim 14, apply to Claim 16 and are incorporated herein by reference. Thus, the method recited in Claim 16 is met by Eggenberger in view of Yang further in view of Takami, Kunde and Singhal.
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) further in view of Wilensky (US 2020/0394773 A1).
Regarding claim 17, Eggenberger teaches an apparatus for media generation, comprising:
a processor (Eggenberger, Para. [0063], devices used herein may include one or more processors); and
a memory including instructions executable by the processor (Eggenberger, Para. [0064], one or more application programs are stored on one or more of the computer readable storage media for execution by the one or more processors) to perform the steps of:
obtaining a media object and context data describing a context of the media object (Eggenberger, Para. [0050], the method comprises receiving an object as input for the GAN. In a next process step, a set of features, in particular contextual attributes or real attributes of the input, are determined. Para. [0044], the object may be an image (in particular, in a digital representation)), wherein the media object comprises one or more modification parameters (Eggenberger, Para. [0044], the set of features may comprise at least one of the group comprising a geometrical form in the image an orientation of the geometrical form, a color value of the geometrical form, a size of the geometrical form, a position of the geometrical form, pixel density, and a brightness value. Para. [0025], the term ‘modification to one feature’ may denote that at least one of the attributes describing the feature may be altered. It may relate to an orientation of the feature, its rotation angle, a change in the underlying color space (varying values of the RGB color space), and application of image filters, or any other observable modification of an appearance of the feature).
Although Eggenberger teaches a feedback value based on a reaction to a modified media object in context (Eggenberger, Para. [0034]), Eggenberger does not explicitly teach “selecting, by a reinforcement learning model, an action for modifying the one or more modification parameters based on a range of values for features of the modified object”. However, in an analogous field of endeavor, Takami teaches a reward function may be a function that varies a reward value, according to a degree that a value of the state parameter relating to the piece of equipment deviates from a target range of the target setting data (i.e.., range of values for features of the object) (Takami, Para. [0057]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger with the teachings of Takami by including computing a reward value based on a range of values for features of the object (i.e., the modified media object of Eggenberger). One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for optimizing a model using a reward function, as recognized by Takami.
Although Eggenberger in view of Takami teaches performing at least one modification to at least one feature of the set of features of the object (Eggenberger, Para. [0052]), they do not explicitly teach “generating, by a media editing application, a modified media object by iterative optimizing the one or more modification parameters based on the action”. However, in an analogous field of endeavor, Wilensky teaches a digital image editing engine that can create, generate, edit, modify, render and/or display an enhanced digital image. For example, the digital image enhancement system can identify modifications and make modifications in relation to a digital image (Wilensky, Para. [0142]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Takami with the teachings of Wilensky by including a media editing application to adjust parameters of a digital image. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for modifying parameter values to generate an enhanced digital image, as recognized by Wilensky. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 18, Eggenberger in view of Takami further in view of Wilensky teaches the apparatus of claim 17, further comprising:
a contextual media interface configured to display the modified media object within the context (Wilensky, Para. [0060], the digital image enhancement system can generate and provide the enhanced digital image for display via the digital image enhancement user interface).
The proposed combination as well as the motivation for combining the Eggenberger, Takami and Wilensky references presented in the rejection of Claim 17, apply to Claim 18 and are incorporated herein by reference. Thus, the system recited in Claim 18 is met by Eggenberger in view of Takami further in view of Wilensky.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) further in view of Wilensky (US 2020/0394773 A1), as applied to claims 17 and 18 above, and further in view of Yang et al. (US 2020/0349119 A1) and Kunde et al. (US 2023/0305544 A1, filed February 13, 2023).
Regarding claim 19, Eggenberger in view of Takami further in view of Wilensky teaches the apparatus of claim 17, further comprising:
wherein the (Eggenberger, Para. [0034], if the object is a digital image and an observer, in particular an evaluating entity in form of a user, looks at the image he may produce a certain reaction. This reaction may be measurable as a feedback value which may be fed back to an optimization algorithm or a reinforcement learning agent into the generative component of the GAN in order to produce immediately another modified object).
Although Eggenberger in view of Takami further in view of Wilensky teaches a reward based on context data (Eggenberger, Para. [0034]), they do not explicitly teach “a reward network comprising a neural network”. However, in an analogous field of endeavor, Yang teaches the reinforcement learning model (i.e., reward network) may include a neural network (Yang, Para. [0047]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Takami further in view of Wilensky with the teachings of Yang by including that the reinforcement model is a neural network. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for promoting better decisions by the reinforcement learning model, as recognized by Yang.
Although Eggenberger in view of Yang further in view of Takami teaches a reward value computed based on the context data (Eggenberger, Para. [0034]), they do not explicitly teach the reward network is “configured to compute a dynamic reward” and “wherein the reward value includes the dynamic reward” However, in an analogous field of endeavor, Kunde teaches determining an optimal set of features using a dynamic reward function that changes across episodes (Kunde, Para. [0062]). The reward of choosing an action depends on the current state. In this scenario, the meta features selected form the state and the reward for selecting the next meta feature depends on the meta features that have already been selected (Kunde, Para. [0069]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Takami further in view of Wilensky and Yang with the teachings of Kunde by including that the reward function computed based on features of the media object is a dynamic reward. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for performing best feature selection using reinforcement learning, as recognized by Kunde. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Eggenberger et al. (US 2021/0350223 A1) in view of Takami et al. (US 2023/0129189 A1, filed October 20, 2022) further in view of Wilensky (US 2020/0394773 A1), Yang et al. (US 2020/0349119 A1) and Kunde et al. (US 2023/0305544 A1, filed February 13, 2023) as applied to claim 19 above, and further in view of Singhal et al. (US 2020/0293586 A1).
Regarding claim 20, Eggenberger in view of Takami further in view of Wilensky, Yang and Kunde teaches the apparatus of claim 19, as described above.
Although Eggenberger in view of Takami further in view of Wilensky, Yang and Kunde teaches reinforcement learning uses rewards and punishments as signals for positive and negative behavior (Eggenberger, Para. [0030]), they do not explicitly teach “a training component configured to update parameters of the reward network based on instructor feedback”. However, in an analogous field of endeavor, Singhal teaches instead or in addition to using a loss function with regard to a classification task, the model may be trained via reinforcement learning with regard to reinforcement signals, e.g., with regard to user satisfaction or user-provided feedback regarding relevance of selected rich segment experiences. In either case, whether loss function or reinforcement learning is used, the system may be adjusted over time to “penalize” incorrect actions by adjusting parameters of the system so that the incorrect actions are less likely in the future given similar inputs, and to “reward” correct actions by adjusting parameters of the system so that the correct actions are more likely in the future (Singhal, Para. [0022]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Eggenberger in view of Takami further in view of Wilensky, Yang and Kunde with the teachings of Singhal by including receiving user-provided feedback on the modified media object and updating the parameters of the system based on the feedback. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a model trained via reinforcement learning that rewards confident, correct answers, as recognized by Singhal. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662