Detailed Action
This Office Action is in response to the remarks entered on 04/07/2026. Claims 1-8 and 11-18 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-7, 11-14, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Shinn et al. (US 20180314942 A1, hereinafter ‘Shinn’) in view of SINGH et al. (US 20220024032 A1, hereinafter ‘Singh’) and further in view of Wild & Srinivasa (US 20180174054 A1, hereinafter ‘Wild’).
Regarding claim 1, Shinn teaches:
A system for calibrating an autonomous agent, the system comprising: ([Shinn, 0003] The autonomous agents in the game (NPCs and events) are implemented using a decision tree and/or state machine. [Fig. 18] discloses the agent controller being updated 1803 based on perception input 1813)
- the autonomous agent having a teachable behavior model assigned thereto, the teachable behavior model comprising ([Shinn, 0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state. [Shinn, 0003] discloses that the state corresponds to the pre-programmed states in a state machine)
- an active state selected from a plurality of pre-programmed behavior states of the teachable behavior model; and ([Shinn, 0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state. [Shinn, 0003] discloses that the state corresponds to the pre-programmed states in a state machine)
- one or more teachable parameter for governing transition of the active state to another one of the plurality of pre-programmed behavior states, wherein descriptive metadata associated with at least one teachable parameter specifies an observable behavioral effect associated with adjustment of the at least one teachable parameter; ([Shinn, 0007-0008; Fig. 3 and 4] and [0027] collectively disclose that the AI neural network models in the agents such as NPCs and events are teachable (i.e., training, testing, and validating) and can be adjusted (i.e., belief update). [0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state (i.e., transitioning of action state to another). [0030] shows that the determination made by the AI is based on input, beliefs, and goals)
- a controlled environment comprising one or more teaching fixture and configured to deploy the autonomous agent thereinto, the one or more teaching fixture comprising one or more sensors installed in the controlled environment and configured to monitor, inspect, or interact with the autonomous agent; ([Shinn, 0027] and [Shinn, 0091, 0095 and Fig. 11] collectively shows that the autonomous AI character receives sensor inputs (i.e., teaching fixtures, interacting with the autonomous agent) and detects movements, voices, and obstacles in the game environment (i.e., the controlled environment). [Shinn, 0095] discloses utilizing the trained agents in the in-game environment 1151 which interacts with objects 1171, 1173, and 1179. [0084-0085 and Fig. 10] Each agent has solver 1021 and the solvers were trained as disclosed in [0094])
- an uncontrolled environment, comprising interacting elements, [Shinn, 0091, 0095 and Fig. 11] shows that the system includes GAME AI platform 1101 and IN GAME ENVIRONMENT 1151. The GAME AI PLATFORM 1101 is interpreted as the uncontrolled environment with interacting elements (planning problem descriptions) 1111, 1113 … 1119)
- a calibration module comprising: ([Shinn, 0132-0133] The agent controller and the AI planning is the calibration module. [0130] The AI platform is implemented using an AI processor)
- one or more calibration processor configured to: ([Shinn, 0132-0133] The agent controller and the AI planning is the calibration module. [0130] The AI platform is implemented using an AI processor)
- receive, from a training agent, a tuning command comprising one or more contextual condition; ([0134] The perception input is created to reflect the detected change, and [0135] a belief is updated, and then [0136] AI planning is performed based on the updated set of beliefs (i.e., tuning command) at 1707)
- into a calibrated behavior model. ([0134] The perception input is created to reflect the detected change, and [0135] a belief is updated, and then [0136] AI planning is performed based on the updated set of beliefs (i.e., tuning command) at 1707. The AI planning process using the updated set of data is equivalent to the calibration process)
- an evaluation module comprising one or more evaluation processor configured to: ([0052], [0057] and [0059] collectively disclose the validation and testing process of the machine learning model. [0130] The AI platform is implemented using an AI processor)
- evaluate, ; ([0052], [0057] and [0059] collectively disclose the validation and testing process of the machine learning model using the validating data points, which are a set of vectors for every k problems in the problem set (interacting elements). [0097] This paragraph further explains the relationship between the planning problems and the objects in the game environment. The actions generated based on the planning problems (interacting elements) changes the game environment including game objects and also each agent)
wherein the one or more teaching fixture is configured to mimic the interacting element configuration ([Shinn, 0027] and [Shinn, 0091, 0095 and Fig. 11] collectively shows that the in-game environment is configured with auditory and visual sensor 1035, and the autonomous AI character receives sensor inputs (i.e., teaching fixtures, interacting with the autonomous agent) and detects movements, voices, and obstacles in the game environment (i.e., the controlled environment). [0097] The actions generated based on the planning problems (interacting elements) changes the game environment including game objects and also each agent. This process changes the In-Game environment (controlled environment) based on the training process performed in the uncontrolled environment)
wherein the target behavior comprises the improvement objective. ([0042] and [0045-0046] collectively disclose creating a set of problem specifications at 109 of Fig. 1 based on the domain specification and problem generator parameters)
However, Shinn does not specifically disclose:
an uncontrolled environment … and configured to deploy the autonomous agent thereinto;
- compute by interpreting the tuning command using the descriptive metadata associated with the at least one teachable parameter, a change to the at least one teachable parameter to reduce a difference between an observed behavior and a target behavior
- evaluate, against a target interaction performance, an observed interaction performance of the autonomous agent with the interacting elements;
- identify, from the interacting elements, an interacting element configuration contributing to the observed interaction performance; and
- identify, from the observed interaction performance, an improvement objective;
when the autonomous agent is deployed in the controlled environment
Singh teaches:
an uncontrolled environment … and configured to deploy the autonomous agent thereinto; ([Singh, 0085] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot (i.e., deploying to the uncontrolled environment) if the retrained AI/ML model meets the one or more performance thresholds at 765)
- evaluate, against a target interaction performance, an observed interaction performance of the autonomous agent with the interacting elements; ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model (i.e., autonomous agents with interacting elements) based on the measured model drift at 780 based on the real data, the model returns to step 755 (deploying to controlled environment) for further retraining)
- identify, from the interacting elements, an interacting element configuration contributing to the observed interaction performance; and ([Singh, 0086] discloses determining whether the new version of the model (i.e., interacting element configuration that interacts with elements, which is real data) contributes to the performance of the AI/ML model. If not, the retrained AI/ML model is discarded. The trained version of the model and previous version of the model are selected based on input data and/or the use case)
- identify, from the observed interaction performance, an improvement objective; ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780 based on the real data (i.e., identifying new improvement objective based on the interacting elements), the model returns to step 755 (deploying to controlled environment) for further retraining)
the autonomous agent is deployed in the controlled environment ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780, the model returns to step 755 (deploying to controlled environment) for further retraining)
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn and Singh to use the method of deploying the model that meets the performance threshold of Singh to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the accuracy of the autonomous agent learning systems by adjusting the model again using the real-world data and to prevent model drift [Singh, ABSTRACT].
However, Shinn in view of Singh does not specifically disclose:
- compute by interpreting the tuning command using the descriptive metadata associated with the at least one teachable parameter, a change to the at least one teachable parameter to reduce a difference between an observed behavior and a target behavior
Wild teaches:
- compute by interpreting the tuning command using the descriptive metadata associated with the at least one teachable parameter, a change to the at least one teachable parameter to reduce a difference between an observed behavior and a target behavior ([Wild, 0029-0030] discloses that Stage 1 weights are updated for subsequent uses of the synapses and Stage 2 weight is used as a reference weight for balancing the learning process and the changes to the Stage 1 weight. This indicates that the Stage 2 weight contributes to the model change which further means that the weights are metadata. [Wild, 0050] The learning algorithm operates to move the weight value to the reference weight value. [Wild, 0051] discloses the reinforcement signal (tuning command) adjusting the weights. [Wild, 0052] If the prior weight change (change to the one or more teachable parameter) was significant (670), then the weight value is persistently memorized)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having both the teachings of Shinn, Singh and Wild to use the method of calculating changes to the one or more teachable parameter of Wild to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the efficiency of the autonomous agent learning systems by introducing features of unsupervised learning method into a generic machine learning method [Wild, 0019].
Regarding claim 2, Shinn in view of Singh teaches:
The system of claim 1, wherein: - the one or more evaluation processor is further configured to: - identify a subsequent improvement objective from the observed interaction performance when the autonomous agent is deployed in the uncontrolled environment. ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. The model drift is measured based on the real data (i.e., interactions with objects). If the model performance is not better than the previous version of the model based on the measured model drift at 780 (Subsequent improvement objective), the model returns to step 755 (deploying to controlled environment) for further retraining)
Regarding claim 3, Shinn in view of Singh teaches:
The system of claim 2 wherein the interacting element configuration comprises an arrangement or state of the interacting elements contributing to the observed interaction performance when the autonomous agent is deployed in the uncontrolled environment. ([Singh, 0086] discloses determining, after deploying the model, whether the new version of the model (i.e., interacting element configuration that interacts with elements, which is real data) contributes to the performance of the AI/ML model. If not, the retrained AI/ML model is discarded. The trained version of the model and previous version of the model are selected based on input data and/or the use case, which indicates that a configuration of a model is selected from a list of two models)
Regarding claim 4, Shinn in view of Singh teaches:
wherein the autonomous agent is an autonomous robot, and the controlled environment is a development environment. ([Singh, 0016, 0018 and 0025] collectively discloses that the invention provides improvement to a robot development environment)
Regarding claim 6, Shinn teaches:
wherein the autonomous agent is a non-playing character in a digital interactive production, and the controlled environment is a development scene. ([Shinn, 0201] shows that the autonomous agents are AI non-player character in a game)
Regarding claim 7, Shinn teaches:
wherein the autonomous agent is a decision agent controlling one or more object of the controlled environment, and the controlled environment is a development scene. ([Shinn, 0201] shows that the autonomous agents are AI non-player character in a game)
Regarding claim 11, Shinn teaches:
a method for calibrating an autonomous agent, the method comprising: ([Shinn, 0003] The autonomous agents in the game (NPCs and events) are implemented using a decision tree and/or state machine. [Fig. 18] discloses the agent controller being updated 1803 based on perception input 1813)
- defining, from an uncalibrated behavior model adjusted by a plurality of model parameters, a teachable behavior model, the plurality of model parameters comprising: ([Shinn, 0064-0065] discloses generating initial machine learning models based on the construction parameters received at 503)
- an active state selected from a plurality of pre-programmed behavior states of the teachable behavior model; and ([Shinn, 0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state. [Shinn, 0003] discloses that the state corresponds to the pre-programmed states in a state machine)
- one or more teachable parameter for governing transition of the active state to another one of the plurality of pre-programmed behavior states, wherein descriptive metadata associated with at least one teachable parameter specifies an observable behavioral effect associated with adjustment of the at least one teachable parameter; and ([Shinn, 0007-0008; Fig. 3 and 4] and [0027] collectively disclose that the AI neural network models in the agents such as NPCs and events are teachable (i.e., training, testing, and validating) and can be adjusted (i.e., belief update). [0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state (i.e., transitioning of action state to another). [0030] shows that the determination made by the AI is based on input, beliefs, and goals)
- calibrating the teachable behavior model into a calibrated behavior model by: ([Shinn, 0132-0133] The agent controller and the AI planning is the calibration module. [0130] The AI platform is implemented using an AI processor)
([Shinn, 0091, 0095 and Fig. 11] shows that the system includes GAME AI platform 1101 and IN GAME ENVIRONMENT 1151. The GAME AI PLATFORM 1101 is interpreted as the uncontrolled environment with interacting elements (planning problem descriptions) 1111, 1113 … 1119. [Shinn, 0095] discloses utilizing the trained agents in the in-game environment 1151 which interacts with objects 1171, 1173, and 1179. [0084-0085 and Fig. 10] Each agent has solver 1021 and the solvers were trained as disclosed in [0094])
- evaluating, ([Shinn, 0052], [0057] and [0059] collectively disclose the validation and testing process of the machine learning model using the validating data points (interacting elements). [0097] This paragraph further explains the relationship between the planning problems and the objects in the game environment. The actions generated based on the planning problems (interacting elements) changes the game environment including game objects and also each agent)
- assembling a controlled environment comprising one or more teaching fixture, the one or more teaching fixture comprising one or more sensors installed in the controlled environment and configured to monitor, inspect, or interact with the autonomous agent; ([Shinn, 0027] and [Shinn, 0091, 0095 and Fig. 11] collectively shows that the in-game environment is configured with auditory and visual sensor 1035, and the autonomous AI character receives sensor inputs (i.e., teaching fixtures, interacting with the autonomous agent) and detects movements, voices, and obstacles in the game environment (i.e., the controlled environment))
- mimicking, in the controlled environment using the one or more teaching fixture, the interacting element configuration; ([Shinn, 0027] and [Shinn, 0091, 0095 and Fig. 11] collectively shows that the in-game environment is configured with auditory and visual sensor 1035, and the autonomous AI character receives sensor inputs (i.e., teaching fixtures, interacting with the autonomous agent) and detects movements, voices, and obstacles in the game environment (i.e., the controlled environment). [0097] The actions generated based on the planning problems (interacting elements) changes the game environment including game objects and also each agent. This process changes the In-Game environment (controlled environment) based on the training process performed in the uncontrolled environment)
[Shinn, 0095] discloses utilizing the trained agents in the in-game environment 1151 which interacts with objects 1171, 1173, and 1179. [0084-0085 and Fig. 10] Each agent has solver 1021 and the solvers were trained as disclosed in [0094])
- receiving, from a training agent, a tuning command comprising one or more contextual condition; and ([Shinn, 0134] The perception input is created to reflect the detected change, and [0135] a belief is updated, and then [0136] AI planning is performed based on the updated set of beliefs (i.e., tuning command) at 1707)
Shinn does not specifically disclose:
- deploying, into an uncontrolled environment comprising interacting elements, the autonomous agent having the teachable behavior model assigned thereto
- evaluating, against a target interaction performance, an observed interaction performance of the autonomous agent with the interacting elements;
- identifying, from the interacting elements, an interacting element configuration contributing to the observed interaction performance;
- identifying, from the observed interaction performance, an improvement objective;
- until a difference between an observed behavior and a target behavior is within a target threshold: - altering the one or more teachable parameter to reduce the difference therebetween.
- computing, by interpreting the tuning command using the descriptive metadata associated with the at least one teachable parameter, a change to the at least one teachable parameter to reduce the difference therebetween
Singh teaches:
- deploying, into an uncontrolled environment comprising interacting elements, the autonomous agent having the teachable behavior model assigned thereto; ([Singh, 0085] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot (i.e., deploying to the uncontrolled environment) if the retrained AI/ML model meets the one or more performance thresholds at 765)
- evaluating, against a target interaction performance, an observed interaction performance of the autonomous agent with the interacting elements; ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780 based on the real data (i.e., interacting elements), the model returns to step 755 (deploying to controlled environment) for further retraining)
- identifying, from the interacting elements, an interacting element configuration contributing to the observed interaction performance; ([Singh, 0086] discloses determining whether the new version of the model (i.e., interacting element configuration that interacts with elements, which is real data) contributes to the performance of the AI/ML model. If not, the retrained AI/ML model is discarded)
- identifying, from the observed interaction performance, an improvement objective; ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780 based on the real data (i.e., identifying new improvement objective based on the interacting elements), the model returns to step 755 (deploying to controlled environment) for further retraining)
- deploying, in the controlled environment, the autonomous agent having the teachable behavior model assigned thereto; ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780, the model returns to step 755 (deploying to controlled environment) for further retraining)
- until : - altering the one or more teachable parameter [Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds (i.e., target threshold) at 765. If the model performance is not better than the previous version of the model based on the measured model drift at 780, the model returns to step 755 (deploying to controlled environment) for further retraining (i.e., altering teachable parameter))
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn and Singh to use the method of deploying the model that meets the performance threshold of Singh to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the accuracy of the autonomous agent learning systems by adjusting the model again using the real-world data and to prevent model drift [Singh, ABSTRACT].
However, Shinn in view of Singh does not specifically disclose:
- until a difference between an observed behavior and a target behavior is within a target threshold: - altering the one or more teachable parameter to reduce the difference therebetween.
- computing, by interpreting the tuning command using the descriptive metadata associated with the at least one teachable parameter, a change to the at least one teachable parameter to reduce the difference therebetween
Wild teaches:
- until a difference between an observed : - altering the one or more teachable parameter to reduce the difference therebetween. ([Wild, 0052] discloses determining whether the difference between the prior weight and the current weight are significant or not (i.e., within the threshold). If the prior weight change (change to the one or more teachable parameter) was significant (670), then the weight value is persistently memorized)
- computing, by interpreting the tuning command using the to reduce the difference therebetween ([Wild, 0029-0030] discloses that Stage 1 weights are updated for subsequent uses of the synapses and Stage 2 weight is used as a reference weight for balancing the learning process and the changes to the Stage 1 weight. This indicates that the Stage 2 weight contributes to the model change which further means that the weights are metadata. [Wild, 0050] The learning algorithm operates to move the weight value to the reference weight value. [Wild, 0051] discloses the reinforcement signal (tuning command) adjusting the weights. [Wild, 0052] If the prior weight change (change to the one or more teachable parameter) was significant (670), then the weight value is persistently memorized)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having both the teachings of Shinn, Singh and Wild to use the method of calculating changes to the one or more teachable parameter of Wild to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the efficiency of the autonomous agent learning systems by introducing features of unsupervised learning method into a generic machine learning method [Wild, 0019].
Regarding claim 12, Shinn in view of Singh teaches:
The method of claim 11, further comprising:
- identifying, from the observed interaction performance, a subsequent improvement objective. ([Singh, 0085-0086] and [Fig. 7A-7B] collectively disclose deploying the trained ML model to the robot if the retrained AI/ML model meets the one or more performance thresholds at 765. The model drift is measured based on the real data (i.e., interactions with objects). If the model performance is not better than the previous version of the model based on the measured model drift at 780 (Subsequent improvement objective), the model returns to step 755 (deploying to controlled environment) for further retraining)
Regarding claim 13, Shinn in view of Singh teaches:
The method of claim 12, wherein the interacting element configuration comprises an arrangement or state of the interacting elements contributing to the observed interaction performance. ([Singh, 0086] discloses determining, after deploying the model, whether the new version of the model (i.e., interacting element configuration that interacts with elements, which is real data) contributes to the performance of the AI/ML model. If not, the retrained AI/ML model is discarded. The trained version of the model and previous version of the model are selected based on input data and/or the use case, which indicates that a configuration of a model is selected from a list of two models)
Regarding claim 14, Shinn in view of Singh teaches:
The method of claim 11, wherein the autonomous agent is an autonomous robot, and the controlled environment is a development environment. ([Singh, 0016, 0018 and 0025] collectively discloses that the invention provides improvement to a robot development environment)
Regarding claim 16, Shinn teaches:
The method of claim 11, wherein the autonomous agent is a non-playing character in a digital interactive production, and the controlled environment is a development scene. ([Shinn, 0201] shows that the autonomous agents are AI non-player character in a game and have many benefits for developers)
Regarding claim 17, Shinn teaches:
The method of claim 11, wherein the autonomous agent is a decision agent controlling one or more object of the controlled environment, and the controlled environment is a development scene. ([Shinn, 0201] shows that the autonomous agents are AI non-player character in a game and have many benefits for developers)
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Shinn in view of Singh in view of Wild and further in view of Kim et al (US 20210390778 A1, hereinafter ‘Kim’).
Regarding claim 5, Shinn in view of Singh and further in view of Wild teaches the system of claim 1.
Shinn in view of Singh and further in view of Wild does not specifically disclose wherein the autonomous agent is a virtual actor in a digital media production, and the controlled environment is a virtual scene.
Kim teaches wherein the autonomous agent is a virtual actor in a digital media production, and the controlled environment is a virtual scene. ([Kim, 0047] discloses that a simulation can be used to test systems such as robotic systems. In at least one embodiment, actions of a virtual agent (or actor, virtual robot, etc.) in a simulation can be learned by watching that agent interact with an environment)
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn, Singh, Wild and Kim to use the method (wherein the autonomous agent is a virtual actor in a digital media production) of Kim to implement the method of Shinn. The suggestion and/or motivation for doing so is to enable the autonomous agent learning systems to be applied in a wider range of fields.
Regarding claim 15, Shinn in view of Singh in view of Wild teaches the method of claim 11.
Shinn in view of Singh and further in view of Wild does not specifically disclose:
wherein the autonomous agent is a virtual actor in a digital media production, and the controlled environment is a virtual scene.
Kim teaches:
wherein the autonomous agent is a virtual actor in a digital media production, and the controlled environment is a virtual scene. ([Kim, 0047] discloses that a simulation can be used to test systems such as robotic systems. In at least one embodiment, actions of a virtual agent (or actor, virtual robot, etc.) in a simulation can be learned by watching that agent interact with an environment)
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn, Singh, Wild, and Kim to use the method (wherein the autonomous agent is a virtual actor in a digital media production) of Kim to implement the method of Shinn. The suggestion and/or motivation for doing so is to enable the autonomous agent learning systems to be applied in a wider range of fields.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Shinn in view of Singh in view of Wild and further in view of Groble (US 7818271 B2, hereinafter ‘Groble’).
Regarding claim 8, Shinn teaches:
wherein the one or more teachable parameter comprises governing transition of the active state to another one of the plurality of pre-programmed behavior states. ([Shinn, 0007-0008; Fig. 3 and 4] and [0027] collectively disclose that the AI neural network models in the agents such as NPCs and events are teachable (i.e., training, testing, and validating) and can be adjusted (i.e., belief update). [0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state (i.e., transitioning of action state to another). [0030] shows that the determination made by the AI is based on input, beliefs, and goals)
However, Shinn, Singh, Wild does not specifically disclose wherein the one or more teachable parameter comprises at least one of a preference score and a rule-based system.
Groble teaches:
wherein the one or more teachable parameter comprises at least one of a preference score and a rule-based system. ([Groble, Claim 1] discloses each of the plurality of parameters of a model representing a user preference (preference score), and the parameter learning which ones of the plurality of interaction policies (rules) are within a specific tolerance of an optimal interaction policy for the plurality of user models)
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn, Singh, Wild, Singh and Groble to use the teachable parameter that comprises preference scores and rule-based system of Groble to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the performance of the autonomous agent tuning system by utilizing more diverse type of information that helps the autonomous agent to mimic user behaviors.
Regarding claim 18, Shinn teaches:
The method of claim 11, wherein the one or more teachable parameter comprises governing transition of the active state to another one of the plurality of pre-programmed behavior states. ([Shinn, 0007-0008; Fig. 3 and 4] and [0027] collectively disclose that the AI neural network models in the agents such as NPCs and events are teachable (i.e., training, testing, and validating) and can be adjusted (i.e., belief update). [0088] discloses AI loop (the teachable behavior model) selecting the best action list for the current state (i.e., transitioning of action state to another). [0030] shows that the determination made by the AI is based on input, beliefs, and goals)
However, Shinn in view of Singh in view of Wild does not specifically disclose:
wherein the one or more teachable parameter comprises at least one of a preference score and a rule-based system
Groble teaches:
wherein the one or more teachable parameter comprises at least one of a preference score and a rule-based system ([Groble, Claim 1] discloses each of the plurality of parameters of a model representing a user preference (preference score), and the parameter learning which ones of the plurality of interaction policies (rules) are within a specific tolerance of an optimal interaction policy for the plurality of user models)
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention, having both the teachings of Shinn, Singh, Wild, and Groble to use the teachable parameter that comprises preference scores and rule-based system of Groble to implement the method of Shinn. The suggestion and/or motivation for doing so is to improve the performance of the autonomous agent tuning system by utilizing more diverse type of information that helps the autonomous agent to mimic user behaviors.
Response to Arguments
35 U.S.C. 103 Rejection
Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUN KWON whose telephone number is (571)272-2072. The examiner can normally be reached Monday – Friday 8:00AM – 5:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUN KWON/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights