DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
The claims being considered in this application are those submitted on 12/18/2024. Claims 1-11 are pending.
Priority
The applicant’s claim to priority of JP2023-216539 on 12/22/2023 is acknowledged.
Information Disclosure Statement
The information disclosure statement(s) filed on 12/18/2024 and 06/20/2025 been annotated and considered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20020016128 A1, IDS) in view of Sakaue et. al. (US 20020165642 A1).
Regarding Claim 1, Saito discloses:
A robot control device comprising one or more processors configured to: (See at least Figure 2 via Control Device 10 and ¶0041 via "The control unit 10 mainly comprises a microcomputer, RAM, and ROM or the like")
cause a robot to make motions in accordance with respective motion patterns each of which is made up of a combination of motion elements, in response to detecting an action from outside with a sensor; (See at least ¶0008 via "an interactive toy comprising a stimulus detecting member for detecting an inputted stimulus, an actuating member for actuating the interactive toy, and a control member for controlling the action member to make the interactive toy take reaction behavior to the stimulus detected from the stimulus detecting member" as well as ¶0040 via "Here, the stimulus sensors 5 are sensors that detect the stimulus received from the outside. A touch sensor, an optical sensor, and a microphone or the like are used therein."; ¶0041 via "The control unit 10 mainly comprises a microcomputer, RAM, and ROM or the like"; ¶0062 and Figure 5 which provide an example of the dog type robot detecting a stimulus (user hitting robot on head - "i-01"), and the randomly selecting a behavior pattern which comprises a combination of motion elements: "…reaction behavior pattern 31 is selected based on a random number, the voice "vce(01)" and the action "act(01)" will be selected. As a result, according to FIGS. 9 and 10, the dog type robot 1 "draws back" yelping "yap!", that is, the dog type robot 1 takes the same action as an actual dog" **Wherein the motion patterns are comprised of motion elements such as voice + action)
derive respective evaluation values of the motion patterns; and (See at least ¶0053 via "The action point is counted (added/subtracted) to the total value of the action points, and the latest total value is stored in the RAM. Here, an "action point" means a generated score caused by the reaction behavior (output) of the dog type robot 1. The total value of the action points corresponds to the level of communication between the dog type robot 1 and a user")
generate a new motion pattern (See at least ¶0057 via " The character state update determination unit 17 suitably updates the value of the character parameter XY based on the total value of the action points." and ¶0065 via "Thus, the basic posture or the character and behavior tendency, or the like, is set to each character parameter XY. In addition, as described later, the character parameter XY in the third stage is updated suitably by the total value of the action points generated according to the reaction behavior (output) performed by the dog type robot 1.").
However, Saito does not explicitly disclose the combining of motion elements to form a new motion pattern.
Nevertheless, Sakaue--who is directed towards a user-machine interface system for enhanced interaction--discloses: generate a new motion pattern by combining, (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
Regarding Claim 2, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein the one or more processors are configured to derive the respective evaluation values of the motion patterns based on detection results by the sensor after the motions are made in accordance with the respective motion patterns (See at least Figure 16 and ¶0078 via "a reaction behavior pattern corresponding to the recognized inputted stimulus is selected (Step 51), the output of the actuators 3 and the speaker 4 are controlled according to the selected reaction behavior pattern (Step 52). Then, the action point VTxyi corresponding to the output control period is calculated (Step 53)" *Wherein the evaluation values (action point) is derived after the executed motions that are made in based on the inputted stimulus)
Regarding Claim 9, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein the motion elements of the motion patterns include at least one of presence or absence of movement of a predetermined part, a speed of the movement of the predetermined part, a pitch of a sound to be output, or a length of the sound to be output (See at least Figure 10 which illustrates various motion elements that correspond to movement of (a) predetermined part(s), also see Figure 9 which illustrates voice lines)
Regarding Claim 10, Modified Saito discloses:
A robot comprising: the robot control device according to Claim 1; and the sensor (See at least Claim 1 rejection, and also see Saito via Figure 1 which illustrates the dog type robot 1 and the stimulus sensors 5).
Regarding Claim 11, Saito discloses:
A robot control method that controls a robot including a sensor that detects an action from outside, comprising: (See at least Figures 12-17 as well as ¶0040 via "Here, the stimulus sensors 5 are sensors that detect the stimulus received from the outside")
causing the robot to make motions in accordance with respective motion patterns each of which is made up of a combination of motion elements, in response to detecting the action from the outside with the sensor; (See at least ¶0008 via "an interactive toy comprising a stimulus detecting member for detecting an inputted stimulus, an actuating member for actuating the interactive toy, and a control member for controlling the action member to make the interactive toy take reaction behavior to the stimulus detected from the stimulus detecting member" as well as ¶0040 via "Here, the stimulus sensors 5 are sensors that detect the stimulus received from the outside. A touch sensor, an optical sensor, and a microphone or the like are used therein."; ¶0041 via "The control unit 10 mainly comprises a microcomputer, RAM, and ROM or the like"; ¶0062 and Figure 5 which provide an example of the dog type robot detecting a stimulus (user hitting robot on head - "i-01"), and the randomly selecting a behavior pattern which comprises a combination of motion elements: "…reaction behavior pattern 31 is selected based on a random number, the voice "vce(01)" and the action "act(01)" will be selected. As a result, according to FIGS. 9 and 10, the dog type robot 1 "draws back" yelping "yap!", that is, the dog type robot 1 takes the same action as an actual dog" **Wherein the motion patterns are comprised of motion elements such as voice + action)
deriving respective evaluation values of the motion patterns; and (See at least ¶0053 via "The action point is counted (added/subtracted) to the total value of the action points, and the latest total value is stored in the RAM. Here, an "action point" means a generated score caused by the reaction behavior (output) of the dog type robot 1. The total value of the action points corresponds to the level of communication between the dog type robot 1 and a user")
However, Saito does not explicitly disclose the combining of motion elements to form a new motion pattern.
Nevertheless, Sakaue--who is directed towards a user-machine interface system for enhanced interaction--discloses: generating a new motion pattern by combining, based on the evaluation values, the motion elements of the motion patterns (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20020016128 A1, IDS) and Sakaue et. al. (US 20020165642 A1) in view of Sabe et. al. (US 20030045203 A1, IDS).
Regarding Claim 3, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein the one or more processors are configured to: select, from among the motion patterns, (See at least ¶0061 and Figures 5-7 which illustrate reaction behavior patterns)
However, Saito does not explicitly disclose the combining of motion elements to form a new motion pattern.
Nevertheless, Sakaue discloses: generate the new motion pattern by combining the motion elements of the at least two motion patterns (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
However, although Saito discloses motion patterns with evaluation values (See at least Saito ¶0053 via "action points") as well as motion patterns being linked to probabilities (See at least Saito ¶0052 via "appearance probability"), Modified Saito does not explicitly disclose the selecting the patterns with higher evaluation values at a higher probability.
Nevertheless, Sabe--who is directed towards a robot apparatus and control method for judging character--discloses: a motion pattern having a higher evaluation value among the motion patterns with a higher probability to select (See at least ¶0108 via "On the basis of this recognition result and a notice from the action switching module 71, the learning module 72 changes transition probability, to which the behavioral model 70.sub.1 to 70.sub.n corresponding thereto in the behavioral model library 70 corresponds in such a manner as to lower, when it "was patted (scolded)," the revelation probability of the action and to raise, when it "was petted (praised)," the revelation probability of the action." **Which illustrates the raising of the probability when there is a positive action (associated with a higher evaluation value)**. )
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito in view of Sabe's higher probability of selecting the patterns with higher evaluation values in order to enable the robot's behavior to smoothly and naturally grow/adapt to the user's preferential behavior: "this robot apparatus is capable of reducing discontinuity in action output before and after change in state space to be used for action generation because the state space to be used for action generation continuously changes. Thereby, output actions can be changed smoothly and naturally, thus making it possible to realize a robot apparatus which improves the entertainment characteristics." [Sabe ¶0017].
Regarding Claim 4, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein the one or more processors are configured to: select (See at least ¶0061 via "After taking this appearance probability into consideration, supposing the reaction behavior pattern 31 is selected based on a random number, the voice "vce(01)" and the action "act(01)" will be selected")
generate the new motion pattern by (See at least ¶0057 via " The character state update determination unit 17 suitably updates the value of the character parameter XY based on the total value of the action points." and ¶0065 via "Thus, the basic posture or the character and behavior tendency, or the like, is set to each character parameter XY. In addition, as described later, the character parameter XY in the third stage is updated suitably by the total value of the action points generated according to the reaction behavior (output) performed by the dog type robot 1.").
However, Saito does not explicitly disclose the combining of motion elements to form a new motion pattern.
Nevertheless, Sakaue discloses: combining the motion elements of the at least two motion patterns (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions.")
as well as at least two motion patterns (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…" **Wherein combining motion patterns requires at least two)
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
However, although modified Saito discloses the evaluation values (See at least Saito via "action points" in ¶0053); modified Saito does not explicitly disclose the selecting the motion patterns in descending order of the evaluation values.
Nevertheless, Sabe discloses: in descending order of the evaluation values; (See at least ¶0108 via "On the basis of this recognition result and a notice from the action switching module 71, the learning module 72 changes transition probability, to which the behavioral model 70.sub.1 to 70.sub.n corresponding thereto in the behavioral model library 70 corresponds in such a manner as to lower, when it "was patted (scolded)," the revelation probability of the action and to raise, when it "was petted (praised)," the revelation probability of the action." **Which illustrates the raising of the probability when there is a positive action (associated with a higher evaluation value), and the preferred selection of higher evaluated actions corresponds to selecting patterns in descending order**)
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito in view of Sabe's higher probability of selecting the patterns with higher evaluation values in order to enable the robot's behavior to smoothly and naturally grow/adapt to the user's preferential behavior: "this robot apparatus is capable of reducing discontinuity in action output before and after change in state space to be used for action generation because the state space to be used for action generation continuously changes. Thereby, output actions can be changed smoothly and naturally, thus making it possible to realize a robot apparatus which improves the entertainment characteristics." [Sabe ¶0017].
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20020016128 A1, IDS) and Sakaue et. al. (US 20020165642 A1) in view of Sabe et. al. (US 20030045203 A1, IDS) and Wei et. al. (US 11360441 B1).
Regarding Claim 5, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, although Saito discloses evaluation values (See Saito ¶0057) and selecting a motion pattern (See Saito ¶0061), Saito does not explicitly disclose selecting two motion patterns.
Nevertheless, Sakaue discloses: wherein the one or more processors are configured to: select two motion patterns from among the motion patterns (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions." **Wherein combining motion patterns requires at least two)
generate the new motion pattern by combining the motion elements each extracted from, of the two motion patterns, (See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions." **Wherein combining motion patterns requires at least two).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
However, Modified Saito does not explicitly disclose the selecting of two motion patterns based on the evaluation values.
Nevertheless, Sabe discloses: based on the evaluation values; and (See at least ¶0108 via "On the basis of this recognition result and a notice from the action switching module 71, the learning module 72 changes transition probability, to which the behavioral model 70.sub.1 to 70.sub.n corresponding thereto in the behavioral model library 70 corresponds in such a manner as to lower, when it "was patted (scolded)," the revelation probability of the action and to raise, when it "was petted (praised)," the revelation probability of the action." **Which illustrates the preferred selection of higher evaluated actions)
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito in view of Sabe's higher probability of selecting the patterns with higher evaluation values in order to enable the robot's behavior to smoothly and naturally grow/adapt to the user's preferential behavior: "this robot apparatus is capable of reducing discontinuity in action output before and after change in state space to be used for action generation because the state space to be used for action generation continuously changes. Thereby, output actions can be changed smoothly and naturally, thus making it possible to realize a robot apparatus which improves the entertainment characteristics." [Sabe ¶0017].
However, Modified Saito does not explicitly disclose, but Wei--who is directed towards self-learning control for a robot--discloses: (See at least Col. 9 Lines 61-63 via " one expansion scheme at block B3 is that it depends on a crossover probability 0<p.sub.c<1 whether a control gain element needs to crossed" as well as Col. 11 Lines 20-26 via "a selection strategy combining roulette with elite retention are adopted to select the control gain element in the expanded control gain set into a new control gain set for next iteration. The roulette strategy enables a control gain element with greater fitness to be selected into a new control gain set with a higher probability…" **Wherein Wei discloses the combination of control gain elements of the sets/populations for robotic control with respect to corresponding crossover probabilities which corresponds to the concept of selecting motion elements from two motion patterns)
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito in view of the element crossover probabilities concept of Wei in order to improve the control and adaptability of the robot: "performing preferential iteration on control gain elements in the first control gain set to obtain a target control gain set" [Wei Col. 2 Lines 13-14] and "enable a control gain element with greater fitness to be selected into a new control gain set with a higher probability" [Wei Col. 11 Lines 20-26].
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20020016128 A1, IDS) and Sakaue et. al. (US 20020165642 A1) in view of Du et. al. (US 20120062399 A1).
Regarding Claim 6, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein each of the motion elements is (See at least ¶0062 and Figure 5 which provide an example of the dog type robot detecting a stimulus (user hitting robot on head - "i-01"), and the randomly selecting a behavior pattern which comprises a combination of motion elements: "…reaction behavior pattern 31 is selected based on a random number, the voice "vce(01)" and the action "act(01)" will be selected. As a result, according to FIGS. 9 and 10, the dog type robot 1 "draws back" yelping "yap!", that is, the dog type robot 1 takes the same action as an actual dog" **Wherein the motion patterns are comprised of motion elements such as voice + action).
However, Modified Saito does not explicitly disclose representing the motion elements by Boolean values.
Nevertheless, Du--who discloses generating improved solutions based on evaluation and bit mutation--discloses: represented by a Boolean value (See at least ¶0035 via "the method can comprise randomly selecting a parent sequence of the parent binary sequences and applying two-point mutation to the parent sequence including generating a predefined number of offspring binary sequences from the parent sequence" and ¶0056 via "… the GA can be coded in a binary string…").\
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito to represent the motion elements as Boolean/binary bit values such as in Du, in order to improve adaptability based on the fitness evaluation (which correspond with the action points of modified Saito): "evolving the biphase sequences of bits with an evolutionary algorithm including bit climbing, wherein the bit climbing includes flipping the bits one by one and evaluating a pre-selected fitness function in connection with determining whether to retain a flipped or non-flipped value" [Du ¶0039]; thus, it would have been obvious to encode modified Saito's motion elements in a similar manner to achieve the same goal of growth/evolution.
Regarding Claim 7, Modified Saito discloses the robot control device according to Claim 6.
Furthermore, Saito discloses: wherein the one or more processors are configured to: generate, as the new motion pattern, (See at least ¶0057 via " The character state update determination unit 17 suitably updates the value of the character parameter XY based on the total value of the action points." and ¶0065 via "Thus, the basic posture or the character and behavior tendency, or the like, is set to each character parameter XY. In addition, as described later, the character parameter XY in the third stage is updated suitably by the total value of the action points generated according to the reaction behavior (output) performed by the dog type robot 1.")
in response to a sum of the evaluation values being equal to or more than a reference value, (See at least ¶0053 via "The point counting unit 15 counts a generated action point caused by the reaction behavior of the dog type robot 1. The action point is counted (added/subtracted) to the total value of the action points, and the latest total value is stored in the RAM. Here, an "action point" means a generated score caused by the reaction behavior (output) of the dog type robot 1. The total value of the action points corresponds to the level of communication between the dog type robot 1 and a user. It also becomes a base parameter related to the update of the character parameter XY, which determines the character state of the dog type robot 1.". Additionally, Saito compares the evaluation value/action point total to a reference value in order to determine when to shift to a next stage in the growth/evolution process: See at least ¶0066 for example: " The aggregate total value VTA corresponds to the amount of communication between a user and the dog type robot 1, and becomes a value for a determination when shifting from the first stage to the second stage." as well as Figures 12-14).
However, Saito does not explicitly disclose the combining of motion elements to form a new motion pattern.
Nevertheless, Sakaue discloses: new motion patterns by combining,(See at least ¶0059 via "As the basic behavior patterns, the most close to loci of the patterns stored is selected or combined…when parameters such as time elapsed and operating frequency by the user exceed a predetermined value, each behavioral pattern is fractionized, which enables more complicated actions.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Saito in view of Sakaue's combination of patterns/fractionized elements in order to enable more complicated actions [Sakaue ¶0059] and for the robot to be able to gradually grow/adapt to a user it interacts with: "[the] robot gradually improves its recognition toward the user and, as a result, the behaviors of the robot gradually change" [Sakaue ¶0014] and "[the] robot becomes 'tameable' to the user, which, in a interactive type of apparatus, corresponds to a concept of 'becoming familiar' or 'becoming mature.'" [Sakaue ¶0081].
However, Modified Saito does not explicitly disclose the inverting the Boolean values of the motion elements.
Nevertheless, Du discloses: invert the Boolean value of each of the (See at least ¶0035 where Du discloses representing elements as binary bits via "the method can comprise randomly selecting a parent sequence of the parent binary sequences and applying two-point mutation to the parent sequence including generating a predefined number of offspring binary sequences from the parent sequence" and ¶0056 via "… the GA can be coded in a binary string…"; additionally see at least ¶0074 via "the bits of the string are flipped by flipping the bits of parent sequences one by one. Their corresponding value in the fitness function f.sub.3 is then evaluated. If the newly flipped sequence has a higher fitness value, the newly flipped sequence replace the previous one in recognition of evolutionary superiority. For a single parent sequence, the fitness function value of L one-bit flipped versions of the parent sequence are computed").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Saito to represent and mutate the motion elements as Boolean/binary bit values such as in Du, in order to improve adaptability based on the fitness evaluation (which correspond with the action points of modified Saito): "evolving the biphase sequences of bits with an evolutionary algorithm including bit climbing, wherein the bit climbing includes flipping the bits one by one and evaluating a pre-selected fitness function in connection with determining whether to retain a flipped or non-flipped value" [Du ¶0039]. Furthermore, the concept of inverting/flipping the bit values as a part of the evolution in Du is working towards the same goal as modified Saito's evolving to the next stage when a sum of the action points are above a reference value, and thus, it would have been obvious to incorporate the bit mutation/inversion into modified Saito's motion elements in a similar manner to achieve the same goal of growth/evolution.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20020016128 A1, IDS) and Sakaue et. al. (US 20020165642 A1) in view of NPL: RL— Introduction to Deep Reinforcement Learning - Jonathan Hui.
Regarding Claim 8, Modified Saito discloses the robot control device according to Claim 1.
Furthermore, Saito discloses: wherein the one or more processors are configured to: determine whether a user has touched the robot based on a detection result by the sensor; and (See at least ¶0040 via "The touch sensor is a sensor that detects whether a user touched a predetermined portion of the dog type robot 1 or not, that is, a sensor for detecting a touch stimulus.")
derive the evaluation values (See at least ¶0053 via " The point counting unit 15 counts a generated action point caused by the reaction behavior of the dog type robot 1. The action point is counted (added/subtracted) to the total value of the action points, and the latest total value is stored in the RAM. Here, an "action point" means a generated score caused by the reaction behavior (output) of the dog type robot 1. The total value of the action points corresponds to the level of communication between the dog type robot 1 and a user.")
Furthermore, Saito also discloses the robot motion (See at least Figure 10 which illustrates various motion elements that correspond to movement )
However, Saito does not explicitly disclose the evaluation value being higher when the time is shorter.
Nevertheless, Hui--who is directed towards reinforcement learning--discloses: such that as a time from when the (See at least Page 3 via "The discount factor γ reduces the weight of future rewards, reflecting the principle that a delayed reward often holds less value than an immediate one.)
Therefore it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modifies Saito in view of Hui's reinforcement learning principles where a delayed reward holds less value than an immediate reward (an immediate reward corresponding to a higher evaluation value) in order to improve the reinforcement learning for a robot to be able to perform more desired actions/behavior control (actions with higher evaluations or higher rewards): "The term “action” is equivalent to “control” in many contexts. Objectives can be framed in two equivalent ways: maximizing rewards or minimizing costs, where costs are simply the negative of rewards." [Hui Page 4] as well as to improve the robot learning stability: "helps some algorithms achieve stable convergence." [Hui Page 3]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAYLA RENEE DOROS whose telephone number is (703)756-1415. The examiner can normally be reached Generally: M-F (8-5) EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.D./Examiner, Art Unit 3657 /ABBY LIN/Supervisory Patent Examiner, Art Unit 3657