The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement filed 12/14/2023 fails to comply with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609 because US patent reference row A2 omits a document number, publication date, and inventor name. All other references have been considered, but applicant is urged to submit a new IDS with any documents that may have been inadvertently omitted. Applicant is advised that the date of any re-submission of any item of information contained in this information disclosure statement or the submission of any missing element(s) will be the date of submission for purposes of determining compliance with the requirements based on the time of filing the statement, including all certification requirements for statements under 37 CFR 1.97(e). See MPEP § 609.05(a).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea to without significantly more. The claims recite a mental process.
Regarding claim 1 and analogous claims 6 and 9:
Per step 1, “Is the claim to a process, machine, manufacture, or composition of matter?”, the claims are directed to a machine, a process, and a machine respectively.
Step 1: Yes
Per step 2A prong 1, “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”, the following limitations recite an abstract idea, i.e. a mental process:
in which an intelligent agent that performs an action based on a mental state is present, and performs autonomous control, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
wherein the apparatus holds an internal model for predicting a future state of an environment based on a state of the environment, a social emotion of an individual with respect to another individual, and an action (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.),
generating, using the observation data, first state information including information related to the intelligent agent present around the apparatus, as a state of an environment grasped by the apparatus; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
generating, using the first state information, second state information including information related to the apparatus and another intelligent agent present around the intelligent agent, as a state of an environment grasped by the intelligent agent; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
deciding a self emotion of the apparatus based on the first state information; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind.)
deciding a self emotion of the intelligent agent based on the second state information; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
deciding a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent based on the self emotion of the apparatus and the self emotion of the intelligent agent; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
generating, using the internal model, first prediction state information that is future prediction of the first state information and second prediction state information that is future prediction of the second state information; (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
and deciding an action to be performed by the apparatus based on the first prediction state information, the second prediction state information, the social emotion of the apparatus with respect to the intelligent agent, and the social emotion of the intelligent agent with respect to the apparatus or another intelligent agent. (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III.)
Step 2A prong 1: yes
Per step 2A prong 2, “Does the claim recite additional elements that integrate the judicial exception into a practical application?”, the following limitations are directed to additional elements which fail to implement the judicial exemption into a practical application:
A computer system to be connected to an apparatus that is present in a space (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the apparatus comprising: a processor; a storage device connected to the processor; and an interface connected to the processor,( (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
acquiring, via the interface, observation data indicating a state of an environment as a state of an environment observed by the apparatus (This constitutes mere data gathering, see MPEP 2106.05(g))
Step 2A prong 2: yes
Per step 2B, “Does the claim recite additional elements that amount to significantly more than the judicial exemption?”, the following additional elements are not sufficient to amount to significantly more than the judicial exemption, both when viewed independently and as an ordered combination, specified at a high level of generality.
A computer system to be connected to an apparatus that is present in a space (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the apparatus comprising: a processor; a storage device connected to the processor; and an interface connected to the processor,( (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
acquiring, via the interface, observation data indicating a state of an environment as a state of an environment observed by the apparatus (This constitutes mere data gathering, see MPEP 2106.05(g))
Step 2B: no
Regarding claim 2 and analogous claim 7:
Per step 1, “Is the claim to a process, machine, manufacture, or composition of matter?”, the claims are directed to a machine, a process, and a machine respectively.
Step 1: Yes
Per step 2A prong 1, “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”, the following limitations recite an abstract idea, i.e. a mental process:
The apparatus according to claim 1,
wherein the apparatus calculates free energy in a free energy principle of the apparatus using the first state information and decides a self emotion of the apparatus based on the free energy of the apparatus, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I). Note that the free energy principle was explicitly introduced as model of human brain function (see Friston K, Kilner J, Harrison L. A free energy principle for the brain. Journal of physiology-Paris. 2006 Jul 1;100(1-3):70-87), i.e. a mental process)
calculates the free energy of the intelligent agent using the second state information and decides a self emotion of the intelligent agent based on the free energy of the intelligent agent, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I); also see analysis above relating to Friston et al.)
and decides a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent based on a combination of the self emotion of the apparatus and the self emotion of the intelligent agent. (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I).)
Step 2A prong 1: yes
Per step 2A prong 2, “Does the claim recite additional elements that integrate the judicial exception into a practical application?”, and step 2B, “Does the claim recite additional elements that amount to significantly more than the judicial exemption?”, the claim does not recite any additional elements. Step 2A prong 2: no
Step 2B: no
Regarding claim 3 and analogous claim 8:
Per step 1, “Is the claim to a process, machine, manufacture, or composition of matter?”, the claims are directed to a machine, a process, and a machine respectively.
Step 1: Yes
Per step 2A prong 1, “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”, the following limitations recite an abstract idea, i.e. a mental process:
The apparatus according to claim 2,
wherein the apparatus calculates expected free energy in the free energy principle of the apparatus using the first prediction state information, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I).)
calculates the expected free energy of the intelligent agent using the second prediction state information, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I).)
and a weight to be multiplied by each of the expected free energy of the apparatus and the expected free energy of the intelligent agent is decided based on a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent. (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I).)
and decides an action of the apparatus based on a weighted sum of the expected free energy of the apparatus and the expected free energy of the intelligent agent, (This is an observation, evaluation, judgement, or opinion, i.e. a concept performed in the human mind. See MPEP 2106.04(a)(2), III. This can further be considered a mathematical formula, equation, or relationship, see MPEP 2106.04(a)(2)(I).)
Step 2A prong 1: yes
Per step 2A prong 2, “Does the claim recite additional elements that integrate the judicial exception into a practical application?”, and step 2B, “Does the claim recite additional elements that amount to significantly more than the judicial exemption?”, the claim does not recite any additional elements.
Step 2A prong 2: no
Step 2B: no
Regarding claim 4:
Per step 1, “Is the claim to a process, machine, manufacture, or composition of matter?”, the claims are directed to a machine, a process, and a machine respectively.
Step 1: Yes
Per step 2A prong 1, “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”, the following limitations recite an abstract idea, i.e. a mental process:
The apparatus according to claim 1,
Step 2A prong 1: yes
Per step 2A prong 2, “Does the claim recite additional elements that integrate the judicial exception into a practical application?”, the following limitations are directed to additional elements which fail to implement the judicial exemption into a practical application:
wherein the intelligent agent includes another device that executes the action decision processing, (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
the apparatus acquires a calculation result calculated in the action decision processing from the other device (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
and the action decision processing is executed using the first state information and the acquired calculation result. (This constitutes merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)Step 2A prong 2: no
Per step 2B, “Does the claim recite additional elements that amount to significantly more than the judicial exemption?”, the claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exemption, both when viewed independently and as an ordered combination, specified at a high level of generality. Per above analysis, each additional element is either a well-understood, routine, and conventional generic computer or mere data gathering.
Step 2B: no
Regarding claim 5:
Per step 1, “Is the claim to a process, machine, manufacture, or composition of matter?”, the claims are directed to a machine, a process, and a machine respectively.
Step 1: Yes
Per step 2A prong 1, “Does the claim recite an abstract idea, law of nature, or natural phenomenon?”, the following limitations recite an abstract idea, i.e. a mental process:
The apparatus according to claim 1,
Step 2A prong 1: yes
Per step 2A prong 2, “Does the claim recite additional elements that integrate the judicial exception into a practical application?”, the following limitations are directed to additional elements which fail to implement the judicial exemption into a practical application:
wherein the apparatus executes learning processing of learning the internal model, and a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent are used in the learning processing. (well-understood, routine, and conventional generic computer, see MPEP 2106.05(f))
Step 2A prong 2: no
Per step 2B, “Does the claim recite additional elements that amount to significantly more than the judicial exemption?”, the claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exemption, both when viewed independently and as an ordered combination, specified at a high level of generality. Per above analysis, each additional element is either a well-understood, routine, and conventional generic computer or mere data gathering.
Step 2B: no
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5, 6, and 9 are rejected under 35 U.S.C. 102(a) as being anticipated by Chrissie's Transcripts web site, Data's Day, 5 July 2008 [Cited 26 August 26 through Wayback Machine]. Available from https://web.archive.org/web/20080705014840/https://www.chakoteya.net/NextGen/185.htm (hereafter referred to as Data’s Day).
Regarding claim 1 and analogous claims 6 and 9:
Data’s Day teaches
An apparatus that is present in a space in which an intelligent agent that performs an action based on a mental state is present, and performs autonomous control, (In Data’s Day, “DATA[OC] [an apparatus]: In reference to your most recent letter, I agree that your study lacks sufficient primary source information on my programming and operation [performs autonomous control, i.e., Data acts independently in the world] […] DATA: All systems normal, sir. Sickbay reports that Lieutenant Juarez went into labour at zero four hundred hours. We remain at station [that is present in a space, i.e., Data is on board the starship Enterprise-D which is at station] awaiting the arrival of Starship Zhukov and guest quarters have been prepared for ambassador T’Pel.[…] KEIKO [an intelligent agent is present, i.e., a human person]: I’ve decided not to go through with it [performs an action based on a mental state, i.e., making a decision to cancel a planned wedding and communicating that decision to Data]”
the apparatus comprising: a processor; a storage device connected to the processor; and an interface connected to the processor, (In Data’s Day, “DATA[OC] [the apparatus]: In reference to your most recent letter [an interface connected to the processor, i.e., Data is able to exchange letters and communicate with others], I agree that your study lacks sufficient primary source information on my programming [a processor, i.e., Data must have a processor because he is able to perform programming; a storage device connected to the processor; i.e., Data must be able to store programming] and operation)
wherein the apparatus holds an internal model for predicting a future state of an environment based on a state of the environment, a social emotion of an individual, and an action, and (In Data’s Day, “DATA [OC]: Eventually, I developed a programme [an internal model] enabling me to predict human emotional responses to specific actions. […] DATA [OC]: My friend Chief O’Brien often says that above all else, he wants to make Keiko happy [a social emotion of an individual]. Since cancelling the wedding [an action] will make her happy [a state of the environment], I must conclude [predicting] the Chief will be pleased at her decision [a future state of an environment]”
the apparatus executes action decision processing including: acquiring, via the interface, observation data indicating a state of an environment observed by the apparatus; (In Data’s Day,
“DATA: It is time for the wedding rehearsal.
KEIKO: I know. Data, I’ve decided not to go through with it.
[acquiring, via the interface, a state of an environment observed by the apparatus, e.g., Data hears Keiko say that she does not intend to go through with something].”)
generating, using the observation data, first state information including information related to the intelligent agent present around the apparatus, as a state of an environment grasped by the apparatus; (In Data’s Day, “DATA: It is time for the wedding rehearsal.
KEIKO [the intelligent agent present around the apparatus]: I know. Data, I’ve decided not to go through with it. DATA: You do not wish to rehearse?
KEIKO: No, I’m calling off the wedding.
[generate, using the observation data, first state information including information related to the intelligent agent e.g., the information that Keiko has called off the wedding] [as a state of an environment grasped by the apparatus, e.g., Data is able to grasp the social life of his spaceship and the relationships between his crewmates]”)
generating, using the first state information, second state information including information related to the apparatus and another intelligent agent present around the intelligent agent, as a state of an environment grasped by the intelligent agent; (In Data’s Day, “
KEIKO [the intelligent agent]: No, I'm calling off the wedding. [the first state information]DATA: May I ask why?
KEIKO: It’s just the right thing to do.
DATA: Have your feelings for Chief O’Brien [another intelligent agent present around the intelligent agent] changed?KEIKO: I’m supposed to be getting married, Data. I should be happy, but I’m not. I just feel this weight pressing down on me.
[generating… second state information, , e.g., the information that Keiko is apprehensive about marrying Chief O’Brien.
as a state of an environment grasped by the intelligent agent, e.g., Data is able to grasp the social life of his spaceship and the relationships between his crewmates]”)
deciding a self emotion of the apparatus based on the first state information; (In Data’s Day, KEIKO: No, I’m calling off the wedding [the first state information]. DATA: May I ask why? [a self emotion of the apparatus, e.g., respectful curiosity]”)
deciding a self emotion of the intelligent agent based on the second state information;
(In Data’s Day, “KEIKO: I’m supposed to be getting married, Data. I should be happy, but I’m not. I just feel this weight pressing down on me [the second state information]
DATA: Will cancelling the wedding make you happy? [deciding a self emotion of the intelligent agent, e.g., deciding that Keiko is unhappy based on her apprehension towards marrying Chief O’Brien”)
deciding a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent based on the self emotion of the apparatus and the self emotion of the intelligent agent; (In Data’s Day, “
DATA: May I ask why? [the self emotion of the apparatus]KEIKO: It's just the right thing to do.DATA: Have your feelings for Chief O'Brien changed?
KEIKO: I'm supposed to be getting married, Data. I should be happy, but I'm not. I just feel this weight pressing down on me.[a self emotion of the intelligent agent]
DATA: Will cancelling the wedding make you happy?KEIKO [the intelligent agent]: Yes. He’ll probably be just as relieved as I am. Data, you introduced us to each other. You mean a lot to both of us. I would really appreciate it if you would tell Miles for me.
DATA [the apparatus]: If you wish. [deciding a social emotion of the apparatus; e.g., conciliation towards Keiko]
[…]
DATA [OC]: My friend Chief O’Brien often says that above all else, he wants to make Keiko happy. Since cancelling the wedding will make her happy [deciding a social emotion of the intelligent agent with respect to another intelligent agent e.g., that cancelling her wedding to O’Brien will make Keiko happy], I must conclude that the Chief will be pleased at her decision.”)
generating, using the internal model, first prediction state information that is future prediction of the first state information and second prediction state information that is future prediction of the second state information; (In Data’s Day, “DATA [OC]: My friend Chief O'Brien often says that above all else, he wants to make Keiko happy. Since cancelling the wedding will make her happy, I must conclude the Chief will be pleased at her decision [first prediction state information that is future prediction of the first state information, e.g., that O’Brien will accede to calling off the wedding]
DATA: I have good news [second prediction state information that is future prediction of the second state information, e.g., that O’Brien will also have positive emotions towards calling off the wedding]”)
and deciding an action to be performed by the apparatus based on the first prediction state information, the second prediction state information, the social emotion of the apparatus with respect to the intelligent agent, and the social emotion of the intelligent agent with respect to the apparatus or another intelligent agent. (In Data’s Day,
DATA: If you wish. [the social emotion of the apparatus with respect to the intelligent agent]
[..]
DATA [OC]: My friend Chief O'Brien often says that above all else, he wants to make Keiko happy. Since cancelling the wedding will make her happy [the social emotion of the intelligent agent with respect to the other intelligent agent], I must conclude the Chief will be pleased at her decision. [first prediction state information]DATA: I have good news.[second prediction state information]O'BRIEN: Oh?
DATA: Keiko has made a decision designed to increase her happiness. She has cancelled the wedding. [deciding an action to be performed by the apparatus, e.g., cheerfully informing O’Brien that the wedding is cancelled based on the earlier conversation and reasoning])
Regarding claim 5:
Data’s Day teaches:
The apparatus according to claim 1
Data’s Day further teaches
wherein the apparatus executes learning processing of learning the internal model (In Data’s Day, “DATA [OC]: Commander Maddox, it would appear that my programme [the internal model] designed to predict the emotional responses needs adjustment [learning processing of learning].”)
and a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent are used in the learning processing (In Data’s Day,
DATA [the apparatus]: If you wish. [deciding a social emotion of the apparatus; e.g., conciliation towards Keiko]
[…]
DATA [OC]: My friend Chief O’Brien often says that above all else, he wants to make Keiko happy. Since cancelling the wedding will make her happy [deciding a social emotion of the intelligent agent with respect to another intelligent agent e.g., that cancelling her wedding to O’Brien will make Keiko happy], I must conclude that the Chief will be pleased at her decision.”)
[…]
DATA: Keiko has made a decision designed to increase her happiness. She has cancelled the wedding.O'BRIEN: She what? Cancelled the wedding? Today? Without even a word? Of all the childish, selfish, irresponsible things to do! (O'Brien storms out)LAFORGE: Next time, maybe I should break the good news.DATA [OC]: Commander Maddox, it would appear that my programme designed to predict the emotional responses needs adjustment [the learning processing, using the social emotions decided during the preceding events].”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-4, 7, and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Data’s Day in view of Kaufmann et al. (Kaufmann R, Gupta P, Taylor J. An active inference model of collective intelligence. Entropy. 2021 Jun 29;23(7):830.7. Hereafter referred to as Kaufmann).
Regarding claim 2 and analogous claim 7:
Data’s Day teaches
The apparatus according to claim 1
Data’s Day further teaches
and decides a social emotion of the apparatus with respect to the intelligent agent and a social emotion of the intelligent agent with respect to the apparatus or another intelligent agent based on a combination of the self emotion of the apparatus and the self emotion of the intelligent agent. (In Data’s Day, “DATA: May I ask why? [the self emotion of the apparatus]
KEIKO: It's just the right thing to do.DATA: Have your feelings for Chief O'Brien changed?
KEIKO: I'm supposed to be getting married, Data. I should be happy, but I'm not. I just feel this weight pressing down on me.[a self emotion of the intelligent agent]
DATA: Will cancelling the wedding make you happy?KEIKO [the intelligent agent]: Yes. He’ll probably be just as relieved as I am. Data, you introduced us to each other. You mean a lot to both of us. I would really appreciate it if you would tell Miles for me.
DATA [the apparatus]: If you wish. [deciding a social emotion of the apparatus based on a combination; e.g., conciliation towards Keiko based on Data’s questioning and Keiko’s apprehension]
[…]
DATA [OC]: My friend Chief O’Brien often says that above all else, he wants to make Keiko happy. Since cancelling the wedding will make her happy [deciding a social emotion of the intelligent agent with respect to another intelligent agent based on a combination; e.g., that cancelling her wedding to O’Brien will make Keiko happy based on Data’s questioning and Keiko’s apprehension], I must conclude that the Chief will be pleased at her decision.”)
Data’s Day fails to teach
wherein the apparatus calculates free energy in a free energy principle of the apparatus using the first state information and decides a self emotion of the apparatus based on the free energy of the apparatus
calculates the free energy of the intelligent agent using the second state information and decides a self emotion of the intelligent agent based on the free energy of the intelligent agent,
Kaufmann teaches
wherein the apparatus calculates free energy in a free energy principle of the apparatus using the first state information and decides a self emotion of the apparatus based on the free energy of the apparatus (On page 9 of Kaufmann, “b = (bown ϵ BN, b*own ϵ BN, bpartner ϵ BN, b*partner ϵ BN) is the agent [the apparatus]’s internal or “belief” state. Bown and b*own respectively are its actual [a self emotion of the apparatus] and desired beliefs about its own position; equivalently, bpartner and b*partner are its actual and desired beliefs about its partners’ position […] These states evolve [decide a self emotion of the apparatus] according to a discrete-time free energy minimization procedure [based on the free energy of the apparatus], extended from McGregor et al.” On page 12 of Kaufmann
PNG
media_image1.png
316
667
media_image1.png
Greyscale
)
calculates the free energy of the intelligent agent using the second state information and decides a self emotion of the intelligent agent based on the free energy of the intelligent agent, (On page 9 of Kaufmann, “s = (sown ϵ {0,1}, Δ ϵ [0… N-1], app ϵ {-1,0,1}) is the agent’s sensory state. Sown is a one-bit sensory input from the environment; Δ is the perceived difference between the agent’s own position and its partner’s, app [the second state information] is the partner’s last action […] These states evolve [decide a self emotion of the intelligent agent] according to a discrete-time free energy minimization procedure [based on the free energy of the intelligent agent], extended from McGregor et al.” On page 11 and 12 of Kaufmann,
PNG
media_image2.png
416
624
media_image2.png
Greyscale
)
Data’s Day and Kaufmann are both related to the same field of endeavor (artificial intelligence capable of functioning in multiagent systems). In view of the teachings of Kaufmann it would have been obvious for a person or ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Kaufmann to Data’s Day because the Free Energy Principle is a well-understood and convenient approach for modeling multiagent systems (On page 3 of Kaufmann, “FEP has recently emerged as a candidate for this type of common mathematical framework for multiscale behavioral processes.[…] The key trick of FEP is that the principle of free energy minimization can be neatly translated into an agent-based process theory, AIF, of approximate Bayesian inference and applied to any self-organizing biological system at any scale.) Additionally, the cited aspects of Kaufmann improve the ability of agents to achieve their goals (On page 24 of Kaufmann, “In model 2, the weak agent possesses ‘Theory of Mind’. This allows it to infer information about their own location in the environment by observing their partner’s actions. This is evidenced by the appearance of two-sharp peaks in the weak agent’s end-state belief distribution. Consequently, we see an improvement in the weak agent’s individual performance (the agent converges faster on an end-state belief faster than in model 1)”).
Regarding claim 3 and analogous claim 8:
Data’s Day and Kaufmann teach
The apparatus according to claim 2
Kaufman further teaches
wherein the apparatus calculates expected free energy in the free energy principle of the apparatus using the first prediction state information, (On page 9 of Kaufmann, “b [using the first prediction state information] = (bown ϵ BN, b*own ϵ BN, bpartner ϵ BN, b*partner ϵ BN) is the agent [the apparatus]’s internal or “belief” state. Bown and b*own [the first prediction state information] respectively are its actual and desired beliefs about its own position; equivalently, bpartner and b*partner are its actual and desired beliefs about its partners’ position.”
On page 10 of Kaufman,
PNG
media_image3.png
369
565
media_image3.png
Greyscale
On page 12 of Kaufmann, “
PNG
media_image4.png
406
599
media_image4.png
Greyscale
On page 13 of Kaufmann, “We interpret eq. 4 as follows: The agent’s sensory and partner models jointly constrain its beliefs both about its own position and its partner’s position. Thus, at each step, the agent: (a) refines its beliefs about both positions, in order to best fit the evidence provided by all its inputs (i.e., its “chemical” sensor for the physical environment and “position and motion” sensor for its partner); and (b) selects the “best” next pair of actions [calculates expected free energy] (for self and partner [the intelligent agent] i.e., that which minimizes the “difference” (the KL divergence) between its present beliefs and the desired beliefs”)
calculates the expected free energy of the intelligent agent using the second prediction state information, (On page 9 of Kaufmann, “b = (bown ϵ BN, b*own ϵ BN, bpartner ϵ BN, b*partner ϵ BN) is the agent [the apparatus]’s internal or “belief” state. Bown and b*own respectively are its actual and desired beliefs about its own position; equivalently, bpartner and b*partner [the second prediction state information] are its actual and desired beliefs about its partners’ position.”
On page 12 of Kaufmann,
PNG
media_image5.png
406
599
media_image5.png
Greyscale
.
On page 13 of Kaufmann, “We interpret eq. 4 as follows: The agent’s sensory and partner models jointly constrain its beliefs both about its own position and its partner’s position. Thus, at each step, the agent: (a) refines its beliefs about both positions, in order to best fit the evidence provided by all its inputs (i.e., its “chemical” sensor for the physical environment and “position and motion” sensor for its partner); and (b) selects the “best” next pair of actions [calculates expected free energy] (for self and partner [the intelligent agent] i.e., that which minimizes the “difference” (the KL divergence) between its present beliefs and the desired beliefs”)
and decides an action of the apparatus based on a weighted sum of the expected free energy of the apparatus and the expected free energy of the intelligent agent, (On pages 12-14 of Kaufmann, “The “rearranging” function, θα, serves to moderate the influence of the partner model on the agent’s own beliefs, and vice-versa. α is an agent-specific parameter, which, as we will see in Subsection 2.9, is identified with each agent’s degree of “alterity. […] Thus, at each step, the agent: (a) refines its beliefs about both positions, in order to best fit the evidence provided by all its inputs (i.e., its “chemical” sensor for the physical environment and “position and motion” sensor for its partner); and (b) selects the “best” next pair of actions [decide an action of] (for self [the apparatus] and partner), i.e., that which minimizes the “difference” (the KL divergence) between its present beliefs and the desired beliefs. […] α [the weight for the sum] can be understood simply as an agent’s degree of alterity, or propensity to see the “other” as an agent like itself. In simulations with values of α close to 0, we expect the partner’s behavior to be dominated by its own “chemical” sensory input. Increasing α, we expect to see an agent’s behavior being more heavily influenced by inputs from its partner, driving qown to become sharper as soon as qpartner does so. Past a certain threshold, this could spill into premature overfitting”.
Further see page 12 of Kaufmann,
PNG
media_image6.png
406
599
media_image6.png
Greyscale
)
and a weight to be multiplied by each of the expected free energy of the apparatus and the expected free energy of the intelligent agent is decided based on the social emotion of the apparatus with respect to the intelligent agent and the social emotion of the intelligent agent with respect to the apparatus or another intelligent agent. (On page 14 of Kaufmann, “This motivates the operationalization of α as a parameter for the intensity to which Theory of Mind shapes the agent’s beliefs. α [a weight to be multiplied by each of the expected free energy of the apparatus and the expected free energy of the intelligent agent is decided] can be understood simply as an agent’s degree of alterity, or propensity to see the “other” as an agent like itself [the social emotion of the apparatus with respect to the intelligent agent]. In simulations with values of α close to 0, we expect the partner’s behavior to be dominated by its own “chemical” sensory input. Increasing α, we expect to see an agent’s behavior being more heavily influenced by inputs from its partner, driving qown to become sharper as soon as qpartner does so. Past a certain threshold, this could spill into premature overfitting. Finally, note the α2 in the second term of agent-level free energy (eq. 4). This represents the notion that the agent is using “second-order theory of mind” or thinking about what its partner might be thinking about it. It seems appropriate to believe the partner to possess the same level of alterity as itself [the social emotion of the intelligent agent with respect to the apparatus]; we then represent this as applying the rearranging function (the “squishing” of the probability distribution) twice, θα × θα = θα2.
Data’s Day and Kaufmann are combinable for the same reasons set out above with respect to claim 2.
Regarding claim 4:
Data’s Day teaches
The apparatus according to claim 1
Data’s Day fails to teach
wherein the intelligent agent includes another device that executes the action decision processing
the apparatus acquires a calculation result calculated in the action decision processing from the other device,
and the action decision processing is executed using the first state information and the acquired calculation result.
Kaufmann teaches
wherein the intelligent agent includes another device that executes the action decision processing (On page 18 of Kaufmann,
PNG
media_image7.png
331
985
media_image7.png
Greyscale
)
the apparatus acquires a calculation result calculated in the action decision processing from the other device, (On page 18 of Kaufmann,
PNG
media_image8.png
319
985
media_image8.png
Greyscale
)
and the action decision processing is executed using the first state information and the acquired calculation result. (On page 18 of Kaufmann,
PNG
media_image9.png
327
985
media_image9.png
Greyscale
Note that the acquired calculation result must pass through multiple other transformations in the systems before eventually being used in action decision processing of agent A; a2 → ψ2 → π12 → b2i → b1 → a1; but is ultimately used in that processing)
Data’s Day and Kaufmann are combinable for the same reasons set out above with respect to claim 2.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. (Chen K, Fong J, Soh H. Mirror: Differentiable deep social projection for assistive human-robot communication. arXiv preprint arXiv:2203.02877. 2022 Mar 6.)
Esaki et al. (Esaki K, Matsumura T, Ito K, Mizuno H. Sensorimotor visual perception on embodied system using free energy principle. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases 2021 Sep 13 (pp. 865-877). Cham: Springer International Publishing.)
Hernandez-Leal et al. (Hernandez-Leal P, Kartal B, Taylor ME. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems. 2019 Nov;33(6):750-97.)
Jaques et al. (Jaques, Natasha, et al. "Social influence as intrinsic motivation for multi-agent deep reinforcement learning." International conference on machine learning. PMLR, 2019.)
Paradox Interactive (Europa Universalis 4 Wiki [Internet]. Stockholm, Sweden: Paradox Interactive; 2021. Relations; 29 November 2021 [Cited 24 August 2026]. Available from: https://eu4.paradoxwikis.com/index.php?title=Relations&oldid=140181)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALMA BENNETT THOMPSON whose telephone number is (571)270-1810. The examiner can normally be reached Monday-Friday 9:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALMA B THOMPSON/ Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights