DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/28/2024.The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgement is made of applicants claim for foreign priority under 35 U.S.C. 119(a)-(d) and (f). The certified copy has been filed in parent application KR10-2024-0044346 filed on 04/01/2024.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 11-12 are rejected under 35 U.S.C. 103 as being unpatentable by Okamori (WO2020026826) in view of KR (KR100905523).
Regarding claim 1, Okamori teaches a robot control apparatus comprising:
at least one processor ([9] disclosing a processor); and
a storage medium storing computer-readable instructions that, when executed by the at least one processor, enable the at least one processor to ([9]-[13]:
apply event information about user activity of a user, which is identified from user
activity data perceived from a robot, and context information about time and space, in which the user activity occurs, to a knowledge (at least [91]-[106] disclosing the association of events information such as a person being in a room with context such as time and TV being on or off being associated with together in a knowledge formed as relationship),
obtain first user intent data regarding intent of the user activity by applying the event instance among instances included in the knowledge graph to a rule creation model for creating information about the intent of the user activity ([91]-[106], [147]-[170] disclosing based on the relationship rules are created for determining the intention of a person such as the person does not want to be disturbed when watching TV in a room), and
control the robot such that the robot performs a target task related to an expected
activity, which follows the user activity, based on the first user intent data ([91]-[106], [147]-[170] disclosing the stopping of the robot based on the intent of a user to not be disturbed and continuing to watch tv at a subsequent time. Also when it is predicted that the person entered the room for few seconds thus the robot is stopped to not interrupt the person who exits the room which is a subsequent action of the person).
KR discloses a knowledge graph for user intention ([30]-[40] disclosing a knowledge graph for the determining of user intention).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Okamari to combine of the graph of KR yielding predictable results improving the determination by following a sequence on the graph that identifies a user intention as taught by KR.
Regarding claim 2, Okamori as modified by KR teaches the apparatus of claim 1, wherein the instructions further enable the at least one
processor to:
identify an event-condition-action (ECA) rule from the rule creation model based on applying the event instance to the rule creation model (Okamori [91]-[106], [147]-[170] discloses an event condition action rule created which is interpreted as from a rule creation model by applying the event to the rule creation model such as when the event is a person entering for a short time is applied to the rule creation model);
obtain a condition regarding the event instance by applying the event instance to the ECA rule (Okamori [91]-[106] and [147]-[170] disclosing the conditions such as the entering, short period of time, person remain in room, are applied to the rule); and
Obtain the first user intent data by applying the context instance paired with the event instance to the condition (Okamori [91]-[106], [147]-[170] disclosing the intention is determined from the context including the paired data of events and conditions such as presence of person and in which room and state of TV and amount of time spent).
Claims 10-11 are rejected for similar reasons as claims 1-2, see above rejection.
Claims 3-8, 12-17 are rejected under 35 U.S.C. 103 as being unpatentable by Okamori (WO2020026826) in view of KR (KR100905523) and Otani (US20220328038) and Korea (KR102140585).
Regarding claim 3, Okamori as modified by KR teaches the apparatus of claim 2, wherein the instructions further enable the at least one processor to: Okamori as modified by KR does not teach identify a neural compositional rule learning (NCRL)-based rule mining model that generates the first user intent data regardless of the context instance based on the event instance among instances included in the knowledge graph from the rule creation model; and update the ECA rule based on an output of the NCRL-based rule mining model and the knowledge graph.
Otani teaches identify a neural compositional rule learning (NCRL)-based rule mining model that generates the first user intent data (abstract, [0007]-[0010], [0021]disclosing the neural generative rule learning indicative of a NCRL based rule mining that generates intention of a user).
The combination of Otani is obvious to one of ordinary skill in the art in order to utilize neural networks, i.e., machine learning to improve the selection of intention of a user as taught by Otani which improves the intention selection as taught by Okamori as modified by KR.
Okamori as modified by KR and Otani does not teach regardless of the context instance based on the event instance among instances included in the knowledge graph from the rule creation model; and update the ECA rule based on an output of the NCRL-based rule mining model and the knowledge graph.
Korea teaches regardless of the context instance based on the event instance among instances included in the knowledge (at least abstract and [13]-[40] disclosing intention is inferred and then the intention is reasoned with context data later in order to obtain the ECA rule, i.e., update the rule).
the combination of the teaching of Korea of regardless of the context data and based on the event instance included in the knowledge from the rule creation and update the ECA rule based on output of the NCRL based rule mining model and graph with the knowledge graph and NCRL based rule mining as taught by Okamori as modified by KR and Otani is obvious in order to create a new rule based on an intention and context via machine learning yielding predictable results and automating the creation.
Regarding claim 4, Okamori as modified by KR and Otani and Korea teaches the apparatus of claim 3, wherein the instructions further enable the at least one processor to:
Specifically, Otani teaches obtain activity sequence data regarding the event instance from the NCRL-based rule mining model (abstract, [0007]-[0010], [0021]disclosing the neural generative rule learning indicative of a NCRL based rule mining that generates intention of a user).
The combination of Otani is obvious to one of ordinary skill in the art in order to utilize neural networks, i.e., machine learning to improve the selection of intention of a user as taught by Otani which improves the intention selection as taught by Okamori as modified by KR.
Korea teaches obtain at least one candidate context instance related to the activity sequence data from the knowledge graph; and update the ECA rule based on the activity sequence data and the at least one candidate context instance (at least abstract and [13]-[40] disclosing intention is inferred and then the intention is reasoned with context data later in order to obtain the ECA rule, i.e., update the rule).
the combination of the teaching of Korea of regardless of the context data and based on the event instance included in the knowledge from the rule creation and update the ECA rule based on output of the NCRL based rule mining model and graph with the knowledge graph and NCRL based rule mining as taught by Okamori as modified by KR and Otani is obvious in order to create a new rule based on an intention and context via machine learning yielding predictable results and automating the creation.
Regarding claim 5, Okamori as modified by KR and Otani and Korea teaches apparatus of claim 4, wherein the instructions further enable the at least one
processor to:
identify a first candidate context instance and a second candidate context instance different from the first candidate context instance from the at least one candidate context instance (Okamori [140]-[170] disclosing at least a first candidate context and a second candidate context which differ by the number of seconds, i.e., counts);
determine a first count, which is a first number of first targets satisfying combination of a first place included in the first candidate context instance and the activity sequence data (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold);
determine a second count, which is a second number of second targets satisfying combination of a second place included in the second candidate context instance and the activity sequence data (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold);
determine one of the first candidate context instance or the second candidate context instance as a target context instance regarding the activity sequence data based on comparing the first count with the second count (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold. The selection of the behavior is based on comparing the number of seconds and if the number is short seconds the behavior of the robot is adjusted based on that to be stopped to not bother the user entering and then leaving quickly);
Regarding claim 6, Okamori as modified by KR and Otani and Korea teaches the apparatus of claim 5, wherein the instructions further enable the at least one processor to update the ECA rule by inserting a second relation between the activity sequence data and the target context instance into the ECA rule based on the target context instance regarding target activity sequence data being determined (Okamori [91]-[104], [140]-[170] disclosing relationship between a sequence of actions and the context instances).
Regarding claim 7, Okamori as modified by KR apparatus of claim 3, wherein the instructions further enable the at least one processor to:
obtain an additional condition regarding an instance of the event information by
applying the instance of the event information to the ECA rule based on the ECA rule being updated (Okamori [91]-[104], [140]-[170] disclosing different intentions are detected based on different contexts to update the rule based on different contexts including TV states, time of a person in a room, a person in the room or not….); and
obtain additional user intent data different from the first user intent data by applying an instance regarding context information paired with the event information to the additional condition (Okamori [91]-[104], [140]-[170] disclosing different intentions are detected based on different contexts to update the rule based on different contexts including TV states, time of a person in a room, a person in the room or not, wherein the intent is updated to determine whether a person is staying in the room based on TV being on or not and based on waiting an amount of time, i.e., different contexts to create different rules);.
Regarding claim 8, Okamori as modified by KR and Otani and Korea teaches the apparatus of claim 7, wherein the instructions further enable the at least one processor to:
determine the target task related to the expected activity based on the first user intent data and the additional user intent data; and control the robot such that the robot performs the target task (Okamori [91]-[104], [140]-[170] disclosing different intentions are detected based on different contexts to update the rule based on different contexts including TV states, time of a person in a room, a person in the room or not, wherein the intent is updated to determine whether a person is staying in the room based on TV being on or not and based on waiting an amount of time, i.e., different contexts to create different rules, wherein the robot is controlled differently to stop cleaning or continue cleaning and resume cleaning based on context);.
Claims 12-17 are rejected for similar reasons as claims 3-8, respectively, see above rejection.
Claims 9, 18, 19 are rejected under 35 U.S.C. 103 as being unpatentable by Okamori (WO2020026826) in view of KR (KR100905523) and Ganhotra (US20190171945).
Regarding claim 9, Okamori as modified by KR and Otani and Korea teaches apparatus of claim 1, wherein the instructions further enable the at least one
processor to:
which includes a relation between the event information and the context information and (Okamori [91]-[104] and [140]-[170] disclosing a knowledge that relates the contexts to events information); and
Okamori does not specifically teach which is in a data format compatible with ontology of the knowledge graph; generate resource description framework (RDF) data, convert the RDF data into the event instance and the context instance and store the event instance and the context instance in the knowledge graph.
Ganhotra teaches which is in a data format compatible with ontology of the knowledge graph; generate resource description framework (RDF) data, convert the RDF data into the event instance and the context instance and store the event instance and the context instance in the knowledge graph ([0044]-[0045] disclosing the knowledge base with the information stored as RDF compatible with the knowledge graph in order to connect them together and validate the extraction of the RDF into the graph).
It would have been obvious to one of ordinary skill in the art to combine the teaching of Ganhotra of the RDF format of the relationships as taught by Okamori and the knowledge graph as taught by KR in order to enable the connection of the validated RDF triplets into a knowledge graph thus improving the graph by expanding the knowledge graph.
Claim 18 is rejected for similar reasons as claim 9, see above rejection.
Regarding claim 19, Okamori teaches a robot control method comprising:
at least one processor ([9] disclosing a processor); and
a storage medium storing computer-readable instructions that, when executed by the at least one processor, enable the at least one processor to ([9]-[13]:
apply event information about user activity of a user, which is identified from user
activity data perceived from a robot, and context information about time and space, in which the user activity occurs, to a knowledge (at least [91]-[106] disclosing the association of events information such as a person being in a room with context such as time and TV being on or off being associated with together in a knowledge formed as relationship),
obtain first user intent data regarding intent of the user activity by applying the event instance among instances included in the knowledge graph to a rule creation model for creating information about the intent of the user activity ([91]-[106], [147]-[170] disclosing based on the relationship rules are created for determining the intention of a person such as the person does not want to be disturbed when watching TV in a room), and
control the robot such that the robot performs a target task related to an expected
activity, which follows the user activity, based on the first user intent data ([91]-[106], [147]-[170] disclosing the stopping of the robot based on the intent of a user to not be disturbed and continuing to watch tv at a subsequent time. Also when it is predicted that the person entered the room for few seconds thus the robot is stopped to not interrupt the person who exits the room which is a subsequent action of the person).
identify an event-condition-action (ECA) rule from the rule creation model based on applying the event instance to the rule creation model (Okamori [91]-[106], [147]-[170] discloses an event condition action rule created which is interpreted as from a rule creation model by applying the event to the rule creation model such as when the event is a person entering for a short time is applied to the rule creation model);
obtain a condition regarding the event instance by applying the event instance to the ECA rule (Okamori [91]-[106] and [147]-[170] disclosing the conditions such as the entering, short period of time, person remain in room, are applied to the rule); and
Obtain the first user intent data by applying the context instance paired with the event instance to the condition (Okamori [91]-[106], [147]-[170] disclosing the intention is determined from the context including the paired data of events and conditions such as presence of person and in which room and state of TV and amount of time spent).
KR discloses a knowledge graph for user intention ([30]-[40] disclosing a knowledge graph for the determining of user intention).
The combination/substitution of the graph of KR is obvious yielding predictable results improving the determination by following a sequence on the graph that identifies a user intention as taught by KR.
Okamori does not specifically teach which is in a data format compatible with ontology of the knowledge graph; generate resource description framework (RDF) data, convert the RDF data into the event instance and the context instance and store the event instance and the context instance in the knowledge graph.
Ganhotra teaches which is in a data format compatible with ontology of the knowledge graph; generate resource description framework (RDF) data, convert the RDF data into the event instance and the context instance and store the event instance and the context instance in the knowledge graph ([0044]-[0045] disclosing the knowledge base with the information stored as RDF compatible with the knowledge graph in order to connect them together and validate the extraction of the RDF into the graph).
It would have been obvious to one of ordinary skill in the art to combine the teaching of Ganhotra of the RDF format of the relationships as taught by Okamori and the knowledge graph as taught by KR in order to enable the connection of the validated RDF triplets into a knowledge graph thus improving the graph by expanding the knowledge graph.
Claim 20 are rejected under 35 U.S.C. 103 as being unpatentable by Okamori (WO2020026826) in view of KR (KR100905523) and Ganhotra (US20190171945) and Otani (US20220328038) and Korea (KR102140585).
Regarding claim 20, Okamori as modified by KR and Ganhotra teaches the method of claim 19, wherein the obtaining the first user intent data comprises:
identify a first candidate context instance and a second candidate context instance different from the first candidate context instance from the at least one candidate context instance (Okamori [140]-[170] disclosing at least a first candidate context and a second candidate context which differ by the number of seconds, i.e., counts);
determine a first count, which is a first number of first targets satisfying combination of a first place included in the first candidate context instance and the activity sequence data (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold);
determine a second count, which is a second number of second targets satisfying combination of a second place included in the second candidate context instance and the activity sequence data (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold);
determine one of the first candidate context instance or the second candidate context instance as a target context instance regarding the activity sequence data based on comparing the first count with the second count (Okamori [140]-[170] disclosing when the number of seconds is short vs the user staying in a room for a longer time to behave differently, i.e., this is equivalent to a number less than a threshold seconds and the other number is greater than the threshold. The selection of the behavior is based on comparing the number of seconds and if the number is short seconds the behavior of the robot is adjusted based on that to be stopped to not bother the user entering and then leaving quickly);
update the ECA rule by inserting a second relation between the activity sequence data and the target context instance into the ECA rule based on the target context instance regarding target activity sequence data being determined (Okamori [91]-[104], [140]-[170] disclosing relationship between a sequence of actions and the context instances).
obtain an additional condition regarding an instance of the event information by
applying the instance of the event information to the ECA rule based on the ECA rule being updated (Okamori [91]-[104], [140]-[170] disclosing different intentions are detected based on different contexts to update the rule based on different contexts including TV states, time of a person in a room, a person in the room or not….); and
obtain additional user intent data different from the first user intent data by applying an instance regarding context information paired with the event information to the additional condition (Okamori [91]-[104], [140]-[170] disclosing different intentions are detected based on different contexts to update the rule based on different contexts including TV states, time of a person in a room, a person in the room or not, wherein the intent is updated to determine whether a person is staying in the room based on TV being on or not and based on waiting an amount of time, i.e., different contexts to create different rules);.
Okamori as modified by KR does not teach identify a neural compositional rule learning (NCRL)-based rule mining model that generates the first user intent data regardless of the context instance based on the event instance among instances included in the knowledge graph from the rule creation model; and update the ECA rule based on an output of the NCRL-based rule mining model and the knowledge graph.
Otani teaches identify a neural compositional rule learning (NCRL)-based rule mining model that generates the first user intent data (abstract, [0007]-[0010], [0021]disclosing the neural generative rule learning indicative of a NCRL based rule mining that generates intention of a user).
The combination of Otani is obvious to one of ordinary skill in the art in order to utilize neural networks, i.e., machine learning to improve the selection of intention of a user as taught by Otani which improves the intention selection as taught by Okamori as modified by KR.
Okamori as modified by KR and Otani does not teach regardless of the context instance based on the event instance among instances included in the knowledge graph from the rule creation model; and update the ECA rule based on an output of the NCRL-based rule mining model and the knowledge graph.
Korea teaches regardless of the context instance based on the event instance among instances included in the knowledge (at least abstract and [13]-[40] disclosing intention is inferred and then the intention is reasoned with context data later in order to obtain the ECA rule, i.e., update the rule).
the combination of the teaching of Korea of regardless of the context data and based on the event instance included in the knowledge from the rule creation and update the ECA rule based on output of the NCRL based rule mining model and graph with the knowledge graph and NCRL based rule mining as taught by Okamori as modified by KR and Otani is obvious in order to create a new rule based on an intention and context via machine learning yielding predictable results and automating the creation.
Conclusion
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure. The prior art cited in PTO-892 and not mentioned above disclose related devices and methods.
US20130211591 disclosing a plurality of ECA rules.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMAD O EL SAYAH whose telephone number is (571)270-7734. The examiner can normally be reached on M-Th 6:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached on (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMAD O EL SAYAH/Primary Examiner, Art Unit 3658B